zeppelin은 처음 기동 시 /tmp에 뭔가 필요한 파일을 생성해서 사용하는 것 같다.
근데 장시간 zeppelin을 사용하지 않는다면 Centos의 tmpwatch에 의해 해당 디렉토리가 삭제되게 되고 수행이 되지 않는다.
해당 에러를 해결하기 위해서는 우선 재기동하면 해결되며,
동일 현상을 피하기 위해서는 tmpwarch에 해당 디렉토리와 파일을 등록해서 삭제되지 않도록 하는 것이 좋다.
[ Zeppein 에러 ]
Fail to execute line 79: myrdd = sc.parallelize(tuple_list)
Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-2151978668833769542.py", line 380, in <module>
File "<stdin>", line 79, in <module>
File "/home/app/packages/zeppelin-0.8.0-bin-all/interpreter/spark/pyspark/pyspark.zip/pyspark/context.py", line 484, in parallelize
tempFile = NamedTemporaryFile(delete=False, dir=self._temp_dir)
File "/usr/lib64/python2.7/tempfile.py", line 458, in NamedTemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
File "/usr/lib64/python2.7/tempfile.py", line 239, in _mkstemp_inner
fd = _os.open(file, flags, 0600)
OSError: [Errno 2] No such file or directory: '/tmp/spark-cbeb1fb9-d60d-45f4-80a9-4fb27e59b54f/pyspark-1cf398c7-7e1d-4999-9324-39bcbf36d23d/tmp1yXe7D'
[ Zeppelin 서버 상태 및 재기동 후 확인 ]
drasim [app{1021} /tmp ] ll
total 0
0 drwxr-xr-x 2 root root 42 Dec 25 16:49 hsperfdata_root/
0 drwx------ 2 app app 24 Feb 28 08:57 ssh-ZNqWo1HJE1/
0 drwx------ 3 root root 16 Oct 17 14:49 systemd-private-a38c92d517f245f99e6bfa2a10848367-colord.service-5y1KLz/
0 drwx------ 3 root root 16 Oct 17 14:49 systemd-private-a38c92d517f245f99e6bfa2a10848367-cups.service-iqszHM/
0 drwx------ 3 root root 16 Oct 17 14:48 systemd-private-a38c92d517f245f99e6bfa2a10848367-rtkit-daemon.service-aJ3THR/
drasim [app{1022} /tmp ]
drasim [app{1033} ~/packages/zeppelin-0.8.0-bin-all/bin ] sudo ./zeppelin-daemon.sh restart
[sudo] password for app:
Zeppelin stop [ OK ]
Zeppelin start [ OK ]
drasim [app{1037} /tmp ] ls -l
total 20
0 drwxr-xr-x 2 root root 30 Feb 28 09:11 hsperfdata_root/
0 drwx------ 4 root root 74 Feb 28 09:11 spark3897722150993516805/
0 drwx------ 2 app app 24 Feb 28 08:57 ssh-ZNqWo1HJE1/
0 drwx------ 3 root root 16 Oct 17 14:49 systemd-private-a38c92d517f245f99e6bfa2a10848367-colord.service-5y1KLz/
0 drwx------ 3 root root 16 Oct 17 14:49 systemd-private-a38c92d517f245f99e6bfa2a10848367-cups.service-iqszHM/
0 drwx------ 3 root root 16 Oct 17 14:48 systemd-private-a38c92d517f245f99e6bfa2a10848367-rtkit-daemon.service-aJ3THR/
4 -rw-r--r-- 1 root root 33 Feb 28 09:11 zeppelin3912350525182699802.txt
0 -rw-r--r-- 1 root root 0 Feb 28 09:11 zeppelin5734086877534788713.txt
16 -rw-r--r-- 1 root root 13518 Feb 28 09:11 zeppelin_pyspark-3091835907407287191.py
drasim [app{1038} /tmp ]