EMR 6.0 Hue(oozie workflow editor) 기능 애러 테크노트

2020-04-26

.

  1. 개요

EMR 6.0 Hue oozie workflow editor 접근 시 발생한 애러에 대한 테크노트임

5.29 ver 에서도 동일한 애러발생, 5.28 ver 이하는 정상작동

  1. 애러식별 내용

1) oozie workflow editor 기능접근 시도 시 can not open oozie workflow editor 현상확인

   exception    ERROR    Internal Server Error: /desktop/workers/aceSqlLocationWorker.js
Traceback (most recent call last):
  File "/usr/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.20-py2.7.egg/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/usr/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.20-py2.7.egg/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/usr/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.20-py2.7.egg/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.20-py2.7.egg/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/hue/build/env/lib/python2.7/site-packages/Django-1.11.20-py2.7.egg/django/utils/decorators.py", line 185, in inner
    return func(*args, **kwargs)
  File "/usr/local/hue/desktop/core/src/desktop/views.py", line 373, in ace_sql_location_worker
    return HttpResponse(render('ace_sql_location_worker.mako', request, None), content_type="application/javascript")
  File "/usr/local/hue/desktop/core/src/desktop/lib/django_util.py", line 225, in render
    data.update({'user': request.user})

애러 재현방법

STEP 1) EMR 6.0을 아래와 같이 구동

1

STEP 2) Hue에 접속하여 아래 그림과 같이 oozie workflow editor 실행

2

STEP 3) 아래 그림과 같이 500 ERROR 확인

3

2) new workflow folder 생성이 안되는 현상

hue IOError: [Errno None] Error accessing directory /user/hue/oozie/workspaces/hue-oozie-1501307582.31

3) 사용자 home directory 생성이 안되는 현상

  1. 트러블 슈팅 : AWS 문의 후 해당오류 내용에 대한 임시 패치파일을 다운받아서 해결

“AWS에서는 해당 애러를 인지하여, 영구적으로 수정보완하는 작업을 진행중이다. 그전까지는 임시로 패치파일(huePatch.diff)을 적용하여 해결할 것”

[답변내용 일부발췌]

It seems that Hue is unable to find a directory for the user admin. 

Please find log snippet below:
------------------
ERROR    All 1 clients failed: {'http://ip-000-000-0-000.ap-northeast-2.compute.internal:14000/webhdfs/v1 ': u'404 Client Error: Not Found for url: http://ip-000-000-0-000.ap-northeast-2.compute.internal:14000/webhdfs/v1/user/admin?op=GETFILESTATUS&user.name=hue&doas=admin\n {"RemoteException":{"message":"File does not exist: \\/user\\/admin","exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException"}}\n'}
------------------

The service team is aware of this issue and it is encountered on EMR version 5.29 and 6.0. The team is currently working on a permanent fix, but in the meantime has provided the following steps as a workaround to mitigate the issue:

1. Download the attached patch file and copy it to the master node of your Cluster
2. SSH into EMR master node and go to hue directory: $cd /usr/lib/hue/
3. Apply the Hue patch: $sudo patch -p1 < ~/huePatch.diff
4. Restart Hue: 
    $sudo /etc/init.d/hue stop
    $sudo /etc/init.d/hue start 
    $sudo /etc/init.d/hue status --> Verify it is ok
5. Open the Hue UI and try creating the Oozie Workflow.

This patch is also tested in my test environment and is working fine. Please find the attached screen-shot for your reference.