在AWS Jupyter EMR中找不到python包

Question

我是AWS EMR Jupyter环境的新手，一直在安装python包。我现在已经设置了一个单节点EMR集群，并通过在节点中执行SSH，在docker容器中安装了这个包looker_sdk。我可以使用iPython/pythonshell查看其已安装和可用，并可以导入它。但当我尝试从Jupyter笔记本导入时，它仍然失败

root@jupyterhub:~# pip freeze | grep looker
WARNING: The directory '/home/jovyan/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
looker-sdk==0.1.3b15

但在朱皮特：

print(sys.path)
import looker_sdk

['/home/notebook/work', '/opt/conda/lib/python37.zip', '/opt/conda/lib/python3.7', '/opt/conda/lib/python3.7/lib-dynload', '', '/opt/conda/lib/python3.7/site-packages', '/opt/conda/lib/python3.7/site-packages/awseditorssparkmonitoringwidget-1.0-py3.7.egg', '/opt/conda/lib/python3.7/site-packages/IPython/extensions', '/home/notebook/.ipython']
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-13-f7a797152935> in <module>
      1 print(sys.path)
----> 2 import looker_sdk

ModuleNotFoundError: No module named 'looker_sdk'

此外，目录列表给出了不同的结果，不同的用户拥有不同的目录

从docker容器中的Shell：

root@jupyterhub:~# ls -ltr /opt/conda/
total 56
-rw-rw-r--  1 jovyan users 11245 Mar  3 01:40 LICENSE.txt
drwsrwsr-x  1 jovyan users    21 Mar 17 21:11 x86_64-conda_cos6-linux-gnu
drwsrwsr-x  3 jovyan users    22 Mar 17 21:11 shell
drwsrwsr-x  2 jovyan users    19 Mar 17 21:16 condabin
drwsrwsr-x  2 jovyan users    30 Mar 17 21:16 compiler_compat
drwsrwsr-x  2 jovyan users   203 Mar 17 21:21 sbin
drwsrwsr-x  1 jovyan users   146 Apr  8 21:36 ssl
drwxr-sr-x  3 jovyan users    19 Apr  8 21:37 var
drwxr-sr-x  2 jovyan users   175 Apr  8 21:37 resources
drwxr-sr-x 26 jovyan users  4096 Apr  8 21:37 qml
drwxr-sr-x  2 jovyan users   252 Apr  8 21:37 phrasebooks
drwxr-sr-x 75 jovyan users  4096 Apr  8 21:37 mkspecs
drwxr-sr-x  3 jovyan users    86 Apr  8 21:37 libexec
drwsrwsr-x  1 jovyan users    23 Apr  8 21:37 lib
drwsrwsr-x  1 jovyan users  4096 Apr  8 21:37 include
drwsrwsr-x  1 jovyan users    21 Apr  8 21:37 etc
drwxr-sr-x  3 jovyan users    20 Apr  8 21:37 doc
drwxr-sr-x  3 jovyan users 12288 Apr  8 21:37 translations
drwsrwsr-x  1 jovyan users    21 Apr  8 21:37 share
drwxr-sr-x  3 jovyan users    17 Apr  8 21:37 qsci
drwxr-sr-x 30 jovyan users  4096 Apr  8 21:37 plugins
drwxr-sr-x  3 jovyan users    18 Apr  8 21:37 man
drwsrwsr-x  1 jovyan users    21 Apr  8 21:38 conda-meta
drwsrwsr-x  1 jovyan users   212 Apr  8 21:38 bin
drwsrwsr-x  1 jovyan users    55 Aug  3 12:16 envs
drwxr-sr-x  1 jovyan users  8192 Aug  3 12:56 pkgs

来自Jupyter：

!ls -ltr /opt/conda
total 124
-rw-rw-r--  1 notebook users 11245 Mar  3 01:41 LICENSE.txt
drwsrwsr-x  1 notebook users  4096 Apr 30 01:44 x86_64-conda_cos6-linux-gnu
drwsrwsr-x  3 notebook users  4096 Apr 30 01:44 man
drwsrwsr-x  3 notebook users  4096 Apr 30 01:44 var
drwsrwsr-x  3 notebook users  4096 Apr 30 01:44 doc
drwsrwsr-x  3 notebook users  4096 Apr 30 01:44 shell
drwsrwsr-x  2 notebook users  4096 Apr 30 01:44 envs
drwsrwsr-x  1 notebook users  4096 Apr 30 01:46 etc
drwsrwsr-x  3 notebook users  4096 Apr 30 01:46 qsci
drwsrwsr-x 30 notebook users  4096 Apr 30 01:46 plugins
drwsrwsr-x  2 notebook users  4096 Apr 30 01:47 condabin
drwsrwsr-x  3 notebook users 12288 Apr 30 01:47 translations
drwsrwsr-x  2 notebook users  4096 Apr 30 01:47 resources
drwsrwsr-x 26 notebook users  4096 Apr 30 01:47 qml
drwsrwsr-x  2 notebook users  4096 Apr 30 01:47 phrasebooks
drwsrwsr-x 75 notebook users  4096 Apr 30 01:47 mkspecs
drwsrwsr-x  4 notebook users  4096 Apr 30 01:47 libexec
drwsrwsr-x  2 notebook users  4096 Apr 30 01:47 compiler_compat
drwsrwsr-x  1 root     users  4096 Apr 30 01:50 locks
drwsrwsr-x  1 notebook users  4096 Apr 30 01:53 ssl
drwsrwsr-x  1 notebook users  4096 Apr 30 01:54 share
drwsrwsr-x  1 notebook users  4096 Apr 30 01:54 sbin
drwsrwsr-x  1 notebook users  4096 Apr 30 01:54 lib
drwsrwsr-x  1 notebook users  4096 Apr 30 01:54 include
drwsrwsr-x  1 notebook users  4096 Apr 30 01:57 bin
drwsrwsr-x  1 notebook users  4096 Apr 30 02:02 pkgs
drwsrwsr-x  1 notebook users  4096 Apr 30 02:02 conda-meta

我试图研究和阅读这方面的内容，但找不到任何东西可以解释这是如何工作的。请帮忙

更新：如果我们直接转到JupyterHub控制台，我们会看到使用了用户“jovyan”。但是如果我们通过连接集群从EMR控制台打开notebook，就会使用用户“notebook”，并且我们在安装内核时所做的任何更改都不可见。我会尽快发布答案，只要我能破解这个