如何在databricks DRE 7.6及更高版本上使用python3/python2 mypthon3.zip编译的zip执行spark提交作业?

2024-06-16 08:59:40 发布

您现在位置:Python中文网/ 问答频道 /正文

我试图在Databricks DRE 7.6及更高版本上使用python3和python2编译的mypthon3.zip执行spark提交作业,但出现以下错误

Traceback (most recent call last):
  File "/dbfs/tmp/WT_SPARK3/Inputsql.py", line 1, in <module>
    from com.test.mymodule import Mymodule
zipimport.ZipImportError: can't find module 'com'

mypython3.zip:使用python3编译并包含带有pycache文件夹的pyc文件

编译命令:python3-m compileall-b package\u dir

mypython3.zip
  |- com
      |- __pycache__ (empty folder)
      |- __init__.pyc
      |- test
           |- __pycache__ (empty folder)
           |- __init__.pyc
           |- mymodule.pyc

Inputsql.py

from com.test.mymodule import Mymodule
import sys
import time
import logging

火花提交命令

[“--py files”、“/dbfs/tmp/mypthon3.zip”、“/dbfs/tmp/Inputsql.py”]

请让我知道如何修理它

谢谢


Tags: frompytestimportcomziptmppython3