在被多进程调用的函数内重新导入Python模块?

0 投票
0 回答
20 浏览
提问于 2025-04-12 10:25

我写了一个Python函数,用来多进程地复制文件夹。我使用的是multiprocess模块,而不是multiprocessing,因为我是在Jupyter Notebook里运行代码。刚开始运行代码的时候,我遇到了一个错误,提示NameError: name 'shutil' is not defined。我花了一些时间才搞明白,原来我需要在main()这个函数里再导入一次shutil。请问这是为什么呢?

from multiprocess import Pool
import shutil

def main(args):
    # Re-import shutil.
    import shutil
    
    # Unpack arguments.
    src_dir = args["src_dir"]
    dst_dir = args["dst_dir"]
    dir_id = args["dir_id"]

    # Copy source directory to a folder in the destination directory.
    shutil.copytree(src_dir, dst_dir / "folder_{}".format(dir_id))

    return "Copied"

# Source and destination directories.
src_directory = Path(r"~\src")
dst_directory = Path(r"~\dst")

# Create argument list.
args = [
    {
     "src_dir": src_directory, 
     "dst_dir": dst_directory, 
     "dir_id": i
    } for i in range(10)
]

# Multiprocess.
with Pool(processes=batch_size) as pool:
    out = pool.map(main, args)

这是在main()函数里导入shutil之前的错误信息。

Traceback (most recent call last):
  File "c:\~\lib\site-packages\multiprocess\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "c:\~\lib\site-packages\multiprocess\pool.py", line 48, in mapstar
    return list(map(*args))
  File "C:\~\AppData\Local\Temp\ipykernel_23252\412796622.py", line 5, in main
NameError: name 'shutil' is not defined

我尝试在main()函数里导入shutil

0 个回答

暂无回答

撰写回答