在被多进程调用的函数内重新导入Python模块?
我写了一个Python函数,用来多进程地复制文件夹。我使用的是multiprocess
模块,而不是multiprocessing
,因为我是在Jupyter Notebook里运行代码。刚开始运行代码的时候,我遇到了一个错误,提示NameError: name 'shutil' is not defined
。我花了一些时间才搞明白,原来我需要在main()
这个函数里再导入一次shutil。请问这是为什么呢?
from multiprocess import Pool
import shutil
def main(args):
# Re-import shutil.
import shutil
# Unpack arguments.
src_dir = args["src_dir"]
dst_dir = args["dst_dir"]
dir_id = args["dir_id"]
# Copy source directory to a folder in the destination directory.
shutil.copytree(src_dir, dst_dir / "folder_{}".format(dir_id))
return "Copied"
# Source and destination directories.
src_directory = Path(r"~\src")
dst_directory = Path(r"~\dst")
# Create argument list.
args = [
{
"src_dir": src_directory,
"dst_dir": dst_directory,
"dir_id": i
} for i in range(10)
]
# Multiprocess.
with Pool(processes=batch_size) as pool:
out = pool.map(main, args)
这是在main()
函数里导入shutil之前的错误信息。
Traceback (most recent call last):
File "c:\~\lib\site-packages\multiprocess\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "c:\~\lib\site-packages\multiprocess\pool.py", line 48, in mapstar
return list(map(*args))
File "C:\~\AppData\Local\Temp\ipykernel_23252\412796622.py", line 5, in main
NameError: name 'shutil' is not defined
我尝试在main()
函数里导入shutil
。
0 个回答
暂无回答