Windows下Python多进程无限循环

Question

我正在尝试使用多个进程进行我的第一次模拟，目前每次模拟大约需要一个小时才能完成。为此，我使用了多进程的导入。

当我在一个循环中运行simulation()时，所有代码都能正常工作。但是，当程序运行时，池中的所有工作者都开始做同样的任务，所以我最后得到了4个相同的副本，而且当它们完成后又会重新开始。此外，当池被创建时，整个程序从第一行开始同时运行四次，而不是只运行模拟。

下面的代码几乎完全是从多进程文档中的一个示例复制粘贴过来的：http://docs.python.org/library/multiprocessing.html

我在使用Windows 7，尽快会在Linux机器上运行，但有人能告诉我这里出了什么问题吗？

非常感谢

附注：Windows 7（64位）+ Python 2.7.1

....

code where: simulation, pulse1, pulse2, sol_ref, model, algorithm and i are defined. Also 
where the various imports are made. Also some print statements.

...


if __name__=='__main__':
    freeze_support()

    def calculate(func, args):
        result = func(*args)
        return '%s says that %s%s = %s' % (
            multiprocessing.current_process().name,
            func.__name__, args, result
            )


    PROCESSES = 4
    print 'Creating pool with %d processes\n' % PROCESSES
    pool = Pool(PROCESSES)
    print 'pool = %s' % pool
    print

    TASKS = [(simulation, (pulse1,pulse2,sol_ref,model,algorithm,i)) for i in delta_tau]

    results = [pool.apply_async(calculate, t) for t in TASKS]
    for r in results:
        print '\t', r.get()
    print

编辑：错误日志

打印模拟中点数的那几行代码来自于if main语句之前的代码。在这段代码中，模拟在一个循环中运行一次，以检查它是否正常工作并给出预期的结果。

C:\Users\HP\Desktop\experiment>python amplifier_parallel.py
Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs

delta_tau:  -6.0
delta_tau:  -3.85714285714
delta_tau:  -1.71428571429
delta_tau:  0.428571428571
delta_tau:  2.57142857143
delta_tau:  4.71428571429
delta_tau:  6.85714285714
delta_tau:  9.0
Simulation Time: 43.1258663953 seconds
Creating pool with 4 processes

pool = <multiprocessing.pool.Pool object at 0x063E66B0>

        Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs
...3 more times...

delta_tau:  -6.0
...3more times
delta_tau:  -3.85714285714
... 3 more tiemes
delta_tau:  -1.71428571429
... 3 more tiemes
delta_tau:  0.428571428571
... 3 more tiemes
delta_tau:  2.57142857143
... 3 more tiemes3
delta_tau:  4.71428571429
... 3 more tiemes
delta_tau:  6.85714285714
... 3 more tiemes
delta_tau:  9.0
... 3 more tiemes
Simulation Time: 63.3316095785 seconds
... 3 more times with slightly different times, depending on the worker who did the job
Process PoolWorker-4:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 232, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 59, in worker
    task = get()
  File "C:\Python27\lib\multiprocessing\queues.py", line 352, in get
    return recv()
AttributeError: 'module' object has no attribute 'calculate'
Number of z points:  169   dist:  20.0  mm  dz:  118.343195266  um
Number of t points:  8192.0   window span:  69.0  ps  dt:  8.4228515625  fs

Process PoolWorker-3:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 232, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Python27\lib\multiprocessing\pool.py", line 59, in worker
    task = get()
  File "C:\Python27\lib\multiprocessing\queues.py", line 352, in get
    return recv()
AttributeError: 'module' object has no attribute 'calculate'
delta_tau:  -6.0
... this kind of stuff goes on for ever until i control-C

代码优化模拟错误调试多进程并行处理任务调度计算性能 windows 7

Windows下Python多进程无限循环

1 个回答

撰写回答