所有子进程并行运行。(为避免这种情况,必须明确等待其完成。)它们甚至可以同时写入日志文件,从而使输出无效。为避免这种情况,您应该让每个进程写入不同的日志文件,并在所有进程完成后收集所有输出。
q = Queue.Queue()
result = {} # used to store the results
for fileName in fileNames:
q.put(fileName)
def worker():
while True:
fileName = q.get()
if fileName is None: # EOF?
return
subprocess_stuff_using(fileName)
wait_for_finishing_subprocess()
checksum = collect_md5_result_for(fileName)
result[fileName] = checksum # store it
threads = [ threading.Thread(target=worker) for _i in range(20) ]
for thread in threads:
thread.start()
q.put(None) # one EOF marker for each thread
之后,结果应存储在中result
。