使用以下方法将方法委派给线程或子流程非常简单BaseEventLoop.run_in_executor
:
import asyncio
import time
from concurrent.futures import ProcessPoolExecutor
def cpu_bound_operation(x):
time.sleep(x) # This is some operation that is cpu-bound
@asyncio.coroutine
def main():
# Run cpu_bound_operation in the ProcessPoolExecutor
# This will make your coroutine block, but won't block
# the event loop; other coroutines can run in meantime.
yield from loop.run_in_executor(p, cpu_bound_operation, 5)
loop = asyncio.get_event_loop()
p = ProcessPoolExecutor(2) # Create a ProcessPool with 2 processes
loop.run_until_complete(main())
至于使用aProcessPoolExecutor
还是ThreadPoolExecutor
,这很难说。腌制一个大物体肯定会消耗一些cpu周期,最初您会认为这ProcessPoolExecutor
是可行的方法。但是,将100MB对象传递到Process
池中的a将需要在主进程中腌制该实例,通过IPC将字节发送到子进程,在子进程中将其取消腌制,然后再次 进行腌制,以便可以将其写入磁盘。鉴于此,我的猜测是,酸洗/去酸洗的开销将足够大ThreadPoolExecutor
,即使使用GIL会对性能造成负面影响,您也最好使用。
也就是说,两种方法的测试都非常简单,并且可以确定找出来,所以您也可以这样做。