您好, 欢迎来到 !    登录 | 注册 | | 设为首页 | 收藏本站

python joblib并行在Windows上工作甚至“如果__name__ ==’__ main__’:”被添加

5b51 2022/1/14 8:22:20 python 字数 3920 阅读 532 来源 www.jb51.cc/python

我在 Windows上运行 Python并行处理.这是我的代码: from joblib import Parallel, delayed def f(x): return sqrt(x) if __name__ == '__main__': a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10)) 这是错误消息: Pr

概述

from joblib import Parallel,delayed

def f(x): 
    return sqrt(x)

if __name__ == '__main__':
    a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10))

这是错误消息:

Process PoolWorker-2:  
Process PoolWorker-1:  
Traceback (most recent call last):    
File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py",line 258,in _bootstrap
self.run()   
File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\process.py",line 114,in run
self._target(*self._args,**self._kwargs)   
File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.5.4.3105.win-x86_64\lib\multiprocessing\pool.py",line 102,in worker
task = get()   
File "C:\Users\yoyo__000.BIGBLACK\AppData\Local\Enthought\Canopy\User\lib\site-packages\joblib\pool.py",line 363,in get
return recv()  
AttributeError: 'module' object has no attribute 'f'

Yes: under linux we are forking,thus their is no need to pickle the
function,and it works fine. Under windows,the function needs to be
pickleable,ie it needs to be imported from another file. This is
actually good practice: making modules pushes for reuse.

我已经尝试过你的代码,它在Linux下运行完美.
在Windows下,如果从脚本运行,它运行正常,如python script_with_your_code.py.但是在交互式python会话中运行时失败了.当我将f函数保存在单独的模块中并将其导入到我的交互式会话中时,它对我有用.

不工作:

互动环节:

>>> from math import sqrt
>>> from joblib import Parallel,delayed

>>> def f(x):
...     return sqrt(x)

>>> if __name__ == '__main__':
...     a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10))
...
Process PoolWorker-1:
Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py",in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py",in run
    self._target(*self._args,**self._kwargs)
  File "C:\Python27\lib\multiprocessing\pool.py",in worker
    task = get()
  File "C:\Python27\lib\site-packages\joblib\pool.py",line 359,in get
    return recv()
AttributeError: 'module' object has no attribute 'f'

工作方式:
fun.py

from math import sqrt

def f(x):
    return sqrt(x)

互动环节:

>>> from joblib import Parallel,delayed
>>> from fun import f

>>> if __name__ == '__main__':
...     a = Parallel(n_jobs=2)(delayed(f)(i) for i in range(10))
...
>>> a
[0.0,1.0,1.4142135623730951,1.7320508075688772,2.0,2.23606797749979,2.449489742783178,2.6457513110645907,2.8284271247461903,3.0]

总结

以上是编程之家为你收集整理的python joblib并行在Windows上工作甚至“如果__name__ ==’__ main__’:”被添加全部内容,希望文章能够帮你解决python joblib并行在Windows上工作甚至“如果__name__ ==’__ main__’:”被添加所遇到的程序开发问题。


如果您也喜欢它,动动您的小指点个赞吧

除非注明,文章均由 laddyq.com 整理发布,欢迎转载。

转载请注明:
链接:http://laddyq.com
来源:laddyq.com
著作权归作者所有。商业转载请联系作者获得授权,非商业转载请注明出处。


联系我
置顶