如何並行執行cython文件編譯？

Question

我想並行執行cython文件編譯。

所以，我看看Cython.Build源文件，並查找以下簽名cythonize功能：

def cythonize(module_list, exclude=None, nthreads=0, aliases=None, 
       quiet=False, force=False, language=None, 
       exclude_failures=False, **options): 

而關於cythonize nthreads選項如下評論：

"For parallel compilation, set the 'nthreads' option to the number of 
concurrent builds." 

所以，我想在我的setup.py文件中使用此選項，如下所示：

from setuptools import setup 
from Cython.Build import cythonize 
from Cython.Distutils.extension import Extension 

EXTENSIONS = [Extension(...) 
       ... 
       Extension(...)] 

setup(name='...', 
     ... 
     ext_modules=cythonize(EXTENSIONS, nthreads=8), 
     ...) 

但我的.pyx文件仍然使用1個線程進行編譯。

我不明白我在這裏做錯了什麼，以及如何使用nthreads選項來並行執行cythonize編譯？

Answer 1

我終於找到了解決編譯我用Cython文件中並行：

from setuptools import setup 
from Cython.Build import cythonize 
from Cython.Distutils.extension import Extension 

NB_COMPILE_JOBS = 8 

EXTENSIONS = [Extension(...) 
       ... 
       Extension(...)] 

def setup_given_extensions(extensions): 
    setup(name='...', 
      ... 
      ext_modules=cythonize(extensions), 
      ...) 

def setup_extensions_in_sequential(): 
    setup_given_extensions(EXTENSIONS) 

def setup_extensions_in_parallel(): 
    cythonize(EXTENSIONS, nthreads=NB_COMPILE_JOBS) 
    pool = multiprocessing.Pool(processes=NB_COMPILE_JOBS) 
    pool.map(setup_given_extensions, EXTENSIONS) 
    pool.close() 
    pool.join() 

if "build_ext" in sys.argv: 
    setup_extensions_in_parallel() 
else: 
    setup_extensions_in_sequential()