2013-09-21 175 views
2

我正在使用scikit-learn解決大量標籤的多標籤分類問題。來自該項目的核心開發人員之一(larsmans)的followed the ideas。它給了我一個運行時警告,然後最終出現內存錯誤。帶特徵選擇的多標籤分類(scikit-learn)

>>> clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),('svm', LinearSVC())]) 
>>> 
>>> multiclf = OneVsRestClassifier(clf, n_jobs=-1) 
>>> 
>>> multiclf.fit(Xtr, ytr) 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: 
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157:    
RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 

然後多個這樣的警告後,它失敗:下面

Process PoolWorker-21: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run 
    self.__target(*self.__args, **self.__kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers 
    pool._maintain_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool 
    self._repopulate_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool 
    w.start() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start 
    self._popen = Popen(self) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__ 
    self.pid = os.fork() 
OSError: [Errno 12] Cannot allocate memory 

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-22: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-23: 
Process PoolWorker-30: 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-31: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-28: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-32: 
Process PoolWorker-29: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-27: 
Process PoolWorker-26: 
Traceback (most recent call last): 
Process PoolWorker-25: 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self.run() 
    self.run() 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-24: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, 'k'. 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-2: 
Traceback (most recent call last): 
Process PoolWorker-3: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-4: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-6: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-5: 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-7: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: unpickling stack underflow 
Process PoolWorker-8: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, ''. 
Process PoolWorker-10: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, '�'. 
Process PoolWorker-21: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 808, in __bootstrap_inner 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/threading.py", line 761, in run 
    self.__target(*self.__args, **self.__kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 325, in _handle_workers 
    pool._maintain_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 229, in _maintain_pool 
    self._repopulate_pool() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 222, in _repopulate_pool 
    w.start() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 130, in start 
    self._popen = Popen(self) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/forking.py", line 121, in __init__ 
    self.pid = os.fork() 
OSError: [Errno 12] Cannot allocate memory 

/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-22: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-23: 
Process PoolWorker-30: 
/home/rsivapr/scikit-learn/sklearn/feature_selection/univariate_selection.py:157: RuntimeWarning: invalid value encountered in divide 
    chisq /= f_exp 
Process PoolWorker-31: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-28: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-32: 
Process PoolWorker-29: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-27: 
Process PoolWorker-26: 
Traceback (most recent call last): 
Process PoolWorker-25: 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self.run() 
    self.run() 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
    return recv() 
MemoryError 
MemoryError 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-24: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, 'k'. 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-1: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-2: 
Traceback (most recent call last): 
Process PoolWorker-3: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-4: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
Process PoolWorker-6: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
Process PoolWorker-5: 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
    return recv() 
MemoryError 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
MemoryError 
Process PoolWorker-7: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: unpickling stack underflow 
Process PoolWorker-8: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, ''. 
Process PoolWorker-10: 
Traceback (most recent call last): 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap 
    self.run() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run 
    self._target(*self._args, **self._kwargs) 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/pool.py", line 102, in worker 
    task = get() 
    File "/home/rsivapr/anaconda/lib/python2.7/multiprocessing/queues.py", line 376, in get 
    return recv() 
UnpicklingError: invalid load key, '�'. 

回答

0

Seing所有MemoryErrors錯誤轉儲,你可能已經耗盡內存。嘗試用n_jobs = 1替換n_jobs = -1。這將控制產生的進程數量(從而控制使用的內存量)。

如果可行,只要你沒有耗盡內存,即n_jobs = 2,n_jobs = 3等等,你就可以增加n_jobs。