Bypass「Array is too big」python error

我有一個csv文件，其中包含超過200 000行的氣象數據。當我想將數據與matplotlib建模，出現此錯誤：Bypass「Array is too big」python error

Traceback (most recent call last): 
    File "try4.py", line 19, in <module> 
    X,Y = meshgrid(data_x,data_y) 
    File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line 3378, in meshgrid 
    mult_fact = np.ones(shape, dtype=int) 
    File "C:\Python27\lib\site-packages\numpy\core\numeric.py", line 148, in ones 
    a = empty(shape, dtype, order) 
    ValueError: array is too big.

我發現有5000行最大一個文件進行處理。如何繞過錯誤以處理200000行的所有文件？這裏是我的代碼：

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 


# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 

# print CSV file headers 
print data.dtype.names 

# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 

X,Y = meshgrid(data_x,data_y) 
U = cos(data_u) 
V = sin(data_v) 


# plot raw data 
Q = quiver(X, Y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
       fontproperties={'weight': 'bold'}) 
title('Current Surface') 

plt.show()

來源

2013-06-26 So4ne

數組元素看起來像什麼？這個答案：http://stackoverflow.com/questions/13652650/matplotlib-plot-array-size-limit似乎認爲沒有一個明顯的數組限制（如果這樣至少在1m +元素） – ydaetskcoR

@ydaetskcoR在這裏是一個示例 //頭X，Y，經緯度，VOGRD_201304100000 //數據292,1,000000，-50.530000,0和200000其他類似的行 – So4ne

您能否提供更多信息？你正在使用哪種'matplotlib'調用等。 – tiago

爲什麼您使用meshgrid（doc）？它很好地生成了一個200k×200k的數組，它與你的u和v數據的尺寸不匹配。我想你想這樣做

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 


# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 

# print CSV file headers 
print data.dtype.names 

# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 

U = cos(data_u) 
V = sin(data_v) 


# plot raw data 
Q = quiver(data_x, data_y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
       fontproperties={'weight': 'bold'}) 
title('Current Surface')

來源

2013-06-26 09:47:46 tacaswell

你是對的，它完美的作品。謝謝 – So4ne

Bypass「Array is too big」python error

回答

相關問題