「爲」迭代與直接索引性能

Question

剛剛得到了一個奇怪的結果，我試圖理解。我有一個約325k行（列表）的數據集，每個約有90個項目（字符串，浮動等 - 它並不重要）。再說了，如果我想爲所有的項目做一些處理，然後我可以將它們用2「爲」結束了迭代：

for eachRow in rows: 
    for eachItem in eachRow: 
     # do something 

在我的系統驗證碼爲41秒執行。但是，如果我用一系列索引訪問（eachRow [0]，eachRowm [1]和到目前爲止的每個Row [89]）替換嵌套循環，則執行時間會降至25秒。

for eachRow in rows: 
    eachRow[0] # do something with this item 
    eachRow[1] # do something with this item 
    .. 
    eachRow[89] # do something with this item 

當然，寫代碼一樣，是不是一個好主意 - 我只是在尋找一種方式來impove數據處理性能，無意中發現這個奇怪的做法。任何意見？

Answer 1

對不起，這是我的錯。我的系統出了問題（這不是一個獨立的Python解釋器，而是一個內置的大系統）。重新啓動整個系統後，我得到了正確的結果 - 兩種變體約爲2.8秒。我覺得我好笨。尋找一種方法來刪除我的問題，因爲無關緊要。

Answer 2

展開過程似乎有輕微的性能優勢，但它可以忽略不計，因此除非您的do_something函數幾乎沒有任何功能，否則不應該看到差異。我有一段艱難的時間，相信不同方法的等效行爲可能達到60％，儘管我總是願意爲我從未想過的某些實現細節感到驚訝。

TL;博士總之，使用的32500，而不是325000，因爲我不耐煩：

do_nothing easy 3.44702410698 
do_nothing indexed 3.99766016006 
do_nothing mapped 4.36127090454 
do_nothing unrolled 3.33416581154 
do_something easy 5.4152610302 
do_something indexed 5.95649385452 
do_something mapped 6.20316290855 
do_something unrolled 5.2877831459 
do_more easy 16.6573209763 
do_more indexed 16.8381450176 
do_more mapped 17.6184959412 
do_more unrolled 16.0713188648 

CPython的2.7.3，代碼：

from timeit import Timer 

nrows = 32500 
ncols = 90 
a = [[1.0*i for i in range(ncols)] for j in range(nrows)] 

def do_nothing(x): 
    pass 

def do_something(x): 
    z = x+3 
    return z 

def do_more(x): 
    z = x**3+x**0.5+4 
    return z 

def easy(rows, action): 
    for eachRow in rows: 
     for eachItem in eachRow: 
      action(eachItem) 

def mapped(rows, action): 
    for eachRow in rows: 
     map(action, eachRow) 

def indexed(rows, action): 
    for eachRow in rows: 
     for i in xrange(len(eachRow)): 
      action(eachRow[i]) 

def unrolled(rows, action): 
    for eachRow in rows: 
     action(eachRow[0]) 
     action(eachRow[1]) 
     action(eachRow[2]) 
     action(eachRow[3]) 
     action(eachRow[4]) 
     action(eachRow[5]) 
     action(eachRow[6]) 
     action(eachRow[7]) 
     action(eachRow[8]) 
     action(eachRow[9]) 
     action(eachRow[10]) 
     action(eachRow[11]) 
     action(eachRow[12]) 
     action(eachRow[13]) 
     action(eachRow[14]) 
     action(eachRow[15]) 
     action(eachRow[16]) 
     action(eachRow[17]) 
     action(eachRow[18]) 
     action(eachRow[19]) 
     action(eachRow[20]) 
     action(eachRow[21]) 
     action(eachRow[22]) 
     action(eachRow[23]) 
     action(eachRow[24]) 
     action(eachRow[25]) 
     action(eachRow[26]) 
     action(eachRow[27]) 
     action(eachRow[28]) 
     action(eachRow[29]) 
     action(eachRow[30]) 
     action(eachRow[31]) 
     action(eachRow[32]) 
     action(eachRow[33]) 
     action(eachRow[34]) 
     action(eachRow[35]) 
     action(eachRow[36]) 
     action(eachRow[37]) 
     action(eachRow[38]) 
     action(eachRow[39]) 
     action(eachRow[40]) 
     action(eachRow[41]) 
     action(eachRow[42]) 
     action(eachRow[43]) 
     action(eachRow[44]) 
     action(eachRow[45]) 
     action(eachRow[46]) 
     action(eachRow[47]) 
     action(eachRow[48]) 
     action(eachRow[49]) 
     action(eachRow[50]) 
     action(eachRow[51]) 
     action(eachRow[52]) 
     action(eachRow[53]) 
     action(eachRow[54]) 
     action(eachRow[55]) 
     action(eachRow[56]) 
     action(eachRow[57]) 
     action(eachRow[58]) 
     action(eachRow[59]) 
     action(eachRow[60]) 
     action(eachRow[61]) 
     action(eachRow[62]) 
     action(eachRow[63]) 
     action(eachRow[64]) 
     action(eachRow[65]) 
     action(eachRow[66]) 
     action(eachRow[67]) 
     action(eachRow[68]) 
     action(eachRow[69]) 
     action(eachRow[70]) 
     action(eachRow[71]) 
     action(eachRow[72]) 
     action(eachRow[73]) 
     action(eachRow[74]) 
     action(eachRow[75]) 
     action(eachRow[76]) 
     action(eachRow[77]) 
     action(eachRow[78]) 
     action(eachRow[79]) 
     action(eachRow[80]) 
     action(eachRow[81]) 
     action(eachRow[82]) 
     action(eachRow[83]) 
     action(eachRow[84]) 
     action(eachRow[85]) 
     action(eachRow[86]) 
     action(eachRow[87]) 
     action(eachRow[88]) 
     action(eachRow[89]) 


def timestuff(): 
    for action in 'do_nothing do_something do_more'.split(): 
     for name in 'easy indexed mapped unrolled'.split(): 
      t = Timer(setup=""" 
from __main__ import {} as fn 
from __main__ import {} as action 
from __main__ import a 
""".format(name, action), 
         stmt="fn(a, action)").timeit(10) 
      print action, name, t 

if __name__ == '__main__': 
    timestuff() 

（請注意，我沒有理會製作因爲我只是試圖衡量變化的可能的規模，即訂單是否變化的統一或不變）。

Answer 3

與其他響應者不同，他定時這一點，我看到一個相當大的di時間上的差異。首先，我的代碼：

import random 
import string 
import timeit 

r = 1000 
outer1 = [[[''.join([random.choice(string.ascii_letters) for j in range(10)])] for k in range(90)] for l in range(r)] 
outer2 = [[[''.join([random.choice(string.ascii_letters) for j in range(10)])] for k in range(90)] for l in range(r)] 
outer3 = [[[''.join([random.choice(string.ascii_letters) for j in range(10)])] for k in range(90)] for l in range(r)] 

def x1(L): 
    for outer in L: 
     for inner in L: 
      inner = inner[:-1] 

def x2(L): 
    for outer in L: 
     for y in range(len(outer)): 
      outer[y] = outer[y][:-1] 

def x3(L): 
    for x in range(len(L)): 
     for y in range(len(L[x])): 
      L[x][y] = L[x][y][:-1] 

print "x1 =",timeit.Timer('x1(outer1)', "from __main__ import x1,outer1").timeit(10) 
print "x2 =",timeit.Timer('x2(outer2)', "from __main__ import x2,outer2").timeit(10) 
print "x3 =",timeit.Timer('x3(outer3)', "from __main__ import x3,outer3").timeit(10) 

注意我跑這些10倍。每個列表都填充有3000個項目，每個項目包含90個項目，每個項目都是10個字母的隨機字符串。

代表性結果：

x1 = 8.0179214353 
x2 = 0.118051644801 
x3 = 0.150409681521 

使用沒有索引（X1）該函數採用66倍更長的時間來執行比確實其使用索引僅在內部循環（X2）的一個。奇怪的是，僅使用內部循環索引（x2）的函數比使用外部循環和內部循環索引（x3）的函數更好。