在字符串長列表上迭代的優化

下面的代碼片段執行時間大約爲18s。據估計，EdgeList的長度爲330K。考慮到我多次調用它，有沒有辦法優化它。在字符串長列表上迭代的優化

我試過優化插入到我的MeanspeedDict。但我猜這是它已經可以最好的了？

EdgeList = traci.edge.getIDList() #Returns a list of Strings 
    for edge in EdgeList: 
     meanspeed = traci.edge.getLastStepMeanSpeed(edge) #returns a float value 
     ''' 
     if edge in MeanspeedDict: 
      MeanspeedDict[edge].append(meanspeed) 
      MeanspeedDict[edge] = MeanspeedDict[edge][-300:] #Only keep the last 300 values 
     else: 
      MeanspeedDict[edge] = [meanspeed] 
     ''' 
     try: 
      MeanspeedDict[edge].append(meanspeed) 
      MeanspeedDict[edge] = MeanspeedDict[edge][-300:] #Only keep the last 300 values 
     except KeyError: 
      MeanspeedDict[edge] = [meanspeed]

配置文件數據按請求。運行它次

  252229348 function calls in 295.056 seconds 

    Ordered by: standard name 

    ncalls tottime percall cumtime percall filename:lineno(function) 
     1 0.000 0.000 295.056 295.056 <string>:1(<module>) 
     1 0.000 0.000 40.158 40.158 __init__.py:101(getVersion) 
     1 0.000 0.000 4.908 4.908 __init__.py:105(close) 
     1 0.000 0.000 0.000 0.000 __init__.py:111(switch) 
     1 0.000 0.000 0.507 0.507 __init__.py:45(connect) 
     1 0.000 0.000 0.000 0.000 __init__.py:49(normalize_encoding) 
     1 0.000 0.000 40.665 40.665 __init__.py:64(init) 
     1 0.000 0.000 0.008 0.008 __init__.py:71(search_function) 
     11 0.000 0.000 1.646 0.150 __init__.py:92(simulationStep) 
    3607912 2.785 0.000 221.939 0.000 _edge.py:151(getLastStepMeanSpeed) 
     1 0.000 0.000 0.000 0.000 codecs.py:92(__new__) 
    3607923 5.689 0.000 16.796 0.000 connection.py:119(_beginMessage) 
    3607923 3.451 0.000 210.529 0.000 connection.py:128(_sendReadOneStringCmd) 
    3607923 8.231 0.000 190.282 0.000 connection.py:152(_checkResult) 
     11 0.000 0.000 1.646 0.150 connection.py:254(simulationStep) 
     1 0.000 0.000 40.158 40.158 connection.py:273(getVersion) 
     1 0.000 0.000 4.908 4.908 connection.py:285(close) 
     1 0.000 0.000 0.507 0.507 connection.py:48(__init__) 
    3607923 4.167 0.000 8.645 0.000 connection.py:64(_packString) 
    3607936 19.808 0.000 108.622 0.000 connection.py:72(_recvExact) 
    3607936 19.507 0.000 200.505 0.000 connection.py:91(_sendExact) 
     15 0.000 0.000 0.000 0.000 copy.py:123(_copy_inst) 
     15 0.000 0.000 0.000 0.000 copy.py:66(copy) 
     15 0.000 0.000 0.000 0.000 domain.py:108(_setConnection) 
    3607923 4.273 0.000 236.643 0.000 domain.py:111(_getUniversal) 
     11 0.004 0.000 17.493 1.590 domain.py:116(getIDList) 
     15 0.000 0.000 0.000 0.000 domain.py:37(__init__) 
     495 0.000 0.000 0.000 0.000 domain.py:47(reset) 
     15 0.000 0.000 0.000 0.000 domain.py:99(_register) 
     1 0.000 0.000 0.000 0.000 latin_1.py:13(Codec) 
     1 0.000 0.000 0.000 0.000 latin_1.py:20(IncrementalEncoder) 
     1 0.000 0.000 0.000 0.000 latin_1.py:24(IncrementalDecoder) 
     1 0.000 0.000 0.000 0.000 latin_1.py:28(StreamWriter) 
     1 0.000 0.000 0.000 0.000 latin_1.py:31(StreamReader) 
     1 0.000 0.000 0.000 0.000 latin_1.py:34(StreamConverter) 
     1 0.000 0.000 0.000 0.000 latin_1.py:41(getregentry) 
     1 0.000 0.000 0.000 0.000 latin_1.py:8(<module>) 
     1 0.000 0.000 0.000 0.000 six.py:180(find_module) 
     1 0.000 0.000 0.000 0.000 six.py:184(find_module) 
     1 0.001 0.001 0.001 0.001 socket.py:189(__init__) 
     1 0.000 0.000 0.000 0.000 socket.py:196(close) 
     2 0.000 0.000 0.506 0.253 socket.py:227(meth) 
    3607936 1.590 0.000 1.590 0.000 storage.py:32(__init__) 
39687197 29.658 0.000 39.432 0.000 storage.py:36(read) 
     12 0.000 0.000 0.000 0.000 storage.py:41(readInt) 
    3607912 1.564 0.000 5.345 0.000 storage.py:44(readDouble) 
    3607924 1.712 0.000 5.389 0.000 storage.py:47(readLength) 
10823772 17.543 0.000 50.373 0.000 storage.py:53(readString) 
     11 1.922 0.175 16.496 1.500 storage.py:57(readStringList) 
     1 0.000 0.000 0.000 0.000 subprocess.py:458(_cleanup) 
     1 0.000 0.000 0.000 0.000 subprocess.py:578(list2cmdline) 
     1 0.000 0.000 0.045 0.045 subprocess.py:651(__init__) 
     1 0.000 0.000 0.000 0.000 subprocess.py:811(_get_handles) 
     3 0.000 0.000 0.000 0.000 subprocess.py:881(_make_inheritable) 
     1 0.000 0.000 0.045 0.045 subprocess.py:905(_execute_child) 
     3 0.000 0.000 0.000 0.000 subprocess.py:946(_close_in_parent) 
     1 6.670 6.670 295.056 295.056 traciTest_meanspeed.py:45(runTraCI) 
     1 0.007 0.007 0.007 0.007 {__import__} 
39687197 3.728 0.000 3.728 0.000 {_struct.calcsize} 
10823795 3.374 0.000 3.374 0.000 {_struct.pack} 
43295133 7.225 0.000 7.225 0.000 {_struct.unpack} 
     1 0.045 0.045 0.045 0.045 {_subprocess.CreateProcess} 
     3 0.000 0.000 0.000 0.000 {_subprocess.DuplicateHandle} 
     6 0.000 0.000 0.000 0.000 {_subprocess.GetCurrentProcess} 
     1 0.000 0.000 0.000 0.000 {_subprocess.GetStdHandle} 
     4 0.000 0.000 0.000 0.000 {built-in method Close} 
     1 0.000 0.000 0.000 0.000 {built-in method __new__ of type object at 0x1E22B4F8} 
     8 0.000 0.000 0.000 0.000 {getattr} 
     61 0.000 0.000 0.000 0.000 {hasattr} 
     6 0.000 0.000 0.000 0.000 {isinstance} 
32471566 2.246 0.000 2.246 0.000 {len} 
     3 0.000 0.000 0.000 0.000 {method 'add' of 'set' objects} 
10496061 2.479 0.000 2.479 0.000 {method 'append' of 'list' objects} 
     990 0.000 0.000 0.000 0.000 {method 'clear' of 'dict' objects} 
     1 0.506 0.506 0.506 0.506 {method 'connect' of '_socket.socket' objects} 
10823772 12.557 0.000 12.565 0.000 {method 'decode' of 'str' objects} 
     1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 
    3607923 3.371 0.000 3.371 0.000 {method 'encode' of 'str' objects} 
     2 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects} 
     17 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects} 
     2 0.000 0.000 0.000 0.000 {method 'join' of 'str' objects} 
    7215956 84.503 0.000 84.503 0.000 {method 'recv' of '_socket.socket' objects} 
     3 0.000 0.000 0.000 0.000 {method 'remove' of 'set' objects} 
    3607936 46.414 0.000 46.414 0.000 {method 'send' of '_socket.socket' objects} 
     1 0.000 0.000 0.000 0.000 {method 'setsockopt' of '_socket.socket' objects} 
     1 0.000 0.000 0.000 0.000 {method 'split' of 'str' objects} 
     1 0.000 0.000 0.000 0.000 {method 'startswith' of 'str' objects} 
     1 0.000 0.000 0.000 0.000 {method 'translate' of 'str' objects} 
     15 0.000 0.000 0.000 0.000 {method 'update' of 'dict' objects} 
     11 0.000 0.000 0.000 0.000 {method 'values' of 'dict' objects} 
     2 0.000 0.000 0.000 0.000 {msvcrt.get_osfhandle} 
     12 0.023 0.002 0.023 0.002 {range} 
     27 0.000 0.000 0.000 0.000 {setattr} 
     1 0.000 0.000 0.000 0.000 {sys.exit} 
     22 0.000 0.000 0.000 0.000 {time.clock}

來源

2016-10-16 Ng Hui Xiong

添加[個人資料數據]（https://docs.python.org/3.6/library/profile.html） – napuzba

from collections import deque, defaultdict 

MeanspeedDict = defaultdict(lambda: deque(maxlen=300)) 

EdgeList = traci.edge.getIDList() 
for edge in EdgeList: 
    MeanspeedDict[edge].append(traci.edge.getLastStepMeanSpeed(edge))

來源

2016-10-16 06:07:54

@我剛剛嘗試過你的建議，但它仍然需要很長時間。我在想，瓶頸可能是第三方執行我的'traci'調用 –

@MaTaKazer是啊，簡要瀏覽配置文件數據，看起來你是對的。瓶頸看起來是讀取。是否有'traci'的任何批量操作？ –

不幸的是，不。他們不支持在給定「邊界列表」的情況下檢索'LastStepMeanSpeed'。因此，我必須使用循環來檢索每個邊的單個值。猜測，我必須通過減少採樣週期來處理它 –

看來getLastStepMeanSpeed太慢：

3607912 2.785 0.000 221.939 0.000 _edge.py:151(getLastStepMeanSpeed)

腳本用於執行這75％的時間（二百九十五分之二百二十一）

來源

2016-10-16 06:26:04 napuzba

在字符串長列表上迭代的優化

回答

相關問題