2014-01-26 22 views
8

Pyparsing運行良好的一個非常小的語法,但語法的不斷壯大,性能下降,並通過屋頂的內存使用情況。pyparsing性能和內存使用

我現在gramar是:

newline = LineEnd() 
minus = Literal ('-') 
plus = Literal ('+') 
star = Literal ('*') 
dash = Literal ('/') 
dashdash = Literal ('//') 
percent = Literal ('%') 
starstar = Literal ('**') 
lparen = Literal ('(') 
rparen = Literal (')') 
dot = Literal ('.') 
comma = Literal (',') 
eq = Literal ('=') 
eqeq = Literal ('==') 
lt = Literal ('<') 
gt = Literal ('>') 
le = Literal ('<=') 
ge = Literal ('>=') 
not_ = Keyword ('not') 
and_ = Keyword ('and') 
or_ = Keyword ('or') 
ident = Word (alphas) 
integer = Word (nums) 

expr = Forward() 
parenthized = Group (lparen + expr + rparen) 
trailer = (dot + ident) 
atom = ident | integer | parenthized 
factor = Forward() 
power = atom + ZeroOrMore (trailer) + Optional (starstar + factor) 
factor << (ZeroOrMore (minus | plus) + power) 
term = ZeroOrMore (factor + (star | dashdash | dash | percent)) + factor 
arith = ZeroOrMore (term + (minus | plus)) + term 
comp = ZeroOrMore (arith + (eqeq | le | ge | lt | gt)) + arith 
boolNot = ZeroOrMore (not_) + comp 
boolAnd = ZeroOrMore (boolNot + and_) + boolNot 
boolOr = ZeroOrMore (boolAnd + or_) + boolAnd 
match = ZeroOrMore (ident + eq) + boolOr 
expr << match 
statement = expr + newline 
program = OneOrMore (statement) 

當我解析以下

print (program.parseString ('3*(1+2*3*(4+5))\n')) 

這需要相當長的:

~/Desktop/m2/pyp$ time python3 slow.py 
['3', '*', ['(', '1', '+', '2', '*', '3', '*', ['(', '4', '+', '5', ')'], ')']] 

real 0m27.280s 
user 0m25.844s 
sys 0m1.364s 

而且內存使用量上升到1.7吉布(原文如此!)。

有我做了一些嚴重的錯誤執行這一語法還是我還能如何保持內存使用情況在可以忍受的利潤率?

+0

以幾分之一秒內lex和yacc同樣的事情。 – Hyperboreus

回答

11

進口pyparsing使packrat解析到memoize的解析行爲後:

ParserElement.enablePackrat() 

這應該使性能有了很大的改進。

+0

謝謝,我會試試看。 – Hyperboreus

+1

根據記錄,這3.5秒鐘後,以0.036秒我的電腦,近100倍的改善上。是否有任何理由不自動開啓memoization--是否在某些邊緣情況下失敗? – Hooked

+1

@Hooked:對於pyparsing解析上packrat詳情,請參閱[在pyparsing FAQ這個項目(http://pyparsing-public.wikispaces.com/FAQs#toc3)。一般來說,請參閱[本SO線程](https://stackoverflow.com/q/1410477/857390)進行packrat解析。 –