2011-10-04 57 views
1

我用pyparsing寫了一個很大的(對我)語法來解析H248消息,這是一種文本格式,約有500行的pypasing結構。如果我向解析器提供許多要解析的消息(我的電腦上的每條消息大約半秒),我發現平均性能是可以接受的。如何提高pyparsing啓動時間?

但是,我的解析器的基本用法是解析一個單一的消息,並轉儲出某個部分,我把它寫成一個獨立的程序。現在,我的x86電腦上花了12s(共享solaria服務器上的1m12s)來解析第一條單條消息。所以我想知道是否有任何技術來解決這個啓動時間?我嘗試過psyco,但它對啓動時間沒有幫助。

我現在所做的是編寫一個長壽的服務器導入解析器並接受解析請求,然後另一個獨立程序(經常觸發)與此服務器進程交互,從而獲得快速響應效果。這樣,我就失去了使用解析結果作爲python對象的機會。我想知道是否有更好的方法來達到我的目的?

順便說一句,如果我手動中斷我的分析器之前12s,堆棧跟蹤顯示它正在遞歸調用streamline()。

謝謝! 凱文

from pyparsing import * 
    ALPHA = alphas 
    DIGIT = nums 
    HEXDIG = "ABCDEF" 
    SafeChar = alphanums + "+-&!_'[email protected]^`~*$\\()%+." 
    RestChar = ";[]{}:,#<>=" 
    COLON = ":" 
    DOT = "." 
    SLASH = "/" 
    DQUOTE = '"' 
    SP = " " 
    HTAB = '\\0x09' 
    HTAB = '' 
    EOL = Suppress(lineEnd) 
    WSP = SP + HTAB 
    COMMENT = Word(";", SafeChar + RestChar + WSP + '"') + EOL 
    LWSP = Suppress(Word(WSP)) 
    nonEscapeChar = "}" + srange("[\\0x01-\\0x7c]") #+ srange("[\\0x7E-\\0xFF]") 
    nonEscapeChar = srange("[\\0x01-\\0x7c]") #+ srange("[\\0x7E-\\0xFF]") 
    octetString = Word(nonEscapeChar) 
    quotedString = dblQuotedString 
    UINT16 = Word(DIGIT, min=1, max=5) 
    UINT32 = Word(DIGIT, min=1, max=10) 
    NAME = Word(ALPHA, alphanums + "_", min=1, max=64) 
    VALUE = quotedString | Word(SafeChar) #| srange("[\\0x80-\\0xFF]") 
    EQUAL = Suppress("=") 
    LBRKT = Suppress("{") 
    RBRKT = Suppress("}") 
    COMMA = Suppress(",") 
    SEP = ((WSP | EOL | COMMENT) + LWSP) 
    AddToken = Suppress(Literal("Add")| "A") 
    AndAUDITSelectToken = Suppress(Literal("ANDLgc")) 
    AuditToken = Suppress(Literal("Audit")| "AT") 
    AuditCapToken = Suppress(Literal("AuditCapability")| "AC") 
    AuditValueToken = Suppress(Literal("AuditValue")| "AV") 
    AuthToken = Suppress(Literal("Authentication")| "AU") 
    BothToken = Suppress(Literal("Both")| "B") 
    BothwayToken = Suppress(Literal("Bothway")| "BW") 
    BriefToken = Suppress(Literal("Brief")| "BR") 
    BufferToken = Suppress(Literal("Buffer")| "BF") 
    CtxToken = Suppress(Literal("Context")| "C") 
    ContextAuditToken = Suppress(Literal("ContextAudit")| "CA") 
    ContextAttrToken = Suppress(Literal("ContextAttr")| "CT") 
    ContextListToken = Suppress(Literal("ContextList")| "CLT") 
    DigitMapToken = Suppress(Literal("DigitMap")| "DM") 
    DirectionToken = Suppress(Literal("SPADirection")| "SPADI") 
    DisconnectedToken = Suppress(Literal("Disconnected")| "DC") 
    DelayToken = Suppress(Literal("Delay")| "DL") 
    DurationToken = Suppress(Literal("Duration")| "DR") 
    EmbedToken = Suppress(Literal("Embed")| "EM") 
    EmergencyToken = Suppress(Literal("Emergency")| "EG") 
    EmergencyOffToken = Suppress(Literal("EmergencyOff")| "EGO") 
    EmergencyValueToken = Suppress(Literal("EmergencyValue")| "EGV") 
    ErrorToken = Suppress(Literal("Error")| "ER") 
    EventBufferToken = Suppress(Literal("EventBuffer")| "EB") 
    EventsToken = Suppress(Literal("Events")| "E") 
    ExternalToken = Suppress(Literal("External")| "EX") 
    FailoverToken = Suppress(Literal("Failover")| "FL") 
    ForcedToken = Suppress(Literal("Forced")| "FO") 
    GracefulToken = Suppress(Literal("Graceful")| "GR") 
    H221Token = Suppress(Literal("H221")) 
    H223Token = Suppress(Literal("H223")) 
    H226Token = Suppress(Literal("H226")) 
    HandOffToken = Suppress(Literal("HandOff")| "HO") 
    IEPSToken = Suppress(Literal("IEPSCall")| "IEPS") 
    ImmAckRequiredToken = Suppress(Literal("ImmAckRequired")| "IA") 
    InternalToken = Suppress(Literal("Internal")| "IT") 
    IntsigDelayToken = Suppress(Literal("Intersignal")| "SPAIS") 
    IsolateToken = Suppress(Literal("Isolate")| "IS") 
    InSvcToken = Suppress(Literal("InService")| "IV") 
    InterruptByEventToken = Suppress(Literal("IntByEvent")| "IBE") 
    InterruptByNewSignalsDescrToken = Suppress(Literal("IntBySigDescr")| "IBS") 
    IterationToken = Suppress(Literal("Iteration")| "IR") 
    KeepActiveToken = Suppress(Literal("KeepActive")| "KA") 
    LocalToken = Suppress(Literal("Local")| "L") 
    LocalControlToken = Suppress(Literal("LocalControl")| "O") 
    LockStepToken = Suppress(Literal("LockStep")| "SP") 
    MediaToken = Suppress(Literal("Media")| "M") 
    MegacopToken = Suppress(Literal("MEGACO")| "!") 
    MessageSegmentToken = Suppress(Literal("Segment")| "SM") 
    MethodToken = Suppress(Literal("Method")| "MT") 
    MgcIdToken = Suppress(Literal("MgcIdToTry")| "MG") 
    ModeToken = Suppress(Literal("Mode")| "MO") 
    ModifyToken = Suppress(Literal("Modify")| "MF") 
    ModemToken = Suppress(Literal("Modem")| "MD") 
    MoveToken = Suppress(Literal("Move")| "MV") 
    MTPToken = Suppress(Literal("MTP")) 
    MuxToken = Suppress(Literal("Mux")| "MX") 
    NeverNotifyToken = Suppress(Literal("NeverNotify")| "NBNN") 
    NotifyToken = Suppress(Literal("Notify")| "N") 
    NotifyCompletionToken = Suppress(Literal("NotifyCompletion")| "NC") 
    NotifyImmediateToken = Suppress(Literal("ImmediateNotify")| "NBIN") 
    NotifyRegulatedToken = Suppress(Literal("RegulatedNotify")| "NBRN") 
    Nx64kToken = Suppress(Literal("Nx64Kservice")| "N64") 
    ObservedEventsToken = Suppress(Literal("ObservedEvents")| "OE") 
    OnewayToken = Suppress(Literal("Oneway")| "OW") 
    OnewayBothToken = Suppress(Literal("OnewayBoth")| "OWB") 
    OnewayExternalToken = Suppress(Literal("OnewayExternal")| "OWE") 
    OnOffToken = Suppress(Literal("OnOff")| "OO") 
    OrAUDITselectToken = Suppress(Literal("ORLgc")) 
    OtherReasonToken = Suppress(Literal("OtherReason")| "OR") 
    OutOfSvcToken = Suppress(Literal("OutOfService")| "OS") 
    PackagesToken = Suppress(Literal("Packages")| "PG") 
    #...many literal tokens 
    TestToken = Suppress(Literal("Test")| "TE") 
    TimeOutToken = Suppress(Literal("TimeOut")| "TO") 
    TopologyToken = Suppress(Literal("Topology") | "TP") 
    TransToken = Suppress(Literal("Transaction") | "T") 
    ResponseAckToken = Suppress(Literal("TransactionResponseAck")| "K") 
    V18Token = Suppress(Literal("V18")) 
    V22Token = Suppress(Literal("V22")) 
    V22bisToken = Suppress(Literal("V22b")) 
    V32Token = Suppress(Literal("V32")) 
    V32bisToken = Suppress(Literal("V32b")) 
    V34Token = Suppress(Literal("V34")) 
    V76Token = Suppress(Literal("V76")) 
    V90Token = Suppress(Literal("V90")) 
    V91Token = Suppress(Literal("V91")) 
    VersionToken = Suppress(Literal("Version") | "V") 
    RSBRKT=Suppress("]") 
    LSBRKT=Suppress("]") 
    INEQUAL=Suppress(LWSP+(Literal(">")|"<"|"#")+LWSP)("INEQUAL") 
    ContextID=Combine((UINT32|"*"|"-"|"$"))("ContextID") 
    ItemID=NAME("ItemID") 
    PackageName=NAME("PackageName") 
    StreamID=UINT16("StreamID") 
    Time=(Word(DIGIT, exact=8))("Time") 
    Date=(Word(DIGIT, exact=8))("Date") 
    RequestID=(UINT32|"*")("RequestID") 
    digitMapName=NAME("digitMapName") 
    sigParameterName=NAME("sigParameterName") 
    signalListId=UINT16("signalListId") 
    pathDomainName=Group(Word(alphanums+"*",alphanums+"-"+"*"+".",max=64))("pathDomainName") 
    pathNAME=Combine(Optional("*")+NAME+Word("/"+"*"+alphanums+"_"+"$")+Optional("@"+pathDomainName))("pathNAME") 
    pkgdName=Combine((PackageName+SLASH+ItemID)|(PackageName+SLASH+"*")|("*"+SLASH+"*")) 
    alternativeValue=Combine((VALUE|LSBRKT+delimitedList(VALUE, COMMA)+RSBRKT|LBRKT+delimitedList(VALUE, COMMA) +RBRKT|LSBRKT+VALUE+COLON+VALUE+RSBRKT))("alternativeValue") 
    parmValue=((EQUAL+alternativeValue|INEQUAL+VALUE))("parmValue") 
    propertyParm=Group(pkgdName+parmValue)("propertyParm") 
    extensionParameter=Group(Literal("X")+(Literal("-")|"+")+Word(ALPHA+DIGIT,min=1,max=6))("extensionParameter") 
    contextIdList=Group(ContextListToken+EQUAL+LBRKT+delimitedList(ContextID, COMMA) +RBRKT)("contextIdList") 
    contextAttrDescriptor=Group(ContextAttrToken+LBRKT+(contextIdList|delimitedList(propertyParm, COMMA))+RBRKT)("contextAttr") 
    emergencyValue=Group(EmergencyValueToken+EQUAL+(EmergencyToken|EmergencyOffToken))("emergencyValue") 
    iepsValue=Group(IEPSToken+EQUAL+(Literal("ON")|"OFF"))("iepsValue") 
    priority=Group(PriorityToken+EQUAL+UINT16)("priority") 
    topologyDirection=Group(BothwayToken|IsolateToken|OnewayToken|OnewayExternalToken|OnewayBothToken)("topologyDirection") 
    TerminationID=(Literal("ROOT")|pathNAME|"$"|"*")("TerminationID") 
    terminationB=(TerminationID)("terminationB") 
    terminationA=(TerminationID)("terminationA") 
    eventStream=Group(StreamToken+EQUAL+StreamID)("eventStream") 
    topologyTriple=Group(terminationA+COMMA+terminationB+COMMA+topologyDirection+Optional(COMMA+eventStream))("topologyTriple") 
    topologyDescriptor=Group(TopologyToken+LBRKT+delimitedList(topologyTriple, COMMA)+RBRKT)("topology") 
    statisticsParameter=Group(pkgdName+Optional(EQUAL+VALUE|(LSBRKT+delimitedList(VALUE, COMMA)+RSBRKT)))("statisticsParameter") 
    statisticsDescriptor=Group(StatsToken+LBRKT+delimitedList(statisticsParameter, COMMA)+RBRKT)("statistics") 
    TimeStamp=Group(Date+"T"+Time)("TimeStamp") 
    packagesItem=Group(NAME+"-"+UINT16)("packagesItem") 
    Version=Group(Word(DIGIT,min=1,max=2))("Version") 
    packagesDescriptor=Group(PackagesToken+LBRKT+delimitedList(packagesItem, COMMA)+RBRKT)("packages") 
    extension=Group(extensionParameter+parmValue)("extension") 
    serviceChangeVersion=Group(VersionToken+EQUAL+Version)("serviceChangeVersion") 
    serviceChangeProfile=Group(ProfileToken+EQUAL+NAME+SLASH+Version)("serviceChangeProfile") 
    deviceName=Group(pathNAME)("deviceName") 
    TransactionID=UINT32("TransactionID") 
    ErrorCode=Word(DIGIT,min=1,max=4)("ErrorCode") 
    mtpAddress=Group(MTPToken+LBRKT+Word(HEXDIG,min=4,max=8)+RBRKT)("mtpAddress") 
    portNumber=Group(UINT16)("portNumber") 
    hex4=Group(Word(HEXDIG,min=1,max=4))("hex4") 
    hexseq=Group(hex4+ZeroOrMore(":"+hex4))("hexseq") 
    hexpart=Group(hexseq+"::"+Optional(hexseq)|"::"+Optional(hexseq)|hexseq)("hexpart") 
    V4hex=Group(Word(DIGIT,min=1,max=3))("V4hex") 
    IPv4address=Group(V4hex+DOT+V4hex+DOT+V4hex+DOT+V4hex)("IPv4address") 
    IPv6address=Group(hexpart+Optional(":"+IPv4address))("IPv6address") 
    domainAddress=Group(Literal("[")+(IPv4address|IPv6address)+"]")("domainAddress") 
    domainName=Group("<"+Word(ALPHA+DIGIT,ALPHA+DIGIT+"-"+".",max=64)+">")("domainName") 
    mId=(((domainAddress|domainName)+Optional(":"+portNumber))|mtpAddress|deviceName)("mId") 
    sigIntsigDelay=Group(IntsigDelayToken+EQUAL+UINT16)("sigIntsigDelay") 
    sigRequestID=(RequestIDToken+EQUAL+RequestID)("sigRequestID") 
    direction=Group(ExternalToken|InternalToken|BothToken)("direction") 
    sigDirection=Group(DirectionToken+EQUAL+direction)("sigDirection") 
    sigDuration=Group(DurationToken+EQUAL+UINT16)("sigDuration") 
    signalType=Group((OnOffToken|TimeOutToken|BriefToken))("signalType") 
    sigSignalType=Group(SignalTypeToken+EQUAL+signalType)("sigSignalType") 
    sigOther=Group(sigParameterName+parmValue)("sigOther") 
    sigStream=Group(StreamToken+EQUAL+StreamID)("sigStream") 
    signalName=Group(pkgdName)("signalName") 
    notificationReason=Group(TimeOutToken|InterruptByEventToken|InterruptByNewSignalsDescrToken|OtherReasonToken|IterationToken)("notificationReason") 
    notifyCompletion=Group(NotifyCompletionToken+EQUAL+(LBRKT+delimitedList(notificationReason, COMMA)+RBRKT))("notifyCompletion") 
    sigParameter=Group(sigStream|sigSignalType|sigDuration|sigOther|notifyCompletion|KeepActiveToken|sigDirection|sigRequestID|sigIntsigDelay)("sigParameter") 
    signalRequest=Group(signalName+Optional(LBRKT+delimitedList(sigParameter, COMMA)+RBRKT))("signalRequest") 
    signalListParm=Group(signalRequest)("signalListParm") 
    signalList=Group(SignalListToken+EQUAL+signalListId+LBRKT+delimitedList(signalListParm,COMMA)+RBRKT)("signalList") 
    signalParm=Group(signalList|signalRequest)("signalParm") 
    signalsDescriptor=Group(SignalsToken+Optional(LBRKT+delimitedList(signalParm,COMMA)+RBRKT))("signals") 
    serviceChangeMgcId=(MgcIdToken+EQUAL+mId)("serviceChangeMgcId") 
    serviceChangeAddress=Group(ServiceChangeAddressToken+EQUAL+(mId|portNumber))("serviceChangeAddress") 
    serviceChangeDelay=Group(DelayToken+EQUAL+UINT32)("serviceChangeDelay") 
    serviceChangeReason=Group(ReasonToken+EQUAL+VALUE)("serviceChangeReason") 
    serviceChangeMethod=Group(MethodToken+EQUAL+(FailoverToken|ForcedToken|GracefulToken|RestartToken|DisconnectedToken|HandOffToken|extensionParameter))("serviceChangeMethod") 
    servChgReplyParm=Group(serviceChangeAddress|serviceChangeMgcId|serviceChangeProfile|serviceChangeVersion|TimeStamp)("servChgReplyParm") 
    serviceChangeReplyDescriptor=Group(ServicesToken+LBRKT+delimitedList(servChgReplyParm, COMMA)+RBRKT)("serviceChangeReply") 
    auditReturnItem=Group((MuxToken|ModemToken|MediaToken|DigitMapToken|StatsToken|ObservedEventsToken|PackagesToken))("auditReturnItem") 
    indAudpackagesDescriptor=Group(PackagesToken+LBRKT+packagesItem+RBRKT)("indAudpackages") 
    indAudstatisticsDescriptor=Group(StatsToken+LBRKT+pkgdName+RBRKT)("indAudstatistics") 
    indAuddigitMapDescriptor=Group(DigitMapToken+EQUAL+(digitMapName))("indAuddigitMap") 
    indAudsignalRequestParm=Group(sigStream|sigRequestID)("indAudsignalRequestParm") 
    indAudsignalRequest=Group(signalName+Optional(LBRKT+delimitedList(indAudsignalRequestParm, COMMA)+RBRKT))("indAudsignalRequest") 
    indAudsignalListParm=Group(indAudsignalRequest)("indAudsignalListParm") 
    indAudsignalList=Group(SignalListToken+EQUAL+signalListId+Optional(LBRKT+indAudsignalListParm+RBRKT))("indAudsignalList") 
    indAudsignalParm=Group(indAudsignalList|indAudsignalRequest)("indAudsignalParm") 
    indAudsignalsDescriptor=Group(SignalsToken+LBRKT+Optional(indAudsignalParm)+RBRKT)("indAudsignals") 
    indAudrequestedEvent=Group(pkgdName)("indAudrequestedEvent") 
    indAudeventsDescriptor=Group(EventsToken+Optional(EQUAL+RequestID)+LBRKT+indAudrequestedEvent+RBRKT)("indAudevents") 
    eventParameterName=Group(NAME)("eventParameterName") 
    eventOther=Group(eventParameterName+parmValue)("eventOther") 
    indAudeventSpecParameter=Group(eventStream|eventParameterName)("indAudeventSpecParameter") 
    indAudeventSpec=Group(pkgdName+Optional(LBRKT+indAudeventSpecParameter+RBRKT))("indAudeventSpec") 
    indAudeventBufferDescriptor=Group(EventBufferToken+LBRKT+indAudeventSpec+RBRKT)("indAudeventBuffer") 
    serviceStatesValue=Group((TestToken|OutOfSvcToken|InSvcToken))("serviceStatesValue") 
    indAudterminationStateParm=Group(pkgdName|propertyParm|ServiceStatesToken+Optional((EQUAL|INEQUAL)+serviceStatesValue)|BufferToken)("indAudterminationStateParm") 
    indAudterminationStateDescriptor=Group(TerminationStateToken+LBRKT+indAudterminationStateParm+RBRKT)("indAudterminationState") 
    streamModes=(SendonlyToken|RecvonlyToken|SendrecvToken|InactiveToken|LoopbackToken)("Mode") 
    indAudlocalParm=Group(ModeToken+Optional((EQUAL|INEQUAL)+streamModes)|pkgdName|propertyParm|ReservedValueToken|ReservedGroupToken)("indAudlocalParm") 
    indAudlocalControlDescriptor=Group(LocalControlToken+LBRKT+delimitedList(indAudlocalParm, COMMA)+RBRKT)("indAudlocalControl") 
    indAudstreamParm=Forward() 
    indAudstreamDescriptor=Group(StreamToken+EQUAL+StreamID+LBRKT+indAudstreamParm+RBRKT)("indAudstream") 
    indAudlocalDescriptor=Group(LocalToken+LBRKT+octetString+RBRKT)("indAudlocal") 
    indAudremoteDescriptor=Group(RemoteToken+LBRKT+octetString+RBRKT)("indAudremote") 
    indAudstreamParm<<(indAudlocalControlDescriptor|indAudstatisticsDescriptor|indAudremoteDescriptor|indAudlocalDescriptor) 
    indAudmediaParm=indAudstreamParm|indAudstreamDescriptor|indAudterminationStateDescriptor 
    indAudmediaDescriptor=Group(MediaToken+LBRKT+delimitedList(indAudmediaParm, COMMA)+RBRKT)("indAudmedia") 
    indAudauditReturnParameter=indAudmediaDescriptor|indAudeventsDescriptor|indAudsignalsDescriptor|indAuddigitMapDescriptor|indAudeventBufferDescriptor|indAudstatisticsDescriptor|indAudpackagesDescriptor 
    indAudterminationAudit=Group(delimitedList(indAudauditReturnParameter, COMMA))("indAudterminationAudit") 
    auditItem=Group(auditReturnItem|SignalsToken|EventBufferToken|EventsToken|indAudterminationAudit)("auditItem") 
    serviceChangeParm=Group(serviceChangeMethod|serviceChangeReason|serviceChangeDelay|serviceChangeAddress|serviceChangeProfile|extension|TimeStamp|serviceChangeMgcId|serviceChangeVersion|ServiceChangeIncompleteToken|auditItem)("serviceChangeParm") 
    serviceChangeDescriptor=Group(ServicesToken+LBRKT+delimitedList(serviceChangeParm,COMMA)+RBRKT)("serviceChange") 
    digitMapLetter=DIGIT+srange("[\\0x41-\\0x4B]")+srange("[\\0x61-\\0x6B]")+"L"+"S"+"T"+"Z" 
    digitLetter=Group(Word(DIGIT+"-"+DIGIT)|Word(digitMapLetter))("digitLetter") 
    digitMapRange=Group((Literal("x")|(LWSP+"["+LWSP+digitLetter+LWSP+"]"+LWSP)))("digitMapRange") 
    digitPosition=Group(digitMapLetter|digitMapRange)("digitPosition") 
    digitStringElement=Group(digitPosition+Optional(DOT))("digitStringElement") 
    digitString=Group(OneOrMore(digitStringElement))("digitString") 
    digitStringList=Group(digitString+ZeroOrMore(LWSP+"|"+LWSP+digitString))("digitStringList") 
    digitMap=Group((digitString|LWSP+"("+LWSP+digitStringList+LWSP+")"+LWSP))("digitMap") 
    Timer=Group(Word(DIGIT,min=1,max=2))("Timer") 
    digitMapValue=Group(Optional("T"+COLON+Timer+COMMA)+Optional("S"+COLON+Timer+COMMA)+Optional("L"+COLON+Timer+COMMA)+Optional("Z"+COLON+Timer+COMMA)+digitMap)("digitMapValue") 
    digitMapDescriptor=Group(DigitMapToken+EQUAL+((LBRKT+digitMapValue+RBRKT)| (digitMapName+Optional(LBRKT+digitMapValue+RBRKT))))("digitMap") 
    modemType=Group(V32bisToken|V22bisToken|V18Token|V22Token|V32Token|V34Token|V90Token|V91Token|SynchISDNToken|extensionParameter)("modemType") 
    modemDescriptor=Group(ModemToken+((EQUAL+modemType)|(LSBRKT+delimitedList(modemType, COMMA)+RSBRKT))+Optional(LBRKT+delimitedList(propertyParm, COMMA)+RBRKT))("modem") 
    observedEventParameter=(eventStream|eventOther) 
    observedEvent=Group(Optional(TimeStamp+LWSP+COLON)+LWSP+pkgdName+Optional(LBRKT+delimitedList(observedEventParameter,COMMA)+RBRKT))("observedEvent") 
    observedEventsDescriptor=Group(ObservedEventsToken+EQUAL+RequestID+LBRKT+SkipTo(RBRKT)+RBRKT)("observedEvents") 
    eventDM=Group(DigitMapToken+EQUAL((digitMapName)|(LBRKT+digitMapValue+RBRKT)))("eventDM") 
    embedSig=Group(EmbedToken+LBRKT+signalsDescriptor+RBRKT)("embedSig") 
    secondEventParameter=Forward() 
    secondRequestedEvent=Group(pkgdName+Optional(LBRKT+delimitedList(secondEventParameter, COMMA)+RBRKT))("secondRequestedEvent") 
    embedFirst=Group(EventsToken+Optional(EQUAL+RequestID+LBRKT+delimitedList(secondRequestedEvent,COMMA)+RBRKT))("embedFirst") 
    embedNoSig=Group(EmbedToken+LBRKT+embedFirst+RBRKT)("embedNoSig") 
    embedWithSig=Group(EmbedToken+LBRKT+signalsDescriptor+Optional(COMMA+embedFirst)+RBRKT)("embedWithSig") 
    notifyRegulated=Group(NotifyRegulatedToken+Optional(LBRKT+(embedWithSig|embedNoSig)+RBRKT))("notifyRegulated") 
    notifyBehaviour=Group(NotifyImmediateToken|notifyRegulated|NeverNotifyToken)("notifyBehaviour") 
    eventParameter=Group((embedWithSig|embedNoSig|KeepActiveToken|eventDM|eventStream|eventOther|notifyBehaviour|ResetEventsDescriptorToken))("eventParameter") 
    secondEventParameter<<(embedSig|KeepActiveToken|eventDM|eventStream|eventOther|notifyBehaviour|ResetEventsDescriptorToken) 
    requestedEvent=Group(pkgdName+Optional(LBRKT+delimitedList(eventParameter, COMMA)+RBRKT))("requestedEvent") 
    eventsDescriptor=Group(EventsToken+Optional(EQUAL+RequestID+LBRKT+delimitedList(requestedEvent,COMMA)+RBRKT))("events") 
    MuxType=Group((H221Token|H223Token|H226Token|V76Token|extensionParameter|Nx64kToken))("MuxType") 
    serviceStates=Group(ServiceStatesToken+EQUAL+serviceStatesValue)("serviceStates") 
    eventBufferControlValue=Group(("OFF"|LockStepToken))("eventBufferControlValue") 
    eventBufferControl=Group(BufferToken+EQUAL+eventBufferControlValue)("eventBufferControl") 
    terminationStateParm=Group((propertyParm|serviceStates|eventBufferControl))("terminationStateParm") 
    terminationStateDescriptor=Group(TerminationStateToken+LBRKT+delimitedList(terminationStateParm, COMMA)+RBRKT)("terminationState") 
    eventSpecParameter=Group((eventStream|eventOther))("eventSpecParameter") 
    eventSpec=Group(pkgdName+Optional(LBRKT+delimitedList(eventSpecParameter,COMMA)+RBRKT))("eventSpec") 
    eventBufferDescriptor=Group(EventBufferToken+Optional(LBRKT+delimitedList(eventSpec, COMMA)+RBRKT))("eventBuffer") 
    remoteDescriptor=(RemoteToken+LBRKT+octetString+RBRKT)("Remote") 
    localDescriptor=(LocalToken+LBRKT+octetString+RBRKT)("Local") 
    streamMode=(ModeToken+EQUAL+streamModes) 
    reservedGroupMode=Group(ReservedGroupToken+EQUAL+(Literal("ON")|"OFF"))("reservedGroupMode") 
    reservedValueMode=Group(ReservedValueToken+EQUAL+(Literal("ON")|"OFF"))("reservedValueMode") 
    localParm=(streamMode|propertyParm|reservedValueMode|reservedGroupMode) 
    localControlDescriptor=Group(LocalControlToken+LBRKT+delimitedList(localParm,COMMA)+RBRKT)("LocalControl") 
    localControlDescriptor=Group(LocalControlToken+LBRKT+octetString+RBRKT)("LocalControl") 
    streamParm=(localDescriptor|remoteDescriptor|localControlDescriptor|statisticsDescriptor) 
    streamDescriptor=Group(StreamToken+EQUAL+StreamID+LBRKT+delimitedList(streamParm, COMMA)+RBRKT)("Stream") 
    mediaParm=streamParm|streamDescriptor|terminationStateDescriptor 
    mediaDescriptor=Group(MediaToken+LBRKT+delimitedList(mediaParm, COMMA)+RBRKT)("Media") 
    terminationIDList=Group(LBRKT+delimitedList(TerminationID, COMMA)+RBRKT)("terminationIDList") 
    muxDescriptor=Group(MuxToken+EQUAL+MuxType+terminationIDList)("mux") 
    termIDList=Group((TerminationID|LSBRKT+TerminationID+OneOrMore(COMMA+TerminationID)+RSBRKT))("termIDList") 
    errorDescriptor=Group(ErrorToken+EQUAL+ErrorCode+LBRKT+Optional(quotedString)+RBRKT)("error") 
    serviceChangeReply=Group(ServiceChangeToken+EQUAL+termIDList+Optional(LBRKT+(errorDescriptor|serviceChangeReplyDescriptor)+RBRKT))("serviceChangeReply") 
    serviceChangeRequest=Group(ServiceChangeToken+EQUAL+termIDList+LBRKT+serviceChangeDescriptor+RBRKT)("serviceChangeRequest") 
    notifyReply=Group(NotifyToken+EQUAL+termIDList+Optional(LBRKT+errorDescriptor+RBRKT))("notifyReply") 
    notifyRequest=Group(NotifyToken+EQUAL+termIDList+LBRKT+(observedEventsDescriptor+Optional(COMMA+errorDescriptor))+RBRKT)("notifyRequest") 
    auditDescriptor=Group(AuditToken+LBRKT+Optional(delimitedList(auditItem,COMMA))+RBRKT)("audit") 
    auditReturnParameter=(mediaDescriptor|modemDescriptor|muxDescriptor|eventsDescriptor|signalsDescriptor|digitMapDescriptor|observedEventsDescriptor|eventBufferDescriptor|statisticsDescriptor|packagesDescriptor|errorDescriptor|auditReturnItem) 
    contextTerminationAudit=Group(EQUAL+CtxToken+(terminationIDList|LBRKT+errorDescriptor+RBRKT))("contextTerminationAudit") 
    terminationAudit=Group(delimitedList(auditReturnParameter,COMMA))("terminationAudit") 
    auditOther=Group(EQUAL+termIDList+Optional(LBRKT+terminationAudit+RBRKT))("auditOther") 
    auditReply=Group((AuditValueToken|AuditCapToken)+(contextTerminationAudit|auditOther))("auditReply") 
    auditRequest=Group((AuditValueToken|AuditCapToken)+EQUAL+termIDList+LBRKT+auditDescriptor+RBRKT)("auditRequest") 
    subtractRequest=Group(SubtractToken+EQUAL+termIDList+Optional(LBRKT+auditDescriptor+RBRKT))("subtractRequest") 
    ammsReply=Group((AddToken|MoveToken|ModifyToken|SubtractToken)+EQUAL+termIDList+Optional(LBRKT+terminationAudit+RBRKT))("ammsReply") 
    ammParameter=mediaDescriptor|modemDescriptor|muxDescriptor|eventsDescriptor|signalsDescriptor|digitMapDescriptor|eventBufferDescriptor|auditDescriptor|statisticsDescriptor 
    ammRequestBody=EQUAL+termIDList+Optional(LBRKT+delimitedList(ammParameter, COMMA)+RBRKT) 
    ammRequest=Group((MoveToken|ModifyToken)+ ammRequestBody)("ammRequest") 
    addRequest=Group(AddToken + ammRequestBody)("Add") 
    segmentNumber=UINT16("segmentNumber") 
    segmentReply=Group(MessageSegmentToken+EQUAL+TransactionID+SLASH+segmentNumber+Optional(SLASH+SegmentationCompleteToken))("segmentReply") 
    commandRequest=addRequest| ammRequest | subtractRequest|auditRequest|notifyRequest|serviceChangeRequest 
    commandRequestList=delimitedList(Optional("O-")+Optional("W-")+commandRequest, COMMA) 
    AndAUDITselectToken=Group(Literal("TODO"))("AndAUDITselectToken") 
    OrAUDITselectToken=Group(Literal("TODO"))("OrAUDITselectToken") 
    auditSelectLogic=Group(Optional(AndAUDITselectToken|OrAUDITselectToken))("auditSelectLogic") 
    contextAuditSelector=Group(priority|emergencyValue|iepsValue|contextAttrDescriptor|auditSelectLogic)("contextAuditSelector") 
    contextAuditProperties=Group((TopologyToken|EmergencyToken|PriorityToken|IEPSToken|pkgdName|contextAuditSelector))("contextAuditProperties") 
    indAudcontextAttrDescriptor=Group(ContextAttrToken+LBRKT+delimitedList(contextAuditProperties, COMMA)+RBRKT)("indAudcontextAttr") 
    contextAudit=Group(ContextAuditToken+LBRKT+(delimitedList(contextAuditProperties,COMMA))|indAudcontextAttrDescriptor+RBRKT)("contextAudit") 
    contextProperty=Group((topologyDescriptor|priority|EmergencyToken|EmergencyOffToken|iepsValue|contextAttrDescriptor))("contextProperty") 
    contextProperties=Group(delimitedList(contextProperty,COMMA))("contextProperties") 
    commandReplys=(serviceChangeReply|auditReply|ammsReply|notifyReply) 
    commandReplyList=Group(delimitedList(commandReplys,COMMA))("commandReplyList") 
    commandReply=Group(((contextProperties+Optional(COMMA+commandReplyList))|commandReplyList))("commandReply") 
    actionReply=Group(CtxToken+EQUAL+ContextID+Optional(LBRKT+(errorDescriptor|commandReply|(commandReply+COMMA+errorDescriptor))+RBRKT))("actionReply") 
    actionReplyList=Group(delimitedList(actionReply,COMMA))("actionReplyList") 
    transactionReply=Group(ReplyToken+EQUAL+TransactionID+Optional(SLASH+segmentNumber+Optional(SLASH+SegmentationCompleteToken))+LBRKT+Optional(ImmAckRequiredToken+COMMA)+(errorDescriptor|actionReplyList)+RBRKT)("TransactionReply") 
    contextRequest=Group(((contextProperties+Optional(COMMA+contextAudit))|contextAudit))("contextRequest") 
    actionRequest=Group(CtxToken+EQUAL+ContextID+LBRKT+((contextRequest+Optional(COMMA+commandRequestList))|commandRequestList)+RBRKT)("actionRequest") 
    transactionRequest=Group(TransToken+EQUAL+TransactionID+LBRKT+delimitedList(actionRequest, COMMA)+RBRKT)("TransactionRequest") 
    transactionAck=Group(TransactionID|(TransactionID+"-"+TransactionID))("TransactionAck") 
    transactionResponseAck=Group(ResponseAckToken+LBRKT+transactionAck+ZeroOrMore(COMMA+transactionAck)+RBRKT)("transactionResponseAck") 
    transactionPending=Group(PendingToken+EQUAL+TransactionID+LBRKT+RBRKT)("transactionPending") 
    transactionList=OneOrMore(transactionRequest|transactionReply|transactionPending|transactionResponseAck|segmentReply) 
    messageBody=(transactionList | errorDescriptor) 
    Message=Group(MegacopToken+SLASH+Version+SkipTo(lineEnd)+messageBody)("Message") 
    AuthData=Group("0x"+Word(HEXDIG,min=24,max=64))("AuthData") 
    SequenceNum=Group("0x"+Word(HEXDIG,exact=8))("SequenceNum") 
    SecurityParmIndex=Group("0x"+Word(HEXDIG,exact=8))("SecurityParmIndex") 
    authenticationHeader=Group(AuthToken+EQUAL+SecurityParmIndex+COLON+SequenceNum+COLON+AuthData)("authenticationHeader") 
    megacoMessage=Group(LWSP+Optional(authenticationHeader+SEP)+Message)("megacoMessage").streamline() 

回答

1

您是否嘗試過酸洗解析器?我之前用一個複雜的解析器(用於Python本身)完成了這一工作,並在啓動時間方面得到了合理的改進。將pickled parser保存到一個文件中,然後解析單個消息,從文件中取出解析器,而不是明確地構建它。如果流水線是設置的昂貴部分,那麼一定要在酸洗之前調用parser.streamline() - 然後當實際解析消息時,可以跳過這個昂貴的簡化步驟。

如果您沒有定義低級基元,但是反覆顯式重新定義相同的表達式,那麼在構建解析器時也可能效率低下。例如,重複使用Word(nums),而不是定義低級表達式integer = Word(nums),然後在需要整數的地方引用integer。這很少會對解析器構建時間產生影響(我只看到其他人實際上有這個問題),但它可能發生。

您是否正在使用函數來定義遵循模式的複雜子解析器?嘗試記憶該函數,以便使用相同參數的重複調用返回相同的解析表達式而不是新的表達式。然後這些重複的表達式只會被簡化一次。

您是否有複雜的低級表達式,使用了許多帶有不同結果名稱的地方?這實際上會在封面下創建該表達的副本。您可以明確地調用streamline作爲定義步驟的一部分,並且副本將提前精簡。如果你有一個真實數字的表達式,我可以想象這種事情。事實上,如果用單個正則表達式(Regex(r'[-+]?\d+(\.\d*)?([Ee][-+]?\d+)?')將對應於前面的表達式)替換通過組裝pyparsing位(類似於Combine(Optional(oneOf('- +')) + Word(nums) + Optional('.' + Word(nums)) + Optional(oneOf('E e') + Optional(oneOf('- +')) + Word(nums)))組成的實數定義,我已經看到了顯着的分析時間改進。儘管如此,pyparsing在解析器表達式中作爲正則表達式做了很多內部表示。

這就是我所能想到的所有事情 - 對於更具體的建議,您需要在某處貼上您的500-liner,以便我們可以看到正在發生的事情的細節。

編輯: 一兩件事你可以做的是嘗試更換所有的字面定義,如:

AddToken = Suppress(Literal("Add")| "A") 

AddToken = oneOf("Add A").suppress() 

這將大大減少你的解析器元素的數量,所以要精簡或醃製的位數少得多。

這看起來像一個錯誤:

digitLetter=Group(Word(DIGIT+"-"+DIGIT)|Word(digitMapLetter))("digitLetter") 

沒有理由重複一個字的定義,裏面的數字串。

不要創建與他們空格的話,你很可能讀取超出你的原意。你使用LWSP也看起來不合適,因爲你試圖做自己的空白跳過。如果您必須做到這一點,改變Word(WSP)White(WSP)。但是,真的,你爲什麼要做自己的空白跳過?

更改註釋:

COMMENT = ';' + restOfLine + lineEnd 
+0

保羅,感謝您的及時回覆。我在這裏發佈了語法,它非常大,所以我跳過了一些文字標記。最後一條streamline()方法花費了我12秒的時間來運行。請幫忙檢查一下是否有問題,謝謝!我會嘗試醃解析器,謝謝你的建議。 –

+0

加載醃製解析器使用8s,醃製文件本身大小44M,59520行。 –