如何調試ANTLR4語法無關的/不匹配的輸入錯誤

Question

我需要解析的規則手冊中包含「demo.rb」文件：

rulebook Titanic-Normalization { 
    version 1 

    meta { 
    description "Test" 
    source "my-rules.xslx" 
    user "joltie" 
    } 

    rule remove-first-line { 
    description "Removes first line when offset is zero" 
    when(present(offset) && offset == 0) then { 
     filter-row-if-true true; 
    } 
    } 
} 

我寫的ANTLR4語法文件Rulebook.g4像下面。目前，它可以很好地解析* .rb文件，但遇到「表達式」/「語句」規則時會引發意外錯誤。

grammar Rulebook; 

rulebookStatement 
    : KWRulebook 
     (GeneralIdentifier | Identifier) 
     '{' 
     KWVersion 
     VersionConstant 
     metaStatement 
     (ruleStatement)+ 
     '}' 
    ; 

metaStatement 
    : KWMeta 
     '{' 
     KWDescription 
     StringLiteral 
     KWSource 
     StringLiteral 
     KWUser 
     StringLiteral 
     '}' 
    ; 

ruleStatement 
    : KWRule 
     (GeneralIdentifier | Identifier) 
     '{' 
     KWDescription 
     StringLiteral 
     whenThenStatement 
     '}' 
    ; 

whenThenStatement 
    : KWWhen '(' expression ')' 
     KWThen '{' statement '}' 
    ; 

primaryExpression 
    : GeneralIdentifier 
    | Identifier 
    | StringLiteral+ 
    | '(' expression ')' 
    ; 

postfixExpression 
    : primaryExpression 
    | postfixExpression '[' expression ']' 
    | postfixExpression '(' argumentExpressionList? ')' 
    | postfixExpression '.' Identifier 
    | postfixExpression '->' Identifier 
    | postfixExpression '++' 
    | postfixExpression '--' 
    ; 

argumentExpressionList 
    : assignmentExpression 
    | argumentExpressionList ',' assignmentExpression 
    ; 

unaryExpression 
    : postfixExpression 
    | '++' unaryExpression 
    | '--' unaryExpression 
    | unaryOperator castExpression 
    ; 

unaryOperator 
    : '&' | '*' | '+' | '-' | '~' | '!' 
    ; 

castExpression 
    : unaryExpression 
    | DigitSequence // for 
    ; 

multiplicativeExpression 
    : castExpression 
    | multiplicativeExpression '*' castExpression 
    | multiplicativeExpression '/' castExpression 
    | multiplicativeExpression '%' castExpression 
    ; 

additiveExpression 
    : multiplicativeExpression 
    | additiveExpression '+' multiplicativeExpression 
    | additiveExpression '-' multiplicativeExpression 
    ; 

shiftExpression 
    : additiveExpression 
    | shiftExpression '<<' additiveExpression 
    | shiftExpression '>>' additiveExpression 
    ; 

relationalExpression 
    : shiftExpression 
    | relationalExpression '<' shiftExpression 
    | relationalExpression '>' shiftExpression 
    | relationalExpression '<=' shiftExpression 
    | relationalExpression '>=' shiftExpression 
    ; 

equalityExpression 
    : relationalExpression 
    | equalityExpression '==' relationalExpression 
    | equalityExpression '!=' relationalExpression 
    ; 

andExpression 
    : equalityExpression 
    | andExpression '&' equalityExpression 
    ; 

exclusiveOrExpression 
    : andExpression 
    | exclusiveOrExpression '^' andExpression 
    ; 

inclusiveOrExpression 
    : exclusiveOrExpression 
    | inclusiveOrExpression '|' exclusiveOrExpression 
    ; 

logicalAndExpression 
    : inclusiveOrExpression 
    | logicalAndExpression '&&' inclusiveOrExpression 
    ; 

logicalOrExpression 
    : logicalAndExpression 
    | logicalOrExpression '||' logicalAndExpression 
    ; 

conditionalExpression 
    : logicalOrExpression ('?' expression ':' conditionalExpression)? 
    ; 

assignmentExpression 
    : conditionalExpression 
    | unaryExpression assignmentOperator assignmentExpression 
    | DigitSequence // for 
    ; 

assignmentOperator 
    : '=' | '*=' | '/=' | '%=' | '+=' | '-=' | '<<=' | '>>=' | '&=' | '^=' | '|=' 
    ; 

expression 
    : assignmentExpression 
    | expression ',' assignmentExpression 
    ; 

statement 
    : expressionStatement 
    ; 

expressionStatement 
    : expression+ ';' 
    ; 


KWRulebook: 'rulebook'; 
KWVersion: 'version'; 
KWMeta: 'meta'; 
KWDescription: 'description'; 
KWSource: 'source'; 
KWUser: 'user'; 
KWRule: 'rule'; 
KWWhen: 'when'; 
KWThen: 'then'; 
KWTrue: 'true'; 
KWFalse: 'false'; 

fragment 
LeftParen : '('; 

fragment 
RightParen : ')'; 

fragment 
LeftBracket : '['; 

fragment 
RightBracket : ']'; 

fragment 
LeftBrace : '{'; 

fragment 
RightBrace : '}'; 


Identifier 
    : IdentifierNondigit 
     ( IdentifierNondigit 
     | Digit 
     )* 
    ; 

GeneralIdentifier 
    : Identifier 
     ('-' Identifier)+ 
    ; 

fragment 
IdentifierNondigit 
    : Nondigit 
    //| // other implementation-defined characters... 
    ; 

VersionConstant 
    : DigitSequence ('.' DigitSequence)* 
    ; 

DigitSequence 
    : Digit+ 
    ; 

fragment 
Nondigit 
    : [a-zA-Z_] 
    ; 

fragment 
Digit 
    : [0-9] 
    ; 

StringLiteral 
    : '"' SCharSequence? '"' 
    | '\'' SCharSequence? '\'' 
    ; 

fragment 
SCharSequence 
    : SChar+ 
    ; 

fragment 
SChar 
    : ~["\\\r\n] 
    | '\\\n' // Added line 
    | '\\\r\n' // Added line 
    ; 

Whitespace 
    : [ \t]+ 
     -> skip 
    ; 

Newline 
    : ( '\r' '\n'? 
     | '\n' 
     ) 
     -> skip 
    ; 

BlockComment 
    : '/*' .*? '*/' 
     -> skip 
    ; 

LineComment 
    : '//' ~[\r\n]* 
     -> skip 
    ; 

我測試的規則手冊解析器與單元測試象下面這樣：

public void testScanRulebookFile() throws IOException { 
     String fileName = "C:\\rulebooks\\demo.rb"; 
     FileInputStream fis = new FileInputStream(fileName); 
     // create a CharStream that reads from standard input 
     CharStream input = CharStreams.fromStream(fis); 

     // create a lexer that feeds off of input CharStream 
     RulebookLexer lexer = new RulebookLexer(input); 

     // create a buffer of tokens pulled from the lexer 
     CommonTokenStream tokens = new CommonTokenStream(lexer); 

     // create a parser that feeds off the tokens buffer 
     RulebookParser parser = new RulebookParser(tokens); 


     RulebookStatementContext context = parser.rulebookStatement(); 
//  WhenThenStatementContext context = parser.whenThenStatement(); 

     System.out.println(context.toStringTree(parser)); 

//  ParseTree tree = parser.getContext(); // begin parsing at init rule 
//  System.out.println(tree.toStringTree(parser)); // print LISP-style tree 
    } 

對於「demo.rb」如上述，解析器得到了錯誤如下。我還以toStringTree的形式打印RulebookStatementContext。

line 12:25 mismatched input '&&' expecting ')' 
(rulebookStatement rulebook Titanic-Normalization { version 1 (metaStatement meta { description "Test" source "my-rules.xslx" user "joltie" }) (ruleStatement rule remove-first-line { description "Removes first line when offset is zero" (whenThenStatement when ((expression (assignmentExpression (conditionalExpression (logicalOrExpression (logicalAndExpression (inclusiveOrExpression (exclusiveOrExpression (andExpression (equalityExpression (relationalExpression (shiftExpression (additiveExpression (multiplicativeExpression (castExpression (unaryExpression (postfixExpression (postfixExpression (primaryExpression present)) ((argumentExpressionList (assignmentExpression (conditionalExpression (logicalOrExpression (logicalAndExpression (inclusiveOrExpression (exclusiveOrExpression (andExpression (equalityExpression (relationalExpression (shiftExpression (additiveExpression (multiplicativeExpression (castExpression (unaryExpression (postfixExpression (primaryExpression offset)))))))))))))))))))))))))))))))))) && offset == 0) then { filter-row-if-true true ;) }) }) 

我也寫單元測試，以測試等"when (offset == 0) then {\n" + "filter-row-if-true true;\n" + "}\n"短輸入上下文調試問題。但它仍然有像錯誤：

line 1:16 mismatched input '0' expecting {'(', '++', '--', '&&', '&', '*', '+', '-', '~', '!', Identifier, GeneralIdentifier, DigitSequence, StringLiteral} 
line 2:19 extraneous input 'true' expecting {'(', '++', '--', '&&', '&', '*', '+', '-', '~', '!', ';', Identifier, GeneralIdentifier, DigitSequence, StringLiteral} 

有兩個一天的嘗試，我沒有得到任何進展。問題是隻要以上，請有人給我一些建議如何調試ANTLR4語法無關/不匹配的輸入錯誤。

Answer 1

我不知道是否有任何更復雜的方法來調試語法/分析器，但這裏的如何我usally做到這一點：

1. 減少引起該問題，以儘可能少的字符 輸入可能。

2. 儘可能減少你的語法，這樣它仍然會在相應的輸入上產生相同的錯誤（大部分時間意味着通過循環原始語法的規則來減少輸入的最小語法（簡化爲儘量）

3. 確保詞法分析器段輸入正確（對於在ANTLRWorks，顯示你的詞法分析器輸出的特徵是偉大的）

4. 看一看分析樹。ANTLR的testRig有個特點以圖形方式顯示ParseTree（您可以通過ANTLRWorks或ANTLR的訪問此功能），所以你可以看看解析器的解釋與你所使用的不同。

5. 做「手動」解析。這意味着你將學習語法並逐步完成輸入，一步一步地嘗試應用邏輯或假設/知識等。在那個過程中。只要按照自己的語法做一臺電腦就可以做到。問題每次採取步驟（是否有另一種方式來匹配輸入），並總是試圖輸入以另一種方式比一個你真正想要它被解析

嘗試修復錯誤的最小匹配語法，然後將解決方案遷移到您的真實語法。