如何使用解析器組合器處理'續行'

Question

我正在嘗試使用Sprache解析器組合器庫編寫一個小解析器。解析器應該能夠解析以單個\結尾的行作爲不重要的空白。

問題

如何創建一個解析器，可以解析可能包含續行字符\的=符號後的值？ 例如

a = b\e,\ 
    c,\ 
    d 

應被解析爲(KeyValuePair (Key, 'a'), (Value, 'b\e, c, d'))。

我是一般使用這個庫和解析器組合器的新手。所以任何正確的方向指針都非常感謝。

我已經試過

測試

public class ConfigurationFileGrammerTest 
{ 
    [Theory] 
    [InlineData("x\\\n y", @"x y")] 
    public void ValueIsAnyStringMayContinuedAccrossLinesWithLineContinuation(
     string input, 
     string expectedKey) 
    { 
     var key = ConfigurationFileGrammer.Value.Parse(input); 
     Assert.Equal(expectedKey, key); 
    } 
} 

生產

嘗試一個

public static readonly Parser<string> Value = 
     from leading in Parse.WhiteSpace.Many() 
     from rest in Parse.AnyChar.Except(Parse.Char('\\')).Many() 
      .Or(Parse.String("\\\n") 
      .Then(chs => Parse.Return(chs))).Or(Parse.AnyChar.Except(Parse.LineEnd).Many()) 
     select new string(rest.ToArray()).TrimEnd(); 

測試輸出

Xunit.Sdk.EqualException: Assert.Equal() Failure 
      ↓ (pos 1) 
Expected: x y 
Actual: x\ 
      ↑ (pos 1) 

嘗試2

public static readonly Parser<string> SingleLineValue = 
     from leading in Parse.WhiteSpace.Many() 
     from rest in Parse.AnyChar.Many().Where(chs => chs.Count() < 2 || !(string.Join(string.Empty, chs.Reverse().Take(2)).Equals("\\\n"))) 
     select new string(rest.ToArray()).TrimEnd(); 

    public static readonly Parser<string> ContinuedValueLines = 
     from firsts in ContinuedValueLine.AtLeastOnce() 
     from last in SingleLineValue 
     select string.Join(" ", firsts) + " " + last; 

    public static readonly Parser<string> Value = SingleLineValue.Once().XOr(ContinuedValueLines.Once()).Select(s => string.Join(" ", s)); 

測試輸出

Xunit.Sdk.EqualException: Assert.Equal() Failure 
      ↓ (pos 1) 
Expected: x y 
Actual: x\\n y 
      ↑ (pos 1) 

Answer 1

您不得在輸出線延續。這是最後一次單元測試的唯一問題。當你解析延續\\\n時，你必須從輸出結果中刪除它並返回空字符串。對不起，我不知道如何使用C＃sprache來做到這一點。也許有類似的東西：

Parse.String("\\\n").Then(chs => Parse.Return('')) 

我解決了使用combinatorix Python庫的問題。這是一個解析器組合器庫。 API使用函數而不是使用鏈接的方法，但想法是相同的。

以下是完整的代碼註釋：

# `apply` return a parser that doesn't consume the input stream. It 
# applies a function (or lambda) to the output result of a parser. 
# The following parser, will remove whitespace from the beginning 
# and the end of what is parsed. 
strip = apply(lambda x: x.strip()) 

# parse a single equal character 
equal = char('=') 

# parse the key part of a configuration line. Since the API is 
# functional it reads "inside-out". Note, the use of the special 
# `unless(predicate, parser)` parser. It is sometime missing from 
# parser combinator libraries. What it does is use `parser` on the 
# input stream if the `predicate` parser fails. It allows to execute 
# under some conditions. It's similar in spirit to negation in prolog. 
# It does parse *anything until an equal sign*, "joins" the characters 
# into a string and strips any space starting or ending the string. 
key = strip(join(one_or_more(unless(equal, anything)))) 

# parse a single carriage return character 
eol = char('\n') 

# returns a parser that return the empty string, this is a constant 
# parser (aka. it always output the same thing). 
return_empty_space = apply(lambda x: '') 
# This will parse a full continuation (ie. including the space 
# starting the new line. It does parse *the continuation string then 
# zero or more spaces* and return the empty string 
continuation = return_empty_space(sequence(string('\\\n'), zero_or_more(char(' ')))) 

# `value` is the parser for the value part. Unless the current char 
# is a `eol` (aka. \n) it tries to parse a continuation, otherwise it 
# parse anything. It does that at least once, ie. the value can not be 
# empty. Then, it "joins" all the chars into a single string and 
# "strip" from any space that start or end the value. 
value = strip(join(one_or_more(unless(eol, either(continuation, anything))))) 

# this basically, remove the element at index 1 and only keep the 
# elements at 0 and 2 in the result. See below. 
kv_apply = apply(lambda x: (x[0], x[2])) 

# This is the final parser for a given kv pair. A kv pair is: 
# 
# - a key part (see key parser) 
# - an equal part (see equal parser) 
# - a value part (see value parser) 
# 
# Those are used to parse the input stream in sequence (one after the 
# other). It will return three values: key, a '=' char and a value. 
# `kv_apply` will only keep the key and value part. 
kv = kv_apply(sequence(key, equal, value)) 


# This is sugar syntax, which turns the string into a stream of chars 
# and execute `kv` parser on it. 
parser = lambda string: combinatorix(string, kv) 


input = 'a = b\\e,\\\n c,\\\n d' 
assert parser(input) == ('a', 'b\\e,c,d')