如何讓RegEx更高效？

Question

在我的代碼，我在斯卡拉做一個正則表達式匹配的情況是這樣的：

line match { 
    case regexp(unix_time, elapsed, remotehost, code_status, bytes, method, url, rfc931, peerstatus_peerhost, file_type) => 
     LogLine(getHumanDate(unix_time), elapsed, remotehost, code_status, bytes, method, url, rfc931, peerstatus_peerhost, file_type) 
    case _ => throw new IllegalArgumentException("Could not parse row: " + line) 
} 

我使用這個正則表達式。

val regexp = """(\d{9,10}\.\d{3})\s*(\d+) (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (\w+\/\d+) (\d+) (\w+) (\S+) (\-) (\w+\/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|\w+\/\-) (\S+)""".r 

我想讓它更具性能。我得到了它的工作，但它不是很快，我想它可以做得更好。

下面是需要符合一些樣品數行：

1393930710.739 278 192.168.1.20 TCP_MISS/200 5848 GET http://www.coderanch.com/templates/default/images/quote.gif - HIER_DIRECT/145.20.133.81 text/plain 

1393930719.989 73 192.168.178.27 TCP_MEM_HIT/200 268805 GET http://sunny:8080/viewapp/classpath/jquery.js - HIER_NONE/- application/x-javascript 

1393997284.209 59287 192.168.1.2 TCP_MISS/503 0 CONNECT 172.104.89.123:5228 - HIER_NONE/- - 

Answer 1

你不可能得到太多使用更快的正則表達式匹配，因爲即使只是在分割空間：

def split_apart(line: String) = line.split("""\s+""") match { 
    case Array(unix_time, elapsed, remotehost, code_status, bytes, method, url, rfc931, peerstatus_peerhost, file_type) => 
    (unix_time, elapsed, remotehost, code_status, bytes, method, url, rfc931, peerstatus_peerhost, file_type) 
    case _ => throw new Exception(":(") 
} 

需要的完整的regex比賽時間的60％。

如果您完全確定您必須關心此問題，那麼您需要手動完成。像這樣的東西是約6倍速度更快（在那裏的Java6子實際上並沒有複製字符串數據，我沒有檢查7）：

def parse(line: String) = { 
    def fail(s: String) = throw new Exception("Could not parse '"+s+"' in "+line) 
    def checkA(s: String) = { 
    if (s.length < 13 || s.length > 14 || s(s.length-4) != '.') fail(s) 
    var i = 0 
    while (i < s.length-4) { if (!s(i).isDigit) fail(s); i += 1 } 
    i += 1 
    while (i < s.length) { if (!s(i).isDigit) fail(s); i += 1 } 
    s 
    } 
    def checkB(s: String) = { 
    if (s.length == 0) fail(s) 
    var i = 0 
    while (i < s.length) { if (!s(i).isDigit) fail(s); i += 1 } 
    s 
    } 
    def checkC(s: String) = { 
    if (s.length < 7) fail(s) 
    var i = 0 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < 1 || i > 3 || s(i) != '.') fail(s) 
    var j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < j+1 || i > j+3 || i >= s.length || s(i) != '.') fail(s) 
    j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < j+1 || i > j+3 || i >= s.length || s(i) != '.') fail(s) 
    j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i != s.length) fail(s) 
    s 
    } 
    def checkD(s: String) = { 
    if (s.length < 3) fail(s) 
    var i = 0 
    while (i < s.length && { var c = s(i); c.isLetterOrDigit || c=='_' }) i += 1 
    if (i+1 >= s.length || !(s(i)=='/')) fail(s) 
    i += 1 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i != s.length) fail(s) 
    s 
    } 
    def checkE(s: String) = checkB(s) 
    def checkF(s: String) = { 
    if (s.length < 0) fail(s) 
    var i = 0 
    while (i < s.length) { var c = s(i); if (!(c.isLetterOrDigit || c=='_')) fail(s); i += 1 } 
    s 
    } 
    def checkG(s: String) = s 
    def checkH(s: String) = { if (s != "-") fail(s); s } 
    def checkI(s: String): String = { 
    if (s.length < 3) fail(s) 
    var i = 0 
    while (i < s.length && { var c = s(i); (c.isLetterOrDigit || c=='_') }) i += 1 
    if (i+1 >= s.length || !(s(i)=='/')) fail(s) 
    i += 1 
    if (s(i) == '-' && i+1 == s.length) return s 
    var j = i 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < j+1 || i > j+3 || s(i) != '.') fail(s) 
    j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < j+1 || i > j+3 || i >= s.length || s(i) != '.') fail(s) 
    j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i < j+1 || i > j+3 || i >= s.length || s(i) != '.') fail(s) 
    j = i+1 
    i = j 
    while (i < s.length && s(i).isDigit) i += 1 
    if (i != s.length) fail(s) 
    s 
    } 
    def checkJ(s: String) = s 
    val cs = line 
    val a0 = 0 
    var a1 = 0 
    while (a0 < line.length && !cs(a1).isWhitespace) a1 += 1 
    var b0 = a1+1 
    while (b0 < line.length && cs(b0).isWhitespace) b0 += 1 
    var b1 = b0+1 
    while (b1 < line.length && !cs(b1).isWhitespace) b1 += 1 
    val c0 = b1+1 
    var c1 = c0+1 
    while (c1 < line.length && !cs(c1).isWhitespace) c1 += 1 
    val d0 = c1+1 
    var d1 = d0+1 
    while (d1 < line.length && !cs(d1).isWhitespace) d1 += 1 
    val e0 = d1+1 
    var e1 = e0+1 
    while (e1 < line.length && !cs(e1).isWhitespace) e1 += 1 
    val f0 = e1+1 
    var f1 = f0+1 
    while (f1 < line.length && !cs(f1).isWhitespace) f1 += 1 
    val g0 = f1+1 
    var g1 = g0+1 
    while (g1 < line.length && !cs(g1).isWhitespace) g1 += 1 
    val h0 = g1+1 
    var h1 = h0+1 
    while (h1 < line.length && !cs(h1).isWhitespace) h1 += 1 
    val i0 = h1+1 
    var i1 = i0+1 
    while (i1 < line.length && !cs(i1).isWhitespace) i1 += 1 
    val j0 = i1+1 
    var j1 = j0+1 
    while (j1 < line.length && !cs(j1).isWhitespace) j1 += 1 
    (checkA(line.substring(a0,a1)), 
    checkB(line.substring(b0,b1)), 
    checkC(line.substring(c0,c1)), 
    checkD(line.substring(d0,d1)), 
    checkE(line.substring(e0,e1)), 
    checkF(line.substring(f0,f1)), 
    checkG(line.substring(g0,g1)), 
    checkH(line.substring(h0,h1)), 
    checkI(line.substring(i0,i1)), 
    checkJ(line.substring(j0,j1)) 
) 
} 

但你最好真的，真的關心有6x加速爲了這樣做。這是一個維護噩夢。

Answer 2

因爲你的目標是一致的線條，你可以做的第一個改進是使用線錨^的開始（這將使模式失敗更快）

^(\d{9,10}\.\d{3})\s*(\d+) (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (\w+/\d+) (\d+) (\w+) (\S+) (-) (\w+/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|\w+/-) (\S+) 

（我已經刪除了所有未完成的轉義）。你可以做一個另一件事就是去除uneeded捕獲組（我不知道那(-)是非常有用的），並試圖把線錨結束$

Answer 3

不知道你能真正獲得性能保持獨特的正則表達式。如果你真的不需要檢查每個零件的格式，如果你使用.split（「」）解析，然後檢查結果長度和只有潛在的不合格零件，它可能會更快......