sed：用下劃線替換引號內的空格

Question

我有一些輸入（例如從OpenBSD上的ifconfig run0 scan）有一些字段被空格分隔，但有些字段本身包含空格（幸運的是，包含空格的字段是總是用引號括起來）。

我需要區分引號內的空格和分隔符空格。這個想法是用下劃線替換引號內的空格。

的樣本數據：

%cat /tmp/ifconfig_scan | fgrep nwid | cut -f3 
nwid Websense chan 6 bssid 00:22:7f:xx:xx:xx 59dB 54M short_preamble,short_slottime 
nwid ZyXEL chan 8 bssid cc:5d:4e:xx:xx:xx 5dB 54M privacy,short_slottime 
nwid "myTouch 4G Hotspot" chan 11 bssid d8:b3:77:xx:xx:xx 49dB 54M privacy,short_slottime 

這並不最終處理方式我想，因爲我還沒有與下劃線引號但內更換的空間：

%cat /tmp/ifconfig_scan | fgrep nwid | cut -f3 |\ 
    cut -s -d ' ' -f 2,4,6,7,8 | sort -n -k4 
"myTouch Hotspot" 11 bssid d8:b3:77:xx:xx:xx 
ZyXEL 8 cc:5d:4e:xx:xx:xx 5dB 54M 
Websense 6 00:22:7f:xx:xx:xx 59dB 54M 

Answer 1

對於sed -only解決方案（我不一定提倡），嘗試：

echo 'a b "c d e" f g "h i"' |\ 
sed ':a;s/^\(\([^"]*"[^"]*"[^"]*\)*[^"]*"[^"]*\) /\1_/;ta' 
a b "c_d_e" f g "h_i" 

翻譯：

• 從行首開始。
• 尋找junk"junk"的模式，重複零次或多次，其中junk沒有報價，然後是junk"junk space。
• 用_替換最後的空格。
• 如果成功，跳回到開頭。

Answer 2

試這樣的：

awk -F'"' '{for(i=2;i<=NF;i++)if(i%2==0)gsub(" ","_",$i);}1' OFS="\"" file 

它適用於在一條線上多的報價部分：

echo '"first part" foo "2nd part" bar "the 3rd part comes" baz'| awk -F'"' '{for(i=2;i<=NF;i++)if(i%2==0)gsub(" ","_",$i);}1' OFS="\"" 
"first_part" foo "2nd_part" bar "the_3rd_part_comes" baz 

編輯替代形式：

awk 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' file 

Answer 3

你最好用perl。代碼是更具可讀性和可維護性：

perl -pe 's:"[^"]*":($x=$&)=~s/ /_/g;$x:ge' 

隨着你的輸入，結果是：

a b "c_d_e" f g "h_i" 

說明：

-p   # enable printing 
-e   # the following expression... 

s    # begin a substitution 

:    # the first substitution delimiter 

"[^"]*"  # match a double quote followed by anything not a double quote any 
       # number of times followed by a double quote 

:    # the second substitution delimiter 

($x=$&)=~s/ /_/g;  # copy the pattern match ($&) into a variable ($x), then 
         # substitute a space for an underscore globally on $x. The 
         # variable $x is needed because capture groups and 
         # patterns are read only variables. 

$x   # return $x as the replacement. 

:    # the last delimiter 

g    # perform the nested substitution globally 
e    # make sure that the replacement is handled as an expression 

---

一些測試：

for i in {1..500000}; do echo 'a b "c d e" f g "h i" j k l "m n o "p q r" s t" u v "w x" y z' >> test; done 

time perl -pe 's:"[^"]*":($x=$&)=~s/ /_/g;$x:ge' test >/dev/null 

real 0m8.301s 
user 0m8.273s 
sys  0m0.020s 

time awk 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' test >/dev/null 

real 0m4.967s 
user 0m4.924s 
sys  0m0.036s 

time awk '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" test >/dev/null 

real 0m4.336s 
user 0m4.244s 
sys  0m0.056s 

time sed ':a;s/^\(\([^"]*"[^"]*"[^"]*\)*[^"]*"[^"]*\) /\1_/;ta' test >/dev/null 

real 2m26.101s 
user 2m25.925s 
sys  0m0.100s 

Answer 4

不是一個答案，萬一有人爲@史蒂夫的Perl代碼只是發表的awk等效代碼的興趣（和幫助我在今後記住這一點）：

@steve發佈：

perl -pe 's:"[^\"]*":($x=$&)=~s/ /_/g;$x:ge' 

，並從閱讀@史蒂夫的解釋最短的AWK相當於Perl代碼（不是首選awk的解決方案 - 見@肯特的答案爲）將是GNU AWK：

gawk '{ 
    head = "" 
    while (match($0,"\"[^\"]*\"")) { 
     head = head substr($0,1,RSTART-1) gensub(/ /,"_","g",substr($0,RSTART,RLENGTH)) 
     $0 = substr($0,RSTART+RLENGTH) 
    } 
    print head $0 
}' 

，我們通過從POSIX awk的解決方案無線開始去個變量：

awk '{ 
    head = "" 
    tail = $0 
    while (match(tail,"\"[^\"]*\"")) { 
     x = substr(tail,RSTART,RLENGTH) 
     gsub(/ /,"_",x) 
     head = head substr(tail,1,RSTART-1) x 
     tail = substr(tail,RSTART+RLENGTH) 
    } 
    print head tail 
}' 

和保存與GNU awk的gensub（）行：

gawk '{ 
    head = "" 
    tail = $0 
    while (match(tail,"\"[^\"]*\"")) { 
     x = gensub(/ /,"_","g",substr(tail,RSTART,RLENGTH)) 
     head = head substr(tail,1,RSTART-1) x 
     tail = substr(tail,RSTART+RLENGTH) 
    } 
    print head tail 
}' 

，然後擺脫變量x的：

gawk '{ 
    head = "" 
    tail = $0 
    while (match(tail,"\"[^\"]*\"")) { 
     head = head substr(tail,1,RSTART-1) gensub(/ /,"_","g",substr(tail,RSTART,RLENGTH)) 
     tail = substr(tail,RSTART+RLENGTH) 
    } 
    print head tail 
}' 

，然後擺脫變量「尾」，如果你不需要$ 0，NF等，則在循環後留下：

gawk '{ 
    head = "" 
    while (match($0,"\"[^\"]*\"")) { 
     head = head substr($0,1,RSTART-1) gensub(/ /,"_","g",substr($0,RSTART,RLENGTH)) 
     $0 = substr($0,RSTART+RLENGTH) 
    } 
    print head $0 
}' 

Answer 5

另一個AWK嘗試：

awk '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" 

卸下報價：

awk '!(NR%2){gsub(FS,"_")}1' RS=\" ORS= 

---

有三重大小的測試文件進一步通過@steve完成之前測試的一些額外的測試。我必須稍微轉換一下sed聲明，以便非GNU sed也可以處理它。我包括awk（bwk）gawk3，gawk4和mawk：

$ for i in {1..1500000}; do echo 'a b "c d e" f g "h i" j k l "m n o "p q r" s t" u v "w x" y z' ; done > test 
$ time perl -pe 's:"[^"]*":($x=$&)=~s/ /_/g;$x:ge' test >/dev/null 

real 0m27.802s 
user 0m27.588s 
sys 0m0.177s 
$ time awk 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' test >/dev/null 

real 0m6.565s 
user 0m6.500s 
sys 0m0.059s 
$ time gawk3 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' test >/dev/null 

real 0m21.486s 
user 0m18.326s 
sys 0m2.658s 
$ time gawk4 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' test >/dev/null 

real 0m14.270s 
user 0m14.173s 
sys 0m0.083s 
$ time mawk 'BEGIN{FS=OFS="\""} {for(i=2;i<NF;i+=2)gsub(" ","_",$i)} 1' test >/dev/null 

real 0m4.251s 
user 0m4.193s 
sys 0m0.053s 
$ time awk '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" test >/dev/null 

real 0m13.229s 
user 0m13.141s 
sys 0m0.075s 
$ time gawk3 '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" test >/dev/null 

real 0m33.965s 
user 0m26.822s 
sys 0m7.108s 
$ time gawk4 '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" test >/dev/null 

real 0m15.437s 
user 0m15.328s 
sys 0m0.087s 
$ time mawk '!(NR%2){gsub(FS,"_")}1' RS=\" ORS=\" test >/dev/null 

real 0m4.002s 
user 0m3.948s 
sys 0m0.051s 
$ time sed -e :a -e 's/^\(\([^"]*"[^"]*"[^"]*\)*[^"]*"[^"]*\) /\1_/;ta' test > /dev/null 

real 5m14.008s 
user 5m13.082s 
sys 0m0.580s 
$ time gsed -e :a -e 's/^\(\([^"]*"[^"]*"[^"]*\)*[^"]*"[^"]*\) /\1_/;ta' test > /dev/null 

real 4m11.026s 
user 4m10.318s 
sys 0m0.463s 

渲染速度最快的結果mawk ...