SED語句更改/修改CSV分隔符和分隔符

Question

我有一個包含逗號分隔值和一些列的值可以包含像,.<>!/\;&

我試圖將CSV轉換爲以逗號分隔的字符一些CSV文件，報價封閉CSV

示例數據：

DateCreated,DateModified,SKU,Name,Category,Description,Url,OriginalUrl,Image,Image50,Image100,Image120,Image200,Image300,Image400,Price,Brand,ModelNumber 
2012-10-19 10:52:50,2013-06-11 02:07:16,34,Austral Foldaway 45 Rotary Clothesline,Home & Garden > Household Supplies > Laundry Supplies > Drying Racks & Hangers,"Watch the Product Video   Plenty of Space to Hang a Family Wash Austral's Foldaway 45 rotary clothesline is a folding head rotary clothes hoist beautifully finished in either Beige or Heritage Green. Even though the Foldaway 45 is compact, you still get a large 45 metres of line space, big enough for a full family wash. If you want the advantage of a rotary hoist, but dont want to lose your yard, then the Austral Foldaway 45 is the clothesline for you.  Installation Note: A core hole is only required when installing into existing concrete, e.g. a pathway. Not required in the ground(grass/soil). To watch video on YouTube, click the following link: Austral Foldaway 45 Rotary Clothesline      //   Customer Video Reviews           ",https://track.commissionfactory.com.au/p/10604/1718695,http://www.lifestyleclotheslines.com.au/austral-foldaway-45-rotary-clothesline/,http://content.commissionfactory.com.au/Products/7228/1718695.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@50x50.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@100x100.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@120x120.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@200x200.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@300x300.jpg,http://content.commissionfactory.com.au/Products/7228/1718695@400x400.jpg,309.9000 AUD,Austral,FA45GR 

而且我試圖達到的輸出

"DateCreated","DateModified","SKU","Name","Category","Description","Url","OriginalUrl","Image","Image50","Image100","Image120","Image200","Image300","Image400","Price","Brand","ModelNumber" 
"2012-10-19 10:52:50","2013-06-11 02:07:16","34","Austral Foldaway 45 Rotary Clothesline","Home & Garden > Household Supplies > Laundry Supplies > Drying Racks & Hangers","Watch the Product Video   Plenty of Space to Hang a Family Wash Austral's Foldaway 45 rotary clothesline is a folding head rotary clothes hoist beautifully finished in either Beige or Heritage Green. Even though the Foldaway 45 is compact, you still get a large 45 metres of line space, big enough for a full family wash. If you want the advantage of a rotary hoist, but dont want to lose your yard, then the Austral Foldaway 45 is the clothesline for you.  Installation Note: A core hole is only required when installing into existing concrete, e.g. a pathway. Not required in the ground(grass/soil). To watch video on YouTube, click the following link: Austral Foldaway 45 Rotary Clothesline      //   Customer Video Reviews           ","https://track.commissionfactory.com.au/p/10604/1718695","http://www.lifestyleclotheslines.com.au/austral-foldaway-45-rotary-clothesline/","http://content.commissionfactory.com.au/Products/7228/1718695.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@50x50.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@100x100.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@120x120.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@200x200.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@300x300.jpg","http://content.commissionfactory.com.au/Products/7228/1718695@400x400.jpg","309.9000 AUD","Austral","FA45GR" 

任何援助非常感謝。

Answer 1

首先，讓我們嘗試瑣碎（和「不夠好」）的解決方案，只是增加了一個雙引號的各個領域（包括那些已經有雙引號這ISN！ 「T你想要的）

sed -r 's/([^,]*)/"\1"/g' 

大，第一部分尋找與他們沒有逗號序列，第二部分增加了他們周圍的雙引號，最終的‘G’是指這樣做不是每行

一次

這會變成

abc,345, some words ,"some text","text,with,commas" 

到 「ABC」， 「345」， 「有些話」 「」 一些文本 「」 「」 文本 「 」用「， 」逗號「，」

幾件事請注意：

• 它正確地圍繞着它們之間有空格的「某些單詞」，但也圍繞着初始和最終空格。我認爲沒關係，但如果沒有，它可以修復

• 如果該字段已經有引號，則會再次引用，這是壞的。需要修復

• 如果該字段已經有引號且內部文本有逗號（這不應該被認爲是字段分隔符），這些逗號也被引用。這也需要修復

因此，我們要匹配兩個不同的正則表達式 - 要麼有一個帶引號的字符串或現場沒有逗號：

sed -r 's/([^,"]*|"[^"]*")/"\1"/g' 

結果現在是

"abc","345"," some words ",""some text"",""text,with,commas"" 

正如您所看到的，我們對最初引用的文字有雙引號。爲此，我們將有一個第二sed命令刪除：

sed -r 's/([^,"]*|"[^"]*")/"\1"/g' | sed 's/""/"/g' 

導致

"abc","345"," some words ","some text","text,with,commas" 

耶！

Answer 2

這聽起來像你希望文件中的每一行都以雙引號開頭和結尾。如果是這樣應該工作：

sed -i.bak 's/^\(.*\)$/"\1"/' filename 

Answer 3

試試這個perl解決方案。它優於我以前的awk，因爲現在我使用了一個解析器來正確處理字段中的逗號。這是neccesary模塊Text::CSV_XS工作：

#!/usr/bin/env perl 

use strict; 
use warnings; 
use Text::CSV_XS; 

die qq|Usage: perl $0 <csv-file>\n| unless @ARGV == 1; 

open my $fh, '<', shift or die qq|ERROR: Could not open input file\n|; 

my $csv = Text::CSV_XS->new({ 
     always_quote => 1, 
}); 

while (my $row = $csv->getline($fh)) { 
     $csv->print(*STDOUT, $row); 
     print "\n"; 
} 
$csv->eof; 
close $fh;