如何在Perl中匹配字符串與變音符號？

Question

例如，匹配「民族報」在「」國際化」沒有額外的模塊，是否有可能在新的Perl版本（5.14，5.15等）？

我找到了答案！感謝tchrist

與UCA匹配分辯溶液（日Thnx到https://stackoverflow.com/users/471272/tchrist）。

# found start/end offsets for matched utf-substring (without intersections) 
use 5.014; 
use strict; 
use warnings; 
use utf8; 
use Unicode::Collate; 
binmode STDOUT, ':encoding(UTF-8)'; 
my $str = "Îñţérñåţîöñåļîžåţîöñ" x 2; 
my $look = "Nation"; 
my $Collator = Unicode::Collate->new(
    normalization => undef, level => 1 
    ); 

my @match = $Collator->match($str, $look); 
if (@match) { 
    my $found = $match[0]; 
    my $f_len = length($found); 
    say "match result: $found (length is $f_len)"; 
    my $offset = 0; 
    while ((my $start = index($str, $found, $offset)) != -1) {             
     my $end = $start + $f_len; 
     say sprintf("found at: %s,%s", $start, $end); 
     $offset = $end + 1; 
    } 
} 

錯誤（但工作）從溶液

的代碼魔術段子：

$str = Unicode::Normalize::NFD($str); $str =~ s/\pM//g; 

代碼示例：

use 5.014; 
    use utf8; 
    use Unicode::Normalize; 

    binmode STDOUT, ':encoding(UTF-8)'; 
    my $str = "Îñţérñåţîöñåļîžåţîöñ"; 
    my $look = "Nation"; 
    say "before: $str\n"; 
    $str = NFD($str); 
    # M is short alias for \p{Mark} (http://perldoc.perl.org/perluniprops.html) 
    $str =~ s/\pM//og; # remove "marks" 
    say "after: $str";¬ 
    say "is_match: ", $str =~ /$look/i || 0; 

Answer 1

右溶液與UCA（日Thnx到tchrist）：

# found start/end offsets for matched s 
use 5.014; 
use utf8; 
use Unicode::Collate; 
binmode STDOUT, ':encoding(UTF-8)'; 
my $str = "Îñţérñåţîöñåļîžåţîöñ" x 2; 
my $look = "Nation"; 
my $Collator = Unicode::Collate->new(
    normalization => undef, level => 1 
    ); 

my @match = $Collator->match($str, $look); 
say "match ok!" if @match; 

P.S. 「假設您可以刪除變音符以獲取基本ASCII字母的代碼是邪惡的，仍然是殘破的，腦損壞的，錯誤的，並且是對死刑的理由。「 ©tchristWhy does modern Perl avoid UTF-8 by default?

Answer 2

你是什麼意思 「沒有額外的模塊」 是什麼意思？

這裏是use Unicode::Normalize;see on perl doc

的解決方案我去掉了「T」，並從您的字符串爲「L」，我的日食沒曾想保存腳本他們。

use strict; 
use warnings; 
use UTF8; 
use Unicode::Normalize; 

my $str = "Îñtérñåtîöñålîžåtîöñ"; 

for ($str) { # the variable we work on 
    ## convert to Unicode first 
    ## if your data comes in Latin-1, then uncomment: 
    #$_ = Encode::decode('iso-8859-1', $_); 
    $_ = NFD($_); ## decompose 
    s/\pM//g;   ## strip combining characters 
    s/[^\0-\x80]//g; ## clear everything else 
} 

if ($str =~ /nation/) { 
    print $str . "\n"; 
} 

輸出是

Internationaliation

的 「Z」 是從字符串刪除，它似乎不是一個組合字符。

的for循環的代碼是從這個方面How to remove diacritic marks from characters

另一個有趣的閱讀是從喬爾斯波斯基The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

更新：

由於@tchrist指出，有存在的算法，這更適合稱爲UCA（Unicode排序算法）。 @nordicdyno已經在他的問題中提供了一個實現。

該算法這裏Unicode Technical Standard #10, Unicode Collation Algorithm

描述perl的模塊這裏描述上perldoc.perl.org