讀取CSV文件並驗證基於UTF-8字符的列

Question

我必須讀取包含列（PersonName，年齡，地址）的CSV文件，並且必須驗證PersonName。 「PersonName只能包含UTF-8字符。」

我正在使用python3.x，因此無法在打開文件後使用解碼方法。

請告訴我如何打開和讀取文件，以便不包含任何UTF-8字符的PersonName可以被忽略，我可以移動到下一行進行驗證。

Answer 1

假設文件的其餘部分不需要檢查或是UTF-8合法（包括ASCII數據），您可以使用open與encoding='utf-8'和errors='replace'的文件。這會將任何無效字節（以UTF-8編碼）更改爲Unicode替換字符\ufffd。或者，要保留數據，可以使用'surrogateescape'作爲errors處理程序，該處理程序使用專用Unicode代碼以稍後可以撤消的方式表示原始值。然後，您可以檢查這些，你去：

with open(csvname, encoding='utf-8', errors='replace', newline='') as f: 
    for PersonName, age, address in csv.reader(f): 
     if '\ufffd' in PersonName: 
      continue 
     ... PersonName was decoded without errors, so process the row ... 

或者與surrogateescape，可以保證任何非UTF-8的數據（如果那是「可能」）在其他領域的恢復上寫：

with open(incsvname, encoding='utf-8', errors='surrogateescape', newline='') as inf,\ 
    open(outcsvname, 'w', encoding='utf-8', errors='surrogateescape', newline='') as outf: 
    csvout = csv.writer(outf) 
    for PersonName, age, address in csv.reader(f): 
     try: 
      # Check for surrogate escapes, and reject PersonNames containing them 
      # Most efficient way to do so is a test encode; surrogates will fail 
      # to encode with default error handler 
      PersonName.encode('utf-8') 
     except UnicodeEncodeError: 
      continue # Had non-UTF-8, skip this row 

     ... PersonName was decoded without surrogate escapes, so process the row ... 

     # You can recover the original file bytes in your code for a field with: 
     #  fieldname.encode('utf-8', errors='surrogateescape') 
     # Or if you're just passing data to a new file, write the same strings 
     # back to a file opened with the same encoding/errors handling; the surrogates 
     # will be restored to their original values: 
     csvout.writerow([PersonName, age, address])