如何將unicode轉換爲拉丁字符python

Question

我想使用python將unicode轉換爲拉丁字符，我有一個大的文本文件，其中包含unicode和所有的tweet。我只想替換4個unicode，比如\ u00f6，\ u015f，.. 我只是想知道tweet是如何被實際推送的（原始語言）。這裏是實際收集推文並保存到文本文件中的代碼。 「＃！/ usr/bin/python

*編碼：ISO 8859-9 _ * _....」我得到這個錯誤「 21，但沒有聲明編碼;見http://www.python.org/peps/pep-0263.html細部」

class listener(StreamListener): 

    def on_data(self,data): 
     try: 
      dirty = open('turkeyjson28.txt','a') 
      encode = data.encode('ascii','ignore') 
      dirty.write(encode) 
      good = tweet.decode("utf-8") """ 
      better = good.decode("utf=8").replace(u"\u00f6", "ö") 
      print better  
      dirty.write('\n') 
      dirty.close() 
      tweet = data.split(',"text":"')[1].split('","source')[0] 
      #saveThis = str(time.time())+'::'+tweet 
      saveFile = open('turkey_clean28.txt','a') 
      saveFile.write(better) 
      saveFile.write('\n') 
      saveFile.write('\n') 
      saveFile.close() 
      return True 
     except BaseException, e: 
      print 'failed ondata,',str(e) 
      time.sleep(5) 
    def on_error(self, status): 
     print status 

auth = OAuthHandler(ckey,csecret) 
auth.set_access_token(atoken,asecret) 
twitterStream = Stream(auth,listener()) 
twitterStream.filter(track = ["turkey"]) 

Answer 1

better = good.decode("utf-8").replace(u"\u00f6", "ö") 

變化

better = good.decode("utf-8").replace(u"\u00f6", u"\u00f6".encode("utf8")) 

或作爲文件的第一行，你需要

#!/usr/bin/python 
# -*- coding: utf8 -*- 

一般來說，我會盡量避免使用的編碼解決方案，只是用Unicode字符編碼，你想怎麼

我會經常更換寫一對輔助函數來協助完成這項工作

def decode(byte_str,encodings=["latin1","utf8","cp1252"]): 
    if not isinstance(byte_str,str) and isinstance(byte_str,unicode): 
     byte_str = encode(byte_str,encodings) 
    for enc in encodings: 
     try: 
      return byte_str.decode(enc) 
     except UnicodeDecodeError: 
      continue 

def encode(unicode_txt,encodings=["latin1","utf8","cp1252"]): 
    if not isinstance(unicode_txt,unicode) and isinstance(unicode_txt,str): 
     unicode_txt = decode(unicode_txt,encodings) 
    for enc in encodings: 
     try: 
      return unicode_txt.encode(enc) 
     except UnicodeDecodeError: 
      continue 

#then you can just do something like 
decode(good).replace(u"\u00f6",decode(u"\u00f6",encodings=["utf8","latin1","ascii"]))