當電子郵件正文中存在unicode字符時，使用python進行Gmail電子郵件解析

我寫了一個腳本來解析電子郵件。當從Mac OS X Mail客戶端接收信件時（這個測試到目前爲止），它工作正常，但是當字母在其身體部分包含unicode字母時，我的分析器失效。當電子郵件正文中存在unicode字符時，使用python進行Gmail電子郵件解析

例如，我發送了一條消息，內容爲ąčę。

這裏是我的腳本的一部分，同時解析正文和附件：

p = FeedParser() 
p.feed(msg) 
msg = p.close() 
attachments = [] 
body = None 
for part in msg.walk(): 
    if part.get_content_type().startswith('multipart/'): 
    continue 
    try: 
    filename = part.get_filename() 
    except: 
    # unicode letters in filename, set default name then 
    filename = 'Mail attachment' 

    if part.get_content_type() == "text/plain" and not body: 
    body = part.get_payload(decode=True) 
    elif filename is not None: 
    content_type = part.get_content_type() 
    attachments.append(ContentFile(part.get_payload(decode=True), filename)) 

if body is None: 
    body = ''

嗯，我提到，它的工作原理與OS X Mail中的字母，但與Gmail的信件它不。

回溯：

Traceback (most recent call last): File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/core/handlers/base.py", line 116, in get_response response = callback(request, *callback_args, **callback_kwargs) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/views/decorators/csrf.py", line 77, in wrapped_view return view_func(*args, **kwargs) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/views/decorators/http.py", line 41, in inner return func(request, *args, **kwargs) File "/Users/aemdy/PycharmProjects/rezervavau/bms/messages/views.py", line 66, in accept Message.accept(request.POST.get('msg')) File "/Users/aemdy/PycharmProjects/rezervavau/bms/messages/models.py", line 261, in accept thread=thread File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/manager.py", line 149, in create return self.get_query_set().create(**kwargs) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/query.py", line 391, in create obj.save(force_insert=True, using=self.db) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/base.py", line 532, in save force_update=force_update, update_fields=update_fields) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/base.py", line 627, in save_base result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert return insert_query(self.model, objs, fields, **kwargs) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/query.py", line 1633, in insert_query return query.get_compiler(using=using).execute_sql(return_id) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 920, in execute_sql cursor.execute(sql, params) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/backends/util.py", line 47, in execute sql = self.db.ops.last_executed_query(self.cursor, sql, params) File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/operations.py", line 201, in last_executed_query return cursor.query.decode('utf-8') File "/Users/aemdy/virtualenvs/django1.5/lib/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xe0 in position 115: invalid continuation byte

我的劇本給我了以下機身��。我怎樣才能解碼它得到ąčę回來？

來源

2012-11-18 aemdy

與您發送特殊字符的字符串的Latin-1編碼和你試圖將其解釋爲utf-8，這顯然失敗了。 –

但我需要一種通用的方式來解析電子郵件的正文。我怎樣才能做到這一點？將latin-1解碼後得到'àèæë' – aemdy

嗯，我自己找到了解決方案。我現在會做一些測試，如果有任何失敗，現在就讓你們參加。

我需要再次身體解碼：

body = part.get_payload(decode=True).decode(part.get_content_charset())

來源

2012-11-18 23:42:19 aemdy

你可能想看看email.iterators（不知道它會解決你的編碼問題，雖然）。

來源

2012-11-18 23:14:34

好吧，我認爲我不能在那裏找到解決方案。它正確迭代，但解碼/編碼失敗困難。 – aemdy

你可以添加你用來測試你的問題的電子郵件嗎？否則很難再現問題。 –

您可能需要使用試試這個：

from email.Iterators import typed_subpart_iterator 


def get_charset(message, default="ascii"): 
    """Get the message charset""" 

    if message.get_content_charset(): 
     return message.get_content_charset() 

    if message.get_charset(): 
     return message.get_charset() 

    return default 

def get_body(message): 
    """Get the body of the email message""" 

    if message.is_multipart(): 
     #get the plain text version only 
     text_parts = [part 
         for part in typed_subpart_iterator(message, 
                 'text', 
                 'plain')] 
     body = [] 
     for part in text_parts: 
      charset = get_charset(part, get_charset(message)) 
      body.append(unicode(part.get_payload(decode=True), 
           charset, 
           "replace")) 

     return u"\n".join(body).strip() 

    else: # if it is not multipart, the payload will be a string 
      # representing the message body 
     body = unicode(message.get_payload(decode=True), 
         get_charset(message), 
         "replace") 
     return body.strip()

來源

2013-03-01 11:48:44

當電子郵件正文中存在unicode字符時，使用python進行Gmail電子郵件解析

回答

相關問題