瓶，SQLAlchemy和Jinja2 - UnicodeDecodeError

Question

我有一個Web應用程序，使用Flask，SQLAlchemy和WTForms，以及必要的Flask擴展，使其一切工作。 MySQL對所有表和列使用utf8_bin。

我插入一些中國文字和phpMyAdmin的正確顯示他們，但每當我試圖打開一個網頁，我得到以下異常：

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)

我明白我應該decode('utf8')我想要顯示的字段，但不該這是SQLAlchemy爲我處理的嗎？

我設法使這項工作的唯一途徑是通過結果列表進行迭代，並做類似的東西：

object.property = object.property.decode('utf8')

但顯然這不應該由手工完成。我錯過了什麼？

更新：SQLAlchemy的映射

class Thread(db.Model): 

    __tablename__ = 'Thread' 

    id = db.Column(db.Integer, primary_key=True) 
    title = db.Column(db.Unicode(255), nullable=False) 
    body = db.Column(db.Text, nullable=True) 
    date_created = db.Column(db.DateTime, nullable=False, default=datetime.now()) 
    created_by = db.Column(db.Integer, ForeignKey(User.id)) 
    user = relationship(User, backref='threads') 
    display_hash = db.Column(db.Unicode(255), nullable=False, unique=True) 
    display_name = db.Column(db.Unicode(255), nullable=True) 
    nsfw = db.Column(db.Boolean, nullable=False, default=False) 
    last_updated = db.Column(db.DateTime, nullable=False) 

    def __init__(self, title=None, body=None, category_id=None, display_name=None): 
     self.title = title 
     self.body = body 
     self.category_id = category_id 
     self.display_name = display_name 
     self.display_hash = custom_uuid() 
     self.last_updated = self.date_created 

    def __repr__(self): 
     return u'<Thread %r>' % (self.title) 

    def url_title(self): 
     """ Generates an ASCII-only slug. """ 

     result = [] 
     for word in _punct_re.split(self.title.lower()): 
      result.extend(unidecode(word).split()) 
     return unicode(u'-'.join(result)) 

更新：堆棧跟蹤

`127.0.0.1 - - [06/Oct/2013 02:37:15] "GET /index HTTP/1.1" 500 - 
Traceback (most recent call last): 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__ 
    return self.wsgi_app(environ, start_response) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app 
    response = self.make_response(self.handle_exception(e)) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception 
    reraise(exc_type, exc_value, tb) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app 
    response = self.full_dispatch_request() 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request 
    rv = self.handle_user_exception(e) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception 
    reraise(exc_type, exc_value, tb) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request 
    rv = self.dispatch_request() 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request 
    return self.view_functions[rule.endpoint](**req.view_args) 
    File "/Users/homedirectory/Projects/Assorted/Fruit Show/app/views.py", line 90, in index 
    return render_template('index.html', threads=threads, pagination=pagination) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/templating.py", line 128, in render_template 
    context, ctx.app) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/flask/templating.py", line 110, in _render 
    rv = template.render(context) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/jinja2/environment.py", line 969, in render 
    return self.environment.handle_exception(exc_info, True) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/jinja2/environment.py", line 742, in handle_exception 
    reraise(exc_type, exc_value, tb) 
    File "/Users/homedirectory/Projects/Assorted/Fruit Show/app/templates/index.html", line 1, in top-level template code 
    {% extends 'base.html' %} 
    File "/Users/homedirectory/Projects/Assorted/Fruit Show/app/templates/base.html", line 50, in top-level template code 
    {% block content %} 
    File "/Users/homedirectory/Projects/Assorted/Fruit Show/app/templates/index.html", line 14, in block "content" 
    <a href="{{ url_for('new_thread') }}/{{ thread.display_hash|safe }}/{{ thread.url_title()|safe }}">{{ thread.title|safe }}</a> 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/jinja2/filters.py", line 747, in do_mark_safe 
    return Markup(value) 
    File "/Users/homedirectory/.virtualenvs/fruitshow/lib/python2.7/site-packages/markupsafe/__init__.py", line 72, in __new__ 
    return text_type.__new__(cls, base) 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 0: ordinal not in range(128)` 

更新：URL項目回購：

https://github.com/ruipacheco/fruitshow

Answer 1

問題是我用的MySQL驅動程序。

我跟着this答案，並將列類型從utf8_bin切換到utf8_general_ci做了竅門。

Answer 2

在連接參數中設置charset只會告訴mysql將列轉換成數據庫中的列，然後將其轉換爲請求的格式編碼。數據仍然以字節的形式在MySQL和客戶端之間傳遞。總之，你必須告訴sqlalchemy「這個特定的」數據是unicode數據（在連接的編碼中）。對於大多數專欄，您已使用Unicode，這就是爲此目的服務的。值得注意的是body，其類型爲Text。你可能想UnicodeText或Text(convert_unicode=True)

Answer 3

有點建議S 012字段在您的型號。

有一個圖書館稱爲Webhelpers（https://pypi.python.org/pypi/WebHelpers），導入和你的標題將自動轉換爲slu。。

安裝WebHelpers，然後導入urlify

from webhelpers.text import urlify 
. 
. 
. 
@property 
def slug(self): 
    return urlify(self.title) 

Answer 4

不太你的答案，但讓我推薦ftfy（固定文本爲你），它修復了一大堆的小Unicode和HTML逃避問題。一種真正煩人的Unicode編碼宗教戰爭是UTF-8無法處理各種單字節字符編碼，如Latin-1。而不是隻是「哦，那肯定是一個簡單的拉丁字符」，解碼器會變得慌亂。當你的數據庫驅動程序觀察到「哦，這適合」時，它會在fatwah創建。