在Django上傳文件時出現UnicodeEncodeError admin

Question

我的問題類似於報告的here，但建議的解決方案對我無效。我試圖通過Django管理應用上傳一個名爲'Testaråäö.txt'的文件。

我在Debian 6服務器上運行Django 1.3.1，使用Gunicorn 0.13.4和Nginx 0.7.6.7。數據庫是PostgreSQL 8.4.9。其他Unicode數據保存到數據庫沒有問題，所以我猜想問題必須以某種方式與文件系統。

我已經在我的nginx.conf設置

http { 
    charset utf-8; 
} 

。 LC_ALL和LANG被設置爲'sv_SE.UTF-8'。運行'語言環境'驗證了這一點。我甚至嘗試在我的nginx初始化腳本中設置LC_ALL和LANG，以確保locale設置正確。

這裏的回溯：

Traceback (most recent call last): 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/handlers/base.py", line 111, in get_response 
response = callback(request, *callback_args, **callback_kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 307, in wrapper 
return self.admin_site.admin_view(view)(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/views/decorators/cache.py", line 79, in _wrapped_view_func 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/sites.py", line 197, in inner 
return view(request, *args, **kwargs) 

File "/srv/django/letebo/app/cms/admin.py", line 81, in change_view 
return super(PageAdmin, self).change_view(request, obj_id) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 28, in _wrapper 
return bound_func(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 93, in _wrapped_view 
response = view_func(request, *args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/utils/decorators.py", line 24, in bound_func 
return func(self, *args2, **kwargs2) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/transaction.py", line 217, in inner 
res = func(*args, **kwargs) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 985, in change_view 
self.save_formset(request, form, formset, change=True) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/contrib/admin/options.py", line 677, in save_formset 
formset.save() 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 482, in save 
return self.save_existing_objects(commit) + self.save_new_objects(commit) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 613, in save_new_objects 
self.new_objects.append(self.save_new(form, commit=commit)) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/forms/models.py", line 717, in save_new 
obj.save() 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 460, in save 
self.save_base(using=using, force_insert=force_insert, force_update=force_update) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 504, in save_base 
self.save_base(cls=parent, origin=org, using=using) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/base.py", line 543, in save_base 
for f in meta.local_fields if not isinstance(f, AutoField)] 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 255, in pre_save 
file.save(file.name, file, save=False) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/db/models/fields/files.py", line 92, in save 
self.name = self.storage.save(name, content) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 48, in save 
name = self.get_available_name(name) 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 74, in get_available_name 
while self.exists(name): 

File "/srv/.virtualenvs/letebo/lib/python2.6/site-packages/django/core/files/storage.py", line 218, in exists 
return os.path.exists(self.path(name)) 

File "/srv/.virtualenvs/letebo/lib/python2.6/genericpath.py", line 18, in exists 
st = os.stat(path) 

UnicodeEncodeError: 'ascii' codec can't encode characters in position 52-54: ordinal not in range(128) 

更新：我試着調試運行Gunicorn開啓，以及文件上傳沒有任何問題的。我想這意味着這個問題與Nginx有關。儘管如此，仍然打敗我去哪裏尋找。以下是Gunicorn和Nginx的，原始的響應頭，如果它使任何意義：

Gunicorn：

HTTP/1.1 302 FOUND 
Server: gunicorn/0.13.4 
Date: Thu, 09 Feb 2012 14:50:27 GMT 
Connection: close 
Transfer-Encoding: chunked 
Expires: Thu, 09 Feb 2012 14:50:27 GMT 
Vary: Cookie 
Last-Modified: Thu, 09 Feb 2012 14:50:27 GMT 
Location: http://my-server.se:8000/admin/cms/page/15/ 
Cache-Control: max-age=0 
Content-Type: text/html; charset=utf-8 
Set-Cookie: messages="yada yada yada"; Path=/ 

Nginx的：

HTTP/1.1 500 INTERNAL SERVER ERROR 
Server: nginx/0.7.67 
Date: Thu, 09 Feb 2012 14:50:57 GMT 
Content-Type: text/html; charset=utf-8 
Transfer-Encoding: chunked 
Connection: close 
Vary: Cookie 

500 

Answer 1

看來，這是服務的一個常見問題運行django服務，無論是apache，gunicorn + supervisor，daemontools等......並不總是使用正確的環境變量。

在這種情況下，gunicorn由主管開始，所以你必須明確地告訴上司使用環境選項可以使用UTF-8：

environment=LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8 

Answer 2

我有genericpath.py給人一種UnicodeEncodeError同樣的問題嘗試上傳帶有非ASCII字符的文件名時。

我使用nginx，uwsgi和django與python 2.7。

一切都在本地，但沒有工作正常的服務器上

下面是我所採取的步驟：

1. 添加到/ etc/nginx的/ nginx的。配置（並沒有解決這個問題）

   http { 
       charset utf-8; 
   } 
   

2. 我加入這行等/默認/區域（未解決的問題）

   LANGUAGE="en_US.UTF-8" 
   

3. 我跟着標題下這裏列出的說明'成功' https://code.djangoproject.com/wiki/ExpectedTestFailures（沒有解決這個問題）

   aptitude install language-pack-en-base 
   

4. 過這個票找到https://code.djangoproject.com/ticket/17816 這表明服務器有什麼用區域設置信息

Django的看法發生

import locale 

def locales(request): 
    """Display the locales""" 
    locales = "Current locale: %s %s -- Default locale: %s %s" % (
     locale.getlocale() + locale.getdefaultlocale()) 
    default_encoding = sys.getdefaultencoding() 
    file_system_encoding = sys.getfilesystemencoding() 

    context = { 
     'locales': locales, 
     'default_encoding': default_encoding, 
     'file_system_encoding': file_system_encoding, # affects file uploads 
    } 
    return render(request, 'testing/locales.html', context) 

Django的模板

<h2>Locales</h2> 
<p>{{ locales }}</p> 

<h2>Default Encoding</h2> 
<p>{{ default_encoding }}</p> 

<h2>File System Encoding</h2> 
<p>{{ file_system_encoding }}</p> 

對於我的測試來看，問題是，我的Ubuntu服務器上沒有語言環境和默認語言環境（儘管我的本地OSX開發機上有它們），然後使用非ASCII文件名稱/路徑將不會正確地上傳與python引發UnicodeEncodeError，但只能在生產服務器上。

解決方案

我將此添加到我的網站，我的網站管理員uwsgi配置文件 例如兩在/ etc/uwsgi皇帝/諸候/我的站點，配置-INI文件

env = LANG=en_US.utf8 

更新

移動到泊塢窗後，我又開始得到同樣的錯誤。 進一步研究後，我意識到在上面的uwsgi django實例import sys; sys.getfilesystemencoding()或{{ file_system_encoding }}返回ANSI_X3.4-1968，但是如果我啓動我自己的python實例並運行import sys; sys.getfilesystemencoding()我會得到UTF-8。 ANSI_X3.4-1968格式是拋出UnicodeEncodeError的。

所以除了上面這個答案列出的uwsgi的解決方案，我也不得不加入到我的Django dockerfile

ENV LANG en_US.UTF-8 
RUN apt-get update && install -y locales && \ 
    sed -i -e "s/# $LANG.*/$LANG UTF-8/" /etc/locale.gen && \ 
    locale-gen --purge &&\ 
    update-locale LANG=$LANG 

或者，如果你不使用泊塢窗你只是在shell中運行這個

$ export LANG=en_US.UTF-8 

然後運行上面的RUN命令（但沒有運行）。

參考文獻：

http://stackoverflow.com/a/37246853/3003438 
https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/uwsgi/