將文本文件內容保存到數據庫：「第1行的列內容」字符串值不正確：'\ xEF \ xBB \ xBF＃W ...'

Question

在我的Django應用程序中，我上傳了一個文本文件，使用file.read（）獲取文件的內容，然後保存到數據庫（使用Django的.save（）方法）。

，我發現了以下錯誤：

Environment: 

Request Method: POST 
Request URL: http://localhost:8000/ 
Django Version: 1.2.5 
Python Version: 2.7.1 
Installed Applications: 
['django.contrib.auth', 
'django.contrib.contenttypes', 
'django.contrib.sessions', 
'django.contrib.sites', 
'django.contrib.messages', 
'django.contrib.admin', 
'django.contrib.markup', 
'files'] 
Installed Middleware: 
('django.middleware.common.CommonMiddleware', 
'django.contrib.sessions.middleware.SessionMiddleware', 
'django.middleware.csrf.CsrfViewMiddleware', 
'django.contrib.auth.middleware.AuthenticationMiddleware', 
'django.contrib.messages.middleware.MessageMiddleware') 


Traceback: 
File "/usr/lib/pymodules/python2.7/django/core/handlers/base.py" in get_response 
    100.      response = callback(request, *callback_args, **callback_kwargs) 
File "/home/mcrittenden/Dropbox/Code/dropdo-django/dropdo/files/views.py" in index 
    31.     return handle_upload(request.FILES['file']) 
File "/home/mcrittenden/Dropbox/Code/dropdo-django/dropdo/files/views.py" in handle_upload 
    60.  file.save() 
File "/usr/lib/pymodules/python2.7/django/db/models/base.py" in save 
    458.   self.save_base(using=using, force_insert=force_insert, force_update=force_update) 
File "/usr/lib/pymodules/python2.7/django/db/models/base.py" in save_base 
    551.      result = manager._insert(values, return_id=update_pk, using=using) 
File "/usr/lib/pymodules/python2.7/django/db/models/manager.py" in _insert 
    195.   return insert_query(self.model, values, **kwargs) 
File "/usr/lib/pymodules/python2.7/django/db/models/query.py" in insert_query 
    1524.  return query.get_compiler(using=using).execute_sql(return_id) 
File "/usr/lib/pymodules/python2.7/django/db/models/sql/compiler.py" in execute_sql 
    788.   cursor = super(SQLInsertCompiler, self).execute_sql(None) 
File "/usr/lib/pymodules/python2.7/django/db/models/sql/compiler.py" in execute_sql 
    732.   cursor.execute(sql, params) 
File "/usr/lib/pymodules/python2.7/django/db/backends/util.py" in execute 
    15.    return self.cursor.execute(sql, params) 
File "/usr/lib/pymodules/python2.7/django/db/backends/mysql/base.py" in execute 
    86.    return self.cursor.execute(query, args) 
File "/usr/lib/pymodules/python2.7/MySQLdb/cursors.py" in execute 
    168.   if not self._defer_warnings: self._warning_check() 
File "/usr/lib/pymodules/python2.7/MySQLdb/cursors.py" in _warning_check 
    82.      warn(w[-1], self.Warning, 3) 

Exception Type: Warning at/
Exception Value: Incorrect string value: '\xEF\xBB\xBF# W...' for column 'contents' at row 1 

我假設（因爲EF BB BF是UTF BOM角色），這是由於在數據庫和文件之間的字符集有區別嗎？這聽起來有效嗎？如果是這樣，我該如何解決它？

Answer 1

你在正確的道路上。檢查你的數據庫的字符集（是utf-8？）。如果不是，你想使用UTF-8，使用SQL命令

alter table yourTableName DEFAULT CHARACTER SET utf8; 

改變的charset關於Python中使用UTF-8讀取this great tutorial如果你想你的UTF-8字符串轉換回。

您可以使用此命令

# Strip the BOM from the beginning of the Unicode string, if it exists
u.lstrip(unicode(codecs.BOM_UTF8, "utf8"))

Answer 2

你是對的，你正在閱讀的文件的前面插入了BOM字符。在傳遞數據之前，您必須檢查並刪除這些字符。該文件的其餘部分將是UTF-8字符。

我不知道如何知道數據庫預期的字符集。