uWSGI無法在日誌文件中寫入重定向到Python日誌文件的Unicode數據

我正在使用uWSGI（2.0.11.2）和Python（3.4.3）在Ubuntu 14.04上爲我的金字塔（1.5.7）應用程序服務。我注意到，我正在與Unicode的解碼在我uWSGI日誌中的錯誤：uWSGI無法在日誌文件中寫入重定向到Python日誌文件的Unicode數據

# 
# one of the situations when exception is raised is 
# when SQLAlchemy (which has set INFO logging level) 
# tries to write an SQL statement containing unicode charater 
# into log file 
# 
2016-02-26 16:01:38,734 INFO [sqlalchemy.engine.base.Engine][b'uWSGIWorker5Core0'] BEGIN (implicit) 
2016-02-26 16:01:38,735 INFO [sqlalchemy.engine.base.Engine][b'uWSGIWorker5Core0'] SELECT * FROM staging WHERE company_name = %(company_name_1)s AND time = %(time_1)s AND ship_name = %(ship_name_1)s 
# exact place (missing line) where SQLAlchemy is trying to print out 
# query parameters, which in this case include unicode character 
--- Logging error --- 
Traceback (most recent call last): 
    File "/usr/lib/python3.4/logging/__init__.py", line 980, in emit 
    stream.write(msg) 
UnicodeEncodeError: 'ascii' codec can't encode character '\xfa' in position 132: ordinal not in range(128) 
Call stack: 
    File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqltap/wsgi.py", line 42, in __call__ 
    return self.app(environ, start_response) 
    File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/pyramid/router.py", line 242, in __call__ 
    response = self.invoke_subrequest(request, use_tweens=True) 
    # 
    # the stack continues... 
    # full stack here -> https://bpaste.net/show/8e12af790372 
    # 
    File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement 
    compiled_sql, distilled_params 
    File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1100, in _execute_context 
    sql_util._repr_params(parameters, batches=10) 
Unable to print the message and arguments - possible formatting error. 
Use the traceback above to help find the error.

我也注意到，寫同一行成金字塔生成日誌文件（不涉及uWSGI）工作完全正常，沒有錯誤無論如何，以及unicode字符都已正確插入。

我用這個命令運行uWSGI：

/usr/local/bin/uwsgi --emperor /etc/uwsgi/vassals

在vassals文件夾我從金字塔的應用程序符號鏈接uWSGI的配置，它看起來像這樣：

[uwsgi] 

host = %h 
username = mk 
project_name = api 
project_root = /shared/projects/python/%(project_name) 

env = PYTHONIOENCODING=UTF-8 

; this env var is generated based on host name 
env = APP_INI_FILE=develop.ini 

; folders config 
home_folder = /home/%(username) 
virtualenv_folder = %(home_folder)/.virtualenvs/%(project_name) 
logs_folder = %(home_folder)/logs/%(project_name) 
chdir = %(project_root) 
socket = %(project_root)/%(project_name).sock 
pidfile = %(project_root)/%(project_name).pid 
virtualenv = %(virtualenv_folder) 
daemonize = %(logs_folder)/uwsgi.log 

; core stuff 
master = true 
vacuum = true 
processes = 5 
enable-threads = true 

; socket conf 
chmod-socket = 666 # invoking the One 
chown-socket = %(username) 
uid = %(username) 
gid = %(username) 

; log conf 
log-reopen = true 
logfile-chown = %(username) 
logfile-chmod = 644 

; app conf 
module = wsgi:application 
harakiri = 120 
max-requests = 500 
post-buffering = 1 
paste = config:%p 
paste-logger = $p

金字塔的配置文件，所有的測井定義如下所示：

### 
# app configuration 
# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html 
### 

[DEFAULT] 
home_dir = /home/mk 

[app:main] 
use = egg:api 

pyramid.reload_templates = false 
pyramid.debug_authorization = false 
pyramid.debug_notfound = false 
pyramid.debug_routematch = false 
pyramid.default_locale_name = en 

sqlalchemy.url = postgresql://XXX:[email protected]:5432/ze_database?client_encoding=utf8 

[server:main] 
use = egg:waitress#main 
host = 0.0.0.0 
port = 6543 

### 
# logging configuration 
# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html 
### 

[loggers] 
keys = root, sqlalchemy 

[handlers] 
keys = console, debuglog 

[formatters] 
keys = generic, short 

[logger_root] 
level = DEBUG 
handlers = console, debuglog 

[logger_sqlalchemy] 
level = INFO 
handlers = 
qualname = sqlalchemy.engine 
# "level = INFO" logs SQL queries. 
# "level = DEBUG" logs SQL queries and results. 
# "level = WARN" logs neither. (Recommended for production systems.) 

[handler_console] 
class = StreamHandler 
args = (sys.stderr,) 
level = DEBUG 
formatter = generic 

[handler_debuglog] 
class = handlers.RotatingFileHandler 
args = ('%(home_dir)s/logs/api/pyramid_debug.log', 'a', 1024000000, 10) 
level = DEBUG 
formatter = generic 

[formatter_generic] 
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s 

[formatter_short] 
format = %(asctime)s %(message)s

And finall Y，我金字塔的wsgi.py文件非常簡單：

import os 
from pyramid.paster import get_app, setup_logging 

here = os.path.dirname(os.path.abspath(__file__)) 
conf = os.path.join(here, os.environ.get('APP_INI_FILE')) # APP_INI_FILE variable is set in uwsgi.ini 
setup_logging(conf) 

application = get_app(conf, 'main')

從本質上講，我重定向我的應用程序的日誌stderr（或stdout，這一切都因爲據我已經注意到了相同的），我也把它寫在一個單獨的文件（pyramid_debug.log）。 stderr在我的情況下是uWSGI守護進程的日誌文件，這就是錯誤發生的地方。

儘管LC_ALL和相關變量在系統上設置爲en_EN.UTF-8，我也嘗試過使用各種與本地化相關的環境變量，並在我的金字塔的wsgi應用程序中明確設置它們，但運氣不大 - 例如僅設置PYTHONIOENCODING=UTF-8 uWSGI配置中的變量解決了我本地機器上的問題，但是一旦部署，我的服務器上就無法解決問題。

這裏我明顯的問題是 - 如何在這種情況下正確處理uWSGI在日誌文件中寫入unicode字符？

來源

2016-02-29 errata

編輯File "/usr/lib/python3.4/logging/__init__.py", line 980和改變

stream.write(msg)

到

stream.write(msg.encode('utf-8'))

最有可能的流類型中，不應該殺了你的UTF-8編碼的能力，但由於Pythonics實際上做的方式被改變。（看起來無論世界尖叫聲「UTF-8」多麼大聲，Python都會忽略所有人。）

例如，如果您處理文件，您嘗試的所有環境變量都將被忽略：

# test.py 
import sys 
sys.stdout.write(u'\u00f6\n')

測試：

max% python test.py 
ö 
max% python test.py > f 
Traceback (most recent call last): 
    File "test.py", line 2, in <module> 
    sys.stdout.write(u'\u00f6\n') 
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 0: ordinal not in range(128)

來源

2017-09-29 17:06:37

uWSGI無法在日誌文件中寫入重定向到Python日誌文件的Unicode數據

回答

相關問題