2016-05-18 78 views
1

我在我的Flask應用程序中進行性能測試。多個請求燒瓶,Gunicorn,Nginx日誌不工作

ngnix -> gunicorn -> flask 

日誌文件:

  • 的access.log OK
  • gunicorn.log OK
  • api.log 只顯示1個請求

我我在用g ab測試工具。我的請求被正確處理,並在我的開發環境(MacOS 10.10.5)中的Flask日誌文件中正確記錄。

ab -c 10 -n 10 -A username:password https://1.1.1.1:8443/api/1.0/echo/ 

當我轉到生產時,(Ubuntu)我看不到Flask日誌文件中記錄的所有10個請求。我只在api日誌文件中看到1個。 ab工具報告所有請求處理正確。

我在Gunnorn的Ngnix看到了10個請求,但在Flask中看不到。

gunicorn啓動腳本(Supervisord)

GUNICORN_LOGFILE=/usr/local/src/imbue/application/imbue/log/gunicorn.log 
NUM_WORKERS=8 
TIMEOUT=60 
WORKER_CONNECTIONS=2000 
BACKLOG=1000 
exec gunicorn api:api_app --bind 0.0.0.0:8081 --log-level=DEBUG --log-file=$GUNICORN_LOGFILE --workers $NUM_WORKERS --worker-connections=$WORKER_CONNECTIONS --backlog=$BACKLOG --timeout $TIMEOUT 

API文件:

from application.app import api_app 

if __name__ == "__main__": 
    api_app.run(debug=False, threaded=True) 

api_app瓶應用

# ========================================================= 
# API Logs 
# ========================================================= 

log = logging_conf.LoggerManager().getLogger("___app___", logging_file=settings.api_logfile) 

# ========================================================= 
# Flask Application imports and database 
# ========================================================= 
from flask import Flask 
from flask.ext import restful 
from werkzeug.contrib.fixers import ProxyFix 

# ========================================================= 
# Flask Main Application 
# ========================================================= 

api_app = Flask(__name__) # Flask Application 
api_app.config.from_pyfile("../../../conf/settings.py") # Flask configuration 

imbue_api = restful.Api(api_app) # Define API 
db = Model.db.init_app(api_app) # Initialize database 

# ========================================================= 
# API Definition 
# ========================================================= 

imbue_api.add_resource(ApiBase, settings.BASE_API_URL) 
imbue_api.add_resource(Echo, '/api/1.0/echo/') 
# ========================================================= 
# API Proxy WSGi for gunicorn 
# ========================================================= 

api_app.wsgi_app = ProxyFix(api_app.wsgi_app) 
log.info('Initializing API >>>') 

api_main

import logging 

# ========================================================= 
# IMBUE imports 
# ========================================================= 

from flask import current_app 
from flask import jsonify, request, Response, url_for 
from flask.ext import restful 
from flask.ext.restful import Resource 

# ========================================================= 
# API Controller 
# ========================================================= 

api = restful.Api 

# ========================================================= 
# API Logs 
# ========================================================= 

log = logging_conf.LoggerManager().getLogger("___app___", logging_file=settings.api_logfile) 

# ========================================================= 
# API Version information 
# ========================================================= 

class Echo(Resource): 
    @authenticator.requires_auth 
    def get(self): 
     """ 

     :return: 
     """ 
     try: 
      # ========================================================= 
      # GET API 
      # ========================================================= 
      log.info(request.remote_addr + ' ' + request.__repr__()) 
      if request.headers['Content-Type'] == 'application/json': 
       # ========================================================= 
       # Send API version information 
       # ========================================================= 
       log.info('api() | GET | Version' + settings.api_version) 
       response = json.dumps('version: ' + settings.api_version) 
       resp = Response(response, status=200, mimetype='application/json') 
       return resp 

版本

gunicorn==19.3.0 
Flask==0.10.1 
Flask-HTTPAuth==2.7.0 
Flask-Limiter==0.9.3 
Flask-Login==0.3.2 
Flask-Mail==0.9.1 
Flask-Migrate==1.6.0 
Flask-OAuthlib==0.9.1 
Flask-Principal==0.4.0 
Flask-RateLimiter==0.2.0 
Flask-RESTful==0.3.5 
Flask-Restless==0.17.0 
Flask-Script==2.0.5 
Flask-Security==1.7.5 
Flask-SQLAlchemy==2.1 
Flask-WTF==0.12 

logging_conf

class Singleton(type): 
    """ 

    """ 
    _instances = {} 

    def __call__(cls, *args, **kwargs): 
     if cls not in cls._instances.keys(): 
      cls._instances[cls] = super(Singleton, cls).__call__(*args, **kwargs) 
     return cls._instances[cls] 


class LoggerManager(object): 
    """ 

    """ 
    __metaclass__ = Singleton 

    _loggers = {} 

    def __init__(self, *args, **kwargs): 
     pass 

    @staticmethod 
    def getLogger(name='___app___', logging_file='imbued.log', **kwargs): 
     """ 

     :param name: 
     :param logging_file: 
     :param kwargs: 
     :return: 
     """ 
     # Define timezone 
     logging.basicConfig(filename=logging_file, 
          filemode='w+', 
          level=logging.INFO, 
          format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s', 
          datefmt='%m/%d/%Y %H:%M:%S') 

     if not name: 
      return logging.getLogger() 
     elif name not in LoggerManager._loggers.keys(): 
      LoggerManager._loggers[name] = logging.getLogger(str(name)) 
     return LoggerManager._loggers[name] 
+0

其中'logging_conf'是正在添加? –

+0

只是更新了原來的問題 – spicyramen

回答

1

在聲明log變量的問題發生。 gunicorn的多個線程調用getLogger的多個實例,然後覆蓋您的日誌文件。當您使用Werkzeug服務器時不會發生這種情況,threaded param app.run(threaded=True)

如果你只需要登錄請求您可以用參數--access-logfile REQUEST_LOG_FILE上gunicorn啓動腳本

GUNICORN_LOGFILE=/usr/local/src/imbue/application/imbue/log/gunicorn.log 
NUM_WORKERS=8 
TIMEOUT=60 
WORKER_CONNECTIONS=2000 
BACKLOG=1000 
REQUEST_LOG_FILE=./request-log-file.log 
exec gunicorn api:api_app --bind 0.0.0.0:8081 --log-level=DEBUG --log-file=$GUNICORN_LOGFILE --workers $NUM_WORKERS --worker-connections=$WORKER_CONNECTIONS --backlog=$BACKLOG --timeout $TIMEOUT --access-logfile REQUEST_LOG_FILE 

查看更多關於洛上documentation

警告:可能不是安全

舉一個簡單的解決方法,你可以改變basicConfig方法內getLogger來,而不是覆蓋文件,追加到現有文件。這是不安全的,多個線程更改同一個文件可能會導致意想不到的結果。

logging.basicConfig(filename=logging_file, 
         filemode='w+', # Open, read and truncate the file. 
         level=logging.INFO, 
         format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s', 
         datefmt='%m/%d/%Y %H:%M:%S') 

logging.basicConfig(filename=logging_file, 
         filemode='a+', # Open, read and append to a file. 
         level=logging.INFO, 
         format='%(asctime)s.%(msecs).03d %(levelname)s %(message)s', 
         datefmt='%m/%d/%Y %H:%M:%S') 

編輯1

顯然logging is thread safe但我不知道這將延長gunicorn工人...