Django pickle.dumps（model.query）擊中db

Question

我嘗試pickle django Query對象以將其保存在Redis中。

materials = Material.objects.prefetch_related('tags_applied').prefetch_related('materialdata_set').prefetch_related('source') 
materials_ids = MaterialData.objects.filter(tag_id__in=tags).values_list('material_id', flat=True) 
materials = materials.filter(pk__in=materials_ids) 
key_name = SAMPLES_UUID + ':' + str(redis_uuid) 
redis_cl.set_key(key_name, pickle.dumps(materials.query)) 
redis_cl.expire(key_name, SAMPLES_TIMEOUT) 

這裏是debug_panel跟蹤（我用懶惰分頁）： 來源查詢：。

選擇 「san_material」， 「ID」， 「san_material」 「created_at」， 「 「san_material」，「title」，「san_material」，「author」，「san_material」，「url」， 「san_material」。「publication_datetime」，「san_material」。「text」， 「san_material」。「size」 san_material「，」source_id「， 」san_material「，」material_type「，」san_material「，」updated_at「， 」san_material「。」status「，」san_material「。」 elastic_sync」， 「san_material」。 「標記」， 「san_material」。 「detection_datetime」， 「san_material」。 「ARTICLE_TITLE」， 「san_material」。 「publication_datetime_article」， 「san_material」。 「author_article」， 「san_material」 。「highlight_data」FROM 「san_material」WHERE（「san_material」。「detection_datetime」BETWEEN '2016-07-01T00：00：00 + 03：00':: timestamptz AND '2016-07-27T10：39：00 +03：00':: timestamptz AND「san_material」。「id」IN （SELECT U0。「material_id」FROM「san_materialdata」U0 WHERE U0。「tag_id」 IN（660）））ORDER BY「san_material」。「 detection_datetime「DESC LIMIT 51

但是，子查詢命中DB：

SELECT U0 「material_id」 FROM 「san_materialdata」 U0 WHERE U0在這裏 「TAG_ID」 IN（660）

：。

/home/maxx/analize/san/utils.py in wrapper(82) 
    result = method_to_decorate(*args, **kwds) 
/home/maxx/analize/san/views/flux.py in flux(111) 
    redis_cl.set_key(key_name, pickle.dumps(materials.query)) 
/usr/lib/python2.7/pickle.py in dumps(1393) 
    Pickler(file, protocol).dump(obj) 
/usr/lib/python2.7/pickle.py in dump(225) 
    self.save(obj) 
/usr/lib/python2.7/pickle.py in save(333) 
    self.save_reduce(obj=obj, *rv) 
/usr/lib/python2.7/pickle.py in save_reduce(421) 
    save(state) 
/usr/lib/python2.7/pickle.py in save(288) 
    f(self, obj) # Call unbound method with explicit self 
/usr/lib/python2.7/pickle.py in save_dict(657) 
    self._batch_setitems(obj.iteritems()) 
/usr/lib/python2.7/pickle.py in _batch_setitems(675) 
    save(v) 
/usr/lib/python2.7/pickle.py in save(333) 
    self.save_reduce(obj=obj, *rv) 
/usr/lib/python2.7/pickle.py in save_reduce(421) 
    save(state) 
/usr/lib/python2.7/pickle.py in save(288) 
    f(self, obj) # Call unbound method with explicit self 
/usr/lib/python2.7/pickle.py in save_dict(657) 
    self._batch_setitems(obj.iteritems()) 
/usr/lib/python2.7/pickle.py in _batch_setitems(675) 
    save(v) 
/usr/lib/python2.7/pickle.py in save(288) 
    f(self, obj) # Call unbound method with explicit self 
/usr/lib/python2.7/pickle.py in save_list(604) 
    self._batch_appends(iter(obj)) 
/usr/lib/python2.7/pickle.py in _batch_appends(620) 
    save(x) 
/usr/lib/python2.7/pickle.py in save(333) 
    self.save_reduce(obj=obj, *rv) 
/usr/lib/python2.7/pickle.py in save_reduce(421) 
    save(state) 
/usr/lib/python2.7/pickle.py in save(288) 
    f(self, obj) # Call unbound method with explicit self 
/usr/lib/python2.7/pickle.py in save_dict(657) 
    self._batch_setitems(obj.iteritems()) 
/usr/lib/python2.7/pickle.py in _batch_setitems(675) 
    save(v) 
/usr/lib/python2.7/pickle.py in save(308) 
    rv = reduce(self.proto) 
/home/maxx/venv/analize/lib/python2.7/copy_reg.py in _reduce_ex(84) 
    dict = getstate() 

我該如何解決？通過2.5秒

('Save obj time:', 2.5215649604797363, 'arg:', 'rhs') 
('Save obj time:', 2.5219039916992188, 'arg:', 'children') 
('Save obj time:', 2.5219550132751465, 'arg:', 'where') 

其3倍：

P.S我測量節省時間參數在DEF _batch_setitems。爲什麼？

Answer 1

Django的查詢的懶惰查詢的，但讓我解釋一下你所寫的內容：

materials = Material.objects.prefetch_related('tags_applied' 
    ).prefetch_related('materialdata_set').prefetch_related('source') 


materials_ids = MaterialData.objects.filter(tag_id__in=tags).values_list('material_id', flat=True) 

# till now materials_id is queryset, means it will not hit DB. 
# as soon it execute next line of code it will hit db, because in next line you are using materials_ids. 

materials = materials.filter(pk__in=materials_ids) 

# So you can avoid hiting db if you are not required to use materials 
key_name = SAMPLES_UUID + ':' + str(redis_uuid) 
redis_cl.set_key(key_name, pickle.dumps(materials.query)) 
redis_cl.expire(key_name, SAMPLES_TIMEOUT) 

您可以通過使用在Django適當加入糾正這一點，

我猜你MaterialData模型材料作爲外鍵材料模型。

materials = MaterialData.objects.filter(tag_id__in=tags).prefetch_related(
'material__tags_applied' 
).prefetch_related('material__materialdata_set').prefetch_related('material__source').values(*all values realted to mateials you can put here by adding materials__ before each material field *) 

# to fetch foreign key attribue you use field followed by duble underscore 

key_name = SAMPLES_UUID + ':' + str(redis_uuid) 
redis_cl.set_key(key_name, pickle.dumps(materials.query))