如何使用elasticsearch-js和elasticsearch進行掃描和滾動5.0

Question

我之前使用elasticsearch-scrolltoend作爲插件，升級到5.0後插件似乎無法工作。如何使用elasticsearch 5.0掃描並滾動大型數據集？

我也收到嘗試時使用的執行在 elasticsearch-js docs錯誤：

json { "error": { "root_cause": [ { "type": "illegal_argument_exception", "reason": "Failed to parse request body" } ], "type": "illegal_argument_exception", "reason": "Failed to parse request body", "caused_by": { "type": "json_parse_exception", "reason": "Unrecognized token 'DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAHKFnV6a2NabEh4VDZLQmdzUzY0Y2tpd0EAAAAAAAAByxZ1emtjWmxIeFQ2S0Jnc1M2NGNraXdBAAAAAAAAAcwWdXprY1psSHhUNktCZ3NTNjRja2l3QQAAAAAAAAHOFnV6a2NabEh4VDZLQmdzUzY0Y2tpd0EAAAAAAAABzRZ1emtjWmxIeFQ2S0Jnc1M2NGNraXdB': was expecting ('true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@67ba4d99; line: 1, column: 457]" } }, "status": 400 }

Answer 1

無法識別的記號響應的錯誤是固定在13.0.0-rc1錯誤 - 見this issue for reference

至於實施，heres how I did it without the elasticsearch-scrolltoend plugin

// methods in some ES6 class 
    ... 

    fetchStuff(min, max = min) { 

    const body = new Bodybuilder(); 

    body.size(3); 
    body.filter('range', 'timestamp', { 
     gte: min, 
     lte: max 
    }); 

    return this.elasticsearch.search({ 
     index: 'my-alias', 
     type: 'my-doc', 
     body: body.build('v2'), // have not upgraded to newer bodybuilder package yet 
     scroll: '15s' 
    }) 
    .then((res) => this._scrollToEnd(res, [])) 
    .then((events) => { 

     // sort by timestamp then _id 
     events = _.sortBy(events, ['_.source.timestamp', '_id']); 

     return events; 

    }); 

    } 

    _scrollToEnd(res, events) { 
    return Promise.resolve().then(() => { 

     events = events.concat(_.get(res, 'hits.hits', [])); 

     if (res.hits.total > events.length) { 
     return this.elasticsearch.scroll({ 
      scrollId: res._scroll_id, 
      scroll: '15s' 
     }) 
     .then((res) => this._scrollToEnd(res, events)); 
     } 

     return events; 

    }); 
    }