0
我正在練習Logstash,我無法理解這個錯誤的來源。 我想通過刪除一個字段並重命名一個來修改索引。 我在Logstash 5.6.2使用logstash更新索引失敗,可能是錯誤?
這是我的條目示例:
{
"_index": "test-shakespeare-italian",
"_type": "logs",
"_id": "AV8QsD5pzoUbREG_8icr",
"_score": 1,
"_source": {
"nome_opera": "Henry IV",
"battuta_numero": "3.2.9",
"battuta": "Make me believe that thou art only markd",
"@timestamp": "2017-10-12T13:05:06.519Z",
"id_battuta": 1841,
"dialogo_numero": 1,
"@version": "1",
"interlocutore": "KING HENRY IV"
}
}
這是我的管道:
# this pipeline updates an existing index by
# changing name of field "nome_opera" in "opera"
# dropping field "dialogo numero"
input {
elasticsearch {
hosts => "localhost"
index => "test-shakespeare-italian"
query => '{"query": {"match_all": {}}}'
}
}
filter {
mutate {
rename => { "nome_opera" => "opera"}
remove_field => "dialogo_numero"
}
}
output {
elasticsearch {
index => "test-shakespeare-italian"
action => "update"
document_id => "OOW-j2DeSCmnsVWVsywOVQ"
hosts => "localhost:9200"
version => "1"
}
#stdout { codec => rubydebug }
}
首先,我沒有使用的版本字段,但這產生了以下錯誤:
[2017-10-13T11:29:25,464][WARN ][logstash.outputs.elasticsearch] Failed action. {:status=>409, :action=>["update", {:_id=>"OOW-j2DeSCmnsVWVsywOVQ", :_index=>"test-shakespeare-italian", :_type=>"logs", :_routing=>nil, :_retry_on_conflict=>1}, 2017-10-12T13:05:08.907Z %{host} %{message}], :response=>{"update"=>{"_index"=>"test-shakespeare-italian", "_type"=>"logs", "_id"=>"OOW-j2DeSCmnsVWVsywOVQ", "status"=>409, "error"=>{"type"=>"version_conflict_engine_exception", "reason"=>"[logs][OOW-j2DeSCmnsVWVsywOVQ]: version conflict, current version [17901] is different than the one provided [17900]", "index_uuid"=>"OOW-j2DeSCmnsVWVsywOVQ", "shard"=>"3", "index"=>"test-shakespeare-italian"}}}}
然後,我添加了版本字段,但仍管道失敗,出現以下錯誤:
[2017-10-13T11:38:39,467][ERROR][logstash.outputs.elasticsearch] Encountered an unexpected error submitting a bulk request! Will retry. {:error_message=>"undefined method `sanitized' for \"http://localhost:9200/_bulk\":String", :class=>"NoMethodError", :backtrace=>["/usr/local/Cellar/logstash/5.6.2/libexec/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.0-java/lib/logstash/outputs/elasticsearch/common.rb:249:in `safe_bulk'", "/usr/local/Cellar/logstash/5.6.2/libexec/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.0-java/lib/logstash/outputs/elasticsearch/common.rb:119:in `submit'", "/usr/local/Cellar/logstash/5.6.2/libexec/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.0-java/lib/logstash/outputs/elasticsearch/common.rb:87:in `retrying_submit'", "/usr/local/Cellar/logstash/5.6.2/libexec/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.0-java/lib/logstash/outputs/elasticsearch/common.rb:38:in `multi_receive'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/output_delegator.rb:49:in `multi_receive'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/pipeline.rb:436:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/pipeline.rb:435:in `output_batch'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/pipeline.rb:381:in `worker_loop'", "/usr/local/Cellar/logstash/5.6.2/libexec/logstash-core/lib/logstash/pipeline.rb:342:in `start_workers'"]}
- 我敢肯定,DOCUMENT_ID是正確的。
- 管道工程完美,如果不是更新我創建一個新的
- --config.test_and_exit指數給出OK
這 https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/612
這導致我認爲這是一個錯誤,不是我的錯。 不幸的是,在這些鏈接中沒有提供真正的解決方案,我是否必須恢復到舊版本的logstash?任何其他想法?
後最終JSON的問題嗎? 順便說一句,我試圖尋找其他方式,我用批量API和腳本字段在SECONDS中完成了這個。 如果用logstash更新索引太複雜了,我想我會每次只創建一個新索引。我不打算經常修改我的數據。 雖然這是個好主意嗎? – user3753342