Elasticsearch模板不能按預期工作

Question

所以我有一個elasticsearch模板，我通過Filebeat傳遞給ES。我有Filebeat發送到logstash以及...這是相關的東西。

Elasticsearch模板

{ 
"template": "filebeat-*", 
"mappings": { 
    "product__name": { 
     "properties": { 
      "@timestamp": { 
       "type": "date", 
       "format": "strict_date_optional_time||epoch_millis" 
      }, 
      "@version": { 
       "type": "text" 
      }, 
      "beat": { 
       "properties": { 
        "hostname": { 
         "type": "text" 
        }, 
        "name": { 
         "type": "text" 
        } 
       } 
      }, 
      "class_method": { 
       "type": "text" 
      }, 
      "class_name": { 
       "type": "text", 
       "index": "true", 
       "fielddata": "true" 
      }, 
      "clientip": { 
       "type": "ip" 
      }, 
      "count": { 
       "type": "long" 
      }, 
      "host": { 
       "type": "text" 
      }, 
      "input_type": { 
       "type": "text" 
      }, 
      "log_level": { 
       "type": "text", 
       "fielddata": "true", 
       "index": "true" 
      }, 
      "log_message": { 
       "type": "text", 
       "index": "true" 
      }, 
      "log_timestamp": { 
       "type": "text" 
      }, 
      "log_ts": { 
       "type": "long" 
      }, 
      "message": { 
       "type": "text" 
      }, 
      "offset": { 
       "type": "long" 
      }, 
      "query_params": { 
       "type": "text", 
       "fielddata": "true", 
       "index": "true" 
      }, 
      "sessionid": { 
       "type": "text", 
       "index": "true" 
      }, 
      "source": { 
       "type": "text" 
      }, 
      "tags": { 
       "type": "text" 
      }, 
      "thread": { 
       "type": "text", 
       "index": "true" 
      }, 
      "type": { 
       "type": "text" 
      }, 
      "user_account_combo": { 
       "type": "text", 
       "index": "true" 
      }, 
      "version": { 
       "type": "text" 
      } 
     } 
    }, 
    "access": { 
     "properties": { 
      "@timestamp": { 
       "type": "date", 
       "format": "strict_date_optional_time||epoch_millis" 
      }, 
      "@version": { 
       "type": "text" 
      }, 
      "beat": { 
       "properties": { 
        "hostname": { 
         "type": "text" 
        }, 
        "name": { 
         "type": "text" 
        } 
       } 
      }, 
      "clientip": { 
       "type": "ip" 
      }, 
      "count": { 
       "type": "long" 
      }, 
      "host": { 
       "type": "text", 
       "index": "true" 
      }, 
      "input_type": { 
       "type": "text" 
      }, 
      "log_timestamp": { 
       "type": "text" 
      }, 
      "log_ts": { 
       "type": "long" 
      }, 
      "message": { 
       "type": "text" 
      }, 
      "offset": { 
       "type": "long" 
      }, 
      "query_params": { 
       "type": "text", 
       "fielddata": "true", 
       "index": "true" 
      }, 
      "response_time": { 
       "type": "long" 
      }, 
      "sessionid": { 
       "type": "text", 
       "index": "true" 
      }, 
      "source": { 
       "type": "text" 
      }, 
      "statuscode": { 
       "type": "long" 
      }, 
      "tags": { 
       "type": "text" 
      }, 
      "thread": { 
       "type": "text", 
       "index": "true" 
      }, 
      "type": { 
       "type": "text", 
       "index": "true" 
      }, 
      "uripath": { 
       "type": "text", 
       "fielddata": "true", 
       "index": "true" 
      }, 
      "user_account_combo": { 
       "type": "text", 
       "index": "true" 
      }, 
      "verb": { 
       "type": "text", 
       "fielddata": "true", 
       "index": "true" 
      } 
     } 
    } 
} 
} 

filebeat配置（修剪）

output.elasticsearch: 
    hosts: ["10.10.43.210:9200"] 
    template: 
    name: filebeat 
    path: "test-template.json" 
    overwrite: true 

output.logstash: 
    hosts: ["10.10.43.210:5044"] 
    worker: 2 
    index: filebeat 

Logstash配置（修剪）

output { 
    stdout { } 
    elasticsearch { 
    hosts => "elasticsearch:9200" 
    manage_template => false 
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}" 
    } 

最後一點......這裏就是在Kibana顯示出來：

所以，問題是：爲什麼我看到的東西.keyword字段時，我已經非常明確的關於我想要的一切，以什麼類型的是不是讓logstash覆蓋我的模板？我錯過了什麼嗎？

Answer 1

Kibana 5.x爲所有string類型字段生成.keyword字段。這使您可以根據整個字段進行聚合。例如，如果您想對host字段的唯一值進行計數，則可以在host.keyword上執行彙總。

因此，這種行爲並不意味着您的模板被覆蓋，這是Kibana允許您在字符串字段上進行聚合的有意行爲。如果你想檢查你的模板是否被Logstash以某種方式覆蓋，請使用 curl elasticsearch:9200/{template_name}?pretty檢查Elasticsearch，它將顯示你的索引映射。這可以幫助您驗證您的索引是否使用了您期望的映射。