ElasticSearch查找磁盤空間使用情況

Question

如何查找Elastic Search用於索引的磁盤空間量？我目前正在本地運行它，並且試圖查看我將在虛擬機上運行多少磁盤空間。

Answer 1

的Elasticsearch辦法做到這一點是使用_cat/shards，並期待在store柱：

curl -XGET "http://localhost:9200/_cat/shards?v" 

index    shard prirep state  docs store ip   node 
myindex_2014_12_19 2  r  STARTED 76661 415.6mb 192.168.1.1 Georgianna Castleberry 
myindex_2014_12_19 2  p  STARTED 76661 417.3mb 192.168.1.2 Frederick Slade 
myindex_2014_12_19 2  r  STARTED 76661 416.9mb 192.168.1.3 Maverick 
myindex_2014_12_19 0  r  STARTED 76984 525.9mb 192.168.1.1 Georgianna Castleberry 
myindex_2014_12_19 0  r  STARTED 76984 527mb 192.168.1.2 Frederick Slade 
myindex_2014_12_19 0  p  STARTED 76984 526mb 192.168.1.3 Maverick 
myindex_2014_12_19 3  r  STARTED 163 208.5kb 192.168.1.1 Georgianna Castleberry 
myindex_2014_12_19 3  p  STARTED 163 191.4kb 192.168.1.2 Frederick Slade 
myindex_2014_12_19 3  r  STARTED 163 181.6kb 192.168.1.3 Maverick 
myindex_2014_12_19 1  p  STARTED 424923 2.1gb 192.168.1.1 Georgianna Castleberry 
myindex_2014_12_19 1  r  STARTED 424923 2.1gb 192.168.1.2 Frederick Slade 
myindex_2014_12_19 1  r  STARTED 424923 2.1gb 192.168.1.3 Maverick 
myindex_2014_12_19 4  r  STARTED 81020 435.9mb 192.168.1.1 Georgianna Castleberry 
myindex_2014_12_19 4  p  STARTED 81020 437.8mb 192.168.1.2 Frederick Slade 
myindex_2014_12_19 4  r  STARTED 81020 437.8mb 192.168.1.3 Maverick 

否則在Linux中來查看文件夾的使用空間：

du -hs /myelasticsearch/data/folder 

或查看由文件系統空間：

df -h 

Answer 2

你可以使用no DES統計休息API
見：https://www.elastic.co/guide/en/elasticsearch/reference/1.6/cluster-nodes-stats.html

做出了FS統計，像這樣的請求： HTTP：//：9200/_nodes /統計/ FS漂亮= 1

，你會看到：

{ 
    "cluster_name" : "<cluster>", 
    "nodes" : { 
    "pEO34wutR7qk3Ix8N7MgyA" : { 
     "timestamp" : 1438880525206, 
     "name" : "<name>", 
     "transport_address" : "inet[/10.128.37.111:9300]", 
     "host" : "<host>", 
     "ip" : [ "inet[/10.128.37.111:9300]", "NONE" ], 
     "fs" : { 
     "timestamp" : 1438880525206, 
     "total" : { 
      "total_in_bytes" : 363667091456, 
      "free_in_bytes" : 185081352192, 
      "available_in_bytes" : 166608117760, 
      "disk_reads" : 154891, 
      "disk_writes" : 482628039, 
      "disk_io_op" : 482782930, 
      "disk_read_size_in_bytes" : 6070391808, 
      "disk_write_size_in_bytes" : 1989713248256, 
      "disk_io_size_in_bytes" : 1995783640064, 
      "disk_queue" : "0", 
      "disk_service_time" : "0" 
     }, 
     "data" : [ { 
      "path" : "/data1/elasticsearch/data/<cluster>/nodes/0", 
      "mount" : "/data1", 
      "dev" : "/dev/sda4", 
      "total_in_bytes" : 363667091456, 
      "free_in_bytes" : 185081352192, 
      "available_in_bytes" : 166608117760, 
      "disk_reads" : 154891, 
      "disk_writes" : 482628039, 
      "disk_io_op" : 482782930, 
      "disk_read_size_in_bytes" : 6070391808, 
      "disk_write_size_in_bytes" : 1989713248256, 
      "disk_io_size_in_bytes" : 1995783640064, 
      "disk_queue" : "0", 
      "disk_service_time" : "0" 
     } ] 
     } 
    } 
    } 
} 

的數據驅動器的空間被列出：

"total" : { 
    "total_in_bytes" : 363667091456, 
    "free_in_bytes" : 185081352192, 
    "available_in_bytes" : 166608117760, 

Answer 3

您可能需要使用nodewise的_cat API磁盤空間使用情況

curl http://host:9200/_cat/nodes?h=h,diskAvail 

參考：https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html

Answer 4

在你不需要每個碎片統計數字顯示/_cat/shards返回情況下，你可以使用

curl -XGET localhost:9200/_cat/allocation?v 

習慣，併爲每個可用的磁盤空間節點。

Answer 5

一個更簡潔的解決方案，以找到索引的大小是使用

curl -XGET 'localhost:9200/_cat/indices?v' 

輸出具有「store.size」列告訴你到底索引的大小。

health status index       uuid     pri rep docs.count docs.deleted store.size pri.store.size 
yellow open logstash-2017.03.01    TfraFM8TQkSXdxjx13CnpQ 5 1 33330000   0  1gb   1gb 
yellow open .monitoring-es-2-2017.03.02  10YscrcfQuGny5wMxeb0TA 1 1  68834   88  30.3mb   30.3mb 
yellow open .kibana       GE6xXV7QT-mNbX7xTPbZ4Q 1 1   3   0  14.5kb   14.5kb 
yellow open .monitoring-es-2-2017.03.01  SPeQNnPlRB6y7G6w1Axokw 1 1  29441   108  14.7mb   14.7mb 
yellow open .monitoring-data-2    LLeWqsD-QE-rPFblwu5K_Q 1 1   3   0  6.9kb   6.9kb 
yellow open .monitoring-kibana-2-2017.03.02 l_MAPERUTmSbq0xbhpnf2Q 1 1  5320   0  1.1mb   1.1mb 
yellow open .monitoring-kibana-2-2017.03.01 UFVg9c7TTA-nbsEd2d4oFw 1 1  2699   0 763.4kb  763.4kb 

此外，您可以通過使用

curl -XGET 'localhost:9200/_nodes/_local/stats/fs' 

下查找的磁盤空間信息瞭解可用的磁盤空間

{ 
    "_nodes": { 
    "total": 1, 
    "successful": 1, 
    "failed": 0 
    }, 
    "cluster_name": "elasticsearch", 
    "nodes": { 
    "MfgVaoRQT9iRAZtAvO549Q": { 
     "fs": { 
     "timestamp": 1488466297268, 
     "total": { 
      "total_in_bytes": 29475753984, 
      "free_in_bytes": 18352095232, 
      "available_in_bytes": 18352095232 
     }, 
     } 
    } 
    } 
} 

我測試過這個「FS」鍵for ElasticSearch version 5.2.1

Answer 6

或者您也可以直接查詢磁盤以測量每個目錄的磁盤空間Elasticsearch節點上的r/var/lib/elasticsearch/[environment name]/nodes/0/indices。

$ du -b --max-depth=1 /var/lib/elasticsearch/[environment name]/nodes/0/indices | sort -rn | numfmt --to=iec --suffix=B --padding=5

17GB的/ var/lib中/ elasticsearch/ENV1 /節點/ 0 /索引 3.8GB的/ var/lib中/ elasticsearch/ENV1 /節點/ 0 /索引/索引1 2.1GB /無功/ lib目錄/ elasticsearch/ENV1 /節點/ 0 /索引/索引2 1.2GB ...

Answer 7

要查看ES集羣的整體磁盤使用率/可用空間，您可以使用下面的命令：

curl -XGET 'localhost:9200/_cat/allocation?v&pretty' 

希望這會有所幫助。