我經過大約370次備份(大約15天)後每小時有一次Elasticsearch備份,我的備份存儲庫大於15G!但總指數的大小隻有500M左右! Elasticsearch是增量備份,但15G VS 500M,差別非常巨大!我想知道索引和備份存儲庫之間是否有這麼大的差異是正常的?備份存儲庫大小遠大於索引大小
是否由我經常備份(每小時)造成的?我使用羣集1中的每小時備份和羣集2中的每小時還原來實時保持兩個ES羣集數據相同。
我使用Elasticsearch備份API來備份
設置回購: 捲曲-XPUT 「http://IP:9200/_snapshot/backup」 -d 「{\」 類型\ 「:\」 FS \」,\ 「設置\」 :{\ 「壓縮\」:真,\ 「位置\」:\ 「備份\」}}」
備份: CURTIME =
date +"%Y-%m-%d %H:%M:%S"
BKTIME = $ {CURTIME // [ - :] /} curl -XPUT「http://IP:9200/snapshot/backup/snapshot $ BKTIME?wait_for_completion = true」 個
我Elasticsearch設置:2個節點,12碎片/節點,2個索引,備份快照存儲到NAS
在Elasticsearch數據目錄的FS型,指數大小:
node1的索引尺寸: [root @ esnode1 indices] $ du -sh
307M。
節點2指數大小 [根@ esnode2指數] $杜-SH
238M。
[根@ esnode1指數] $杜-LH
8.0K ./index1/10/translog 8.0K ./index1/10/_state 2.9M ./index1/10/index 2.9M ./index1/10 12K ./index1/5/translog 8.0K ./index1/5/_state 1.5M ./index1/5/index 1.5M ./index1/5 8.0K ./index1/4 /超越對 8.0K ./index1/4/_state 2.9M ./index1/4/index 2.9M ./index1/4 8.0K ./index1/_state 8.0K ./index1/7/translog 8.0K ./index1/7/_state 2.9M ./index1/7/index 2.9M ./index1/7 8.0K ./index1/1/translog 8.0K ./index1/1/_state 2.9中號./index1/1/index 2.9M ./index1/1 8.0K ./index1/2/translog 8.0K ./index1/2/_state 2.9M ./index1/2/index 2.9M。 /索引1/2 8.0K ./index1/6/translog 8.0K ./index1/6/_state 3.0M ./index1/6/index 3.0M ./index1/6 8.0K ./index1/0/translog 8.0K ./index1/0/_state 1.5M ./index1/0/index 1.5M ./index1/0 8.0K ./index1/8/translog 8.0K。/索引1/8/_STATE 1.5M ./index1/8/index 1.5M ./index1/8 8.0K ./index1/11/translog 8.0K ./index1/11/_state 2.9M ./index1/11 /索引 2.9M ./index1/11 12K ./index1/9/translog 8.0K ./index1/9/_state 3.0M ./index1/9/index 3.0M ./index1/9 8.0K ./index1/3/translog 8.0K ./index1/3/_state 3.0M ./index1/3/index 3.0M ./index1/3 31M ./index1 16K ./index2/10/ translog 8.0K ./index2/10/_state 16M ./index2/10/index 16M ./index2/10 36K ./index2/5/translog 8.0K ./index2/5/_state 43M ./index2/5/index 43M ./index2/5 20K ./index2/4/超越對 8.0K ./index2/4/_state 17M ./index2/4/index 18M ./index2/4 8.0K ./index2/_state 40K ./index2/7/translog 8.0K ./index2/7/_STATE 32M ./index2/7/index 32M ./index2/7 68K ./index2/1/translog 8.0K ./index2/1/_state 21M ./index2/1/index 21M ./index2/1 64K ./index2/2/translog 8.0K ./index2/2/_state 19M ./index2/2/index 19M ./index2/2 116K ./index2/6/translog 8.0K ./index2/6/_state 22M ./index2/6 /索引 22M ./index2/6 24K ./index2/0/translog 8.0K ./index2/0/_state 17M ./index2/0/index 17M ./index2/0 128K ./索引2/8 /超越對 8.0K ./index2/8/_state 34M ./index2/8/index 34M ./index2/8 72K ./index2/11/translog 8.0K ./index2/11/_state 20M ./index2/11/index 20M ./index2/11 88K ./index2/9/translog 8.0K ./index2/9/_state 22M ./index2/9/index 22M ./index2/9 76K ./index2/3/translog 8.0K ./index2/3/_state 16M ./index2/3/index 16M ./index2/3 277M ./index2 307M。
在備份庫,大小: [根@ esnode1備份] $杜-lh
114M ./backup/indices/index1/0
112M ./backup/indices/index1/5
114M ./backup/indices/index1/11
114M ./backup/indices/index1/10
111M ./backup/indices/index1/8
116M ./backup/indices/index1/4
120M ./backup/indices/index1/9
118M。/備份/索引/索引1/3
114M ./backup/indices/index1/2
115M ./backup/indices/index1/7
115M ./backup/indices/index1/1
112M ./backup/indices/index1/6
1.4G ./backup/indices/index1
747M ./backup/indices/index2/0
1.6G ./backup/indices/index2/5
887M ./backup/indices/index2/11
743M ./backup/indices/index2/10
2.1G ./備份/索引/索引2/8
801M ./backup/indices/index2/4
1.3G ./backup/indices/index2/9
8.78 ./backup/ind冰/索引2/3
951M ./backup/indices/index2/2
1.2G ./backup/indices/index2/7
953M ./backup/indices/index2/1
943M ./backup/indices/index2/6
13G ./backup/indices/index2
15G ./backup/indices
15G ./backup
1.1M ./backuplogs
15G。
====== https://www.elastic.co/blog/introducing-snapshot-restore 備份和恢復操作是增量,這意味着只有自上次快照更改的文件將被複制到存儲庫或恢復成一個索引。 增量快照允許按照需要頻繁執行快照操作,而無需太多的磁盤空間開銷。用戶現在可以輕鬆地在升級之前創建快照,或者在羣集中進行有風險的更改,並在出現問題時快速回滾到以前的索引狀態。快照/恢復機制還可用於同步「熱」集羣和不同地理位置的遠程「冷」備份集羣之間的數據,以實現快速災難恢復。
從上面,我的情況是一個真正的問題,任何人都可以幫助我嗎?提前致謝 !
你如何觸發你的小時備份? – Val
在/ var/spool/cron/root中添加一個linux cron作業 0 1 * * * /opt/es/ESBackup.sh >>/backup/log/backup _ $(date + \%Y - \%m- \ %d).log – caiyufei
您的shell腳本是使用快照/還原還是隻是複製數據文件夾?在後一種情況下,請知道,如果不先複製索引,則不應複製數據文件夾,否則會冒着複製不一致數據的風險。 – Val