我們有一個Percona Xtradb-v2集羣,其中包含3個節點。Percona Xtradb - 長時間停機後無法啓動節點
一切工作和同步,當我們關閉節點2和3,只留下節點1.節點待了一個星期,在此期間,數據庫的大小增長了100GB。
當我們試圖重新啓動2和3,啓動時的初始SST過程中失敗的節點,之後不到一分鐘。我已經嘗試徹底刪除/ var/lib/mysql並重新啓動,但它具有相同的效果。
錯誤日誌似乎顯示的問題與初始SST,這可能是由於需要將被移送的初始啓動的數據量。我們有足夠的磁盤空間,並且文件權限是正確的。 xtrabackup軟件包已安裝並可用(並且以前工作過)。
日誌顯示「沒有這樣的文件或目錄」
木日誌顯示: Dec 15 01:21:51 xm1adb05 mysqld: #011Group state: 67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb:5766440 Dec 15 01:21:51 xm1adb05 mysqld: #011Local state: 00000000-0000-0000-0000-000000000000:-1 Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Note] WSREP: New cluster view: global state: 67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb:5766440, view# 54: Primary, number of nodes: 2, my index: 1, protocol version 3 Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Warning] WSREP: Gap in state sequence. Need state transfer. Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.23.40.115' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '13029' '' ' Dec 15 01:21:51 xm1adb05 mysqld: WSREP_SST: [INFO] Logging all stderr of SST/Innobackupex to syslog (20161215 01:21:51.575) Dec 15 01:21:51 xm1adb05 -wsrep-sst-joiner: Streaming with xbstream Dec 15 01:21:51 xm1adb05 -wsrep-sst-joiner: Using socat as streamer ... Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb): 1 (Operation not permitted) Dec 15 01:21:51 xm1adb05 mysqld: #011 at galera/src/replicator_str.cpp:prepare_for_IST():507. IST will be unavailable. ... Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Note] WSREP: Member 1.0 (xm1adb05) requested state transfer from '*any*'. Selected 0.0 (xm1adb04)(SYNCED) as donor. Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 5766440) Dec 15 01:21:51 xm1adb05 mysqld: 2016-12-15 01:21:51 13029 [Note] WSREP: Requesting state transfer: success, donor: 0 Dec 15 01:21:51 xm1adb05 mysql-systemd: State transfer in progress, setting sleep higher ... Dec 15 01:22:02 xm1adb05 -wsrep-sst-joiner: xtrabackup_checkpoints missing, failed innobackupex/SST on donor Dec 15 01:22:02 xm1adb05 -wsrep-sst-joiner: Cleanup after exit with status:2 Dec 15 01:22:02 xm1adb05 mysqld: 2016-12-15 01:22:02 13029 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.23.40.115' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '13029' '' : 2 (No such file or directory) Dec 15 01:22:02 xm1adb05 mysqld: 2016-12-15 01:22:02 13029 [ERROR] WSREP: Failed to read uuid:seqno from joiner script. Dec 15 01:22:02 xm1adb05 mysqld: 2016-12-15 01:22:02 13029 [ERROR] WSREP: SST script aborted with error 2 (No such file or directory) Dec 15 01:22:02 xm1adb05 mysqld: 2016-12-15 01:22:02 13029 [ERROR] WSREP: SST failed: 2 (No such file or directory) Dec 15 01:22:02 xm1adb05 mysqld: 2016-12-15 01:22:02 13029 [ERROR] Aborting
捐助日誌顯示: Dec 15 01:22:02 xm1adb04 mysqld: 2016-12-15 01:22:02 6531 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup-v2 --role 'donor' --address '10.23.40.115:4444/xtrabackup_sst//1' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' '' --gtid '67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb:5766440' Dec 15 01:22:02 xm1adb04 mysqld: 2016-12-15 01:22:02 6531 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'donor' --address '10.23.40.115:4444/xtrabackup_sst//1' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' '' --gtid '67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb:5766440': 22 (Invalid argument) Dec 15 01:22:03 xm1adb04 mysqld: 2016-12-15 01:22:03 6531 [ERROR] WSREP: Command did not run: wsrep_sst_xtrabackup-v2 --role 'donor' --address '10.23.40.115:4444/xtrabackup_sst//1' --socket '/var/lib/mysql/mysql.sock' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' '' --gtid '67e7e56d-8e95-11e6-a9d2-ce8abe8f95bb:5766440'
類似行動成功啓動另一個輔助節點(多更小)的數據庫,所以看起來大小可能是問題。
誰能給我們如何可以初始化並重新啓動其他節點一些幫助?