MYSQL巨大的SQL文件插入| MyISAM的速度突然減慢插入（奇怪的問題）

Question

我面臨很奇怪的問題，我在這裏問question關於加快在MYSql中插入，特別是有關插入大小超過GB的巨大SQL文件。他們建議我使用MyISAM引擎。我做了以下內容：

• ALTER TABLE revision ENGINE=MyISAM;
• 使用ALTER TABLE .. DISABLE KEYS。
• （僅MyISAM）將bulk_insert_buffer_size設置爲500M。
• （僅MyISAM）設置unique_checks = 0。 未檢查。
• SET autocommit=0; ... SQL import statements ... COMMIT;
• SET foreign_key_checks=0;

它加快的過程中，以前花了2個小時，讓我印象深刻5分鐘。但現在，當我試圖與其他表同樣的事情，那麼就沒有加速和再次需要幾個小時：（...

當最初我得到了成功比我CPU使用率在90％左右和插入只用了5分，但是現在按照同樣的程序我CPU佔用率在5％左右在最大這說明什麼問題..

我也驗證了我的表引擎是MyISAM的通過如下：

SHOW TABLE STATUS WHERE Name = 'xxx'; 

注意：我正在使用維基百科數據庫模式。我在維基百科數據集的Categorylinks表中獲得成功。我沒有成功（非常慢的插入速度）在revision,page和text表。

請幫我解決這個奇怪的問題。

Answer 1

我還沒有找到理想的原因，背後的問題。但是當我深入研究時，我發現它的表現通常與關係豐富的表相關。雖然我已經禁用鍵..但我仍然認爲beileve以下設置是最適合大文件的插入爲下面的設置給出多數表最佳性能：關於把住perofmance調音

- ALTER TABLE revision ENGINE=MyISAM; 
- Use ALTER TABLE .. DISABLE KEYS . 
- (MyISAM only) Set bulk_insert_buffer_size to 500M. 
- (MyISAM only) Set unique_checks = 0 . not checked. 
- SET autocommit=0; ... SQL import 
- statements ... COMMIT; 
- SET foreign_key_checks=0; 

其它的改進：請看到完整的my.cnf文件..

# 
# The MySQL database server configuration file. 
# 
# You can copy this to one of: 
# - "/etc/mysql/my.cnf" to set global options, 
# - "~/.my.cnf" to set user-specific options. 
# 
# One can use all long options that the program supports. 
# Run program with --help to get a list of available options and with 
# --print-defaults to see which it would actually understand and use. 
# 
# For explanations see 
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html 

# This will be passed to all mysql clients 
# It has been reported that passwords should be enclosed with ticks/quotes 
# escpecially if they contain "#" chars... 
# Remember to edit /etc/mysql/debian.cnf when changing the socket location. 
[client] 
port  = 3306 
socket  = /var/run/mysqld/mysqld.sock 

# Here is entries for some specific programs 
# The following values assume you have at least 32M ram 

# This was formally known as [safe_mysqld]. Both versions are currently parsed. 
[mysqld_safe] 
socket  = /var/run/mysqld/mysqld.sock 
nice  = 0 

[mysqld] 
# 
# * Basic Settings 
# 
user  = mysql 
pid-file = /var/run/mysqld/mysqld.pid 
socket  = /var/run/mysqld/mysqld.sock 
port  = 3306 
basedir  = /usr 
datadir  = /var/lib/mysql 
tmpdir  = /tmp 
bulk_insert_buffer_size = 1G 
lc-messages-dir = /usr/share/mysql 
skip-grant-tables 
skip-networking 
skip-external-locking 
init_connect='SET autocommit=0' 
innodb_buffer_pool_size = 1G 
innodb_flush_log_at_trx_commit = 0 
max_allowed_packet = 500M 
table_open_cache = 512 
max_connections=100 

query_cache_size=32M 

table_cache=512 

tmp_table_size=64M 

thread_cache_size=8 

myisam_max_sort_file_size=100G 

myisam_sort_buffer_size=256M 

key_buffer_size=512M 
# 
# Instead of skip-networking the default is now to listen only on 
# localhost which is more compatible and is not less secure. 
bind-address  = 127.0.0.1 
# 
# * Fine Tuning 
# 
key_buffer  = 16M 
max_allowed_packet = 16M 
thread_stack  = 192K 
thread_cache_size  = 8 
# This replaces the startup script and checks MyISAM tables if needed 
# the first time they are touched 
myisam-recover   = BACKUP 
#max_connections  = 100 
#table_cache   = 64 
#thread_concurrency  = 10 
# 
# * Query Cache Configuration 
# 
query_cache_limit = 1M 
query_cache_size  = 16M 
# 
# * Logging and Replication 
# 
# Both location gets rotated by the cronjob. 
# Be aware that this log type is a performance killer. 
# As of 5.1 you can enable the log at runtime! 
#general_log_file  = /var/log/mysql/mysql.log 
general_log=0 
# 
# Error log - should be very few entries. 
# 
log_error = /var/log/mysql/error.log 
# 
# Here you can see queries with especially long duration 
#log_slow_queries = /var/log/mysql/mysql-slow.log 
#long_query_time = 2 
#log-queries-not-using-indexes 
# 
# The following can be used as easy to replay backup logs or for replication. 
# note: if you are setting up a replication slave, see README.Debian about 
#  other settings you may need to change. 
#server-id  = 1 
#log_bin   = /var/log/mysql/mysql-bin.log 
expire_logs_days = 10 
max_binlog_size   = 1G 
#binlog_do_db  = include_database_name 
#binlog_ignore_db = include_database_name 
# 
# * InnoDB 
# 
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. 
# Read the manual for more InnoDB related options. There are many! 
# 
# * Security Features 
# 
# Read the manual, too, if you want chroot! 
# chroot = /var/lib/mysql/ 
# 
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca". 
# 
# ssl-ca=/etc/mysql/cacert.pem 
# ssl-cert=/etc/mysql/server-cert.pem 
# ssl-key=/etc/mysql/server-key.pem 



[mysqldump] 
quick 
quote-names 
max_allowed_packet = 16M 

[mysql] 
#no-auto-rehash # faster start of mysql but no tab completition 

[isamchk] 
key_buffer  = 16M 

# 
# * IMPORTANT: Additional settings that can override those from this file! 
# The files must end with '.cnf', otherwise they'll be ignored. 
# 
!includedir /etc/mysql/conf.d/ 

所以總體上這些設置是建議，你會得到相當的性能優勢。