2017-09-07 122 views
0

當我運行查詢以獲取它在5分鐘內返回結果的51M記錄時,我遇到了一個與memsql集羣有關的問題,但是它過去需要15分鐘以上的時間數據插入與讀取平行。Memsql花費大量的時間來執行查詢

我測量了磁盤io並且沒問題,磁盤是hdd磁盤。

有到memsql和CPU沒有其他的連接也與64的核心機採用15%

下面是我varaiables

Variable_name        | Value                  | 
+----------------------------------------------+------------------------------------------------------------------------------+ 
| aggregator_failure_detection     | ON                   | 
| auto_replicate        | OFF                   | 
| autocommit         | ON                   | 
| basedir          | /data/master-3306               | 
| character_set_client       | utf8                   | 
| character_set_connection      | utf8                   | 
| character_set_filesystem      | binary                  | 
| character_set_results      | utf8                   | 
| character_set_server       | utf8                   | 
| character_sets_dir       | /data/master-3306/share/charsets/           | 
| collation_connection       | utf8_general_ci                | 
| collation_database       | utf8_general_ci                | 
| collation_server        | utf8_general_ci                | 
| columnar_segment_rows      | 102400                  | 
| columnstore_window_size      | 2147483648                 | 
| compile_only         | OFF                   | 
| connect_timeout        | 10                   | 
| core_file         | ON                   | 
| core_file_mode        | PARTIAL                  | 
| critical_diagnostics       | ON                   | 
| datadir          | /data/master-3306/data              | 
| default_partitions_per_leaf     | 16                   | 
| enable_experimental_metrics     | OFF                   | 
| error_count         | 0                   | 
| explain_expression_limit      | 500                   | 
| external_user        |                    | 
| flush_before_replicate      | OFF                   | 
| general_log         | OFF                   | 
| geo_sphere_radius       | 6367444.657120                | 
| hostname          | ****             | 
| identity          | 0                   | 
| kerberos_server_keytab      |                    | 
| lc_messages         | en_US                  | 
| lc_messages_dir        | /data/master-3306/share              | 
| leaf_failure_detection      | ON                   | 
| load_data_max_buffer_size     | 1073741823                 | 
| load_data_read_size       | 8192                   | 
| load_data_write_size       | 8192                   | 
| lock_wait_timeout       | 60                   | 
| master_aggregator       | self                   | 
| max_allowed_packet       | 104857600                 | 
| max_connection_threads      | 192                   | 
| max_connections        | 100000                  | 
| max_pooled_connections      | 4096                   | 
| max_prefetch_threads       | 1                   | 
| max_prepared_stmt_count      | 16382                  | 
| max_user_connections       | 0                   | 
| maximum_memory        | 506602                  | 
| maximum_table_memory       | 455941                  | 
| memsql_id         | **          | 
| memsql_version        | 5.7.2                  | 
| memsql_version_date       | Thu Jan 26 12:34:22 2017 -0800            | 
| memsql_version_hash       | 03e5e3581e96d65caa30756f191323437a3840f0          | 
| minimal_disk_space       | 100                   | 
| multi_insert_tuple_count      | 20000                  | 
| net_buffer_length       | 102400                  | 
| net_read_timeout        | 3600                   | 
| net_write_timeout       | 3600                   | 
| pid_file          | /data/master-3306/data/memsqld.pid           | 
| pipelines_batches_metadata_to_keep   | 1000                   | 
| pipelines_extractor_debug_logging   | OFF                   | 
| pipelines_kafka_version      | 0.8.2.2                  | 
| pipelines_max_errors_per_partition   | 1000                   | 
| pipelines_max_offsets_per_batch_partition | 1000000                  | 
| pipelines_max_retries_per_batch_partition | 4                   | 
| pipelines_stderr_bufsize      | 65535                  | 
| pipelines_stop_on_error      | ON                   | 
| plan_expiration_minutes      | 720                   | 
| port           | 3306                   | 
| protocol_version        | 10                   | 
| proxy_user         |                    | 
| query_parallelism       | 0                   | 
| redundancy_level        | 1                   | 
| reported_hostname       |                    | 
| secure_file_priv        |                    | 
| show_query_parameters      | ON                   | 
| skip_name_resolve       | AUTO                   | 
| snapshot_trigger_size      | 268435456                 | 
| snapshots_to_keep       | 2                   | 
| socket          | /data/master-3306/data/memsql.sock           | 
| sql_quote_show_create      | ON                   | 
| ssl_ca          |                    | 
| ssl_capath         |                    | 
| ssl_cert          |                    | 
| ssl_cipher         |                    | 
| ssl_key          |                    | 
| sync_slave_timeout       | 20000                  | 
| system_time_zone        | UTC                   | 
| thread_cache_size       | 0                   | 
| thread_handling        | one-thread-per-connection             | 
| thread_stack         | 1048576                  | 
| time_zone         | SYSTEM                  | 
| timestamp         | 1504799067.127069               | 
| tls_version         | TLSv1,TLSv1.1,TLSv1.2              | 
| tmpdir          | .                   | 
| transaction_buffer       | 67108864                  | 
| tx_isolation         | READ-COMMITTED                | 
| use_join_bucket_bit_vector     | ON                   | 
| use_vectorized_join       | ON                   | 
| version          | 5.5.8                  | 
| version_comment        | MemSQL source distribution (compatible; MySQL Enterprise & MySQL Commercial) | 
| version_compile_machine      | x86_64                  | 
| version_compile_os       | Linux                  | 
| warn_level         | WARNINGS                  | 
| warning_count        | 0                   | 
| workload_management       | ON                   | 
| workload_management_expected_aggregators  | 1                   | 
| workload_management_max_connections_per_leaf | 1024                   | 
| workload_management_max_queue_depth   | 100                   | 
| workload_management_max_threads_per_leaf  | 8192                   | 
| workload_management_queue_time_warning_ratio | 0.500000                  | 
| workload_management_queue_timeout   | 3600   

回答

0

的幾點思考:

+0

感謝您的回覆。 網絡IO不能作爲我的主人,並且兩個葉子都在同一臺機器上。 我做了查詢分析和正如我所說的查詢用於執行並在不到一分鐘內給我51M記錄,但查詢只需要更多的時間,當讀取並行。 – user3536104

+0

如果查詢全部使用CPU,則可能會出現這種情況。檢查並行運行查詢時CPU的使用情況。 –

0

傑克,我做的工作負荷分析和找出lock_time_ms是相當高的,當NETWORK_LOGICAL_SEND_B高

LAST_FINISHED_TIMESTAMP,LOCK_TIME_MS,LOCK_ROW_TIME_MS,ACTIVITY_NAME,NETWORK_LOGICAL_RECV_B,NETWORK_LOGICAL_SEND_B,ACTIVITY_NAME,數據庫名稱,partition_id時,左(Q .query_text,50)

'2017年9月11日10時41分31秒', '39988', '0', 'InsertSelect_AggregatedHourly_temp_7aug_KING__et_al_c44dc7ab56d56280', '0', '538154753', 'InsertSelect_AggregatedHourly_temp_7aug_KING__et_al_c44dc7ab56d56280', '數據倉庫', '7','選擇\ n combinedday AS天,\ n`lineitem'

相關問題