1. 首頁
  2. 問答
  3. NoHostAvailableException，而批量加載數據到卡桑德拉

NoHostAvailableException，而批量加載數據到卡桑德拉

2014-03-25 61 views
1

我試圖加載1百萬LUBM數據集到卡桑德拉。但是，在大約200,000次插入後，我不斷收到此異常。NoHostAvailableException，而批量加載數據到卡桑德拉

異常線程 「main」 com.datastax.driver.core.exceptions.NoHostAvailableException：所有主機（S）試了查詢失敗（嘗試：本地主機/ 127.0.0.1（com.datastax.driver.core。 exceptions.DriverException：讀取期間超時））

服務器仍然活着。我仍然可以使用Datastax開發人員中心執行CQL查詢。在此之前，我是越來越WriteTimeoutException，所以我調整在.yaml

這裏是一些選項是我的系統的規格：

CPU-英特爾I5 - 2芯| Memory-4GB | 硬盤 - 500 GB 4個驅動器。

我有一個從中讀取數據並插入Cassandra的OWL文件。我有混合的工作量。有時，根據條件，我需要從行中檢索數據並將新數據附加到它並將其寫入同一行。這不能成爲這個問題的原因，因爲在200k插入期間這種情況一定發生了很多次。

我正在使用外部硬盤存儲提交日誌。 concurrent_reads設置爲16. concurrent_writes設置爲32. read_timeout已設置爲10000，因此爲write_timeout。 MAX_HEAP_SIZE = 「2G」 HEAP_NEWSIZE = 「200M」

只要我拿到帽子例外，我看到這篇日誌這種模式

INFO [MemoryMeter:1] 2014-03-25 16:28:56,498 Memtable.java (line 449) CFS(Keyspace='rdf', ColumnFamily='lubm1m.lubm1m_teacherof_idx') liveRatio is 15.733535462912831 (just-counted was 15.733535462912831). calculation took 91ms for 665 cells 
INFO [MemoryMeter:1] 2014-03-25 16:30:01,560 Memtable.java (line 449) CFS(Keyspace='system', ColumnFamily='sstable_activity') liveRatio is 15.896188158961882 (just-counted was 15.896188158961882). calculation took 37ms for 411 cells 
INFO [ScheduledTasks:1] 2014-03-25 16:33:22,252 GCInspector.java (line 116) GC for ParNew: 15581 ms for 1 collections, 395151608 used; max is 1038876672 
INFO [ScheduledTasks:1] 2014-03-25 16:33:24,894 StatusLogger.java (line 55) Pool Name     Active Pending  Completed Blocked All Time Blocked 
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,956 StatusLogger.java (line 70) ReadStage       0   0   343094   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,957 StatusLogger.java (line 70) RequestResponseStage    0   0    1   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,996 StatusLogger.java (line 70) ReadRepairStage     0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,004 StatusLogger.java (line 70) MutationStage      0   0   601747   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,024 StatusLogger.java (line 70) ReplicateOnWriteStage    0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,031 StatusLogger.java (line 70) GossipStage      0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,066 StatusLogger.java (line 70) AntiEntropyStage     0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,067 StatusLogger.java (line 70) MigrationStage     0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,098 StatusLogger.java (line 70) MemtablePostFlusher    0   0    52   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,099 StatusLogger.java (line 70) MemoryMeter      0   0   139   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,100 StatusLogger.java (line 70) FlushWriter      0   0    31   0    16 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,101 StatusLogger.java (line 70) MiscStage       0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,130 StatusLogger.java (line 70) PendingRangeCalculator   0   0    1   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,133 StatusLogger.java (line 70) commitlog_archiver    0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,134 StatusLogger.java (line 70) InternalResponseStage    0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,142 StatusLogger.java (line 70) HintedHandoff      0   0    0   0     0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,223 StatusLogger.java (line 79) CompactionManager     0   0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,232 StatusLogger.java (line 81) Commitlog      n/a   0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 93) MessagingService    n/a  0/0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 103) Cache Type      Size     Capacity    KeysToSave 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,287 StatusLogger.java (line 105) KeyCache      17212     51380224      all 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,335 StatusLogger.java (line 111) RowCache       0      0      all 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 118) ColumnFamily    Memtable ops,data 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 121) system.schema_triggers     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.local        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.peers        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.batchlog       0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.NodeIdInfo       0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.compactions_in_progress     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.schema_keyspaces    4,1130 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.compaction_history   38,11220 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.paxos        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.schema_columns    3361,891818 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.schema_columnfamilies   1426,394018 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.IndexInfo       0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,342 StatusLogger.java (line 121) system.range_xfers      0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.peer_events      0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.hints        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) system.sstable_activity   1542,196000 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_NUM     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO_REVERSE     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_DT     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P_REVERSE     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.SPOC      0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.POSC      0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.counter     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,349 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_O_PROPS     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_D_PROPS     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.PREFIX_TO_NS     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_DT     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_NUM     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_CLASSES     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.OSPC      0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.animal        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.lubm20        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.bsbm50        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.tim         0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.bsbm250        0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.lubm1m     152700,22140112 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) rdf.lubm3         0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) system_traces.sessions     0,0 
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,359 StatusLogger.java (line 121) system_traces.events      0,0

可能是什麼問題呢？

來源

2014-03-25 SanjayV

回答

1

使用SocketOptions增加套接字超時。您也可以嘗試在datastax客戶端和服務器端進行配置。另外，看看降低一致性是否可以提供幫助？

-Vivek

來源

2014-03-26 08:00:01

+0

謝謝。我解決了它。我遵循這個：http://www.opensourceconnections.com/2013/08/31/building-the-perfect-cassandra-test-environment/ 而且我增加了concurrent_reads到64（硬件特定）。沒有更多的崩潰和WriteTimeoutException。 – SanjayV

相關問題

 相關問題