Hadoop地圖作業失敗，出現com.datastax.driver.core.exceptions.NoHostAvailableException

Question

我正在嘗試使用hadoop map-reduce在cassandra中存儲的數據運行分析。爲此，我使用可通過maven依賴項cassandra-all獲得的類CqlInputFormat。目前，我們在生產環境中一直使用這種依賴關係的2.0.10版本。另外，我們使用版本爲2.1.1的caassandra-driver-core。

現在，當我向我的jobtracker提交一個簡單的map-reduce作業時，我的所有mapper任務都會失敗，並顯示以下異常。

這裏需要注意的另一件重要的事情是，如果我使用cassandra-all 2.0.6中的CqlPagingInputFormat，那麼一切正常。但是這個輸入格式在後面的版本中已經被刪除了。

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections))) 
    at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:65) 
    at com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:258) 
    at com.datastax.driver.core.Cluster.connect(Cluster.java:267) 
    at org.apache.cassandra.hadoop.cql3.CqlRecordReader.initialize(CqlRecordReader.java:137) 
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
    at org.apache.hadoop.mapred.Child.main(Child.java:249) 
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: 10.40.242.174:9042 (com.datastax.driver.core.exceptions.DriverException: Timeout while trying to acquire available connection (you may want to increase the driver number of per-host connections))) 
    at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103) 
    at com.datastax.driver.core.SessionManager.execute(SessionManager.java:446) 
    at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:482) 
    at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:88) 
    at com.datastax.driver.core.AbstractSession.executeAsync(AbstractSession.java:60) 
    at com.datastax.driver.core.Cluster.connect(Cluster.java:260) 
    ... 9 more 

任何輸入是最受歡迎的。

Answer 1

我在使用cassandra-all-2.0.10時遇到了完全相同的問題，但問題已通過使用版本cassandra-all-2.0.12得到解決。

我想修正這一承諾： Potentially use non-local hosts in CqlConfigHelper