無法使用sqoop導入mysql表到hadoop

Question

我想從mysql表導入表使用sqoop，但我收到錯誤沒有找到數據庫。

雖然運行以下命令：

sqoop import --connect jdbc:mysql://localhost/EOD --username xxx --password xxx --table sqoop --target-dir /data/small/book2 


java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:416) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
    at org.apache.hadoop.mapred.Child.main(Child.java:249) 
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193) 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162) 
    ... 9 more 
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532) 
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) 
    at com.mysql.jdbc.Util.getInstance(Util.java:386) 
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053) 
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4096) 
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4028) 
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:919) 
    at com.mysql.jdbc.MysqlIO.secureAuth411(MysqlIO.java:4612) 
    at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1296) 
    at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2412) 
    at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2445) 
    at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2230) 
    at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:813) 
    at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:532) 
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) 
    at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:399) 
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:334) 
    at java.sql.DriverManager.getConnection(DriverManager.java:620) 
    at java.sql.DriverManager.getConnection(DriverManager.java:200) 
    at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:294) 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187) 
    ... 10 more 

13/11/28 10:55:03 WARN mapred.JobClient: Error reading task outputlocalhost.localdomain 
13/11/28 10:55:03 WARN mapred.JobClient: Error reading task outputlocalhost.localdomain 
13/11/28 10:55:03 INFO mapred.JobClient: Task Id : attempt_201311271328_0011_m_000001_0, Status : FAILED 
java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:167) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:416) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
    at org.apache.hadoop.mapred.Child.main(Child.java:249) 
Caused by: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:193) 
    at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162) 
    ... 9 more 


I am able to get the list of databases using below command. 

    sqoop list-databases --connect "jdbc:mysql://localhost" --username xxx --password xxxx 

BTM

EOD

請任何一個可以幫助。

Answer 1

我相信根異常是這個：

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'EOD' 

即表明您正在連接的數據庫不存在。我會建議驗證你的JDBC URL。如果您可以附加下一次使用參數--verbose生成的整個Sqoop日誌，這也會非常有幫助。

另一個相當可疑的事實是使用「localhost」作爲數據庫主機。考慮到Sqoop將從您的hadoop羣集中可能的所有節點連接到您的數據庫，使用「localhost」通常不是正確的做法。

Answer 2

嘗試：sqoop import --connect jdbc:mysql://localhost:3306/EOD --username xxx --password xxx --table sqoop --target-dir /data/small/book2

，用來說明你的mysql監聽的端口使用本地主機尤其是必要的。