使用遠程java客戶端的Sqoop導入

Question

我正在爲從sqoop（1.4.5）從mysql導入到HDFS（hadoop-1.2.1）編寫遠程java客戶端。

這是我的代碼：

Configuration config = new Configuration(); 
config.set("fs.default.name","hdfs://x.y.z.w:8020"); 
config.set("mapred.job.tracker", "x.y.z.w:9101"); 
SqoopOptions options = new SqoopOptions(config); 
options.setConnectString("jdbc:mysql://x.y.z.w:3306/testdb"); 
options.setUsername("user"); 
options.setPassword("password"); 
options.setTableName("test"); 
options.setTargetDir("/testOut"); 
options.setNumMappers(1); 
int ret = new ImportTool().run(options);` 

我收到以下錯誤：

ERROR security.UserGroupInformation: PriviledgedActionException as:xxx cause:java.net.UnknownHostException: unknown host: xxxx ERROR tool.ImportTool: Encountered IOException running import job: java.net.UnknownHostException: unknown host: xxxx at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:236) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1239) at org.apache.hadoop.ipc.Client.call(Client.java:1093) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at com.sun.proxy.$Proxy2.getProtocolVersion(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62) at com.sun.proxy.$Proxy2.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.checkVersion(RPC.java:422) at org.apache.hadoop.hdfs.DFSClient.createNamenode(DFSClient.java:183) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:281) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:245) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1446) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1464) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:263) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187) at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:103) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936) at org.apache.hadoop.mapreduce.Job.submit(Job.java:550) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580) at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:119) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:179) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:413) at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:97) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:381) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:454)

Hadoop日誌顯示以下信息：

NameNode的日誌：

INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8020: readAndProcess threw exception java.io.IOException: Connection reset by peer. Co unt of bytes read: 0 java.io.IOException: Connection reset by peer

JobTracker的日誌：

INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9101: readAndProcess threw exception java.io.IOException: Connection reset by peer. Co unt of bytes read: 0 java.io.IOException: Connection reset by peer

任何人可以在這方面幫助？

Answer 1

問題可能與主機名未正確解析，請檢查您的主機名。 另請檢查您是否已授予您在sqoop中使用的特定主機名和用戶從數據庫導入的權限。