在Sqoop獲取錯誤，當進口全表中的Cloudera快速入門VM

Question

我收到以下錯誤，當我嘗試通過sqoop自營全表：

sqoop進口的所有桌-m 12 - -connect在此處輸入代碼爲 「jdbc：mysql的：//quickstart.cloudera：3306/retail_db」 --username = retail_dba --password = Cloudera的--warehouse-DIR =/R/Cloudera公司/ sqoop_import

   Please set $ACCUMULO_HOME to the root of your Accumulo installation. 
      17/04/23 15:29:27 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.8.0 
      17/04/23 15:29:27 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 
      17/04/23 15:29:27 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 
      17/04/23 15:29:27 INFO tool.CodeGenTool: Beginning code generation 
      17/04/23 15:29:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1 
      17/04/23 15:29:27 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1 
      17/04/23 15:29:27 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce 
      Note: /tmp/sqoop-cloudera/compile/e8e72a2e112fced2b0f3251b5666473d/categories.java uses or overrides a deprecated API. 
      Note: Recompile with -Xlint:deprecation for details. 
      17/04/23 15:29:30 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/e8e72a2e112fced2b0f3251b5666473d/categories.jar 
      17/04/23 15:29:30 WARN manager.MySQLManager: It looks like you are importing from mysql. 
      17/04/23 15:29:30 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 
      17/04/23 15:29:30 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 
      17/04/23 15:29:30 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 
      17/04/23 15:29:30 INFO mapreduce.ImportJobBase: Beginning import of categories 
      17/04/23 15:29:31 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 
      17/04/23 15:29:32 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 
      17/04/23 15:29:32 INFO client.RMProxy: Connecting to ResourceManager at quickstart.cloudera/192.168.40.134:8032 
      17/04/23 15:29:37 INFO db.DBInputFormat: Using read commited transaction isolation 
      17/04/23 15:29:37 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`category_id`), MAX(`category_id`) FROM `categories` 
      17/04/23 15:29:37 INFO db.IntegerSplitter: Split size: 4; Num splits: 12 from: 1 to: 58 
      17/04/23 15:29:38 INFO mapreduce.JobSubmitter: number of splits:12 
      17/04/23 15:29:38 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1492945339848_0010 
      17/04/23 15:29:39 INFO impl.YarnClientImpl: Submitted application application_1492945339848_0010 
      17/04/23 15:29:39 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1492945339848_0010/ 
      17/04/23 15:29:39 INFO mapreduce.Job: Running job: job_1492945339848_0010 
      17/04/23 15:29:52 INFO mapreduce.Job: Job job_1492945339848_0010 running in uber mode : false 
      17/04/23 15:29:52 INFO mapreduce.Job: map 0% reduce 0% 
      17/04/23 15:29:52 INFO mapreduce.Job: Job job_1492945339848_0010 failed with state FAILED due to: Application application_1492945339848_0010 failed 2 times due to AM Container for appattempt_1492945339848_0010_000002 exited with exitCode: 1 
      For more detailed output, check application tracking page:http://quickstart.cloudera:8088/proxy/application_1492945339848_0010/Then, click on links to logs of each attempt. 
      Diagnostics: Exception from container-launch. 
      Container id: container_1492945339848_0010_02_000001 
      Exit code: 1 
      Stack trace: ExitCodeException exitCode=1: 
       at org.apache.hadoop.util.Shell.runCommand(Shell.java:578) 
       at org.apache.hadoop.util.Shell.run(Shell.java:481) 
       at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763) 
       at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213) 
       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) 
       at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) 
       at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
       at java.lang.Thread.run(Thread.java:745) 


      Container exited with a non-zero exit code 1 
      Failing this attempt. Failing the application. 
      17/04/23 15:29:52 INFO mapreduce.Job: Counters: 0 
      17/04/23 15:29:52 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead 
      17/04/23 15:29:52 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 19.6175 seconds (0 bytes/sec) 
      17/04/23 15:29:52 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 
      17/04/23 15:29:52 INFO mapreduce.ImportJobBase: Retrieved 0 records. 
      17/04/23 15:29:52 ERROR tool.ImportAllTablesTool: Error during import: Import job failed!`enter 

Answer 1

外貌像應用程序大師們一再被殺死的意思是，他們沒有獲得像他們想要的那麼多的記憶。如果你只是嘗試在cloudera虛擬機上使用sqoop，不要使用-m 12，這會嘗試產生你（單個）機器可能無法處理的12個並行映射任務。完全忽略該設置，或者使用--direct代替。另外什麼是--warehousedir=/r/cloudera/sqoop_import？是/r/因爲錯字或應當/user/

試試這個：

sqoop import-all-tables \ 
--connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" \ 
--warehouse-dir=/user/cloudera/sqoop_import 
--username=retail_dba \ 
--direct 
--password=cloudera; 

Answer 2

嘗試加載第一個表，而不是進口的所有桌，也可以嘗試同時採用進口全表來限制你的映射器，12個映射程序正在阻礙VM上的內存。

sqoop import-all-tables \ 
--connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" \ 
--warehouse-dir=/user/cloudera/sqoop_import 
--username=retail_dba \ 
--password=cloudera 
-m 2