我正在嘗試運行Ruby Hadoop流式處理程序,該程序在「Ruby的權威指南」中給出。爲什麼我的流式傳輸命令在MapReduce基礎程序中失敗?
這是我所使用的命令:
的文件路徑是正確的。當我運行命令,我得到這樣的錯誤:
packageJobJar: [/var/lib/hadoop-0.20/cache/cloudera/hadoop-unjar7505856816337735719/] [] /tmp/streamjob5440613434295562761.jar tmpDir=null
11/03/14 22:58:56 INFO mapred.FileInputFormat: Total input paths to process : 1
11/03/14 22:58:56 INFO streaming.StreamJob: getLocalDirs(): [/var/lib/hadoop-0.20/cache/cloudera/mapred/local]
11/03/14 22:58:56 INFO streaming.StreamJob: Running job: job_201103140235_0015
11/03/14 22:58:56 INFO streaming.StreamJob: To kill this job, run:
11/03/14 22:58:56 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=localhost:8021 -kill job_201103140235_0015
11/03/14 22:58:56 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp...
11/03/14 22:58:57 INFO streaming.StreamJob: map 0% reduce 0%
11/03/14 22:59:54 INFO streaming.StreamJob: map 100% reduce 100%
11/03/14 22:59:54 INFO streaming.StreamJob: To kill this job, run:
11/03/14 22:59:54 INFO streaming.StreamJob: /usr/lib/hadoop-0.20/bin/hadoop job -Dmapred.job.tracker=localhost:8021 -kill job_201103140235_0015
11/03/14 22:59:54 INFO streaming.StreamJob: Tracking URL: http://localhost:50030/jobdetails.jsp...
11/03/14 22:59:54 ERROR streaming.StreamJob: Job not Successful!
11/03/14 22:59:54 INFO streaming.StreamJob: killJob...
Streaming Command Failed!
我看着地圖上的日誌文件,這就是錯誤:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:379)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:317)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
at org.apache.hadoop.mapred.Child.main(Child.java:211)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:230)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/home/cloudera/projects/max_temp/map.rb": java.io.IOException: error=13, Permission denied
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:214)
... 23 more
Caused by: java.io.IOException: java.io.IOException: error=13, Permission denied
at java.lang.UNIXProcess.(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 24 more