2017-05-23 68 views
-1

我正在使用Cygwin啓動namenode和jobtracker。 Namenode運行成功,但是當我試圖打開作業跟蹤器,失敗。hadoop中的Jobtracker未運行

我拿出了日誌PFB。 可否請你幫助我理解通過日誌,哪裏是失敗。

2017-05-23 11:19:05,862 INFO org.apache.hadoop.mapred.JobTracker: STARTUP_MSG: 
/************************************************************ 
STARTUP_MSG: Starting JobTracker 
STARTUP_MSG: host = ADMINIB-QKB8E8D/9.78.115.117 
STARTUP_MSG: args = [] 
STARTUP_MSG: version = 1.2.1 
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013 
STARTUP_MSG: java = 1.7.0_80 
************************************************************/ 
2017-05-23 11:19:05,980 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 
2017-05-23 11:19:05,993 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 
2017-05-23 11:19:05,994 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 
2017-05-23 11:19:05,994 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics system started 
2017-05-23 11:19:06,113 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source QueueMetrics,q=default registered. 
2017-05-23 11:19:06,395 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 
2017-05-23 11:19:06,396 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 
2017-05-23 11:19:06,397 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 
2017-05-23 11:19:06,399 INFO org.apache.hadoop.mapred.JobTracker: Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) 
2017-05-23 11:19:06,400 INFO org.apache.hadoop.util.HostsFileReader: Refreshing hosts (include/exclude) list 
2017-05-23 11:19:06,401 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 min(s) 
2017-05-23 11:19:06,401 INFO org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens 
2017-05-23 11:19:06,412 INFO org.apache.hadoop.mapred.JobTracker: Starting jobtracker with owner as IBM 
2017-05-23 11:19:06,441 INFO org.apache.hadoop.ipc.Server: Starting SocketReader 
2017-05-23 11:19:06,444 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcDetailedActivityForPort50001 registered. 
2017-05-23 11:19:06,445 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source RpcActivityForPort50001 registered. 
2017-05-23 11:19:06,526 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 
2017-05-23 11:19:06,724 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 
2017-05-23 11:19:06,727 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 50030 
2017-05-23 11:19:06,728 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 50030 webServer.getConnectors()[0].getLocalPort() returned 50030 
2017-05-23 11:19:06,728 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50030 
2017-05-23 11:19:06,728 INFO org.mortbay.log: jetty-6.1.26 
2017-05-23 11:19:07,061 INFO org.mortbay.log: Started [email protected]:50030 
2017-05-23 11:19:07,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 
2017-05-23 11:19:07,064 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source JobTrackerMetrics registered. 
2017-05-23 11:19:07,074 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 50001 
2017-05-23 11:19:07,074 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030 
2017-05-23 11:19:07,102 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 
2017-05-23 11:19:07,102 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50001: starting 
2017-05-23 11:19:07,103 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 50001: starting 
2017-05-23 11:19:07,103 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 50001: starting 
2017-05-23 11:19:07,104 INFO org.apache.hadoop.mapred.JobTracker: Setting safe mode to true. Requested by : IBM 
2017-05-23 11:19:07,190 INFO org.apache.hadoop.mapred.JobTracker: Setting safe mode to false. Requested by : IBM 
2017-05-23 11:19:07,392 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory 
2017-05-23 11:19:07,461 INFO org.apache.hadoop.mapred.JobHistory: Creating DONE folder at file:/C:/cygwin64/home/hadoop-1.2.1/logs/history/done 
2017-05-23 11:19:07,464 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
2017-05-23 11:19:07,465 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:IBM cause:java.io.IOException: Failed to set permissions of path: C:\cygwin64\home\hadoop-1.2.1\logs\history\done to 0755 
2017-05-23 11:19:07,479 FATAL org.apache.hadoop.mapred.JobTracker: java.io.IOException: Failed to set permissions of path: C:\cygwin64\home\hadoop-1.2.1\logs\history\done to 0755 
    at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691) 
    at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:672) 
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514) 
    at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349) 
    at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193) 
    at org.apache.hadoop.mapred.JobHistory.initDone(JobHistory.java:564) 
    at org.apache.hadoop.mapred.JobHistory.initDone(JobHistory.java:540) 
    at org.apache.hadoop.mapred.JobTracker$4.run(JobTracker.java:1823) 
    at org.apache.hadoop.mapred.JobTracker$4.run(JobTracker.java:1821) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
    at org.apache.hadoop.mapred.JobTracker.initialize(JobTracker.java:1820) 
    at org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2147) 
    at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4711) 

2017-05-23 11:19:07,480 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG: 
/************************************************************ 
SHUTDOWN_MSG: Shutting down JobTracker at ADMINIB-QKB8E8D/9.78.115.117 
************************************************************/ 

回答

1

它失敗了,因爲hadoop正在嘗試寫入的日誌目錄沒有權限。只需更改文件夾的文件權限(即提供寫訪問權限),它應該可以工作。