Hadoop計數器文檔？

Question

我MapReduce作業結束後，我收到了一大堆的Counter信息：

File System Counters 
       FILE: Number of bytes read=4386096368 
       FILE: Number of bytes written=8805370803 
       FILE: Number of read operations=0 
       FILE: Number of large read operations=0 
       FILE: Number of write operations=0 
       HDFS: Number of bytes read=54583718086 
       HDFS: Number of bytes written=4382090874 
       HDFS: Number of read operations=1479 
       HDFS: Number of large read operations=0 
       HDFS: Number of write operations=2 
     Job Counters 
       Launched map tasks=369 
       Launched reduce tasks=1 
       Data-local map tasks=369 
       Total time spent by all maps in occupied slots (ms)=34288552 
       Total time spent by all reduces in occupied slots (ms)=232084 
       Total time spent by all map tasks (ms)=8572138 
       Total time spent by all reduce tasks (ms)=58021 
       Total vcore-seconds taken by all map tasks=8572138 
       Total vcore-seconds taken by all reduce tasks=58021 
       Total megabyte-seconds taken by all map tasks=35111477248 
       Total megabyte-seconds taken by all reduce tasks=237654016 
     Map-Reduce Framework 
       Map input records=14753874 
       Map output records=666776 
       Map output bytes=4383426830 
       Map output materialized bytes=4386098552 
       Input split bytes=47970 
       Combine input records=0 
       Combine output records=0 
       Reduce input groups=1 
       Reduce shuffle bytes=4386098552 
       Reduce input records=666776 
       Reduce output records=666776 
       Spilled Records=1333552 
       Shuffled Maps =369 
       Failed Shuffles=0 
       Merged Map outputs=369 
       GC time elapsed (ms)=1121584 
       CPU time spent (ms)=23707900 
       Physical memory (bytes) snapshot=152915259392 
       Virtual memory (bytes) snapshot=2370755190784 
       Total committed heap usage (bytes)=126644912128 
     Shuffle Errors 
       BAD_ID=0 
       CONNECTION=0 
       IO_ERROR=0 
       WRONG_LENGTH=0 
       WRONG_MAP=0 
       WRONG_REDUCE=0 
     File Input Format Counters 
       Bytes Read=49449743227 
     File Output Format Counters 
       Bytes Written=4382090874 

我在哪裏可以找到什麼這些字段的意思解釋？其中一些非常明顯（Number of bytes read），但其他一些更模糊（Total time spent by all maps in occupied slots vs Total time spent by all map tasks）。

我發現了一個list of all the default counters，但我似乎無法找到他們的解釋或描述。

我很驚訝，我似乎無法輕鬆找到有關此輸出的文檔。任何人都可以提供鏈接或解釋？

Answer 1

Hadoop: The Definitive Guide的第8章（完整的PDF鏈接華盛頓州立大學），提供了計數器的詳細信息，與MapReduce。這從225頁開始，並在表8-1中列出。此資源的更新版本（第4版）可在Safari Books Online獲得（您需要先登錄）。