TeraSort 之 Metrics

基本 CountersFile System Counters FILE:Number of bytes read460639246FILE:Number of bytes written461201571FILE:Number of read operations0FILE:Number of large read operations0FILE:Number of write operations0HDFS:Number of bytes read1048576122HDFS:Number of bytes written1048576000HDFS:Number of read operations8HDFS:Number of large read operations0HDFS:Number of write operations2HDFS:Number of bytes read erasure-coded0Job Counters Launched map tasks1Launched reduce tasks1Data-local map tasks1Total time spent by all maps in occupiedslots(ms)27102Total time spent by all reduces in occupiedslots(ms)7534Total time spent by all maptasks(ms)13551Total time spent by all reducetasks(ms)7534Total vcore-milliseconds taken by all map tasks13551Total vcore-milliseconds taken by all reduce tasks7534Total megabyte-milliseconds taken by all map tasks55504896Total megabyte-milliseconds taken by all reduce tasks15429632Map-Reduce Framework Map input records10485760Map output records10485760Map output bytes1048576000Map output materialized bytes460639246Input split bytes122Combine input records0Combine output records0Reduce input groups10485760Reduce shuffle bytes460639246Reduce input records10485760Reduce output records10485760Spilled Records20971520Shuffled Maps1Failed Shuffles0Merged Map outputs1GC timeelapsed(ms)91CPU timespent(ms)21280Physicalmemory(bytes)snapshot2016575488Virtualmemory(bytes)snapshot6662488064Total committed heapusage(bytes)3006529536Peak Map Physicalmemory(bytes)1909489664Peak Map Virtualmemory(bytes)4690046976Peak Reduce Physicalmemory(bytes)999567360Peak Reduce Virtualmemory(bytes)3314622464Shuffle Errors BAD_ID0CONNECTION0IO_ERROR0WRONG_LENGTH0WRONG_MAP0WRONG_REDUCE0File Input Format Counters Bytes Read1048576000File Output Format Counters Bytes Written1048576000具体metrics 研究Total time spent by all maps in occupiedslots(ms)27102Total time spent by all reduces in occupiedslots(ms)7534Total time spent by all maptasks(ms)13551Total time spent by all reducetasks(ms)7534