hadoop的java.lang.InterruptedException

2019/7/21 21:36:19 人评论 次浏览 分类:学习教程

知识共享许可协议 版权声明:署名,允许他人基于本文进行创作,且必须基于与原先许可协议相同的许可协议分发本文 (Creative Commons)

 运行hadoop的时候,爆出来java.lang.InterruptedException:

[root@node-1 text]# hadoop jar hadoop-04-1.0-SNAPSHOT.jar
19/07/21 20:41:48 INFO client.RMProxy: Connecting to ResourceManager at node-1/192.168.52.100:8032
19/07/21 20:41:49 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
19/07/21 20:41:50 INFO input.FileInputFormat: Total input paths to process : 1
19/07/21 20:41:50 WARN hdfs.DFSClient: Caught exception 
java.lang.InterruptedException
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1252)
        at java.lang.Thread.join(Thread.java:1326)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:967)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:705)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:894)
19/07/21 20:41:50 WARN hdfs.DFSClient: Caught exception 
java.lang.InterruptedException
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1252)
        at java.lang.Thread.join(Thread.java:1326)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeResponder(DFSOutputStream.java:967)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.endBlock(DFSOutputStream.java:705)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:894)
19/07/21 20:41:50 INFO mapreduce.JobSubmitter: number of splits:1
19/07/21 20:41:51 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1563691024394_0002
19/07/21 20:41:52 INFO impl.YarnClientImpl: Submitted application application_1563691024394_0002
19/07/21 20:41:52 INFO mapreduce.Job: The url to track the job: http://node-1:8088/proxy/application_1563691024394_0002/
19/07/21 20:41:52 INFO mapreduce.Job: Running job: job_1563691024394_0002
19/07/21 20:42:10 INFO mapreduce.Job: Job job_1563691024394_0002 running in uber mode : true
19/07/21 20:42:10 INFO mapreduce.Job:  map 0% reduce 0%
19/07/21 20:42:12 INFO mapreduce.Job:  map 100% reduce 0%
19/07/21 20:42:14 INFO mapreduce.Job:  map 100% reduce 100%
19/07/21 20:42:14 INFO mapreduce.Job: Job job_1563691024394_0002 completed successfully
19/07/21 20:42:15 INFO mapreduce.Job: Counters: 52
        File System Counters
                FILE: Number of bytes read=194
                FILE: Number of bytes written=307
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=424
                HDFS: Number of bytes written=306983
                HDFS: Number of read operations=37
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=14
        Job Counters 
                Launched map tasks=1
                Launched reduce tasks=1
                Other local map tasks=1
                Total time spent by all maps in occupied slots (ms)=0
                Total time spent by all reduces in occupied slots (ms)=0
                TOTAL_LAUNCHED_UBERTASKS=2
                NUM_UBER_SUBMAPS=1
                NUM_UBER_SUBREDUCES=1
                Total time spent by all map tasks (ms)=2567
                Total time spent by all reduce tasks (ms)=1905
                Total vcore-milliseconds taken by all map tasks=0
                Total vcore-milliseconds taken by all reduce tasks=0
                Total megabyte-milliseconds taken by all map tasks=0
                Total megabyte-milliseconds taken by all reduce tasks=0
        Map-Reduce Framework
                Map input records=3
                Map output records=6
                Map output bytes=63
                Map output materialized bytes=81
                Input split bytes=104
                Combine input records=0
                Combine output records=0
                Reduce input groups=4
                Reduce shuffle bytes=81
                Reduce input records=6
                Reduce output records=4
                Spilled Records=12
                Shuffled Maps =1
                Failed Shuffles=0
                Merged Map outputs=1
                GC time elapsed (ms)=242
                CPU time spent (ms)=3000
                Physical memory (bytes) snapshot=545247232
                Virtual memory (bytes) snapshot=6020751360
                Total committed heap usage (bytes)=270802944
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters 
                Bytes Read=41
        File Output Format Counters 
                Bytes Written=33

解释:

原因是:最初,DataStreamer :: closeResponder始终打印有关InterruptedException的警告; 自从HDFS-9812以来,DFSOutputStream :: closeImpl总是强制线程关闭,这会导致InterruptedException。

 

相关资讯

    暂无相关的资讯...

共有访客发表了评论 网友评论

验证码: 看不清楚?
    -->