在线客服 关于我们 联系我们
博天堂918-918.com
新闻动态
联系我们
销售热线:
Contact Hotline
15899999999
传真:

E-mail:

公司地址:
当前位置: 官网首页 > 新闻动态 > 行业资讯 >
sqoop导入MySQL数据出现Call From hadoop
 
ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: java.net.ConnectException: Call From hadoop/172.16.59.90 to hadoop:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: https://wiki.918.comapache.org/hadoop/ConnectionRefused
 at org.apache.hadoop.mapred.ClientServiceDelegate.invoke
 at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus
 at org.apache.hadoop.mapred.YARNRunner.getJobStatus
 at org.apache.hadoop.mapreduce.Job$1.run
 at org.apache.hadoop.mapreduce.Job$1.run
 at java.security.AccessController.doPrivileged
 at javax.security.auth.Subject.doAs
 at org.apache.hadoop.security.UserGroupInformation.doAs
 at org.apache.hadoop.mapreduce.Job.updateStatus
 at org.apache.hadoop.mapreduce.Job.isSuccessful
 at org.apache.hadoop.mapreduce.Job.waitForCompletion
 at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob
 at org.apache.sqoop.mapreduce.ImportJobBase.runJob
 at org.apache.sqoop.mapreduce.ImportJobBase.runImport
 at org.apache.sqoop.manager.SqlManager.importTable
 at org.apache.sqoop.manager.MySQLManager.importTable
 at org.apache.sqoop.tool.ImportTool.importTable
 at org.apache.sqoop.tool.ImportTool.run
 at org.apache.sqoop.Sqoop.run
 at org.apache.hadoop.util.ToolRunner.run
 at org.apache.sqoop.Sqoop.runSqoop
 at org.apache.sqoop.Sqoop.runTool
 at org.apache.sqoop.Sqoop.runTool
 at org.apache.sqoop.Sqoop.main
Caused by: java.net.ConnectException: Call From hadoop/172.16.59.90 to hadoop:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: https://wiki.apache.org/hadoop/ConnectionRefused
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0
 at sun.reflect.NativeConstructorAccessorImpl.newInstance
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance
 at java.lang.reflect.Constructor.newInstance
 at org.apache.hadoop.net.NetUtils.wrapWithMessage
 at org.apache.hadoop.net.NetUtils.wrapException
 at org.apache.hadoop.ipc.Client.call
 at org.apache.hadoop.ipc.Client.call
 at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke
 at com.sun.proxy.$Proxy14.getJobReport
 at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport
 at sun.reflect.GeneratedMethodAccessor9.invoke
 at sun.reflect.DelegatingMethodAccessorImpl.invoke
 at java.lang.reflect.Method.invoke
 at org.apache.hadoop.mapred.ClientServiceDelegate.invoke
 ... 23 more
Caused by: java.net.ConnectException: Connection refused
 at sun.nio.ch.SocketChannelImpl.checkConnect
 at sun.nio.ch.SocketChannelImpl.finishConnect
 at org.apache.hadoop.net.SocketIOWithTimeout.connect
 at org.apache.hadoop.net.NetUtils.connect
 at org.apache.hadoop.net.NetUtils.connect
 at org.apache.hadoop.ipc.Client$Connection.setupConnection
 at org.apache.hadoop.ipc.Client$Connection.setupIOstreams
 at org.apache.hadoop.ipc.Client$Connection.access$2800
 at org.apache.hadoop.ipc.Client.getConnection
 at org.apache.hadoop.ipc.Client.call
 ... 31 more

错误说明DataNode 需要访问 MapReduce JobHistory Server,而我已经配置了

{hadoop_home}/etc/hadoop/mapred-site.xml

配置如下

 mapreduce.jobhistory.address
 {namenode}:10020

配置完成之后要启动

{hadoop_dir}/sbin/mr-jobhistory-daemon.sh start historyserver

而我没有启动JobHistory 服务。

所以发生了此错误,如果启动JobHistory服务后还发生如下错误;

ERROR org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Error while trying to scan the directory hdfs://namenode:9000/tmp/had
oop-yarn/staging/history/done_intermediate/hadoop

那么修改${HADOOP_DIR}/etc/hadoop/mapred-env.sh这个配置文件的heap大小

export HADOOP_JOB_HISTORYSERVER_HEAPSIZE=2000

然后在重新启动JobHistory服务,问题解决!

在线客服1
在线客服2
关注官方微信
返回顶部