Info

This question is closed. Reopen it to edit or answer.

How to solve "The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information."

1 view (last 30 days)
Hadoop version 1.2.1 Matlab version 2015a
Linux ubuntu 14.
At first, I try to run them at a cluster (one master and two slaves). I can run the example 'wordcount'in Hadoop. And it is success to read the data from the HDFS through the Matlab.
But when I try to run the example in Matlab 'Run mapreduce on a Hadoop Cluster',I failed.
It shows that the Map 0% and Reduce 0%.
Error using mapreduce (line 100)
The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information.
Error in run_mapreduce_on_a_hadoop (line 24)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,...
There is my Matlab codes
setenv('HADOOP_HOME','/usr/local/hadoop');
cluster = parallel.cluster.Hadoop;
cluster.HadoopProperties('mapred.job.tracker') = 'ubuntu:50031';
cluster.HadoopProperties('fs.default.name') = 'hdfs://ubuntu:8020';
outputFolder = '/home/rjy/logs/hadooplog';
mr = mapreducer(cluster);
ds = datastore('airlinesmall.csv','TreatAsMissing','NA',... 'SelectedVariableNames','ArrDelay','ReadSize',1000);
preview(ds)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,... 'OutputFolder',outputFolder)
I don't know what it means in this logs as follows,
2015-08-14 22:41:40,981 INFO org.apache.hadoop.mapred.JobInProgress: Choosing a failed task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
'attempt_201508142238_0001_m_000000_3' to tip task_201508142238_0001_m_000000, for tracker
'tracker_slave1:localhost/127.0.0.1:41664'
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201508142238_0001_m_000000_2'
2015-08-14 22:41:42,855 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201508142238_0001_m_000000_3:
com.mathworks.toolbox.parallel.hadoop.MatlabStartupFailureException: parallel:mapreduce:HadoopTaskCouldNotFindMatlab
Caused by: java.lang.ClassNotFoundException: com.mathworks.toolbox.distcomp.util.Version
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
How to solve this problem? Thanks a lot.
I attach my Hadoop logs and Matlab logs.

Answers (0)

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!