Clear Filters
Clear Filters

Info

This question is closed. Reopen it to edit or answer.

How to solve "The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information."

1 view (last 30 days)
Hadoop version 1.2.1 Matlab version 2015a
Linux ubuntu 14.
At first, I try to run them at a cluster (one master and two slaves). I can run the example 'wordcount'in Hadoop. And it is success to read the data from the HDFS through the Matlab.
But when I try to run the example in Matlab 'Run mapreduce on a Hadoop Cluster',I failed.
It shows that the Map 0% and Reduce 0%.
Error using mapreduce (line 100)
The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information.
Error in run_mapreduce_on_a_hadoop (line 24)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,...
There is my Matlab codes
setenv('HADOOP_HOME','/usr/local/hadoop');
cluster = parallel.cluster.Hadoop;
cluster.HadoopProperties('mapred.job.tracker') = 'ubuntu:50031';
cluster.HadoopProperties('fs.default.name') = 'hdfs://ubuntu:8020';
outputFolder = '/home/rjy/logs/hadooplog';
mr = mapreducer(cluster);
ds = datastore('airlinesmall.csv','TreatAsMissing','NA',... 'SelectedVariableNames','ArrDelay','ReadSize',1000);
preview(ds)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,... 'OutputFolder',outputFolder)
I don't know what it means in this logs as follows,
2015-08-14 22:41:40,981 INFO org.apache.hadoop.mapred.JobInProgress: Choosing a failed task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
'attempt_201508142238_0001_m_000000_3' to tip task_201508142238_0001_m_000000, for tracker
'tracker_slave1:localhost/127.0.0.1:41664'
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201508142238_0001_m_000000_2'
2015-08-14 22:41:42,855 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201508142238_0001_m_000000_3:
com.mathworks.toolbox.parallel.hadoop.MatlabStartupFailureException: parallel:mapreduce:HadoopTaskCouldNotFindMatlab
Caused by: java.lang.ClassNotFoundException: com.mathworks.toolbox.distcomp.util.Version
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
How to solve this problem? Thanks a lot.
I attach my Hadoop logs and Matlab logs.

Answers (0)

This question is closed.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!