java - Parallelization of map reduce -


i have 2 independent mapreduce jobs in single java program. how can make them run in parallel. system configuration hadoop 2.7.1 3 node 16 physical cores

instead of using job.waitforcompletion(true) use job.submit() method submitting job. later submit job , return immediately. job.waitforcompletion(true) submit job , waits completion.

then should use iscomplete() checking completion status of jobs. remember non-blocking call, might want put while loop while sleep time.

job job1 = new job("job1"); job job2 = new job("job2"); ....  // code configure both job objects. //now submit both jobs. job1.submit(); job2.submit(); // wait completion. while(!job1.iscomplete() && !job2.iscomplete()) {      thread.sleep(10000); } 

Comments

Popular posts from this blog

php - Invalid Cofiguration - yii\base\InvalidConfigException - Yii2 -

How to show in django cms breadcrumbs full path? -

ruby on rails - npm error: tunneling socket could not be established, cause=connect ETIMEDOUT -