Using Oozie 4.4.0 with Hadoop 2.2

The current version of Oozie (4.0.0) doesn’t build correctly when you try and target Hadoop 2.2. The Oozie team have a fix going into release 4.0.1 (see OOZIE-1551), but until then you can hack the Maven files to get it working with 4.0.0.

First download the 4.0.0 version from https://oozie.apache.org/, and then unpackage it. Next run the following command to change the Hadoop version being targeted:

cd oozie-4.0.0/
find . -name pom.xml | xargs sed -ri 's/(2.2.0\-SNAPSHOT)/2.2.0/'

Now all you need to do is target the hadoop-2 profile in Maven and you’ll be all set:

mvn -DskipTests=true -P hadoop-2 clean package assembly:single

About the author

Hadoop in Practice, Second Edition

Alex Holmes works on tough big-data problems. He is a software engineer, author, speaker, and blogger specializing in large-scale Hadoop projects. He is the author of Hadoop in Practice, a book published by Manning Publications. He has presented multiple times at JavaOne, and is a JavaOne Rock Star.

If you want to see what Alex is up to you can check out his work on GitHub, or follow him on Twitter or Google+.

comments powered by Disqus


Full post archive