Hortonworks Hadoop 2.0 Developer Certification

Horotonworks Certification Tips and Guidelines

I successfully completed this certification on Oct 24, 2014 with a passing score of 88%.  I am sharing the experience I gained on this certification. I have given all the required materials what I have gone through for this certification.  Please have some sandbox level hands on experience on these topics before you appear for the examination.

Read all the answers of a question very carefully before you select the correct answer. Because these are little tricky.  Also 90 minutes is more than enough for this exam as you can easily complete in 45 to 60 mins. All the Best

Certification 1 – Hortonworks Certified Apache Hadoop Developer (Pig and Hive)

Exam Format

• Registration Cost – $200/attempt (Unlimited attempts are allowed)
• 1The exam consists of approximately 50 multiple-choice questions. The exam is delivered in English.
• You have to clear 38 questions (75%) to get certified
• Certification References @ Hortonworks — > http://hortonworks.com/training/hadoop-2-0-developer-certification/

Course Curriculum

Objective 1.1 – HDFS and Hadoop 2.0

• Explain Hadoop 2.0 and YARN
• Explain how HDFS Federation works in Hadoop 2.0
• Explain the various tools and frameworks in the Hadoop 2.0 ecosystem
• Use the Hadoop client to input data into HDFS
• Using HDFS commands

Various Study materials I have referred for this objective:

• Hadoop Definitive Guide by Tom White (Chapters 2,3 )
• YARN by Arun C. Murthy (Chapters 3,4)
• http://m.dummies.com/how-to/content/how-to-launch-a-yarnbased-application.html
• http://hortonworks.com/hadoop/yarn/
• http://hortonworks.com/blog/an-introduction-to-hdfs-federation/
• http://hortonworks.com/blog/namenode-high-availability-in-hdp-2-0/
• http://hortonworks.com/hadoop-tutorial/using-commandline-manage-files-hdfs/
• http://hortonworks.com/hadoop/hdfs/

Objective 2.1 – MapReduce and YARN

• Explain the architecture of MapReduce
• Run a MapReduce job on Hadoop
• Monitor a MapReduce job

Various Study materials I have referred for this objective:

• http://hortonworks.com/hadoop/mapreduce/
• http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.3/bk_using-apache-hadoop/content/running_mapreduce_examples_on_yarn.html

Objective 3.1 – Pig

• Write a Pig script to explore and transform data in HDFS
• Define advanced Pig relations
• Use Pig to apply structure to unstructured Big Data
• Invoke a Pig User-Defined Function
• Compute Quartiles with Pig
• Explore data with Pig
• Split a dataset with Pig
• Join datasets with Pig
• Use Pig to prepare data for Hive

Objective 4.1 – Hive and HCatalog

• Write a Hive query
• Understand how Hive tables are defined and implemented
• Use Hive to run SQL-like queries to perform data analysis
• Perform a multi-table select in Hive
• Design a proper schema for Hive
• Explain the uses and purpose of HCatalog ™
• Use HCatalog with Pig and Hive
• Computing ngrams with Hive
• Analyzing Big Data with Hive
• Understanding MapReduce in Hive
• Joining datasets with Hive
• Streaming data with Hive and Python

Various Study materials I have referred for the objectives 3.1 and 4.1:

• Programming Pig by Alan Gates (Chapters 4,5,6) http://datafu.incubator.apache.org/docs/datafu/guide/statistics.html
• Programming Hive by Jason Rutherglen, Dean Wampler, Edward Caprilo (Chapters 3,4,5,6,14,15) https://cwiki.apache.org/confluence/display/Hive/StatisticsAndDataMining

Objective 5.1 – Hadoop Tools

• Use Sqoop to transfer data between Hadoop and a relational database·
• Using Sqoop to transfer data between HDFS and a RDBMS
• Using HCatalog with Pig
• Define a workflow using Oozie

Various Study materials I have referred for the above objective.

• http://hortonworks.com/hadoop/sqoop/
• http://hortonworks.com/hadoop-tutorial/import-microsoft-sql-server-hortonworks-sandbox-using-sqoop/
• http://hortonworks.com/hadoop/oozie/
• https://cwiki.apache.org/confluence/display/Hive/HCatalog+UsingHCat

Additional Oozie References:

Oozie

Oozie_another reference

Focus More on:

• Functional Flow of YARN — >> Client — > Resource Manager — > Application Master —  >Containers.
• Hadoop fs get (copyToLocal), put (copyFromLocal), cat commands
• Pig JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN (Please find the attachment below)
• Pig Replicated, Merge, Skew Joins
• REGISTER and INVOKE Pig UDFs
• Hive Managed vs External Tables and It’s file transmission to /apps/hive/warehouse directory.
• HCatLoader,  HCatStorer
• Hive SORT BY
• MR Default Partitioner
• WebHDFS open, CREATE, MKDIRS, LISTSTATUS, GETFILESTATUS commands
• Hive ngrams
• HDFS Federation vs DataNodes vs Namenode Failures.
• Get a Clear Programming Level understanding of Pig and Hive Scripts
• Apache HCatalog

Pig Joins

Some More Notables:

1.Pig Picks the file from current working directory

2.Pig Sample Command

3.Top 50 BiGrams

4. HCatLoader and HCatStorer

5.NameNode Federation

6. YARN Life Cycle

Am sure you must be able to follow my guidelines on this Pig Hive Certification. Wish you all the Best and Advanced Congratulations.