# |
Hadoop - Course Contents |
1 |
Basics |
|
What is Big data? |
|
How Data is Organised and Analyzed |
|
Different characteristics of Big data |
|
Reasons behind learning Hadoop |
|
What is Hadoop? |
|
Hadoop History and Evolution |
|
Principles of Hadoop |
|
Different Features of Hadoop |
2 |
HDFS Concepts |
|
What is HDFS? |
|
Understanding Blocks in HDFS |
|
Understanding DataNode |
|
Understanding NameNode |
|
Metadata in NameNode |
|
Understanding Secondary NameNode |
|
How Read Operation works in HDFS |
|
How Write Operation works in HDFS |
3 |
Linux Commands |
|
Introduction to a Linux Shell Environment - What is Shell? |
|
Learning User commands like pwd, who, etc. |
|
Learning commands related to Calendar |
|
Commands related to creating a new file |
|
Commands related to displaying the contents of a file |
|
Commands related to listing directories and files |
|
Commands related to creating a new directory |
|
Commands related to copy, move and comparing different
files |
|
Commands related to counting lines, words and characters in
a file |
|
Commands related to running Processes and their Statistics
including grep |
|
Practice sessions on Linux Commands |
4 |
HDFS Commands |
|
Commands to start and stop dfs and yarn |
|
Commands to list directories and files in Hadoop |
|
Commands to find size and empty space in directories |
|
Commands to create directories, put files into HDFS and
later remove files |
|
Commands to move files, get files and list the files in
HDFS |
|
Commands to view content of a file in HDFS |
|
Commands to copyFromLocal, copyToLocal, moveFromLocal and
moveToLocal |
|
Commands to checksum and count the number of directories,
files and bytes |
|
Commands related to Access Control Lists (ACLs) |
|
Commands to merge different files |
|
Commands to change group and their permissions |
|
Commands to list file statistics |
|
Commands to how to append a file to another file |
|
Commands to create a zero byte file |
|
Commands to empty the trash folder in HDFS |
|
Commands to reduce the size of a file (using truncate) |
|
Commands to check file system consistency |
|
Commands to check the classpath of HDFS |
|
Commands to verify missing and corrupted blocks of a file |
|
Commands to get configurations of NameNode and secondary
NameNode |
|
Commands to create, list and delete snapshots |
|
Practice Sessions on HDFS Commands |
5 |
Hadoop I/O |
|
Introduction to working of Serialization and
Deserialization |
|
Will learn about Writable Interface |
|
Will learn about Writable Comaparables and Comparators |
|
Will learn about Writable classes |
|
Will learn about IntWritable class and methods |
|
Will learn about FloatWritable class and methods |
|
Will learn about ByteWritable and BooleanWritable classes
and methods |
|
Will learn about Text class and methods |
|
Will learn about BytesWritable, NullWritable and
ObjectWritable classes |
|
Different practice programs on Writable classes and methods |
|
Will learn about Configuration, Job and JobConf classes |
6 |
MapReduce |
|
What is MapReduce? |
|
Different terminology used in MapReduce |
|
Working Procedure of MapReduce |
|
Will learn about JobClient in MapReduce |
|
Working process of JobTracker in MapReduce |
|
TaskTracker procedure in MapReduce |
|
Will learn about MapTask in MapReduce |
|
Will learn about ReduceTask in MapReduce |
|
How the Combiner works in MapReduce |
|
Advantages of Partitioners in MapReduce |
7 |
Solving WordCount Problem in MapReduce |
|
Understanding a WordCount problem |
|
Working with WordCount program |
|
Will learn to write the code in Driver class |
|
Will learn to write the code in Mapper class |
|
Will learn to write the code in Reducer class |
8 |
Solving Marks Computation Problem in MapReduce |
|
Understanding a Marks Average problem |
|
Will learn to write a program on how to find average marks
of each student |
|
Will learn to write the code in Driver, Mapper and Reducer
classes for similar class of problems |
9 |
Solving Electricity Units Consumption Problem in MapReduce |
|
Understanding a problem related to consumption of Average
Electricity Units |
|
Will learn to write a program on how to find Average
Consumption of Electricity Units greater than 30 |
|
Will learn to write the code in Driver, Mapper and Reducer
classes for similar class of problems |
10 |
Solving Maximum Temperature Problem in MapReduce |
|
Understanding a problem related to computation of Maximum
Temperature |
|
Will learn to write a program on how to find Maximum
Temperature of any year using MapReduce |
|
Will learn to write the code in Driver, Mapper and Reducer
classes for similar class of problems |
11 |
Solving WordCount Problem using Multiple Reducers in
MapReduce |
|
Understanding a problem related to WordCount using Multiple
Reducers |
|
Will learn to write a program using Partitioner |
|
Will learn to write the code in Driver, Mapper, Reducer and
Partitioner classes for similar class of problems |
12 |
Solving Votes Count Problem using MapReduce |
|
Understanding a problem related to Counting Votes |
|
Will learn to write a program on counting of votes in a
General Election using MapReduce |
|
Will learn to write the code in Driver, Mapper and Reducer
classes for similar class of problems |
13 |
Solving Matrix Multiplication Problem using MapReduce |
|
Understanding a problem related to Matrix Multiplication |
|
Will learn to write a program on Matrix Multiplication
Problem using MapReduce |
|
Will learn to write the code in Driver, Mapper and Reducer
classes for similar class of problems |
14 |
Apache Pig Introduction |
|
Understanding Pig and PigLatin |
|
Different Datatypes in PigLatin |
|
Will learn about Schema in PigLatin |
|
Will learn about Schema definition for Tuple, Bag and Map
datatype |
15 |
Grunt Shell in Pig |
|
Will learn to execute PigLatin in Local mode and MapReduce
moode |
|
Will learn executing PigLatin Script |
|
Will learn ShShell Commands in Grunt |
|
Will learn FsShell Commands in Grunt |
16 |
Operators in Pig |
|
Understanding how to use Load Operator |
|
Will learn about Store and Dump Operator |
|
Will learn how to use Describe Operator |
|
Will learn about Foreach Operator |
|
with Filter Operator |
|
Will learn usage of Group and Cogroup Operators in Pig |
|
Understanding Order By Operator |
|
Will learn how to use Distinct Operator |
|
Will learn about Join Operator |
|
Working with Limit Operator |
|
Will learn usage of Parallel Operator in Pig |
|
Understanding Flatten Operator |
|
Will learn how to use Specialized Join Operator |
|
Will learn to work with Union and Cross Operator |
17 |
Hive |
|
Understanding Hive |
|
Architecture and different components of Hive |
|
Numeric Datatypes in Hive |
|
String Datatypes in Hive |
|
Date and Miscellaneous datatypes |
|
Will learn about Datatype conversions in Hive |
|
Will learn different Complex Datatypes |
|
Understanding Arithematic Operators |
|
Understanding Predicate and Logical Operators |
|
Will learn different Mathematical and Aggreagate functions |
|
Will learn Collection and Date functions |
|
Will learn different String functions |
18 |
Hive Queries |
|
Queries on Create, Alter and Drop Databases |
|
Creating different types of tables like managed, external
and partitioned |
|
Queries on Alter and Drop Tables |
|
Queries on how to load input data to table from Local
filesystem and HDFS |
|
Queries using SELECT clause |
|
Queries on how to Insert data into tables |
|
Queries using WHERE clause |
|
Queries using CREATE TABLE AS SELECT(CTAS) |
|
Queries using LIMIT clause |
|
Queries using Nested SELECT |
|
Queries on creating column INDEX of a table |
|
Queries on Inner JOIN and Outer JOIN |
|
Queries using ORDER BY and UNION ALL clause |
The complete course includes extensive
coding and practice exercises. |