Professional Certificate course in Data Engineering 

offered by IIT Kanpur

Module 1

Introduction to DE

What is Data Engineering

Role of Data Engineers in the Industry

Importance of Data Engineering in Data-driven Organizations

Overview of Data Engineering Tools and Technologies

Career Paths and Opportunities in Data Engineering

 

Module 2

Python

Introduction to Python

Basic Syntax and Data Types

Control Structures (Conditional Statements and Looping)

Functions

Lambda Functions

Data Structures (Lists, Tuples, Dictionaries, Sets)

File Handling

Error Handling (try and except)

List Comprehensions

Decorators

NumPy

Pandas

Regex

Code optimisation

 

Module 3

RDBMS

Introduction to Databases

MYSQL -Introduction & Installation

SQL KEYS

PRIMARY KEY

FOREIGN KEY

UNIQUE KEY

Composite key

Normalization and Denormalization

ACID Properties

 

Module 4 

SQL

Basic SQL Queries

Advanced SQL Queries

Joins (INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL JOIN)

Data Manipulation Language (DML): INSERT, UPDATE, DELETE

Data Definition Language (DDL): CREATE, ALTER, DROP

Data Control Language (DCL): GRANT, REVOKE

Aggregate Functions (SUM, AVG, COUNT, MAX, MIN)

Grouping Data with GROUP BY

Filtering Groups with HAVING

Subqueries

Views

Indexes

Transactions and Concurrency Control

Stored Procedures and Functions

Triggers

Stored procedures

 

Module 5

Mongo DB

Introduction to NoSQL and MongoDB

Installation and Setup of MongoDB

MongoDB Data Model (Documents, Collections, Databases)

CRUD Operations (Create, Read, Update, Delete)

Querying Data with MongoDB

Indexing and Performance Optimization

Aggregation Framework

Data Modeling and Schema Design

Working with Embedded Documents and Arrays

Transactions and Atomic Operations

Security in MongoDB (Authentication, Authorization)

Replication and High Availability

Sharding and Scalability

 

Module 6

Shell Script

Introduction to Shell Scripting

Basics of Shell Scripting (Variables, Comments, Quoting)

Input/Output in Shell Scripts

Control Structures (Conditional Statements, Loops)

Functions and Scripts Organization

Command Line Arguments and Options

String Manipulation

File and Directory Operations

Process Management (Running Commands, Background Processes)

Text Processing (grep, sed, awk)

Error Handling and Exit Status

Environment Variables

 

Module 7

GIT

Introduction to Version Control Systems (VCS) and Git

Installation and Setup of Git

Basic Git Concepts (Repositories, Commits, Branches, Merging)

Git Workflow (Local and Remote Repositories)

Creating and Cloning Repositories

Git Configuration (Global and Repository-specific Settings)

Tracking Changes with Git (git add, git commit)

Viewing Commit History (git log)

Branching and Merging (git branch, git merge)

Resolving Merge Conflicts

 

Module 8

Cloud

Introduction to Cloud Computing and Data Engineering

Overview of Cloud Providers (AWS and Azure)

Cloud Storage Solutions (AWS S3, Azure Blob Storage)

Cloud Database Services (AWS RDS, Azure SQL Database)

Data Warehousing in the Cloud (AWS Redshift, Azure Synapse Analytics)

Cloud Data Integration and ETL (AWS Glue, Azure Data Factory)