Duke University - Linux and Bash for Data Engineering 

Welcome to Linux and Bash for Data Engineering

Meet your Course Instructor: Noah Gift

Overview of Key Concepts

Introduction to Working with Linux

Introduction to the Linux Shell

Installing and Accessing Linux

Getting Started with Github Codespaces

Comparing Linux Shell Environments

Writing 'Hello World' Bash Script

Popular Terminal Commands in Linux

Introduction to Shell Piping

What is Shell Piping?

Walkthrough of Shell Pipeline Examples

Introduction to SSH

What is SSH?

Creating SSH Keys and using with GitHub

Using SSH to Connect to Remote Server in AWS Cloud

Walkthrough of SSH Tunneling Example

Meet your Supporting Instructors: Kennedy Behrman and Alfredo Deza

Course Structure and Discussion Etiquette

Using Linux

Introduction to Working with Bash

Introduction to Configuring your Bash Shell Environment

What are Shell Configuration Files?

Configuring .bashrc

Configuring .zshrc and Third-Party Tool oh-my-zsh

Introduction to Working with Shell Variables

What are Shell Variables?

Exporting Shell Variables in an Interactive Prompt

Using Shell Variables in a Bash Script

Introduction to Standard Streams

What are Standard Streams?

Using Standard Out

Using Standard In

Using Standard Error

Using Bash

Introduction to Building Bash Scripts

Introduction to Shell Logic and Control Flow

What is Shell Control Flow?

Using Shell Control Flow to Solve Problems in Bash

Using Shell Loops in Bash

Evaluating Conditions in Bash

Introduction to Manipulating Data in the Bash Shell

Shell Techniques for Data in the Bash Shell

Truncating Data in Bash

Filtering Data in Bash

Searching Data in Bash

Introduction to Writing Scripts and Command-Line Tools in Bash

What are Scripts and Command-Line Tools?

Building a Bash Script

Building a Bash Function

Building a Bash Command-Line Tool

Bash-Adjacent Tools: Makefiles and Dockerfiles

Data Structures in Bash

Building Bash Scripts

Introduction to Composing File and Data Management Solutions in Linux

Introduction to Searching a Filesystem in Linux

Methods for Searching a Filesystem in Linux

Using the Locate Command to Find Files in Linux

Using the Find Command to Find Files in Linux

Using xargs to Extend Search Capabilities in Linux

Using mdfind on OS X

Introduction to Modifying Files, Directories, Permissions and Archiving in Linux

Overview of Modifying the Filesystem in Linux

Moving Files and Directories in Linux

Setting Permissions on Files and Directories in Linux

Archiving Data in Linux

Introduction to Processing Text in Linux

Benefits of Processing Text in Linux

Using grep, cut, sort and unique in Linux

Editing with Truncation, awk and sed in Linux

Using Regular Expressions (regex) in Linux

Composing File and Data Management Solutions with Linux