DSEng 107 – Data Engineering & Pipelines

(Weeks 13–15 | Lec 9 Hrs / Lab 27 Hrs / Ext 0 Hrs | 36 Total Hrs | 1.2 Credit Hrs)
Students will:

Design robust and scalable data systems
Prerequisite: DSEng 106 – Deep Learning & Generative AI
Tools: Apache Airflow, Apache Spark, Databricks

Build ETL and ELT pipelines using modern frameworks

Automate data workflows with Apache Airflow and Prefect

Manage large datasets using Spark and Databricks