Course title

Data engineering

Lecture type

Elective

Course code

21-02-506

Semester

1

ECTS

5

Course overview

This module will expose students to a particular experience when dealing with data engineering problems in a practical way both individually and in teams

The objectives of this module are to enable students to learn to:
• Analyze and organize raw data
• Build data systems and pipelines
• Evaluate business needs and objectives
• Interpret trends and patterns
• Prepare data for prescriptive and predictive modelling
Build and maintain an organization’s data ecosystem, including; data sources and
databases to data storage solutions.

The aim of this module is for students to demonstrate their knowledge and understanding of basic methods of data preparation that includes methods of cleaning, transforming, introverting, normalizing and aggregating data, time series transformation, work with missing values as well as basic data reduction methods such as feature reduction, sample reduction, and discretization.

It is important for students to take this module to gain a better understanding of how critical data engineering is, often described as backbone of data science. Once data science process begins, the first people to interact with data are data engineers. The more efficient they are at filtering, cleaning, and directing that data, the more efficient everything else can be as the data flows further down the project funnel and towards other team members. The knowledge and understanding students acquire in this module will contribute to the overall skillset for their future employment as data scientists.

Literature

Essential reading:
1. Crickard, P (2020) Data Engineering with Python: Work with massive datasets to design data models and automate data pipelines using Python, Birmingham: Packt Publishing,
2. Algebra University College (2020), Data Engineering Handbook, Zagreb: Algebra University College

Recommended reading:
1. Garcia, S., Luengo, J., Herrera, F. (2016) Data Preprocessing in Data Mining, Cham: Springer International Publishing
2. Balamurugan, A.S., Christopher, A.B. (2012) Insight into Data Preprocessing: Theory and Practice: Data Mining Perspective Chisinau: Lap lambert Academic Publishing

Further reading:
1. Chakrabarti, S., Cox E., Eibe, F., Hartmut, RG, Han, J., Jiang, X., Kamber, M., Lightstone, S.S. (2009) Data Mining: Know It All, Massachusetts: Morgan Kaufmann