Data engineering
- About
- e-Leadership MBA
- Graduate Professional Program
- Joint Graduate Study Programme in Computer Science - Internet of Things and Artificial Intelligence
- Lecture Plan
- Computing
- Undergraduate University Program
- Learning outcomes on the program level
- Learning outcomes on the program level
- Admission
- Learning outcomes on the program level
- Learning outcomes on the program level
- Apply now
- Learning outcomes on the program level
- Learning outcomes on the program level
- Learning outcomes on the program level
- Learning outcomes on the program level
- Useful Information
- Learning outcomes on the program level
- Learning outcomes on the program level
- Useful Information
- Professional Title Acquired
- 3D Design
- Design
- Duration of the Study Program
- Undergraduate Professional Program
- Lecture Plan
- Lecture Plan
- Lecture Plan
- Useful Information
- Useful Information
- Software Package Included in the Price of the Study
- Software Package Included in the Price of the Study
- Course Schedules
- Course Schedules
- Professional Title Acquired
- Duration of the Study Program
- Digital Marketing
- Software Package Included in the Price of the Study
- Duration of the Study Program
- Lecture Plan
- Professional Title Acquired
- Useful Information
- Course Schedules
- Multimedia Computing
- Industry Certifications
- Course Schedules
- Professional Title Acquired
- Software Package Included in the Price of the Study
- System Engineering
- Duration of the Study Program
- Lecture plan
- Course Schedules
- Professional Title Acquired
- Software Package Included in the Price of the Study
- Industry Certifications
- Useful Information
- Software Engineering
- Duration of the Study Program
- Software Package Included in the Price of the Study
- Course Schedules
- Lecture Plan
- Industry Certifications
- Useful Information
- Professional Title Acquired
- Creative Market Communications Management
- Duration of the Study Program
- Course Schedules
- Industry Certifications
- Lecture Plan
- Useful Information
- Duration of the Study Program
- Software Package Included in the Price of the Study
- Digital Marketing
- Who Can Enroll
- Professional Title Acquired
- Software Package Included in the Price of the Study
- Course Schedules
- Lecture Plan
- Professional Title Acquired
- Useful Information
- Useful Information
- Data Science
- Industry Certifications
- Course Schedules
- Jobs and necessary knowledge and skills that graduates will be prepared for
- Course Schedules
- Duration of the Study Program
- Software Package Included in the Price of the Study
- Lecture Plan
- Duration of the Study
- Game Development
- Professional Title Acquired
- Who Can Enroll
- Professional Title Acquired
- MITx MicroMasters® program
- Useful Information
- Lecture Plan
- Software Package Included in the Price of the Study
- System Engineering
- Duration of the Study
- Who can Apply
- Course Schedules
- Lecture Plan
- Software Engineering
- Software Engineering
- Useful Information
- Study Duration
- Lecture Plan
- Lecture Plan
- Useful Information
- Useful Information
- Learning outcomes on the program level
- Learning outcomes on the program level
- Industry Certifications
- Industry Certifications
- Professional Title Acquired
- Professional Title Acquired
- Duration of the Study Program
- Duration of the Study Program
- Course Schedules
- Course Schedules
- Learning outcomes on the program level
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Partnership with Goldsmiths, University of London
- Class 30
- Practice 30
- Independent work 90
Course title
Data engineering
Lecture type
Elective
Course code
21-02-506
Semester
1
ECTS
5
Lecturers and associates
Course overview
This module will expose students to a particular experience when dealing with data engineering problems in a practical way both individually and in teams
The objectives of this module are to enable students to learn to:
• Analyze and organize raw data
• Build data systems and pipelines
• Evaluate business needs and objectives
• Interpret trends and patterns
• Prepare data for prescriptive and predictive modelling
Build and maintain an organization’s data ecosystem, including; data sources and
databases to data storage solutions.
The aim of this module is for students to demonstrate their knowledge and understanding of basic methods of data preparation that includes methods of cleaning, transforming, introverting, normalizing and aggregating data, time series transformation, work with missing values as well as basic data reduction methods such as feature reduction, sample reduction, and discretization.
It is important for students to take this module to gain a better understanding of how critical data engineering is, often described as backbone of data science. Once data science process begins, the first people to interact with data are data engineers. The more efficient they are at filtering, cleaning, and directing that data, the more efficient everything else can be as the data flows further down the project funnel and towards other team members. The knowledge and understanding students acquire in this module will contribute to the overall skillset for their future employment as data scientists.
Literature
Essential reading:
1. Crickard, P (2020) Data Engineering with Python: Work with massive datasets to design data models and automate data pipelines using Python, Birmingham: Packt Publishing,
2. Algebra University College (2020), Data Engineering Handbook, Zagreb: Algebra University College
Recommended reading:
1. Garcia, S., Luengo, J., Herrera, F. (2016) Data Preprocessing in Data Mining, Cham: Springer International Publishing
2. Balamurugan, A.S., Christopher, A.B. (2012) Insight into Data Preprocessing: Theory and Practice: Data Mining Perspective Chisinau: Lap lambert Academic Publishing
Further reading:
1. Chakrabarti, S., Cox E., Eibe, F., Hartmut, RG, Han, J., Jiang, X., Kamber, M., Lightstone, S.S. (2009) Data Mining: Know It All, Massachusetts: Morgan Kaufmann
Minimal learning outcomes
- Describe possible solutions to data preparation problems.
- Discuss differences between methods for working with missing data and data transformation methods.
- Explain the impact of selected newer technologies on the data preparation process.
- Identify different aggregation functions and methods of time series transformation.
- Explain possible solution for a particular problem in the process of integration, normalization and discretization of data
- Explain available basic methods of feature and pattern reduction.
Preferred learning outcomes
- Recommend optimal solutions to data preparation problems.
- Distinguish between an adequate method for working with missing data and data transformation methods.
- Judge the impact of newer technologies on the data preparation process.
- Select adequate aggregation functions and methods of time series transformation.
- Choose an adequate solution for a particular problem in the process of integration, normalization and discretization of data.
- Apply adequate basic methods of feature and pattern reduction.