The module equips you with big data technologies for handling large-scale, varied, and real-time data. It focuses on data management and the properties of modern data storage solutions, and their relevance in the context of enterprise systems. You will gain knowledge about creating data pipelines for analytics and to make informed platform choices for designing and implementing solutions in diverse data scenarios. The module also covers the identification and documentation of relevant data hierarchies or taxonomies. You will develop the necessary skills and knowledge to become proficient data engineers, capable of ensuring data is organised and accessible for analysis and decision-making.

Pre-requisite modules: None.

Co-requisite modules: None.

This module is a part of MSc Digital and Technology Solutions (Integrated Degree Apprenticeship) programme. It cannot be taken as an elective by students on other programmes.

The apprenticeship standard and other documentation relating to the Level 7 Digital and Technology Solutions (Data Analyst Specialist) Apprenticeship can be found here: https://www.instituteforapprenticeships.org/apprenticeship-standards/digital-and-technology-solutions-specialist-integrated-degree/

This module covers engineering aspects of data science and big data, hence the main objective of the module is to provide you with specialised knowledge and critical understanding of the process of collecting, organising, retrieving and storing large scale and real-time complex datasets with help of databases and data storage solutions and technologies such as Hadoop and Spark. Consequently, an important aim of the module is to equip you with practical knowledge needed to work with modern database management system including both relational databases and non-relational databases (NoSQL). You also need to know how to develop code in a programming language (Python) and use its various libraries for big data processing, managing the coordination and creating data pipelines for analytics that transform data into actionable decisions.

On successful completion of this module you should be able to:

Module Specific Skills and Knowledge

Discipline Specific Skills and Knowledge

Personal and Key Transferable / Employment Skills and Knowledge

Whilst the module’s precise content may vary from year to year, an example of an overall structure is as follows:

• Big data, foundation, infrastructures and Platforms
• Data representation, information modelling and mapping
• Data storage solutions, relational and non-relational databases systems
• Transactions and their use in integrity and recovery management
• Distributed systems, cloud computing and Data
• Data orchestration, integration and ETL/ELT pipelines
• CAP theorem and BASE database design principle
• Business intelligence, data quality and data hierarchies in enterprise data management
• Python for API programming of various big data platforms and applications  

Basic reading:

• Soufian, M. (2014) Notes on STFC Big Data and Analytics Summer School 2014, Daresbury Laboratories, Warrington, UK.
• Buyya, R., Calheiros, R.N., Dastjerdi, A.V. (2016), Big Data: Principles and Paradigms. Morgan Kaufman
• Berman, J.J. (2018), Principles and Practice of Big Data: Preparing, Sharing, and Analyzing Complex Information. 2nd ed. Academic Press
• Kleppmann, M (2016) Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, O'Reilly
• Ceder, N. (2018) The Quick Python Book. Third Edition, Manning Publications Co

Reading list for this module: