UCOVI - Organise

Discipline Overview

Organise your data into a trustworthy analytical model that respects data management principles and best practice but fits the needs of your business. Design and build the model so that processing errors are simple to flag up and rectify, and new features are straightforward to build.

Sample questions and considerations

What manual processes can we automate? What checks are in place for errors and gaps in data? Do we have a warehouse load error reporting system? How do we validate the values in our lookup tables and minimise data redundancy?

People and Teams to Involve

IT and security teams, data engineers, data scientists and analysts.

Discipline Overview

Coding and engineering with data: designing data models and architecture that derive insightful features from raw data points.

Common situations faced

Debugging and fixing ETL failures, designing incremental data processing systems, building history and dimension tables in ways that minimise storage and maximise information retained.

Technologies to learn and specialisms to hone

SQL, Python, Alteryx, DBT, Data Bricks. Database normalisation principles, batch processing, debugging, query optimisation.