Day 2 of data school, we started our learning with the concept of modern analytics architecture.
Firstly, what is analytics architecture? This is a framework that organizations use to implement analytical processes and achieve data-driven insights. E.g. collect data, store, prepare, analyse and visualise. The way organizations manage data has evolved throughout time. From traditional to modern analytics architecture.
Modern analytics architecture - infrastructure, tools and processes that enable storage, organization and analysis of an organization's data. It differs from traditional architecture by the way they operate (more business led), the tools they use (e.g. tableau and alteryx) and how the different teams of the organization work internally (agile). Within the modern analytics architecture we have the analytics pipeline
(Modern) analytics pipeline - This term refers to the the 'stages' that the data goes trough. The aim is to convert raw data into digestible visuals.
- Raw data - This is the very beginning. You have access to data that has not been processed for use. The data is collected from various sources
- Ingestion - Then, you will need to move the data to a storage space. E.g. SQL or APIs (application programming interface) can be used.
- Central storage - The ingestion stage is pretty much interconnected with this one. Using the necessary software, all the data will be stored in the same place. (This will also include the data that will be prepd and the trusted data.)
- Prepared data - This is a stage that data analyst spend some time. Data will be cleansed, filtered, formatted and transformed. At the end of this stage the data should be accurate and consistent, which leads to the next stage.
- Trusted data - the trusted data is the type of data that you can literally 'trust' to be right. All the mistakes and errors are out of the way. It will also be stored in the central storage.
- Visualised data - Lastly, data is displayed in a digestible format, such as charts.
Understanding these concepts is important to better what is going to come next during learning.