When first looking at a dataset, I bet nearly everyone’s first instincts will be to open up their chosen analysis tool and jump straight into the task at hand. I know that when I first started my application, I was playing around with the data to try and trying to see what insights I could call from the dataset. But this is far from what a proper consultant should do.
Firstly, a brief introduction for everyone (as this is my first blog 👀). Hi! 😊My name is Victor and I am a recent starter of the Data School program, joining the London Office as a part of the DS49 Cohort. But that is enough about me.
In my first week at the Data School, one key emphasis that was stressed by all the coaches was the importance of good documentation and thorough planning before taking any action. By maintaining proper documentation and a clear plan, the transition or handover of a project becomes much smoother. Planning ahead before starting a task also helps in identifying potential mistakes early on, allowing for timely discussions with clients to clarify any data queries. This proactive approach ensures that issues are addressed upfront, reducing the risk of having to start over at a later time.
To put this into practice, I will use my first Friday project as an example and explain how I approached documenting each step of the process. For some context, this task revolved around preparing a dataset from raw data files for an insurance company.
Stage 1: Creating a Project Plan
The first step is to create a project plan. This helps keep track of the tasks you aim to accomplish as per the project brief. To aid with the planning of the project, you can also assign time against the task to ensure that you don’t overrun on a particular area, ensuring you have enough time to complete high-priority tasks.
Stage 2: Data Profiling the Raw Data
The next step I undertook was profiling of the raw data. This is crucial as it allows for us to assess the available data and spot any questions that we have on the data even before we start the actual workflow to prep the data. Now Tableau Prep is a great tool for this as their clean step view gives us a summary of the data types which we can use to fill in our data profile.
Now this step was extremely useful in this case as during the data profiling, I spotted an error in the data, where it appeared that two customers were inputted under the same ID. If left uncorrected, this could have caused some issues in the analysis, however as this was spotted early (and before any actual work on the data was done), I was able to go back to the client to discuss the impacts and how they would like to approach this issue.
Stage 3: Plans for Preparing Data
The next steps for preparing the data is planning how you are going to transform your raw data into your desired state.
This comprises of multiple stages:
Step 1 – Do you know your data?
• Sketch out the dataset
• Identify the categories and measures
Step 2 – What is your desired state?
• Sketch out what you need
• Identify the data fields, data types and sample values
Step 3 – How will you get to your desired state?
• Annotate the desired state to identify the changes you need
• List out the changes to be made
Stage 4: Creating the Workflow
Finally, we are starting to use Tableau Prep to create our workflow and prep the data. For this I followed the steps as outlined in my plan to clean the data. I also added descriptions and captions in Tableau Prep, making it easier for other to follow my workflow.
Stage 5: Completing Project Documentation
Once all the outputs are finalized, it is time finish the documentation of the process. As previously mentioned, documenting each step helps consolidate the steps taken while also ensuring that anyone who takes over the project can easily understand the workflow and make necessary adjustments if needed.
The final documentation and plan for this project can be found here visualised in a GIF
Stage 6: Complete Project Log
The final stage for this weekly project was completing the project log and creating a reflective statement regarding the work done. This serves as a record of the work done and highlights key learning points pick up during the process. Additionally, it supports a smooth project handover by ensuring the next person can easily understand and build upon the work.
Here are some of the key things this log should talk about:
• What was the initial objective?
• What was the client's request?
• What did I actually accomplish?
• What did I leave behind for the next person?
• Can another consultant pick up my work and understand it?
• What are the next steps?
• What challenges might arise?
• What were the overarching goals?
• Can an Account Manager easily understand the project’s progress and outcomes?
So that's it! That is all for the stages I took to plan and document my work during my first project. While this is by no means perfect, I hope it provides some useful guidance and tips as to how we should approach a new task.
Thank you for reading! 😊
Victor