Day 3 of Dashboard Week arrived, and the task at hand was diving into the #GamesNightViz community project, focusing on Street Fighter V. Like any project, it all starts with data discovery, and that’s exactly where I began.
We were given two links to stats on the game’s characters, their usage over time, and the freedom to pull in additional resources if needed. Naturally, my curiosity led me to explore tournaments, players, character choices, winning conditions, and all those intriguing questions in between. The data I wanted, however, was online, meaning only one thing: web scraping.
The Thrill—and Challenge—of Web Scraping
I’ve web scraped before, using tools like Excel, Alteryx, and Power BI. But one thing I’ve learned through experience is that no two scraping projects are the same. A method that worked seamlessly before might hit a roadblock today. And that's precisely what happened here.
The webpage held most of the information I needed, but getting it into a clean, usable format wasn’t as straightforward as I had hoped. As anyone who’s done this before knows, the process can be frustrating when you hit obstacles. But I had a plan, and I wasn’t about to let this challenge derail it.
Tip: Always Have a Backup Plan
Here’s the biggest takeaway for anyone working on a data project: Plan to succeed, but also plan for when things don’t go as expected. We often forget that setbacks are inevitable. The real trick is preparing for them. I did exactly that by splitting my project into two parts—simple, foundational questions I could answer easily, and more complex queries that required detailed analysis and extra time.
This way, even if the advanced part of the project failed, I could still deliver something valuable. I knew that having a fallback would allow me to move forward, rather than get stuck trying to solve everything at once.
The Scraping Struggle
I began scraping using Alteryx, but within an hour, I ran into data cleaning issues that prevented me from moving forward. I spent another hour troubleshooting, using AI tools, researching solutions, and even consulting with colleagues. Despite all the effort, I still couldn’t get the data in the shape I needed.
It was frustrating, but here’s the thing: Sometimes, the best decision is knowing when to shift gears. After nearly three hours, I made the tough call to abandon the complex scraping in favour of focusing on a simpler aspect of the project—tournament prize data.
The Importance of Flexibility
If I had been stubborn, I would have wasted the entire day trying to crack the scraping problem and ended up with nothing to present. Instead, by pivoting, I was able to create a simple yet effective visualisation on tournament prizes. While it wasn’t the full analysis I had originally planned, it was still a meaningful part of the project and a deliverable that I could stand behind.
The Lesson: It’s Never Game Over
This project reinforced an important lesson for me: It’s never truly game over. Sure, I didn’t achieve everything I wanted, but I adapted, made progress, and delivered something valuable. The ability to adjust your expectations and focus on what can be achieved—rather than what went wrong—makes all the difference.
The game hasn’t ended; it’s just begun. And while today wasn’t a perfect win, it was a reminder that success is about playing smart, not just playing hard.