Dashboard Week Day 3 was my favorite so far - in part due to the requirement that we use Cheese.com as our data source, and in larger part because it required more time in Alteryx doing some web-scraping (which I always enjoy).
The joy I felt building batch macros and writing Regex dimmed right around lunchtime, however. Up until that point I felt as though I had scoped the project well and was moving briskly through what I needed to do. There are only 2,006 cheeses on the Cheese.com website, which is a lot of cheeses to be sure, but in terms of a simple dataset to build is *nothing* (or so I thought to myself).
As it turns out, hitting the Cheese.com website 2,006 times does take a lot of time and effort from Alteryx (and my computer’s physical memory). In retrospect, I estimate it took about 6 seconds per cheese - a total of 22 minutes to run the workflow I’d built to scrape that data (and that was the *second* scrape I had to do, the first being one just to get the URLs to do that second scrape).
But I didn’t run that 22-minute workflow just once. And herein is my lesson for Dashboard Week Day 3:
If your workflow will take a long time to run, always test it first.
My workflow had one small error that meant when I ran it the first time, I produced a dataset with 2,006 records of identical data. 22 minutes isn’t that long of a time, but when you only have 5.5 hours to do ideate a dashboard, get data, clean that data, build a dashboard, and prepare a presentation…22 minutes is a lot to lose. And 44 minutes is even worse. Looking back, testing my workflow on a small sample of data would have been easy, and would have given me the time I wish I’d had to complete my dashboard’s functionality and formatting.
I’m glad to have learned that lesson now and not while working with a client, and I’m looking forward to coming back to my Cheeseboard Builder Dashboard someday in the near future to finish and publish it!