Creating an iterative Alteryx App using the flickr API “getPublicPhotos” method

For this project, I wanted to download data about photos stored on flickr – the photo sharing website owned by Yahoo.

The flickr API uses pagination to return results meaning that multiple calls to the API are made until your complete dataset is returned, given the parameters you supplied to the API call.

As we are not looping through a predetermined number of items, we do not use a batch macro in this case, but instead use an Alteryx iterative macro, which will run until a certain condition is met – in this case, that the total number of pages of results has been returned.  The macro itself is contained in one workflow and that is then called from a separate workflow which is made to be an “Analytic App” in Alteryx.  Below is the Analytic App, which we’ll look at in more detail in my next blog post:

The “f” icon in the middle is the iterative app (we’ll look at the workflow for this shortly).  On the left are the inputs that the user can dynamically enter when the App is run – these include the flickr ID and a flickr API key (this can be obtained from flickr free of charge – see the resource links at the end of this blog for details).  To the right is the output, which is a “run command” tool that opens Tableau and also saves the output of the macro (the data) to a .TDE file, that Tableau will access.

Let’s take a closer look at the macro workflow, which is shown below:

When building a macro you might first start with a normal workflow as I did in this case and dropped a Text Input tool on the canvas, as shown below, which holds the URL for the API call we will make:

When I later converted the workflow into a macro, this was done by right clicking on Text Input tool and selecting “Convert to Macro Input” as below:

This automatically converts your workflow to a macro and changes the icon to a Macro Input icon:

However, we must tell Alteryx which type of macro we want.  To do this, click on the canvas and then in the Configuration pane, select the “Workflow” tab and select “Iterative Macro” from the drop-down:

The properties for this API call (the web address and parameters) are set within this first Text Input tool:

The remainder of the first part of the macro is shown below, with each tool having a comment to explain its purpose.

Parse the Photo data rows

The JSON data that is returned from the API call holds both photo data rows – one row for each photo – and header data rows.  The photo data rows are split out and sent to a macro output tool.  At first glance this seems like a black hole as there is little configuration for this tool:

However, the destination file for this data is defined in the App, which we’ll return to in my next blog post.

Parse the Header rows

As this is an iterative macro, it will run until a predefined condition is met.  In this case the condition is: “is the current page number of results less than or equal to the total number of pages in the full result set”.  This condition is checked within a formula tool, the configuration of which is shown below:

If there are more pages to download, a new URL is constructed with the same format as was used at the very start of the macro but with the “Page” parameter incremented by 1.

You can see from the tool configuration that the formula tool only continues the workflow if the condition is True (the “T” output).  If the condition is False, no data flows forward and therefore no new URL is formed.  In this case, the macro stops and returns control to the App that initiated the macro.

My next blog post will look at how to use the output of a macro within an Analytic App to automatically open Tableau and connect to the data created by the macro.

Resources

The flickr API resource webpage

The getPublicPhotos method webpage