I’ve heard the phrases full refresh and incremental refresh being bounded about and I didn’t really have a full understanding of what the differences were between the two until our server refresher today with Jmac. A refresh refers to the refresh of a data extract on the server. This is great for a variety of data, for example sales data, so when new data comes in, it can be automatically embedded in your viz. For example if you use MAX DATE, when new values come in it will automatically refer to the latest date in the data set without you having to manually select it.
When you publish a data source to the server you get an option to refresh the data from the database and you get two options; Full refresh and Incremental refresh.
Full Refresh
- Full refresh will send a query to the database to return ALL EXISTING ROWS of data from the previous query as well as the new rows
- This will take longer than an incremental refresh
- Will look at changes in the underlying data as well (for example if a sales value was to change in your data, the full refresh would pick this up)
Incremental Refresh
- An incremental refresh will feed in ONLY NEW data in the data set
- This is usually a lot quicker than a full refresh as it is not querying data already in the extract
- This will not look at any changes in the original data set, it will only load up new rows of data entered into the database
- To learn more about an incremental refresh be sure to check out Micol’s blog.
When should you use what?
Use a full refresh if your data could change at any point and current values could be subject to change. It’s a good idea to run a full refresh once a week (possibly over the weekend) to ensure that any changed data or deleted rows are captured within the refresh.
Use an incremental refresh if your current, existing data is not subject to change and you’re only interested in bringing in new rows of data. An incremental refresh is also quicker so it could be run once a day in the early morning to ensure that your daily reports are showing the most up to date figures.