Filter support for time-series data 💪

We are happy to bring a significant data overhaul that we have done to time-series data sources. We give you fully accessible, per post, and filterable time-series data. Why is this huge? Let's find out.

Humble beginnings

The sheer amount of data queried in our platform at each given moment creates challenges when it comes to the loading times and performance. The time-series data was most affected, so we have decided to propose a solution - aggregation tables. In a nutshell, instead of getting information on the posts, we stored the critical metrics with timestamps in those aggregation tables. This was a perfect solution for many years, but when there is progress, there are welcomed changes and challenges - we have introduced powerful post tags and filters.   

The aggregation tables were a great solution at the time. Still, with the introduction of tags and filters, it was impossible to add this functionality to those tables as the aggregation concept works on the number of posts and key metrics and not on the individual posts themselves. That meant that metrics based on aggregation tables could not take advantage of the advanced post filtering when needed. 

Over the last few months, we have improved our data infrastructure, which allowed us to revisit the topic of how to support filters for time-series data. Today we can happily say that we finalized it.  

What does it mean to you?

Simply put, any metric based on post data can now be further filtered down by our advanced filters in real-time. Would you like to see how tagged posts, hashtags or mentions did over time? You have a social media campaign running, and you want to do competitive benchmarking on top-level KPIs? Now you can. In terms of functionality, nothing changes. You can still use and build the same metrics, with the difference that popular metrics like "Key Metrics Table" and "Number of Posts", if there are filters applied, the total aggregated results will show only the filtered posts where requested. This change will allow you to build even more robust dashboards that can start with the most generic KPIs, drilling down to post level to tell the full zoomed-in story. It is worth mentioning that this also applies to our auto-tagging filter.

There are a few small prints that we need to list below.

• This change only applies to post related metrics as it operates on the post titles and descriptions, so followers and user metrics won't be affected.
• Filtering by tag won't be available for User Generated Content (members posts on your timeline), but the post text filter will. Our system will recognize this for you and present it accordingly.
• Due to the sheer amount of data that has to be parsed ad hoc, in a very rare scenario (we are talking hundreds of profiles with thousands of posts) that the generation of the result might take more than 20 seconds

What the future holds

This update was an important missing piece to provide you with genuinely consistent data. It also standardizes all the processes within quintly and removes unnecessary maintenance. Our advanced filters will only get more powerful over time (we have a few things planned), so this alignment is really welcomed. Most importantly, this improvement allows you, our users, to zoom in and filter out the noise where it matters the most. We cannot wait to see what you come up with and how you will use it. That's our biggest win.