Insight is a very complex platform and tool. While its main objective is to remove noise and complexity for publishers, there's a lot of challenges that comes with normalizing massive data sets from numerous data providers because each has their own reporting schedules, metrics, etc.
To provide more transparency on what goes on in the background so publishers can have a normalized view to fully understand performance, the following sections explain different technical details on how and when Insight collects data and the challenges that come with it.
For engagement metrics, Insight operates according to Central European Time (CET - Paris, Madrid, etc.).
For revenue metrics, Insight operates according to Pacific Standard Time (PST - Los Angeles, San Fransisco, Seattle, etc.).
Marfeel aggregates the revenue of each revenue stream respecting its specific timezone. Revenue metrics are not normalized according to PST. This is because revenue streams don't provide hourly revenue updates and timezones can't be shifted.
This can create minor mismatches at times when calculating ARPU (average revenue per user) because revenue from one timezone is divided by visits from another timezone. It's also a by-product of working with a myriad of revenue streams who themselves don't even have all the right data points to normalize metrics.
While a mismatch in the data might exist at low-frequency levels (for example, when looking at one day), this variance does not exist when looking at longer periods of time (for example, weeks and months).
Insight queries and retrieves data from revenue streams every 15 minutes for the current and previous day. That means that 24 hours after closing a day, that day's revenue metrics are still subject to change as Insight will continue to query for and collect data into the next day. For example, the revenue for the 1st of the month can continue to be updated well into the the 2nd of that same month.
Normally, during the first hours of the day, users can see the major moments of revenue generation for the previous day, with cents being added as the previous day continues to be updated.
Revenue stream reporting delay
Theoretically, all revenue metrics should be up-to-date. However some revenue streams report with a delay. For example, when queried at 6 pm, some revenue streams will only provide the metrics from 2 pm. This is a persistent issue with the majority of revenue streams within the industry with only the length of the reporting delay varying between them.
Some revenue streams even have a 24-hour delay in reporting – without providing updates throughout the day – that further complicates the issue. That means that revenue generated on the 29th of a given month is not available until the next day – the 30th.
While Marfeel can accurately anticipate and predict revenue streams' reporting patterns, there have even been cases where a revenue stream has not provided reporting data for up to 4 days, thereby introducing unpredictable reporting patterns and adding complexity to collecting and normalizing metrics.
On the 1st of every month, Marfeel also re-queries all revenue streams regarding the previous month, normally adding positive corrections. It's not uncommon to have several Euros or USDs added due to this re-query simply because revenue streams report this with a significant delay.
Geos and channels
In some cases, publishers will see mismatches when drilling into their revenue and engagement metrics based on geos or channels.
The reason for this is because revenue streams don't provide this level of reporting. These consolidated metrics don't come from revenue streams or analytics providers; they are not from raw and pure sources such as visits which come from a specific analytics vendor or revenue that comes from a revenue stream.
Metrics for geos and channels are essentially collected from DFP, yet there is still a lot instrumentation that happens in the background because these figures from DFP are not absolutely precise and reliable. For that reason, Insight crunches massive data sets and performs data extrapolations based on key-values to provide this feature to publishers at the expense of absolute precision.
Other major data providers are afflicted by the same issue and challenges and provide even larger discrepancies.
However this feature is designed to be indicative so publishers can see a clear trend and get the insight of how they are performing based on these dimensions. In that regard, while there might be slight mismatches, the indication and trend is accurate and provides the insight that empowers decision making.