Content Synchronization SLA
Marfeel uses a complex crawling system to make sure that the mobile version of the content is always synced as soon as possible after the desktop version is updated.
However, as the data is kept in a different database, a certain delay is unavoidable. In this article we explain which are the different service level agreements (SLA) that we can expect depending on the circumstances.
NOTE: All this document guidelines apply just the same regardless of customer having Marfeel's CDN implemented or not.
Summary
| Method \ content Type | Home section | Menu section | Tag | Article |
|---|---|---|---|---|
| Invalidation API | 30s | 30s | 30s | 10s |
| Mrf-Cache-Control meta | - | - | - | 1m (configurable) |
| On demand | 3m30s | 1h (configurable) | 1h | 1m - 1d |
SLA for sections
Marfeel uses 3 different strategies to keep the sections of its customers updated.
Invalidations API
Whenever a customer updates a section, usually by publishing a new article included in it, they can call the invalidations API to request Marfeel to update it. This is the fastest possible way. In this case, Marfeel will:
- Obtain the content from the customer’s servers and transform it into a consumable content by Marfeel. The duration of this may vary very much depending on the customer’s response time, going from 1 second to more than 30 in some cases.
- Purge all Marfeel caches so the new content is spread through the network. This takes around 10 seconds.
Therefore, the SLA for section invalidations using the Invalidations API is around 30 seconds.
This is why many customers integrate calls to the Invalidations API in an automatic way after every publication, and customers using MarfeelPress have this automation embedded in it.
Crawling recurrent jobs
To keep in sync the content of users who don’t use the Invalidations API, we put in place recurrent crawling of their sections (only the ones in mobile’s menu, specified in the customer’s definition.json). The recurrence period of these jobs depend on the section itself and the customer’s Marfeel configuration. The default values are:
- Home sections: every 3 minutes.
- Other sections: every 60 minutes.
We add this recurrence to the previous SLA (as both extraction and purge also need to be completed) and we obtain an SLA of 3’30’’ for home sections, and 1 hour for the rest of them. This values, though, are configurable for each different section.
Dynamic sections / Tags
Dynamic sections, as they are called in Marfeel, are the ones that are defined using a URL regex, such as domain/topic/*. An example for these would be the usual author pages, or Wordpress tags.
For dynamic sections, they will be updated whenever a user requests them and it has been more than 1 hour since last update.
This makes an SLA for dynamic sections of 1 hour.
SLA for articles
The system to update articles is a little bit more complex than the one for sections, but we also have 3 different strategies for it.
On demand
Articles may be invalidated whenever some user requests them. This happens of course when an article is brand new, making it a 0 sync time for new articles.
After the first article extraction, the probability that the next user request will force us to update the article will depend on whether we detect changes in the article itself or not. If an article does not ever change (vast majority of the cases), this “recurrence” will diminish until we only update it once per day. On the other hand, if the article changes constantly, we will keep updating it as frequently as once per minute.
Summarizing, article updates SLA is somewhere between 1 minute and 1 day, depending on the article’s age and whether this article changes frequently or not.
Invalidations API
Whenever our ‘On demand’ strategy does not fit into a certain content needs, for example in the case of an update for an old article where we don’t want to wait for hours for it to be synced, customers may again use the Invalidations API to force an almost instant sync. In this case, we still need to add both extraction and purge steps, but article extractions take normally much less than section ones, usually less than 1 second. This brings SLA for articles using Invalidations API to around 10 seconds.
Also, whenever the Invalidations API is used upon an article, its recurrence for “On Demand” strategy is restarted and therefore subsequent requests should trigger frequent updates as well.
Finally, keep in mind that MarfeelPress customers have Invalidations API automated for articles as well, triggering it every time a change on an article is detected.
Meta Mrf-Cache-Control
Whenever customer plans to frequently update an article, for instance, an article of a football game minute-by-minute result, they can instruct Marfeel to increase the crawling frequency of it. This is done by adding the meta Mrf-Cache-Control to its HTML:
<meta name="Mrf-Cache-Control" content="60" />Value in 'content' is configurable and in seconds, but Marfeel does not support values lower than 60 seconds.
Everytime someone demands the article, if time since last refresh is higher than the configured value, Marfeel will crawl it again, making it a configurable crawling time.
Warning: Setting a low value for many articles may cause Marfeel to crawl the client's servers too frequently and put them at risk.