“Cache Everything” On Cloudflare The RIGHT Way

Everyone knows that CDNs are only for static content. CSS, JS, images, and font files for the most part. The “static” is what makes CDNs efficient. The idea is that a piece of content needs to be retrieved only once per EDGE server (not once in general), and then users are served a copy of that file instead of from the origin.

It takes a while to “prime” or “warm” all the EDGE nodes – especially if your CDN provider has hundreds of servers all over the world. Each of them needs to be hit with at least one request for each static file before it’s available in the cache. This can take some time. Let’s see how much.

Requests Needed to Prime the Cache

Let’s say your CDN provider has 150 POP server locations. It’s not an unusual number. And let’s say you have 20 static files (a very conservative number). The total number of “unprimed” or “uncached” requests will be:

150*20 = 3000 requests

So 3000 requests will be served from your origin. Keep in mind that those 3000 requests will be slightly slower than a normal request because it’s going through an additional stop point along the way. Based on your traffic, it can take several days before you see the impact of a CDN on your site performance!

And if you ever clear the cache completely – say you change your CSS files, the whole process will start again.

Moral of the story: Don’t clear your cache often!

That’s Why Cloudflare’s “Cache Everything” Generally Doesn’t Work

Cloudflare is a very special CDN. It has a unique “cache everything” page rule, which includes the dynamic HTML. That drastically improves your Time to First Byte (TTFB) and server response times.

But only if your pages are already being cached at that particular EDGE server.

The previous calculation assumed only 20 static files. Lots of websites however, have hundreds of pages. And if we wanted to prime 150 POP locations with 400 page – well, that’s 60,000 requests! And considering that all CDNs purge their caches automatically when a file hasn’t been requested from a particular EDGE server for a while…you can see that the “cache everything” option can be a disaster.

Use Case Scenario for “Cache Everything” on Cloudflare

I’d explained in more detail in an earlier article why Cloudflare’s “Cache Everything” isn’t a good idea. And if you just turn it on and make it cache every page, you’ll have a pretty bad experience. Not to mention horrible problems with anonymous visitors accidentally seeing admin bars etc.

But it can work nicely in a specific situation.

Let’s say you have ONE page on your site that generates most of your revenue. Or all of it! One basic landing page which is the cornerstone of your revenue strategy. It need not contain the shopping cart itself but can lead to it. Or it converts really well.

I’ve found that we can speed up the delivery of this single page by using Cloudflare “Cache Everything” page rule. It will work under the following conditions:

1. Contains static content that’s the same for all anonymous visitors
2. Doesn’t change very often

Priming the Cache Efficiently

But sometimes the page will change. Let’s say once a week you make an update to it. We need to prime all the EDGE servers. What I do is I subscribe to Cloudflare’s “Argo” service which uses tiered caching. This means that under some circumstances, Cloudflare will be able to get a copy of your page from nearby POPs instead of directly from the origin server. This helps with the priming problem.

In addition, after making a change to my page, I log onto my VPN and ping my site from 6-7 locations around the world – one in each continent at least. I usually choose:

1. US
2. South America
3. Europe
4. Middle East
5. South Africa
6. India
7. Hong Kong

This way, I get my page onto all distant caches, and maximize the changes of Argo’s tiered cache system picking up my page from one of these POP servers instead of from my origin. After doing this for a while, my server response times and TTFB have increased dramatically!