Cache-busting and assets

Hi

I am using assets to reference css and js files.
{% do assets.addCss('theme://css/styles.css', 100) %}

I would like to force the clients to reload these files when I modify them (cache busting).
For this I added a version number in the file name:

<link rel="stylesheet" href="style.123.css">

Then, in order to keep my physical files without version numbers in their name, I added a rule in my .htaccess:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+).(\d+).(js|css)$ $1.$3 [L]

This is working well when writing "<link rel="stylesheet" href="style.123.css">" directly in my twig templates, but it is not working when I write "{% do assets.addCss('theme://css/styles.123.css', 100) %}" because the asset manager needs to confirm that the file exists (and the physical file doesn't the version number).

Is there any solution?

Thanks

I don't understand the problem. When you update the file, Grav should automatically invalidate the cache for that file. You shouldn't have to do anything weird. I certainly never have problems when I'm developing a theme, for example. I update the CSS file, refresh, and the changes are there.

The nuclear option is to bin/grav clearcache after you update the CSS file, but that shouldn't be necessary.

Perlkonig

Thank you for you reply.

I may have missed some Grav options but how does Grav (server side) force refresh the CSS on the client browsers cache? I do have the problem that when I update a CSS file I have to force refreshing the page (CTRL+F5) to see the modifications.

In my present configuration (which is the default), I don't see any query strings appended to the css file names in the HTML produced by Grav. In system.yaml, there is an option "enable_asset_timestamp" which allows to bust the cache by adding a timestamp to the asset names, but this is too aggressive and I would rather control this on a version basis.

You can read about the different cache busting techniques on this page: https://css-tricks.com/strategies-for-cache-busting-css/

You only have to Ctrl+F5 because you're developing in real time. Assuming someone doesn't visit your site multiple times a day, this just shouldn't be a problem. A well behaving client will check the freshness of its cache periodically. A simple HEAD request will show that it has changed, and it will refresh it.

These sorts of cache busting techniques are (in my opinion) rude, for lack of a better word. They subvert the whole purpose of the standard and of caching in general. CSS files are hardly critical. The client should have the right to choose how frequently to refresh their cache. And if they see the site isn't looking right, they can Ctrl+F5 themselves and refresh it directly.

Whether it is "ethical" or not to do cache busting is another discussion.
In my opinion, the <link> html tag lacks of some kind of timestamp attribute that would indicate to the browser if it's necessary to request the CSS via HTTP.
I think this would be smoother for both the client and the developer.

Now I am a bit confused about what you are saying because that is a bit different to what I observed.

@Perlkonig:
A well behaving client will check the freshness of its cache periodically. A simple HEAD request will show that it has changed, and it will refresh it.

Are you talking about the browser here? I often see cases where the cache is not refreshed for several days.

@Perlkonig:
The client should have the right to choose how frequently to refresh their cache. And if they see the site isn’t looking right, they can Ctrl+F5 themselves and refresh it directly.

You also used "client" in this sentence, but I guess you refer to the user here. Well, I think that many users don't even know about "Ctrl+F5". And even if they know it, is it their role to investigate whether the styles look right or not? This is assuming they know how the styles should look... (Btw, I know users who would clear the whole cache of the browser, which in this case would certainly subvert the purpose of caching). I think it's fairly reasonable to assume that the developer, who edit the files, is the best placed to indicate when a new version is published. Of course this requires some self-discipline to do a proper versioning. On the other hand, the grav option “enable_asset_timestamp” is (in my opinion) not the good policy as it doesn't reflect the changes made on the files.

To go back to the original question: How can I solve the issue of the first post if the file "styles.123.css" doesn't exist physically?

I apologize for sidetracking the conversation. I do not know of a native way to make Grav recognize these "virtual" files. If your method is deterministic, then you could maybe create a plugin that does some mapping on the back end. Or if you just create the version numbers manually, then maybe the plugin could consult a map file or something. Having never done this myself, I don't know off the top of my head. I'd start with the Grav Lifecycle (in particular the onAssetsInitialized event) and then look at other plugins to see what sorts of things others have done with that event. And maybe someone else will reply to this post.

Ignore this paragraph if you wish. Just responding to your questions: "Client" refers to the software—the browser. It comes with certain defaults and can be configured by a user to refresh the cache after different periods of time. My only point is that it's not the server's role to force things on the client. The HTTP standard makes mechanisms available to signal to the client that resources have changed. A "well-behaved" server leverages the standard and lets the client do its job. There's no need for a "timestamp" attribute, for example. The standard has ways of managing this very thing. You can set the cache-control header, for example, to suggest to the client to check more often than it normally would. I was just suggesting that maybe there are simpler solutions to your problem.

Thank you for your reply. It's not a problem at all to drift a bit from the original question. I think the cache discussion is actually very interesting.

I get your standpoint that the server should not force the client to update stuffs. I agree that always forcing to load CSSs or arbitrarily force at regular interval would be bad.. I was more thinking about a way to indicate to the client when a new version of the asset is published (it is always more efficient to inform than having to check multiple times at regular interval). If there was a timestamp (or a version) attribute in the link tag, the browser would not need such thing like regularly send a HEAD request to the server to check the file's status. It would simply compare the timestamp (date of the new version) to the last loaded time and update it if necessary. This would btw be a better technique than what I want to do because then the file could be replaced in the cache instead of cluttering it.. but anyway it was just a thought and it doesn't exist.

Of course the client (the browser) can surely be configured, but most users won't do it and might not get updated for several days when a new script or CSS is published. Updating the cache-control header to check more often would make the problem "less worse"... but would also increase unnecessary requests in normal time (when nothing is updated).

I will look at the onAssetsInitialized event. Developing a plugin might be interesting. I heard that WordPress natively has a way to control cache and assets versioning via query strings, and I think that could be a good thing to have a similar mechanism in Grav.

@mathmax I may be mistaken, but it seems that server side caching by Grav and client side caching by the browser are a bit mixed-up in the discussion.

TL;DR: Enabling Grav's Assets pipelines leads to cache busting.

I guess the OP is talking about busting the browser cache by renaming the resource (js, css), or by adding a querystring when a resource has changed. When the page is refreshed, the document contains references to the new resources. The browser can no longer use the css/js from cache and has to request the new css/js file from the server.

When enabling Grav's Assets pipelines, cache busting will happen automatically. When the css/js pipelines are enabled, Grav will process the resources and if any file has changed, a new filename will be created. This will force the browser to fetch the new css/js file when the browser has refreshed the page from the server.

Renaming files are just one part of the whole. Cache expiration, caching directives and cache validators/validation are another part.

The following are a few good reads:

• HTTP caching – Ilya Grigorik. This is a short-read providing a good overview and also discusses the OP's question.
• HTTP caching by Mozilla Developer Network
• Caching Tutorial – Mark Nottingham
• RedBot, a tool to check your cache-related HTTP headers.

Hope this helps.

I want to note, that I've implemented multiple solutions for cache busting with grav here:
https://github.com/getgrav/grav/issues/1330#issuecomment-752031229