Follow

New blog post: Cache busting in Node.js dynamic ESM imports

ar.al/2021/02/22/cache-busting

(Includes link to a tiny module I made that generalises this that you probably don’t even need to use: npmjs.com/package/@small-tech/)

· · Web · 3 · 2 · 3
@aral I'm mostly clueless about ESM, but do previous version of the dataset (with a different ?update= date) hang around in some kind of cache, or is ESM smarter about that?

I see you can explicitly delete the cache entry in CommonJS, but I'm wondering whether ESM supports anything like that. (Although I guess with your 227 MB dataset, memory leaks would be pretty apparent.)

@kristof Good question. I don’t know. But I’ll look into it. (Thanks) :)

@aral this is super funny as I was researching this exact issue and was writing up a response here, for updating 11ty (a static site generator) to work with ESM :))
github.com/11ty/eleventy/issue

This is an excellent reference, thank you for writing it up and now I can safely delete half of my work-in-progress comment 😂

@flakoot w00t! Very happy to hear it helped (and my condolences for your half comment. May it live forever happy in half-comment heaven).

@aral also fyi there's a follow-up conversation on the birdsite about this problem space that you might be interested in! twitter.com/slsoftworks/status

@aral This is a neat technique, but doesn't it potentially introduce a memory leak compared to the `delete require.cache` approach? I imagine Node is still caching the old versions of the module and they're never deleted. Probably a very tiny leak, but could matter if the module size is large and the cache is frequently busted.

BTW love your posts on ESM! I learn a lot reading these. 🙂

@nolan I’m actually look into that right now and, yeah, it doesn’t look good.

@nolan Added a note to the blog post about the memory leak inherent in this approach. Sadly, not a solved problem. See: github.com/nodejs/modules/issu

@aral That's really disappointing. It seems like something that should be solved at the spec level, since you might want to clear the cache in both Node and the browser.

@nolan @aral yeah the TL;DR seems to be (based on the conversation in twitter.com/davidmarkclem/stat) is that a large chunk of (/almost all?) use cases seem to be around hot module replacement, and solving that issue specifically can solve the leakage without having to expose the cache.

Sign in to participate in the conversation
Aral’s Mastodon

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!