Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.ar.al is one of the many independent Mastodon servers you can use to participate in the fediverse.
This is my personal fediverse server.

Administered by:

Server stats:

1
active users

mastodon.ar.al: AboutProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.4.3

Aral Balkan

New blog post: Cache busting in Node.js dynamic ESM imports

ar.al/2021/02/22/cache-busting

(Includes link to a tiny module I made that generalises this that you probably don’t even need to use: npmjs.com/package/@small-tech/)

ar.alCache busting in Node.js dynamic ESM importsI’m porting JSDB to EcmaScript Modules (ESM) and one of the issues I had to look into was module cache invalidation. JSDB is my little in-memory native JavaScript Database that writes JavaScript operations to append-only JavaScript logs that have UMD headers. And it loads in these tables either via a dynamic require() call, or, for very large tables, by streaming them in and evaluating them line by line1. When a table is loaded in, it is stored in the require cache.
#esm#js#NodeJS
··Web
2boosts·3favorites
Kristóf Marussy
@aral I'm mostly clueless about ESM, but do previous version of the dataset (with a different ?update= date) hang around in some kind of cache, or is ESM smarter about that?

I see you can explicitly delete the cache entry in CommonJS, but I'm wondering whether ESM supports anything like that. (Although I guess with your 227 MB dataset, memory leaks would be pretty apparent.)
Aral Balkan

@kristof@pleroma.marussy.com Good question. I don’t know. But I’ll look into it. (Thanks) :)

Nolan Lawson

@aral This is a neat technique, but doesn't it potentially introduce a memory leak compared to the `delete require.cache` approach? I imagine Node is still caching the old versions of the module and they're never deleted. Probably a very tiny leak, but could matter if the module size is large and the cache is frequently busted.

BTW love your posts on ESM! I learn a lot reading these. 🙂

Aral Balkan

@nolan I’m actually look into that right now and, yeah, it doesn’t look good.

Nolan Lawson

@aral That's really disappointing. It seems like something that should be solved at the spec level, since you might want to clear the cache in both Node and the browser.

Flaki

@nolan @aral yeah the TL;DR seems to be (based on the conversation in twitter.com/davidmarkclem/stat) is that a large chunk of (/almost all?) use cases seem to be around hot module replacement, and solving that issue specifically can solve the leakage without having to expose the cache.

TwitterDavid Mark Clements on TwitterBy David Mark Clements
Flaki

@aral this is super funny as I was researching this exact issue and was writing up a response here, for updating 11ty (a static site generator) to work with ESM :))
github.com/11ty/eleventy/issue

This is an excellent reference, thank you for writing it up and now I can safely delete half of my work-in-progress comment 😂

github.comModify Eleventy to work with ECMAScript Modules (ESM) by default · Issue #836 · 11ty/eleventyNode 13 projects can switch to ECMAScript Modules using "type": "module" in package.json, using .mjs files, or --input-type https://nodejs.org/docs/latest-v13.x/api/esm.html#esm...
Aral Balkan

@flakoot w00t! Very happy to hear it helped (and my condolences for your half comment. May it live forever happy in half-comment heaven).

Flaki

@aral also fyi there's a follow-up conversation on the birdsite about this problem space that you might be interested in! twitter.com/slsoftworks/status

Twitterflaki on TwitterBy flaki
TrendingLive feeds
About