Node.js devs, imagine npm doesn’t exist. Instead, if you’re nice, you make a list of your dependencies in your README and everyone installs them manually. (Seriously.) Also, you can list names any way you like. So, if a library is called libclutter-1.0-dev, list it as clutter, libclutter, clutter-1.0, … and people will magically know you meant libclutter-1.0-dev (or, more likely, they’ll do an, e.g., apt search clutter to try and find out wtf you meant). Welcome to C/Vala development in 2021.


· · Web · 7 · 8 · 17

(Oh, and every library you install is essentially a global install that is shared with every project you’re trying to build. So basically everything is ~ an npm install --global.)

This is really something an operating system like elementary OS has to tackle for its apps at least if it wants to attract developers (even if, say, the GNOME, etc., folks are happy with the status quo and/or enjoy the right of passage it represents for keeping out the hoi polloi).

(Ironically, I just had to manually install two dependencies to compile a work-in-progress(?) Vala dependency manager app.) :)

@aral Which one was this, if I may ask? I’m guessing Vanat. Do share your thoughts on it.

I agree that a simpler dependency management solution would really help Vala. Flatpak kinda solves this for whatever can be Flatpak’d at all. But if something needs to e.g. install a system service or access the list of processes, Flatpak is not applicable.

Having something as simple as npm is a huge undertaking, because it requires either centralizing all packages into a repo or supporting pointing to git URLs and building those — which raises the question of how to build those, since they all use different build systems.

There are a lot of moving parts, and not a lot of coordinated effort going on in this direction.

I don’t have a point here. Just sharing some thoughts.

CC @colinkiama @nahuelwexd

@dubiousdisc @aral @colinkiama Something like this is what I was talking about with Prince a while ago on Vala's Discord server. While it is true that within the community there are very few resources, we need an NPM-style package manager for Vala

Maybe it doesn't even need to be centralized, maybe just taking the URLs from the GitHub repositories might be enough, like Go does, but we need to do it.

Otherwise, more painful experiences will be shared by devs who try to use Vala

@nahuelwexd @aral @dubiousdisc We need a proof of concept ASAP!

I’m all for improving the developer experience but maybe there’s something we’re missing here. Why do some Vala developers feel hostile about package managers?

@nahuelwexd @dubiousdisc @colinkiama Good to hear there’s thought and discussion around this. Any thoughts on jhbuild?

Would be great not to create yet another standard that relies on Microsoft (perhaps git URLs instead of GitHub URLs?) At this point I’d probably settle for a simple standard that just apt installs the dependencies even. (Although, ideally, a more robust solution would keep a separate dev environment to your system environment.)


@aral I would prefer that you could join the server and be able to chat in a more fluid way, although you may not like the idea as Discord is proprietary software

We have been discussing more in depth today together with @dubiousdisc and @colinkiama and have come to some pretty good conclusions IMO

Maybe later with more discussions we can get something solid (like a spec) and see to start working on it. Ideally it would be nice to reuse Vanat's work, so @colinkiama left some issues there

@aral Obviously all this is done in each one's free time, so... It will take a while. For this reason, the more people helping, the better :)

@nahuelwexd @dubiousdisc @colinkiama Cool, will do. I’ve learned to hold my nose and use Discord when necessary :P

@dubiousdisc @colinkiama @nahuelwexd Yeah, vanat. Couldn’t get it to run with a simple case.

Since all app distribution on elementary OS is Flatpak, I wonder if that’s just the way to go. Use Flatpak for everything, including during dev…

… I’m looking into that but it seems (a) slower (compared to just using ninja/meson during dev) and (b) seems to be harder to debug / integrate with IDEs (like VSCode). On the other hand, your dev env matches deployment + only one tool is simpler conceptually.

@aral @dubiousdisc @colinkiama To make developing with Flatpak effective, you need an IDE with built-in support

I use GNOME Builder :)

@nahuelwexd @dubiousdisc @colinkiama I’m going to try out a Flatpak + Builder workflow for what I’m working on. It Flatpak works fine for dev with Builder, that might be the route to recommend for elementary OS app dev. (Will also look into integration with VSCodium but didn’t look trivial at first glance to get debugging, etc., going. Still on holiday so will have a better look in a couple of days.)

@aral That reminds me of two months ago when i tried to make a local build of Geary to maybe do some visual tweaks. I just couldn't get it to work because of dependency hell / version mismatching, gave up after two days.

@ff0000 I’ve got a better one for you: I got Geary to build and added Fastmail to the list of accounts you could set up easily alongside the surveillance-based Gmail, etc. Pull request was denied. Still legitimises/encourages people to use Gmail to this day. (So maybe you dodged a bullet?)

@aral Sometimes i think people just don't want to even (simply) offer alternatives to people. Not even when they come in the form of PR's.

@ff0000 I’ve been told it has nothing to do with Google being a sponsor of GNOME.

@aral They couldn't see the PR approve button because of the huge wall of Google money

@ff0000 @aral thats really sad. i liked my fastmail account when i had it.

@aral @ff0000 What was the reason for denial of that pull request? Surely it cant be that they only want to support gmail?

@aral @aramloosman ooof a lot of words just to say: neh, just make so that *insert small ethical company*'s integration is just as easy as multi billion company with endless money Google.

@aral @aramloosman @ff0000 @rra That was frustrating to read. Sure, adding easy selection of competitors does nothing to boost competition, okay.

Google adding DDG and Qwant as alternative search options to Chrome, and Apple adding alternative browser options to iOS, must have had nothing to do with antitrust legal action closing in.

Because who would ever think making competition accessible helps reduce anti-competitiveness?

Also perplexing why MS are making it harder to set a new browser. 🤔

@aral @ff0000
Yeah thats uncool :/
To be fair to the dev, I would also say that if one would start to add more providers than 3+more, one would need to redesign the page.
But I completely agree that it is not a solution to simply deny your work on it because it would question some previous work. Otherwise when will anything ever change?

@krag @celia @ff0000 Interesting. I wonder what’s common with these types of pull requests 🤔

@praveen Aha, thanks! I wonder why I’m not seeing it used more. Will have a play :)

@aral Remember that Unix systems are not Javascript to be managed by NPM.

Dependency section should not be for end users, but rather for either packagers or other developers. And you should install those packages. Or, idx, try Nix.

C doesn't need it, most systems don't need it. I don't know what Vala needs

@yyp You say that and all I hear is C/Vala/elementary OS doesn’t need developers.

At least for the last one, given the state of the AppCenter, I tend to disagree.

@aral The solution would be to fix AppCenter ¯\_(ツ)_/¯

@yyp What is the problem you see with AppCenter? As far as I’m concerned it’s the only solution on a free/open operating system for potentially having a sustainable app ecosystem with independent devs in the future. (Sure, there are issues, like it being at the mercy of Microsoft currently) but if you know of any better alternatives please do let me know as I’d be very interested in trying it out.

@aral i know this is meant to be about how shitty c dev is; but this honestly sounds like a huge improvement. npm is a trashfire and a more informal dependency management situation would force library authors not to break api compatibility every 3 seconds.

@zens Odd, my experience with Node.js over the last seven/eight years has been: clone repo, npm install, run app.

My experience with c/Vala in 2021 (last time with former was when I was 8/9 years old) so far as been: clone repo, ffs (try to figure out/install the fucking depedencies), maybe run app.

@aral my experience has been; clone repo; npm install; 300 security bugs! please use fix audit!
mutually incompatible dependency tree; plesse fix by making upstrram pull requests ro random ass libraries

confflicting versions!

did you install this with a version of node that’s too new? use nodemv to install older versions

whoops, the library you just fixed needs a newer version.

the webpack config format is different in from the one you installed!!!!

@aral i ince got to the bottom of this shit pile and the newest version of webpack changed its format *again*

@aral so; i don’t know what crazy magic version of node or npm you’re using is; but that frictionless experience you’re talking about has never once happened for me.

@aral at least with C; the number of dependencies is usually counrable on two hands

@zens @aral C/C++ projects with a large number of hyper-specific deps are code smell for badly engineered software. Or a web browser.

@zens @aral Maybe sub NPM for Cargo. Node is a unique form of hell on earth, but Cargo works broadly similarly to NPM with far fewer of the footguns and annoyances because the language is a bit less volatile (a bit).

@klardotsh @zens I can count on the fingers of one hand where I’ve had an issue with npm in seven/eight years of Node.js development. But sure, npm/cargo/you get the idea :)

@aral I realize this was already mentioned to you in another thread, but GNU Guix and Nix do exactly this for C (and any language really). I highly recommend giving them a try. My experience with Guix has been extremely pleasant, and I’d be willing to work with you on it if you need pointers.

it almost sounds like you're arguing for centralized package management, naming and distribution.
freedom and decentralization promote diversity, which brings some complexity with it, but I still like it better than the alternative. indeed, IMHO such centralized package management systems and easy dependency management are an easy way for you to have no clue as to what your program really depends on, promoting technical and legal insecurity

@lxo We already have centralised package management (what’s apt?) What I’m arguing for is a standard for dependency management for development to simplify elementary OS app development and to get more developers contributing with new (and to existing) apps. Doesn’t have to centralised. In fact, all the better if not. But anything is better than nothing.

it's not really centralized. apt, yum, pacman, etc repositories are a bit like nodes in a federated social network. some are more popular than others, some won't federate with others, and though one has authority over the node one runs, there's no single central authority on package naming, or packaging standards, and that's exactly where the complexity you complain about comes from: different distros, even ones using the same packaging format, may name packages differently, have different package versioning standards, carry different compat packages (if any), etc. that's diversity that follows from freedom

@aral I agree. The status quo is pretty horrible.

As @robby already mentioned though, #nix (and #guix but I haven't tried it) kinda solve this problem without the need to develop yet another package manager.

I recently started working on a big corporate project with a very complex dev environment. Since nix is being used though I actually didn't have to setup anything. Just entered the repo directory and everything set itself up (after I allowed it) automagically. Pretty awesome.

@AbbieNormal That was a good read. Thanks.

Personally, I think #npm is the absolute worst of the mainstream package managers. It's not uncommon to see 30.000 dependencies in an npm project. That's before de-duplication. But even the de-deplucated list is usually massive and what is all the duplication there for in the first place anyway?

@iooioio @AbbieNormal

It’s not uncommon to see 30.000 dependencies in an npm project.

Wait what. That’s madness. Do you have examples?

@robby Good thing you asked. Looks like I probably got something wrong there.

One package with lots of deps is react-native-template-vife. When resolving the dependencies with yarn I saw a progress bar that went up to 28k or something like that. I simply presumed that was the number of dependencies. Doing `yarn list | wc -l` merely yields 4176 though.

webche also looks pretty big. But I can't get the deps to resolve right now because "linux is unsupported" or something.

@robby @AbbieNormal @iooioio If you want to quickly throw together a React app you'll run the boilerplate maker create-react-app.

That's 1956 packages deduplicated, but de-deduplicated it comes out to around 3300, within the same order of magnitude.

If you start at 2000 packages to dynamically generate some pages, it's not hard to imagine that adding some integration with this, some fancy widgets for that will easily drag in a few thousand more.

$ npx create-react-app app npx: installed 67 in 17.02s Creating a new React app in .../app. Installing packages. This might take a couple of minutes. Installing react, react-dom, and react-scripts with cra-template... [ . . . ] + react-scripts@4.0.3 + cra-template@1.1.2 + react@17.0.2 + react-dom@17.0.2 added 1913 packages from 718 contributors and audited 1913 packages in 320.486s [ . . . ] + @testing-library/jest-dom@5.14.1 + @testing-library/react@11.2.7 + web-vitals@1.1.2 + @testing-library/user-event@12.8.3 added 43 packages from 86 contributors and audited 1956 packages in 90.003s [ . . . ]

@iooioio @AbbieNormal @aral @robby

if only the tiny company that owns npm didn't directly benefit from the complexity of it by being able to sell solutions for it. 🙃

@iooioio @aral @robby

Fascinating analysis

But I disagree with the final remarks

In my opinion we should try to avoid this mess

Introduce more semantic information in the html so less Javascript is needed, use native apps

The web is lost

@AbbieNormal I agree with the final remarks of the article. Be the change. Anyone is free to start publishing things that have fewer dependencies and do something relevant. Dare to refuse depending on 'odd' and 'even'.

The difference between pypi and npm is mostly cultural, even though I agree with the analysis that little nudges in the tools contributed to those cultural differences.

@aral @iooioio @robby
Sign in to participate in the conversation
Aral’s Mastodon

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!