Aral Balkan @aral

Should Data Scientists Adhere to a Hippocratic Oath?

A growing amount of my time is being spent countering Silicon Valley spin:

Aral Balkan argues that an ethics code like that drafted this week could actually worsen societal harms caused by technology. He fears it will be used by corporations as a signal of virtue, while they continue business as usual. “What we should be exploring is how we can stop this mass farming of human data for profit…”

· Web · 15 · 10

@aral Corporations don't need a code of ethics. They need democratic governments holding guns to their heads, ready to shut them down at the first sign of malfeasance.

Getting there, of course, is the hard part.

@starbreaker Boom! (Literally and figuratively) :)

@aral IMO, the US government didn't go for enough when they settled for breaking up the Bell System. (

They should have utterly destroyed AT&T. They should have nationalized the POTS infrastructure, made Bell Labs a civilian R&D agency, and put all of AT&T's patents, copyrights, and other IP into the public domain, including UNIX.

I think this would have solved a great many problems, and prevented future problems by setting a precedent.

@starbreaker @aral I strongly agree that a democracy must have a lot of power to protect itself against corporate misdeeds.

And it's an interesting idea: what if the consequences of violating the public good were to be nationalized with no return for shareholders?

@byronalley @aral

You'd have a lot of conservatives crapping themselves in public over "government overreach" and "punishing businesses for succeeding", for starters.

@starbreaker @aral So you call it capital punishment, for corporations.


@byronalley @aral

I call it bringing back the "corporate death penalty". Apparently corporations used to get nuked for malfeasance on a regular basis in the 19th century, which was why you had really big business concerns like Standard Oil organized as trusts (hence the term antitrust).

@starbreaker @aral It's edgy-trendy rn to rail against "capitalism" but really the problem is we're doing it so wrong. Corporations that have thousands of shareholders and that protect them from ethical responsibility for the corporation's actions while forcing their officers to prioritize shareholder value... That's not inherent in capitalism. It's a perversion.

Limited liability is good for emboldening new business ideas, but it must not be carte blanche.

@byronalley @aral

No, limited liability should not be carte blanche.

As for anti-capitalism being edgy: when I see people with more money than they could possibly spend using that money to buy influence in government, I become a lot less sympathetic to talk of capitalism being "perverted".

Money is power. Power corrupts.
Excessive concentration of wealth is inimical to individual rights and the rule of law, which is reason enough to forcibly redistribute it.

@starbreaker @aral Anti-capitalism doesn't mean "I think wealth distribution shouldn't be too unequal" it means "I don't believe people should be allowed to own things or hire people."

Capitalism is a good compromise among economic systems, but it depends on law to decide the boundaries and details of ownership, which can be done well or badly. Eg. our corporate law: bad.

I agree in any political/economic system we need processes that decentralize power, and social safety nets.

@byronalley @aral Many conservatives and capitalists regard any opposition to the status quo as both a personal assault and an attack on capitalism.

I'm content to use such a loose definition of anti-capitalism because offending conservatives amuses me.

That said, I do think that most things should not be subject to private ownership, and that those who wish to exploit the labor of others for a fixed wage should be made to pay dearly for the privilege.

@starbreaker @aral Well cons and caps are not one thing...

A world without private ownership is an unhappy one. I mean it's the first thing that happens in prison: even "your" clothing is not your own. Humans are territorial. Kids understand ownership first and are then taught to share.

And people who start businesses that employ others *do* pay dearly for the privilege, usually working anywhere from 50% to 100% longer hours than their employees.

Exploitation, well, that's a different story.

@byronalley @aral

> A world without private ownership is an unhappy one.

You're conflating ownership of land, resources, and machinery with personal possessions and living spaces. They're not the same.

Socialists don't want your clothes, or your toys, or your apartment. They don't even want your house, unless you're living alone in a friggin' mansion.

We don't want your cattle. But we won't let you monopolize grazing land so that other people can't feed their herds.

@byronalley @aral

> And people who start businesses that employ others *do* pay dearly for the privilege, usually working anywhere from 50% to 100% longer hours than their employees.

That's not good enough. Wage labor is bullshit.

If you want other people to help you realize your dream, you can damn well take them on as full partners. Give them an equal stake, an equal share of the profits, and an equal say in the decisions -- or cultivate the virtue of self-reliance.

@starbreaker @aral Power corrupts in some ways and less in others--eg. there's a reason for philanthropists.

Simon Sinek has useful points about how we don't mind people having power as long as they're good leaders, and a major weakness of large-scale politico-economic systems is giving power to the ambitious. You don't have to be moral to make money and that's a problem when money is power.

But all systems do that. The key is building better safeguards into the system.

@aral @starbreaker Basically we need:
1) To ensure money doesn't overpower democracy
2) For ill-gotten gains to be forfeited completely (not just wrist-slap fines)
3) Social programs and safety nets
4) Intrinsic systematic limiting of high-end wealth accumulation (taxation being an example)
5) For businesses to actually pay the social/environmental costs they incur
6) Mechanisms of decentralization to be cultivated

@byronalley @aral @starbreaker

The first two seem completely impossible as current systems of government and corporationment exist

the third is being stripped out

the fourth is automatically gonna get vetoed by the rich and the "we're not rich we only make six figures a year" rich

the fifth seems unlikely since the first two seem completely impossible

the sixth is being done in various ways, at least? :\

sorry if this seems bleak, I'm told I can be a wet squib

@byronalley @aral @starbreaker
I tend to favor replace for-profit corporations with worker collectives.

@starbreaker @aral democratic governments don't need code of ethics. They need democratic go- wait this does not work

@digital @aral Democratic governments need a knife at their throats held by the people from whose consent they derive their lawful power.

The only justification for government is that it exists because the people as a whole have delegated limited authority for the purpose of creating a peaceful society where all are protected and can prosper.

@starbreaker @aral we should not need this much violence to keep things running.

look around you, it's not working

@digital @aral

What do you think government is?

Last time I checked, government is nothing but institutionalized systematic violence.

Government should not exist, and neither should corporations, but we're stuck with both.

@aral Actually, there already exists such an oath (see Archimedean Oath here [1]). I guess it is more or less well known depending on the country, but when I finished my MSc, my former university proposed us to signed it.


@Bromind Ah, would they ever use that, given it starts with “I shall practice for the good of mankind, respecting human rights” :)

@aral Supposing you are a proud newly graduated student, still having faith in human kind, I think most people would swear. Imho, universities should promote it, not companies: Hippocratic Oath works pretty well because doctors swear before they enter private companies.

@aral Software Engineering as a profession has a liability issue -- namely there doesn't appear to be any.

If a PE's dam collapses, there can be criminal repercussions and at the very least they can have their license revoked so they can no longer practice their profession.

If a SE's software shows gross security negligence or unethical behavior, nothing happens.

Until the SE takes on personal liability for their decisions they is no real backbone behind them pushing back against employers.


> how we can stop this mass farming of human data for profit

this question seems analogous to "how do we stop microwave ovens from becoming more common"

it's technology

it usually can't be stopped, it can only be bent

in the history of humanity there have been very few examples of a technology that "stopped"

the closest you're going to get is war crime stuff like mustard gas that became off limits as an option

that's about it

this doesn't dissolve lungs, so it's gonna keep growing

@sydneyfalk Modern technology works by collecting data. True. We are not going to change that. Also true.

The question is who owns and controls the data and technology by which it is gathered, analysed, and acted upon?

If the answer is corporations, we have a feudalism.

If the answer is us as individuals, there is no problem. We have a system compatible with human rights and democracy.

Technology is not an objective vector from past to future; it’s character is shaped by ideology & funding.


ah! so the "for profit" was the issue, my mistake and apologies

(I mean, that'll be difficult but I suspect it isn't quite impossible ^_^)