Follow-up
Follow-up Conversation with Jackson Kernion
Jackson writes:
Credit scores, but for online behavior.
So: you try to sign up for some online service, but it checks your score first. Maybe some services only allow you to create an account if you have a score above a certain threshold. Or maybe the service unlocks different priveleges/features depending on your score.
If you 'misbehave' on some service, by being blocked, being reported, or misusing the service, the service can then report back that behavior to the scoring agency. It's up to the scoring agency to determine if/how that purported misbehvior should affect your score.
As I see it, this would modularize and commoditize a key "platform integrity" function, making it cheaper and easier for a broad array of services to secure their platform.
A generic "is this account not a bot/bad actor" score would be widey applicable. But the same basic technology could be used for other kinds of reputation tracking...
To which I reply:
What you describe is in deployed in practice in China. It's fairly dystopian:
- https://newyork.cbslocal.com/2018/04/24/china-assigns-every-citizen-a-social-credit-score-to-identify-who-is-and-isnt-trustworthy/
- https://www.wired.co.uk/article/china-social-credit
The idea, like any technology, is not intrinsically bad. The problem is, such an idea requires protections to prevent individuals from being oppressed by those who control computing power / are controlling flows of resources and access to services.
Corporate interests resist regulation like the plague. And without the right regulations to protect people, this scheme will result in massive disenfranchisement. e.g. In the 80's the way for insurance companies to compete was covering as many people as possible (competing in a zero sum game). In the 2000's, because most people are covered, the strategy has switched to algorithmically reducing liability -- i.e. using computing resources to exclude anyone who is high risk from being covered. It wasn't until recently that the consumer received temporary reprieve via policy which prevents insurance companies from excluding people based on pre-existing conditions. But my guess is (like the 2015 Open Internet Order, established under the same administration) that this policy will also be short lived and succumb to corporate pressure.
The most relevant book on the subject is Jaron Lanier's, "Who Owns the Future" https://openlibrary.org/.../OL16801714W/Who_Owns_the_Future
Tristan Harris describes such scenarios (e.g. those where corporations might access / compute "scores" for people and classify people into buckets) as “a supercomputer [...] playing chess with your mind”. Spoiler alert: you as the patron/consumer don't win.
A potential alternative universe might look like this (pseudonymous identities) The Web in 2040