A Coup: Universal Messaging Protocol for All

2018-08-05 Sunday

AOL Instant Messenger (AIM) was one of the first systems to make chat truly ubiquitous and mainstream on the Internet. It pioneered and mainstreamed a handful of important concepts (much on the coattails of irc 1988), like the chat room, the buddy list, emoticons, and bots (the ability to program interactive chat bots over TOC2).

AIM also did something else amazing -- as of 1998[1] they made your data accessible in Jabber/XMPP[2] (Extensible Messaging and Presence Protocol) format (xmpp.oscar.aol.com). XMPP is a mouthful, but its actually a *very* special internet standard. It's ostensibly a contract, that any chat server (e.g. MSN, Gchat, AIM) which agrees to use it is making their data available in the same, interoperable way. This is kind of like how email works. Facebook chat and gmail, for instance, are not interoperable (you can't send facebook chat to someone's email), but you *can* have a yahoo email address and/or a gmail address and they can all send messages to each other. This is because mail services are similarly interoperable; they use a protocol which predates XMPP called IMAP & POP3. Text messaging operates similarly and uses a standard called SMPP which powers the phrase you've likely heard SMS.

Because several chat services (aim, msn, etc) all started making their service(s) available in XMPP, open source programmers like you started creating their own chat interfaces capable of logging into any and all of your chat services / identities at once (e.g. the "Pidgin" client) and merging them into a universal chat interface. Now you only needed one app to talk to all of your friends.

It wasn't all fun and games[3,4]. MSN and AOL quickly experienced the contradiction between the pressures of adopting open standards and "protecting" (retaining) their users from being siphoned by competing services.

Today, said generously, chat interoperability has fallen by the wayside. If not for policy like GDPR, users would struggle even access complete data dumps of their facebook activity. The general trend is, the number of winners online in shrinking, the number of apps being downloaded is falling[5], and we're living a zero sum race for people's attention.

People who choose google (gchat, gmail) I imagine are interested in the interoperability between their services (google offers docs, mail, messaging, video calls). People who use skype likely have similar tie-in to the microsoft suite of tools. People who use whatsapp care about text/msg interoperability and encryption, but it doesn't really matter given its owned by Facebook. People use facebook messenger because their eyeballs are already on the fb news feed and by metcalfe's law, most of their friends are already on the platform. There isn't a whole lot of room for other players who aren't solving a specific (e.g. business, language/community, or specific multimedia) use case, like slack, wechat, or snapchat. If you're anything like me, you probably have a handful of these services on your phone because there's always one friend who refuses to use one of these services. But why should the app I use be tied to the app my friend wants to use? This doesn't make sense.

We should be free to use whatever application we want: there should be a clear separation between apps and the data they require[6,7] and these apps should be able to communicate via protocols which follow at least some basic fundamental interoperability principles (e.g. at minimum they make their data available via a json endpoint/feed and publish a schema which describes this data.

How do we achieve this without creating yet another standard?[8]

To a degree, you kind of don't. You can look to old standards, like XMPP, but in this case the problem lends itself well to a shift in paradigm. In this case, I see the relevant paradigm being P2P (peer-to-peer). This is effectively how skype started. It comes from the perspective of asking the questions, (a) "Wait, why do we need facebook as an intermediary in order to send messages to each other?" and (b) "What prevents someone from sending a message to anyone?". The answers to both (a,b) are that you don't need facebook, or any intermediary for that matter, to send a message between two people (so long as a TCP/IP pathway exists to transmit the message).

So why do so many people use facebook messenger (or whichever [Name your app])? Well, identity, collective action, and discovery are hard (e.g. finding and adding people to a chat without a "phonebook" like facebook). And Facebook does these things well. That and, people are going to need *some* app to message each other, whether its P2P or through an intermediary, and they already have the facebook app (and are unlikely to get rid of it any time soon). But it's important as a technical note that there isn't anything which makes Facebook a necessary requirement for sending messages peer-to-peer. It just makes it easier (reducing friction) to find friends and integrates into activities (e.g. reading news feed) which we're already doing. These are important benefits (for adoption of any competing software) and I (and others) would do well not to underestimate their value proposition.

However, what's to prevent one from bootstrapping identities from Facebook (google, etc) to seed identities for a peer to peer model? Could a decentralized P2P chat system emerge which makes it easy for peers to discover each other, verify each other's identities, and to communicate without requiring us all to be locked into the same user interface and data security preferences?

What if there was an open source, decentralized electron chat app called "Howler" (why not) which behaved similarly to Signal messenger, offering encryption. Howler might use some sort of variant of XMPP and therefore users would download this app (or any other type of chat app that suits their needs).

Howler would perform authentication *from the user's client machine* (i.e. no middleware gets your data) and log in to facebook, gchat, skype, signal, and all the user's other chat services in order to fetch/download/sync all their pre-existing chat contexts and previous messages. It would also aggregate a "buddy list" where each user on the buddy list would have a unique identifier / signature which is based on the service/id the user came from. So, Howler would know about a Michael E Karpeles (Mek) backed by facebook.com/mekarpeles and perhaps a google michael.karpeles@gmail.com and allow me to merge these two identities (as their owner -- verified through a secure signing mechanism during the auth process). Identities would be encrypted and published via a decentralized storage layer like Dat, IPFS (Interplanetary File System), or SSBC (Secure Scuttlebutt) and identities could be secretly shared across all clients of this system. That is to say, the only way this system would work (at first) is bootstrapping off of the trust of identities on systems like facebook et al.

The system would suck at first because of Metcalfe's law (i.e. until the network size became large enough). It would be difficult to get users to switch (in the same way that GPG/PGP hit difficulties in adoption). It would also require a tremendous amount of effort to marshal the archived messaging data formats from various systems (like facebook, google, etc) into a format like XMPP (in lieu of there being no native interoperable interchange layer between these services -- the client-side app would have to do this marshaling itself, possibly as a maintained open source project which programmers could contribute to support the various services they rely on).

Imagine that MEK authenticates against Facebook and the result is a Howler buddy list which creates ~800 encrypted identities (for his friends) over decentralized storage. How do his peers join? First off, this decentralized chat system shouldn't be mutually exclusive to the centralized version. They should be synced until its actually more convenient to use Howler (because you can e.g. locally search all of your messages across services, which becomes an increasingly powerful value proposition over time). When users become "synced", Howler can optionally send a message on your behalf (over e.g. fb messager) which allows the recipient to download Howler and to claim their identity (and populate their buddy list). This would be a painful, spammy, and exploited process (I'm sure), but eventually, it would bootstrap itself into a truly peer2peer chat system (the only one you need) which allows communication between any parties. Or at least that's the general idea. Someone smarter than myself can find the errors in the logic and propose better solutions to address technical details.