Posted on December 3rd, 2012 No comments
During the question and answer section of the panel I recently spoke on at DCWeek 2012, one questioner asked the panel to describe an API that had “disappointed” us at some point. I replied: Twitter. Though he was angling for technical reasons – poor design, bad documentation, or insufficient abstraction – I had different reasons in mind.
Twitter’s Successful API
Twitter’s primary API is without a doubt a hallmark of good design. It is singularly focused on one particular task: broadcasting small messages from one account to all following accounts as close to real-time as possible. That extreme focus led to simplicity, and that simplicity meant it is easy for developers to code and test their applications. Interactions with the API’s abstraction are straight-forward and often self-documenting. When coupled with Twitter’s excellent documentation and sense of community, the early years meant that developers were free to explore and experiment, leading to a plethora of interesting – and sometimes terrible – Twitter clients (including my own Java IRC bot JackBot).
Coincidentally, the explosion of smart phones, social networking, and always-on Internet connectivity meant Twitter’s raison d’être was also a means to explosive growth. The Fail Whale was an all-too-familiar sight during those early growing pains, but the same focus and simplicity that made it an easy API for developers to use also made it possible for Twitter to dramatically improve the implementation. Today, Twitter serves over 300 million messages daily – up several orders of magnitude from when I joined – yet our favorite marine mammal surfaces rarely.
Twitter’s early business model is a familiar story. A cool idea formed the basis of a company, funded by venture-capital and outside investment. There was little thought given to how to turn a profit. Seeing themselves in competition with the already-huge Facebook, growing the user-base was the only real concern. For many years, Twitter continued to foster its community: In a symbiotic relationship with developers and users – who were often the same – Twitter expanded and modified the API, improved the implementation, and actively encouraged new developers to explore new and different ways of interacting with the Twitter systems. So important was this relationship that even things like the term “tweet”, the concept of the re-tweet, and even Twitter’s trademarked blue bird logo all originated with third-parties.
But the good times can’t roll forever; eventually the investors want a return, and the company began seeking a method to make money. Seeing itself as a social network, advertising was the obvious choice. But there was a problem: the company’s own policy and openness had made advertising difficult to implement. Here’s what I wrote in December 2009:
More than 70% of users on Twitter post from third-party applications that aren’t controlled by Twitter. Some of those applications are other services – sites like TwitterFeed that syndicate information pulled from other places on the web (this blog, included). Others are robots like JackBot, my Java IRC bot which tweets the topics of conversation for a channel I frequent.
Advertisers purchase users’ attention, and if you can’t guarantee that access, you can’t sell ads. But what third-party client is going to show ads on behalf of Twitter? Users – particularly the developers creating those third-party apps – don’t want to see ads if they can avoid it. You won’t make much money selling ads to only 30% of your users (who are also likely the least savvy 30%). What’s a little blue birdie to do?
The chosen path was to limit – and perhaps eliminate entirely – third-party clients. The recent 100,000 limit on client tokens is an obvious technological step, and they are already completely cutting off access for some developers. Additionally, where technological restrictions are difficult, changes to the terms of service have placed legal restrictions on how clients may interact with the API, display tweets, and even in how they may interact with other similar services. (Twitter clients are not allowed to “intermingle” messages from Twitter’s API with messages from other services.) It seems likely that the screws will continue to tighten.
A Way Forward: Get On The Bus
Twitter has built the first ubiquitous, Internet-wide, scalable pub-sub messaging bus. Today that bus primarily carries human-language messages from person to person, but there are no technical limitations preventing its broader use. The system could be enhanced and expanded to provide additional features – security, reliability, bursty-ness, quantity of messages, quantity of followers, to name just a few – and then Twitter can charge companies for access to those features. Industrial control and just-in-time manufacturing, stock quotes and financial data, and broadcast and media conglomerates would all have benefited from a general-purpose, simple message exchange API.
Such a generalized service would be far more useful to the world at large than just another mechanism for shoving ads in my face, and I would bet that the potential profits from becoming the de facto worldwide messaging bus would dwarf even the wildest projections for ad revenues. It wouldn’t be easy: highly available, super-scalable systems are fraught with difficulty – just ask Amazon – but Twitter is closer to it than anyone else, and their lead and mindshare would give them a huge network-effect advantage in the marketplace.
With this new model replacing the advertising money, third-party clients would no longer be an existential threat. Twitter could remove the pillow from the face of their ecosystem and breath new life back into their slowly-suffocating community.
Will they take this path? I doubt it. The company’s actions in the past several months clearly telegraph their intentions. Twitter’s API teaches us an important lesson that, no matter how well designed, documented, and supported an platform is, there will always be people behind it making business decisions. Those decisions can affect the usability of the API just as deeply as bad design, and often much more suddenly. Caveat programmer!
Posted on November 10th, 2012 No comments
I had the opportunity to speak on a panel at DCWeek 2012 this past week: “Five Crucial APIs to Know About”. (I am not listed on the speakers page, as I was a rather last-minute addition.) Conversation ranged from what goes into making a good API – dogfooding, documentation, focus – to pitfalls to be aware of when building your business on an external API. It was a fun and informative discussion, and I walked away with plenty to chew on.
An API is all about two things: Abstraction and Interaction. It takes something messy, abstracts away some of the details, and then you, as a programmer, interact with that abstraction. That interaction causes the underlying code to do something (and hopefully making your life easier). If you interact with it differently, you’ll get different results. Understanding an API, then, requires understanding both the abstraction as well as how you are meant to interact with it.
Now, DCWeek focuses primarily on the startup scene. As such, I expected that most of my fellow panelists would be focusing on web-exposed APIs. Sure enough, there was plenty of talk on Facebook, Twilio, Twitter, and laundry list of other HTTP-accessible APIs. All of which are great! Note, though, that these APIs share one common thing: They are all network-reliant APIs. As such, they are built on a whole bunch of other APIs, but at the end of the day, they all route through one specific API (or a clone): Berkley Sockets.
Why should you care about a 30-year-old API when you care about tweets and friends and phone calls? Stop for a moment and think about what those high-level APIs are built on: a network. Worse – the Internet. A series of tubes. Leaky, lossy, variable-bandwidth tubes. And it’s only getting worse – sometimes you’re on a high-bandwidth wifi connection; other times you’re on a crappy, intermittent cellular connection in a subway tunnel.
The user’s experience with a high-level network API is going to be directly impacted by socket options chosen several layers down – often just by default – but different experiences require different expectations from the network. Do you have a low-latency API that provides immediate user-interactive feedback in super-short bursts? Then you might want to learn about Nagel’s Algorithm and TCP_NODELAY. Does your app require a user to sit and stare at a throbber while you make a network call? You might want to consider adjusting your connection, send, and receive timeouts to provide more prompt feedback when the network fails.
And believe me: the network will fail. But how do you handle it? As programmers, we tend to focus on the so-called “happy path”, relegating failure handling to second-class status. Unfortunately, treating failure as unlikely is simply not acceptable in a world of ubiquitous networking and web services. Not all network failures are the same, and providing the best user experience requires understanding the difference between the various types of failures in the specific context of what you were attempting to accomplish.
So take a moment and do some research. If you’re using a networked API that exposes network details, learn about them and tweak them for the specific task at hand. If you’re writing an API, consider how users will be accessing it, and provide them guidance with how to achieve the best possible experience over the network. The people using your apps will thank you.
I’d like to thank my fellow panelists – Greg Cypes, Sasha Laundy, and Evgeny Popov – for such an interesting discussion, as well as to thank our moderator Matt Hunckler for keeping us on track. I hope we can do it again in the future.