News

Twitter: How do the new API rules effect you?

Twitter is one of the most popular technologies this millennium and millions of users, designers, and programmers rely on its API. Things have changed, though, and the new rules are bound to affect you

Twitter

Twitter is more popular than ever. Why? One of the main reasons is how open the API is and Twitter’s focus on service, not creating products. This created an enormous opportunity for others to create and innovate. This innovation opened new markets and expanded demand. It was a great partnership: except Twitter was not making money.

A bit of history here. Twitter (or twttr as it was originally called) was first built as an SMS communication platform; only later did it turn into a web-based product with simple APIs. Since a very large userbase of phones already existed that can only text, Twitter was often the only way to engage in social media without a computer. Keep in mind that this was before the iPhone and Android began their run to take over the phone market.

When the web API was released, there was a flood of hobby programmers, just like yours truly, who could create a cool fun little app around twitter in a weekend. For example, during the South by Southwest conference in 2009, a fun game was created called Assassin that had people assigning targets and reporting hits on their targets, all through Twitter. So overnight I wrote a simple little program that created a webpage of players, targets and Xs over successful hits. This application had nothing to do with Twitter’s original mission, but because of the openness and simplicity of the 140 character length, fun little ideas like that were popping up seemingly every day. As Twitter grew, so did the API, and that sparked even more innovation, from allowing creatives to build Twitter tools or applications using Twitter data to do everything from creating superior Twitter web clients; to tracking disease vectors, to playing, to creating calendar and to-do lists, and integrating into other applications that use Twitter as part of a greater function, like integration with other social media feeds. But the times, they are a-changing.

Why do I care, and how will this affect me?

Whether you’re a programmer, designer or a user, these changes will affect you. Clearly, as a programmer, you will have your hands full making changes to deal with greater OAuth requirements, small changes in how some API calls return data, and quite a few headaches if you did all your programming expecting XML to be returned instead of JSON. If you are a designer, you do not get off free either. Before 1.1 there were only recommendations as to how a tweet will be displayed; now it is a regulation. You will have to go back to your designs and UI layouts and make sure you are compliant with the new rules.
Finally as a user, you may or may not be affected depending on what Twitter tools you use and if your website uses Twitter information. You may wake up one day and find that your favourite old tools no longer work, or a feature in an app is no longer functional. It’s up to the app owners to decide if it’s worth spending development money to comply with all the changes in 1.1. Many people may decide it’s not worth it, and on 5 March, 2013, you may have lost your favourite app.

Per-endpoint rate limiting

In version 1.0 of the Twitter API, there was a 150 limit on the number of API calls you could make per hour that rose to 350. If you had a whitelisted application, that can be as much as 2,000 per hour. However, Twitter no longer supports whitelisting. Instead, Twitter has changed how it counts the API calls that are made to its service. Instead of the blanket 350 API calls, Twitter now has API buckets with different limits and most API calls set to 60 calls per hour. Twitter also changed the measure of how API calls are metered. Instead of every hour, the API measures every 15 minutes, so in this case 60 calls an hour is 15 calls every 15 minutes. This change also limits API bursts. By forcing the client to 15 calls per 15 minutes, demand on the system will be spread out over time, making life easier on the service. However, since per hour has been the standard since the early days, we will stay with that measure.

This may sound like a step backwards, but it really isn’t for most app developers that have a niche market. Remember the way it used to work is you would get 350 calls per hour regardless of the API you are calling, so the downside is you can make quite a few calls depending on the sophistication of your application and what other information you want to gather besides just a tweet. Although many calls are limited to 60 an hour, high-volume calls related to tweet display, profile display, user lookup and user search will be able to make up to 720 calls per hour, per endpoint.

As such, it’s very important to understand which calls are going to come out of which bucket. If you feel that your application is going to need to pull from the low bucket more than 60 times per hour, then you are going to need to get your application approved by Twitter. Once approved, you will have a larger pool of API calls. However, if you plan on consuming a lot of tweets, then you need to think about accessing the Twitter stream.

As we can see from Table 1 below, the small bucket has calls that normally would not be made that often, whereas the larger bucket would. However, this totally depends on the type of application you are writing. Say you want an app that only displays tweets within your area. Well, notice the GET geo/search API call is limited to 60 per hour. Thus, the only way your app is going to survive the 60 limit is to instead pull from the stream and sort it by Geo on your own. Further, if you have an application that monitors a brand or brands, you will run into this limitation as well since GET statuses/mentions_timeline is also in the small bucket. In both cases you could try to access the steam and sort yourself, but you might want to try to get an exception from Twitter. Good luck with that if they feel your product can be a threat to them.

This is not the only change however, even if you can manage to support your application’s needs within the API call budget, there is another limitation. If you have an application that replicates the Twitter client, then you cannot have more than 100,000 access tokens, which is another way of saying you cannot support more than 100,000 users. If you already have an application that has more than 100,000 users, then you will be limited a growth of 200 per cent of your current userbase. Again, this is only if you have or are building a Twitter client application that is accessing the home timeline, account settings or direct messages API. If you’re doing something completely different, like analysing tweets, you should be fine. For now.

API changes for the designer

So let’s say you have an application that does not hit the Twitter API that often, nor have near 100,000 users; don’t worry, there are changes in the API for you as well. These changes are more about policy then programming, and you’re not going to be happy.
A major change with Twitter is the restriction of HOW you can display tweets. There are plenty of Twitter clients out there that have their own take on how Twitter information should be displayed, and how the user can interact with it. Odds are all of those apps and applications will see the end of the road in March 2013. Twitter is coming down hard on any client application that does not follow Twitters display guidelines, and since all API calls must be authenticated, they know who you are and can turn off the water at any time. The guidelines are pretty strict and do not provide much wiggle room. Let’s have a look at Figure 1 (below) to the left, and then the high points…

As we can see from these first four points, Twitter wants to be sure that as many links as possible take the user back to Twitter.

User_mentions must always link to the mentioned user’s profile
Hashtags must link to a twitter.com search with the hashtag as the query
Links in tweet text must be displayed using the display_url field in the URL entities API response, and link to the original t.co URL field
Reply, Retweet, and Favourite action icons must always be visible for the user to interact with the tweet. These actions must be implemented using Web Intents or with the authenticated Twitter API.

The next four points highlights that Twitter want to make sure that core features are available so the user senses these functions have proven to improve the life of a Twitter thread.

No other social or third-party actions similar to Follow, Reply, Retweet and Favourite may be attached to a tweet
It must always be clear to the viewing user that they are looking at a tweet, and that the content is from Twitter
The Twitter logo or Follow button for the tweet author must always be displayed
The Twitter logo must link to Twitter.

The final four points, below, show just how exacting Twitter wants information to be displayed and how the user is to interact with it.

Reply, Retweet and Favourite Tweet actions must always be available to the user when interacting with the tweet on the timeline. eg, select, hover, touch, swipe
For tweets that have been sent in the last 24 hours, use the short-form timestamp relative to the current time, for example 20s for a Tweet sent 20 seconds ago, 3m for three minutes ago, 5h for five hours ago
Tweets older than 24 hours should show a short-form date including the day and month, eg, 6 Jun
If the tweet being displayed is a retweet, the name of the user who retweeted it and the Retweet icon must be displayed under the tweet text – eg, Retweeted by Josh Brewer. The name should link to the retweeting user’s profile, unless your application is displaying tweets on a mobile platform that has clear physical or technical limitations that stop it doing this.
The above four points show just how exacting Twitter wants information to be displayed and how the user is to interact with it. However, the detailed outlines for how to display a date is beyond me.
Tweets that are grouped together into a timeline must not be rendered with non-Twitter content. eg, comments, updates from other networks.

This last point (Tweets that are grouped together) is important. Twitter is very keen to insure that when displaying a Twitter stream, all posts in that stream are shown as Twitter would like, and that is including sponsored posts, although that is not mentioned.
Again, the list provided previously is just a few of the high points. The actual design specification goes into great detail and includes restrictions on what can go inside a tweet, how links must work, not putting any other links inside a tweet, and always making sure there is a Twitter logo somewhere near the display. If you have any thoughts of taking a tweet or stream of tweets and doing something even remotely interesting, or heck, anything at all, you will find both hands tied behind your back. And Twitter wants it this way. You may be asking yourself, ‘basically there is no room to experiment or innovate. Why would Twitter do this’? Why indeed.

Why on earth did Twitter make these changes?

According to Twitter’s website, these changes have been enacted in order to make the system more stable, more secure, and the user experience more consistent. As stated, some of the changes with the API actually have nothing to do with the API at all, but instead have to do with the change of style recommendations to style requirements. So, what does this mean? Conspiracy theories aside, Twitter is essentially trying to gain better control of its product in order to better support and monetise in some way. And depending on where your product or a service you depend on falls in the aforementioned diagram, you could find that your application is no longer working quite as well as it once did with Twitter.

As you can see in Figure 3 above, Twitter is trying to carve up the current Twitter market into what they have branded competitive and non-competitive segments.
On the left-hand side of the grid are applications that are targeted at businesses
In the lower-left quadrant, are apps and services that serve the business market, with analytics products based on Twitter content
In the upper-left quadrant are providers of tools that help businesses engage with Twitter, like social CRM providers
On the right-hand side of the grid are applications that are targeted at consumers.
In the lower-right quadrant are services that utilise Twitter content for social influence ranking, such as the popular Klout service
In the upper right-hand quadrant are services that enable users to interact with tweets, especially those that ‘mimic or reproduce the mainstream Twitter consumer client experience’. You do not want to be there.

In this case, the traditional Twitter clients and syndication segment is considered completive while the rest are not… for now. The main reason for this is twofold. First, Twitter wants better control over where and how its content is displayed because, second, they need to make money.

And there we get to the core.

Changes to the API for performance, monitoring bots and in general making the system more robust are a needed and understood thing. If it was just about the API in itself, I’m not sure there would be a story here. But, by insisting on how third parties can display Twitter data and enforcing a growth limit so not any one product can become more popular than the Twitter client itself, indicates that they are doing all they can to maximise monetisation opportunities through the Twitter stream and on the Twitter client.

There is only one problem with this. There are quite a few companies that have been building off the Twitter API, helping to make Twitter more useful, and thus grow, who are going to get cut off at the knees. There are three really nice quotes from a money.cnn.com post. (money.cnn.com/2012/08/23/technology/twitter-api-tumblr/index.html).

“They have a right to do what they want with their own API, but it changes what Twitter is about,” said Burt Herman, co-founder of social curation site Storify.

“Twitter was built up on the backs of developers. And some apps are arguably a better use of Twitter for users than Twitter is itself.”

“Even some of Twitter’s own employees are vocally displeased. ‘This @tumblr business just stinks,’ Twitter engineer Alex Choi tweeted.”

That confusion fuelled a post from Instapaper creator Marco Arment, who took to his influential blog to detail his take on the new guidelines. He had particularly harsh words for the 100,000-user limit before app makers require special permissions from Twitter.
“Translation: ‘Once you get big enough for us to notice, we’re going to require you to adhere to more strict, unpublished rules to make sure you don’t compete with us or take too much value from our network,’” Arment wrote. As you can gather from that little selection of opinions there, not everybody is happy with these changes.

What does the Future hold? Will there be more changes?

As you can probably ascertain by now, there are quite a few companies that have been basically handed their hat with the changes in the API, limits, and display guidelines. This will also continue to pour cold water on any new initiative to create innovative Twitter products – much less Twitter clients. This begs the questions; is Twitter abandoning its once-neutral stance? And what legal actions, if any, can already established companies take? The short answers are ‘Yes’ and ‘None’.

Twitter still needs to pay its bills. It has a nice new space in San Francisco, has hired lots of people, and spends a good chunk of money on servers. That is not free you know; not to mention investors who are waiting for their payback.
The problem is how do you make money on Twitter? This has been an ongoing discussion for years, with the early answer of ‘we will build it first, and then worry about making money later.’ Well it’s later now, and plenty of companies have been doing fine creating successful services on top of the Twitter ecosystem.

The obvious revenue stream is through sponsored tweets and adverts on the Twitter web and mobile clients. Knowing they are pushing hard to funnel as many users through a controlled experience as possible explains pretty much most of the changes we have discussed, and as such, you can imagine that any changes needed in the future to further maximise this revenue stream will be entertained. This includes sacrificing a few companies on the way.

The problem is that this laid-back approach has been going on for a long time. Too long. And now Twitter has very little choice but to close the doors on its once wide-open system in order to create revenues through adverts and special access – and if people get hurt, so be it.

A general and valid concern out there is that if Twitter does not feel these current changes are enough to improve their bottom line, they may extend their reach even further into non-core markets. Taking lessons from Apple, they may decide that any company profiting from the Twitter service will be required to pay Twitter a percentage, or a licensing fee. Perhaps Twitter will decided the real money is in analytics and create a new rule that all current and future analytic products are deemed competitive and degrade their ability to compete with native Twitter products. Perhaps they will come out with their own iPod as well, but that might be looking too far down the Apple model.

Regardless of whether you own a Twitter-based product, or you are a user that favours something other than the official Twitter clients for your tweets, you are going to feel the big chill come March
2013. That is, unless, you are eaten in the Great Zombie Apocalypse at the end of 2012. But, that’s another story.

Author: Dr Christopher Peri

×