News

Handling Twitter’s three-legged OAuth process

Building and signing requests according to OAuth can be tricky. We explain the authentication process for Python and Flask

The OAuth settings page for your new Twitter app

Once upon a time, Twitter was a favourite API for developers looking to write a quick weekend hack. Setup was quick, documentation was plentiful and the data available was vast. Fun times! But now things are a little different. With the Twitter API v1.1, authentication is not quite as easy as it once was (though the documentation is still plentiful and the data vast). Twitter now requires adherence to OAuth protocol v1 for all requests made to its API. It’s tempting to install a library and ignore all the complicated stuff – and entirely appropriate if you’re putting together a quick hack project. However, understanding what’s going on in the authentication process will help you along when it comes to debugging problems you may encounter later if you have to start handling multiple users or requests. For this tutorial, we’ll be using Flask – a great, lightweight Python server framework to define routes, create sessions and handle parameters passed to and from Twitter. Flask is perfect for this project: as we’ll see shortly, defining routes and handling parameters is a piece of cake and saves us having to deal with a lot of the overheads that come with frameworks such as Django.

The OAuth settings page for your new Twitter app
The OAuth settings page for your new Twitter app

Resources

Python 2.7+
Flask 0.10.0
Flask GitHub
A Twitter account
Full code files for this tutorial to follow along with

Step-by-step

Step 01 Install Flask

So before we start writing code, let’s deal with the prerequisites. If you have pip on your system then installing is as simple as entering sudo pip install flask into your terminal. If you’re into building from source, you can head over to Flask’s GitHub and grab the latest version there.

Step 02 Create a Twitter app

Before we can do anything with Twitter, we need to create an app for their service. Go to https://dev.twitter.com and sign in. If you look in the top-right corner, you’ll see your Twitter profile picture; hover over it and a drop-down will appear. Select ‘My Apps’ and then click the ‘Create new app‘ button that appears on the next screen. Follow the instructions to set up the app. The information you enter here isn’t important right now – at the end of the process you’ll see a field titled ‘OAuth Settings’ with a table of settings. We’ll need those in a little bit.

Step 03 Importing Python modules

These are the modules we’ll be using for this project. With the exception of Flask, all of the modules should be included with all versions of Python 2.7+. These modules are all used in the process of generating, signing and encoding each request we send to and from Twitter in the auth process. We need to be specific when importing sessions, requests etc from Flask, otherwise those aspects of the library won’t be included in the scope of the endpoints.

Step 04 Create global variables for Twitter keys

Back in step 2 we created an app with Twitter. Now we need those variables that we saw in the OAuth Settings area of the ‘Create my app’ page. Fill in the consumer_key and consumer_secret variables. For now, leave the oauth_secret and oauth_token variables as blank strings – we’ll assign those later.

Step 05 Assigning a namespace to the Flask app

Now we create an instance of the Flask class. This variable will be how we access all of the classes and properties associated with the Flask package. The __name__ property in this instance gives our Flask class an idea of the scope it’s operating in.

Step 06 Creating our first route

In order to authorise our user, we need to create an endpoint that they can visit to trigger the process. This is where Flask comes in handy. Using the routes decorator (@app.route()) we can determine paths that trigger functions or actions. The first endpoint we want to create is @app.route(‘authenticate’). The function we declare immediately after the decorator will be executed when that endpoint is called. Note that although we’ve done so here, you do not need to name the function the same as the endpoint. Flask can deliver static files as well as use template rendering engines, but for the sake of brevity we’re going to use return at the end of the function to deliver a simple text response showing our progress.

Step 07 Creating a session with Flask

In order to maintain various properties and values specific to multiple users, we’re going to be storing our values in Flask sessions. A Flask session is almost exactly the same as using a cookie to store values, but with sessions we get a layer of cryptography to help keep user credentials secure. The values in our sessions behave just like a dictionary, so we can call all of the methods that dictionaries have. For example, we can call session.clear() if we want to log a user out of a service or restart the authentication process. We need to make sure that before we make any calls to our sessions, we’ve set our session key, otherwise we’ll get an error. In this example we’ve chosen to insert the key just before app.run() so it will be set just before our server is executed. Like most keys, you must keep this key secret.

Step 08 Preparing to authorise our app

Now we need to start thinking about how we are going to submit data to Twitter. The OAuth protocol expects every request to be signed with a SHA1 hash so Twitter can check the integrity of the data it has received before allowing access to any personal information. In order to sign the request properly, we need to order the parameters of the request alphabetically, URL-encode them, concatenate them and then sign against them each time. This is where the headache of OAuth begins. Because we’ll be doing this for most every request, we’re going to write a convenience function that does all of this for us: sign_request().

Step 09 Creating our signature

So, what’s going on in our sign_request() function? First, we need to URL-encode the base URL of the request, which we can do with urllib.quote(). Next, we create an ordered dictionary (OrderedDict) which sorts the parameters we’ve passed through to the function alphabetically according to the key value. This is crucial as each request must be signed with all of the parameters concatenated alphabetically in order to be valid. Next, we create the variable requestString to build the first part of our request body. This is made up of the method we’ll be using for the request and the URL we’ll be submitting to. We then iterate through the keys in the OrderedDict, concatenating and URL-encoding the key value pairs of the dictionary so we have something like key=value. Finally, we check whether or not we’re at the last key in the dictionary; if we aren’t then we’ll append a ‘&’ to the parameter string in preperation for the next one to follow. Ultimately, we’ll have a string that will appear as:

GET&http%3A%2F%2Fapi.twitter.com%2Fetc 12 %2F?key1=value1&key2=value2&key3=value3

Step 10The signing key

Now we need to create a signing key for our signature. As the code shows, it’s just our consumer_secret + oauth_secret with an ampersand in the middle. However, you may notice that we don’t have an OAuth secret yet. That’s okay: for this first signing request we don’t need one. You may have noticed in our /authenticate endpoint that after we popped the session values, we reassigned session[‘oauth_secret’] to “”. This way we won’t get a KeyError message when we try to sign our request. When we get our callback in our /authorised endpoint, we’ll assign our session[‘oauth_secret’] for future requests.

Step 11 Hashing the key

Finally, we’re ready to hash the key using Python’s HMAC library, we pass our signing key, our result string (the concatenation of our escaped base URL and method with our parameters string) and from Python’s hashlib library, the SHA1 hash function. This results in a binary buffer with our signature – that’s great, but we can’t send a buffer to Twitter’s server. On the next line, we pass our hashed variable through to the binascii library to convert it to a Base64 value. Now we have our signature which we’ll return to the function that called it.

Step 12 Obtaining the request token

With Twitter API v1.1, we can no longer just request an access token to access user data. First, we need to obtain a request token and then exchange that for our first access token. We don’t need the user to interact with anything at this point, so we can trigger this on the back-end and handle it ourselves. In the requestParams dictionary, we prepare the values we’ll need to sign against and then submit these to Twitter to receive a request token. The oauth_callback field won’t do anything right now, but it’s where our app will redirect after we obtain our first access token. We want to point it to our route /authorised. On line 27 we pass the requestParams through to the sign_request method. We then add the signature returned to the requestParameters that passed through to the signing function, as we’ll be using these to build our authorisation headers next.

Step 13 Building the OAuth authorisation headers

Much like when we sign our requests, we need to properly encode and format our authorisation headers, too. The main difference between sign_request() and create_oauth_headers() is that the signature base string (the string we build to generate our SHA1 signature) is all of the variables joined with ‘&’, whereas our auth headers are joined with ‘,’ and prepended with the word ‘Auth’. Other than that, the two serve the same purpose. Here we’ve separated the two out to emphasise the importance of the difference – if either of these two functions returns a malformed string, the request will fail. Using the urllib2 module, we can prepare the request we’ll submit. Notice the second parameter of the request function on line 31. If we omit this parameter, the request method will be GET; if it is included – regardless of the content – it will be a POST request. The parameter’s main function is to assign the body property of a request; it’s a little clumsy, but it’s something we’ll need in a bit.

Step 14 Making the first request

Once we’ve gotten the prepared auth headers back, we add them to the HTTP request and then call the open() method and wait for a response. If all has gone well, we should receive a string in the request response with our first auth token and auth secret. To save us having to individually parse the string for properties, the function getParameters() will split the string up and return a dictionary we can access. We’ll then store these values in our session for later.

Step 15 Redirecting the user

Now we have our first set of credentials we can redirect our user to the Twitter authorisations screen to let them grant access to their data. Because we haven’t sent anything to our client yet, we can use Flask’s redirect() method to send our user to the Twitter authorisation view. This request is one of the few that we can use URL parameters to send tokens to Twitter.

Step 16 Handling access

If the user grants our application access, they will be redirected to the callback URL we defined in step 12. The redirection will include URL parameters that are our access token and a token verifier string. The Twitter documents state that we should check the access token we received in the URL parameters is the same as the one we obtained moments ago. If all has gone well, Our /authenticate endpoint will have stored our token in the session dictionary. If our stored token and the one Twitter returned matches, we can prepare to exchange our access token (which currently only has restricted privileges) for a fully fledged access token with read/write permissions (depending on the permissions you requested when you first created the app).

Step 17 Third (and final) token exchange

So, same as before: we prepare our parameters for submission with sign_request(), build our auth headers and then fire off our request. Just like when we first retrieved our request token, this is something that we can do in the background without user interaction. Providing everything has gone swimmingly, you will then be returned a fully authorised access token that you can start building your own requests with.

Step 18 Fire up your server

As you can see, we have ended our flask.py file with:

if __name__ == ‘__main__’:
   app.secret_key = ‘R4nDOMCRypt0gr4ph1cK3yf0R5355i0N’
   app.run(host=’0.0.0.0’, debug=True)

If we didn’t include any parameters in the run function then our server would run and be accessible at 127.0.0.1: 5000, but it wouldn’t be accessible to external machines – kind of defeats the point of having a server. By passing host=’0,0,0,0’, Flask will listen on any IP address it can for potential requests with a default port of 5000. debug=True is an error message beautifier that Flask includes. Rather than crashing our server, an error page will be delivered detailing what went wrong in place of our expected return.

×