nav-left cat-right
cat-right

HTTP Signed Requests with PHP

I thought I'd write a quick primer on a basic implementation of HTTP request signing with PHP. I see a lot of posts dealing with the topic, especially by people writing homebrew REST services.

What are signed HTTP requests?

Signed HTTP requests are simply a normal HTTP request, such as a GET or a POST, that happens to include a signature as part of the request.  A signature is just a string of characters generated in a meaningful way.  This signature can be used by the receiving party (ie. a REST service) to validate the request and ensure that it hasn't been tampered with.

What are signed requests good for?

In a nutshell, validating an HTTP request where restricted access to resources are necessary. For example, a service that allows someone to update their personal information needs to ensure that people can only update their own personal information. Signed requests ensure that the service that receives the data can verify that the sender is who they claim they are (or at least possesses the proper secret key, a de-facto assumption it is the right person).

Obviously, having someone log in and create a session is another method of verifying user identity, however, this is not always efficient or possible, especially with web services. As an example, every Facebook application developer gets a secret key when they create an application on Facebook. When Facebook sends information to someone's application, it will include a signed request that is only valid with that developer's key. By checking the signature, a developer can verify that a request came from Facebook, and isn't someone trying to fool their application. They also don't need to provide a complicated session procedure or have Facebook "log in" to their application; the signed request is enough.

 What aren't signed requests good for?

Signed requests are not a form of encryption. If the data being sent is sensitive in nature, such that you wouldn't want it being sniffed in transit over a network connection, then SSL encrypted communication (HTTPS) is the way to go. For simple REST services however, often you just need to limit requests based on a user or other similar scope, and signed requests are sufficient for that.

A problem that signed requests can solve

Here is an example situation that has a security problem that can be solved by the use of signed requests. Let's say you operate a network of beer brewing web sites, and the brewers are allowed to send updates via a REST service you have provided. Each brewer has their own user id that they get when they sign up to your service. Bob's Brewery (user id 1234) wants to update their email address. The simplest of implementations might allow for a request to be sent like this: /user/update?email=newbob@example.com&userid=1234. However, if this was the only thing the REST service needed to update an email address, it would be trivial to cause serious mayhem. For example, Mr. Evil's Brewery could send in a simple request to change Bob's email address, just by finding Bob's ID somewhere, or even trying random ones:

/user/update?email=evil@example.org&userid=1
/user/update?email=evil@example.org&userid=12
/user/update?email=evil@example.org&userid=1234
etc.

Too easy! Obviously more security is needed, and this is where the concept of secret keys comes in.

Secret Keys

It is a common practice to assign each user a secret key that they use to interact with a web service, often called an API key, generally consisting of randomly generated characters. In this case, you could give Bob a secret key when he signs up with the brewery network, and it would be stored in your database along with the rest of his account information.

Here's where I often see confusion setting in. Many people think it is logical that the secret key be sent as part of the API request, because after all, it uniquely identifies the user and it was given only to that particular person. However, this is a security problem!

The thing about secret keys is that they have to remain secret, just like any password. As soon as a key is leaked, whoever finds it can impersonate that account, just as easily as Mr. Evil's Brewery did in the previous example. If you send a secret key as part of an HTTP request, it is no longer secret. You have just leaked it to whoever cared to be listening! A network sniffer could pick it up, or it could be recorded in proxy logs or web server logs that aren't secured. Perhaps your ISP is doing deep packet inspection, and a rogue employee makes off with the logs. Who knows!

The following is an example of this type of insecurity. Bob is sending his update request along with the secret key:

/user/update?email=newbob@example.com&userid=1234&secret_key=bobs-super-secret-key

Now, pretend that Bob goes to his local Tarborks coffee shop and jumps on the free open wireless connection. Mr. Evil happens to be there, and fires up his network sniffer program and starts watching all the HTTP traffic on the network. He sees Bob's request go out, since HTTP connections are not secure. Now he has Bob's secret key, and can make any kind of request he wants with it. Maybe he could even delete Bob's account with a request to /user/delete?userid=1234&secret_key=bobs-super-secret-key, or send insults to Bob's customers using a mail endpoint!

Sending the secret key as part of a network request is NOT safe. So how do we properly identify the sender if the user id can't be trusted and they can't send us their key? Well, you knew I'd get to it eventually… signed requests!

Signed HTTP Requests

To produce a signed HTTP request, the sender and the receiver must both know the rules on how to generate a signature. It can be any crazy method you care to dream up, but a common one is as follows:

  1. The sender organizes the data they want to send in a logical way, such as sorting it alphabetically.
  2. The data is run through a hashing algorithm using the secret key to produce a hash. Hashing algorithms produce short strings of characters that vary based on the input data, and the output is sufficiently unique such that varying the input data by even one character produces a completely different hash.
  3. The hash is added to the original data and sent.
  4. The receiver identifies the user and gets the secret key from their associated account information, and recreates the hash on the received data.
  5. If the recreated hash matches the one included in the request, the request is valid.

To return to our Tarborks coffee shop scenario, the request going out might look like this now:

/user/update?email=newbob@example.com&userid=1234&sig=x1zz645

If Mr. Evil sniffs this request, he might be pretty pleased with himself and try to change it to /user/update?email=evil@example.org&userid=1234&sig=x1zz645. This request would be rejected by the server however, since the signature is now invalid! When Mr. Evil changed the request, he needed to change the signature to match. But since he doesn't have Bob's secret key, he can't generate a correct signature, for this request or any other he cares to make up. The only valid request he can possibly make is the exact same one he just sniffed, and that's not very malicious at all, since it's what Bob was trying to do anyway.

Get to the code already

Assuming Bob's user id is 1234, and the secret key you gave him is "bobs-super-secret-key", he might write the following PHP code.

User code

[php]
$USER_ID = "1234";
$SECRET_KEY = "bobs-super-secret-key";

/**
* @param array $data Array of key/value pairs of data
* @param string $secretKey
* @return string A generated signature for the $data based on $secretKey
*/
function generateSignature($data,$secretKey)
{
//sort data array alphabetically by key
ksort($data);
//combine keys and values into one long string
$dataString = ";
foreach($data as $key => $value) {
$dataString .= $key.$value;
}
//lowercase everything
$dataString = strtolower($dataString);
//generate signature using the SHA256 hashing algorithm
return hash_hmac("sha256",$dataString,$secretKey);
}

$bobsData = array(
"userid" => $USER_ID,
"email" => "newbob@example.com"
);

$sig = generateSignature($bobsData,$SECRET_KEY);
//add signature to the outgoing data
$bobsData['sig'] = $sig;
//generate HTTP query string
$queryString = http_build_query($bobsData);

echo $queryString;
[/php]

The code above outputs userid=1234&email=newbob%40example.com&sig=efffd9cc30a220f2981b5124e1caa44d91b85aa2d2181f5331f48ca719983c1d.  That's the HTTP query string that Bob would send to the /user/update endpoint to change his email address.  So how does the service verify the request?

Server-side code

[php]
if (empty($_REQUEST['userid'])) {
throw new Exception("No user id was sent with the request.");
}

//look up the account associated with the value in $_REQUEST['userid']
//and get the secret key for that account -- implement as necessary
$secretKey = getSecretKeyFromUserId($_REQUEST['userid']);

$data = $_REQUEST;
$receivedSignature = $data['sig'];
//generate a signature using the data sent by the user, without the 'sig'
//parameter of course. Note that the generateSignature() function is the
//SAME ONE that the users would use!
unset($data['sig']);
$generatedSignature = generateSignature($data,$secretKey);

if ($generatedSignature != $receivedSignature) {
throw new Exception("Received signature is invalid!");
}
else {
//continue on, knowing it is the right user making the request.
}
[/php]

There you have it. Signed requests!

Advanced Usage

The signature for a signed request can be sent in different ways. Some services, such as Amazon's S3 REST API, puts the signature in the HTTP headers. This is arguably a bit cleaner than including it in the parameters of an HTTP request, since the signature doesn't get mixed in with the data, and has implications for caching as well (browser caches and proxies). If you want to do it that way, you might have the user set a header as part of their HTTP request:

[php]
header("X-Brewery-Sig: ".$sig);
[/php]

And the receiving server, instead of looking for the sig parameter in $_REQUEST['sig'] (and having to remove it before running the data through the generator function), would find it in:
[php]
$_SERVER['HTTP_X_BREWERY_SIG'];
[/php]

Hope you found this useful!

Be Sociable, Share!

6 Responses to “HTTP Signed Requests with PHP”

  1. Hi Blake,

    Thanks for this nice layout of requests signing. The only thing I'm always worried about with signatures is how portable the signing mechanisms are. If SHA1() is insecure now (they say it can be hacked in a matter of hours through brute force now), at least it can be used from any language. As web services tend to improve the cross-languages communication, it would be sad to have one signing method not supported by a specific language.
    Would you know of any page which would reference all the hashing algorithms and the programming languages in which they are supported?

    Happy new year, by the way.

  2. blake says:

    Hi Yannick,

    Good question. I don't know of such a page, and unfortunately couldn't find one with some quick googling. That would definitely be a valid concern for someone writing a public API or REST service, and would be worth researching as part of a project design.

    On the plus side, hashing algorithms are generally known and publicly available. With a little work, they can be implemented in any language, so I would expect the most popular algorithms to make it in to most or all of the major languages either with core support or 3rd-party support. SHA256 is ok for now, although some people theorize it may be "broken" (ie. take less time to find collisions than you would expect for the size of the hash length) within a few years, and future versions of SHA are already under development.

    Since hashing evolves over time, it would not be a bad idea to plan a REST service to include different acceptable hash functions. In that way you add functions in the future, and deprecate and remove older ones. It could be as simple as having the user pass a "hash=sha256" parameter with requests that did not use the default preferred hash method.

    Cheers, and all the best in 2012!

  3. Gavin says:

    Thanks a lot for this simple explanation and the codesample. Very much appreciated!

  4. Brian says:

    Thank you for the great tutorial and examples. It's just what I was looking for.

  5. Luke says:

    Good stuff.

    This obviously doesn't work for js web applications since the code is viewable. The attacker can just look at the implementation of generateSignature and copy it to generate a correct signature.

    How do you secure api calls when the code is public?

  6. Dmitri says:

    Luke, access token should be requested from the server and used to identify the user.

Leave a Reply