On API Correctness

Developing APIs is hard.

You pour your blood, sweat, and tears into this interface that bares the soul of your company and of your product to the world. The machinery under the hood, though, is often a lot less polished than the fancy paint job would lead the rest of the world to believe. You have to be careful, then, not to inflict your own rough edges on the people you expect to be consuming your API because…

Using APIs is hard.

As an app developer you’re trying to take someone else’s product and somehow integrate it into whatever vision you have in your head. Whether it’s simply getting a list of things from another service (such as embedding a reading list) or wrapping your entire product around another product (using Amazon S3 as your primary binary storage mechanism, for example), you have a lot of things to reconcile.

You have your own programming language (or languages) that you’re using. There’s the use case you have in mind, and the ones the remote devs had in mind for the API. There’s the programming language they used to create the API (and that they used to test it). Finally, don’t forget the encoding or representation of the data — and its limitations. Reconciling all of the slight (or major) differences between these elements is a real challenge sometimes. Despite years of attempts at best practices and industry standards, things just don’t always fit together like we pretend that they will.

As a developer providing an API it’s important to remember three things. There are obviously many other things to consider, but these three things are more universal than most.

#1 You want people to use your API.

Unless you’re developing a completely internal API, you’re hoping that the world sees your API as something amazing, and that your functionality starts popping up in other magical places without any further effort on your part.

#2 You have no control over what tools others are using.

Are you using a language that has little or no variable type enforcement? Some people aren’t. Some of those people still want to use your product. Did you come up with your own way of doing things with custom code instead of using widely-adopted industry standards (which, being widely deployed, come with battle-tested libraries in many languages)? Did that cause you to release a client in your own language (how about Clojure, how about Erlang, how about C++, how about Perl, how about…)?

#3 Your API is a promise.

It’s easy to forget (especially for those of us who spend our time in a forgiving language such as PHP or Python) that the API we provide is a promise to the rest of the world. What it promises is this: “When you provide me with ${this} I will provide you with ${that}”.

The super-important (and insidiously non-obvious) thing about this is that if you do not provide a written promise (in the form of your API’s documentation), then the behavior of your API becomes the implicit promise.

The most important thing to note here is that when your documentation is wrong, the promise of your actual behavior wins every single time.

Keep your promises

When your promises don’t match your actual results things get hairy.

Let’s take a look at a completely hypothetical situation.

  1. You have an API that is documented to return a json object with a success member which should be a boolean value.
  2. You have a case (maybe all cases) where success is actually rendered as an integer (0 for false, 1 for true).
  3. John has an app written in a strongly-typed language that works around this by defining success as an integer type instead of a boolean type. Because John was busy, he never got around to letting you know. Or maybe John never knew because he simply inspected your API and worked backwards from the responses that you gave. Now John’s app has 100k users depending on this functionality.
  4. Mary is writing an app, and because Mary doesn’t like to play fast and loose (and she doesn’t want her app to break later on) she submits an issue pointing out that you are returning the wrong type.

At this point you are trapped. The existing user base (and by extension their user base) is committed to integers. And you only have four options.

  1. You can cripple an existing and deployed application enjoyed by 100k users.
  2. You can version your API — an entire new version to correct what should be a boolean value.
  3. You can work with John to roll out a new version of the app which can handle both (but maybe his app is in the iOS app store, and getting everyone to update is impossible, takes a long time, and/or would require a lengthy, and potentially costly, review process by yet another party).
  4. You make a really sad face and change your promise — to reflect that you are going to do what is actually the less correct thing, forever.

Because you wrote an API whose promise was wrong, or whose promise was missing, you have painted yourself into a very undesirable corner. You’re now in a place where doing the right thing for the right reasons is the wrong move.

So do yourself, and everyone else, one of two favors — depending on the position in which you find yourself.

If you’re producing an API, take extra care to make sure that your results match your documentation (and you need to have documentation).

If you’re consuming an API, don’t be like John. Don’t work backwards from the data — work forwards from the docs. And if the docs are wrong you should submit a ticket and wait for it to be fixed (or at the very, very least, make sure your workaround deals with both the documented expectation and the actual incorrect return value).

In conclusion

Just like a child, it takes a village to raise a good, decent, hard working API.