If you’ve ever worked on a coding project that grew to any size, you know it doesn’t take that long for it to become impossible for any single person to know all its inner workings. In order to deal with the cognitive limitations of human beings, it becomes necessary to divide the project up into individual components which are developed semi-independently of each other. As this process continues the teams working on each component begin to develop their own workflows, codebases, and dynamics. At the extreme, it can become difficult for the various components to work together smoothly.
And yet, if the overall project is to be a success this is precisely what has to happen. Having great backend and front end code written doesn’t do any good if none of it fits together. So how do we get everything to coordinate?
This is a deep question with a deep answer, but today we’re going to focus on just one piece of technology that helps make the modern world of technology possible:
Application Programming Interfaces (APIs).
If you’re looking to get into a technology career of any kind, knowing how APIs work is going to be an advantage. They are used all over the place for solving problems of almost every kind.
As it turns out, communication isn’t just essential in establishing healthy relationships between human beings. It’s no less important in making sure chunks of code get along. In the introduction I tried to set up my discussion of APIs by noting that within single projects complexity can cross a threshold beyond which there must be established rules for how one component talks to another. Well, the exact same thing is true at a much higher level between projects.
So what we have is a situation in which there needs to be layers both inside and outside a given project whose entire purpose is to make it so that someone can talk to the project’s software in a clear, rule-based way.
And that’s what an API is. One popular example of an API is code which enables you to grab the contents of a webpage (I describe just such an API below), but they come in many more shapes.
Twitter has multiple APIs for accessing its database of tweets, which let you do things like search for specific hashtags. SERPapi is a 3rd-party resource that lets you grab Google search results in a JSON object; it returns datetime information, text snippets, titles, links, and pretty much everything else Google would. I could write an entire article just expanding this list.
There’s a pretty good chance you work with APIs everyday without even knowing it.
Why Use an API?
I want to start answering this question by first drilling down into an example which is similar to my first experience with an API.
Imagine you’d like to grab the headlines, contents, and metadata for all the articles published by the New York Times in the past 5 years for natural language processing. If you couldn’t do any clever programming, how would you do this?
You’d probably have to set aside a week to spend all your time going backwards through the New York Times’ website, painstakingly copying out the data you need by hand into some kind of database.
Can you conceive of a more ridiculous waste of time?
Luckily, like almost all major websites, media outlets, applications, and software, the New York Times has an API which lets you accomplish this very thing in much less time. If I recall correctly, it took me a few hours of fiddling to get years worth of text data.
With this story having been relayed I doubt much else needs to be said. You use an API for the same reason you use a keyboard or a wheelbarrow or an atlatl: it makes your life easier.
Why Build an API?
And we’ve also gone most of the way towards answering the question from the other side. While I doubt the New York Times cares all that much about whether a fledgling data scientist saves time gathering their articles together. But most other application developers don’t have much choice. If they want their product to be used and integrated with other applications, they have to make an API to facilitate that process.
But there are subtler reasons. It sometimes winds up being possible to utilize applications through APIs in surprising ways that even the developers didn’t foresee. The famous automation platform If This Then That (IFTTT) is basically nothing but a way of lashing together hundreds of APIs to make it possible to do things like automatically email your spouse when you’re still logged into your work machine after a certain time or send out a tweet every time you bookmark an article.
Famously, Amazon began slicing their code up into smaller pieces that became exposed through APIs, which paved the way for what would later become Amazon Web Services, the most powerful cloud-computing service in the word.
If you write good code and follow best practices, you never know the ways in which human ingenuity will carry your vision further than you thought possible.
Some Common API Questions.
This could become an entire series of articles by itself, but there is a small set of typical confusions about APIs which it’s worth addressing.
Is an API a Web Service?
We’ve covered APIs in some detail. A web service is subtly different, facilitating machine-to-machine interaction over a network through the use of a Web Services Description Language (WSDL).
These do look quite similar, and the biggest distinction to be made is that web services are a subset of APIs. While every web service is an API, there are APIs which are not web services. Web services require a network and must adhere to a specific design protocol, but there are APIs which can work offline, and they can be built in accordance with any protocol.
What Is a RESTful API?
Building an API according to Representational State Transfer (REST) principles has been compared to using Graphical User Interfaces; so common that most people don’t even realize there are alternatives. REST APIs were introduced by Roy Fielding, and they essentially separate client-side concerns from server-side concerns.
This has a few implications. First, client and server applications can evolve independently of each other. Second, the flow of information becomes a matter of requesting and responding. Instead of there being a continuous exchange between them, they don’t communicate unless one of them requests something from the other. Finally, REST APIs can cache information so that they can more quickly produce it if and when it’s required.
This barely scratches the surface, read REST vs Streaming APIs for a more thorough breakdown.
Learning More about APIs
Given how common and powerful APIs are it’s a good idea to at least familiarize yourself with how they’re used and built.
Here are some resources to get you started doing just that.
- Intro to APIs. This Upwork piece has great explanations backed with lots of compelling graphs.
- Since most people seem to be building REST APIs, check out this tutorial aimed at beginners.
- As is true for almost everything, someone posted a lengthy explanatory article on Medium. It covers JSON as well as building your own API.
- If you’d like to build a REST API with Flask or Python, check out this Udemy course.
- An outstanding article laying out historical examples of companies that have succeeded by incorporating APIs. It does a great job of motivating the need for building good interfaces.
- Characteristically, O’Reilly has some books which cover the subject in detail. “APIs: A Strategy Guide” by Jacobson, Brail, and Woods discusses APIs from a business perspective, as potentially profitable long-term investments. “Designing Web APIs” by Jin, Sahni, and Shevat takes a more technical approach to building, scaling, and improving your web API.
APIs are important, and getting more so all the time. If you’d like to truly stand out as you build your technology career, be sure and learn to work with them, write them, and leverage them to add value.