Abstract: The concepts and techniques you learn in this tutorial will allow you to practice with any API you like, and use Python for any API use requirements you may have.

This article is from Python and API: successful combinations of reading public data by Yuchuan.

To understand the API

API stands for Application Programming interface. In essence, an API acts as a communication layer, or as the name suggests, an interface that allows different systems to communicate with each other without knowing exactly what each other does.

Apis can take many forms or shapes. They can be operating system apis for things like opening the camera and audio to join a Zoom call. Or they could be web apis for web-centric operations, such as liking an image on Instagram or getting the latest tweets.

All apis function roughly the same, regardless of type. You typically request information or data, and the API returns a response based on your request. For example, every time you open Twitter or scroll down an Instagram feed, you’re basically making a request to the API behind the application and getting a response in return. This is also called calling the API.

In this tutorial, you’ll focus more on high-level apis for cross-network communication, also known as Web apis.

SOAP with REST with GraphQL

Although some of the examples mentioned above are for newer platforms or applications, Web apis have been around for a long time. In the late 1990s and early 2000s, two different design models became the norm for open data:

1. **SOAP (Simple Object Access Protocol) ** is generally associated with the enterprise world, has a stricter contract-based usage, and is designed primarily around operations.

2. **REST (Representational State Transfer) ** is commonly used with public apis and is ideal for retrieving data from the Web. It is much lighter than SOAP and closer to the HTTP specification.

Now, there’s a new kid in town: GraphQL. Created by Facebook, GraphQL is a very flexible API query language that allows clients to decide exactly what they want to fetch from the server, rather than the server deciding what to send.

If you want to learn more about the differences between these three design models, here are some great resources:

  • What is soap?

  • What is rest?

  • API 101: SOAP and REST

  • GraphQL profile

  • Compare API architectural styles: SOAP vs RESTvs GraphQL vs RPC

While GraphQL is emerging and being adopted by larger and larger companies, including GitHub and Shopify, the fact remains that most public apis are still REST apis. Therefore, for the purposes of this tutorial, you will only learn about REST apis and how to use them using Python.

Requests and apis: a match made in heaven

To use the API with Python, you only need one library: requests. With it, you should be able to perform most, if not all, of the operations you need to use any public API.

You can install requests by running the following command from the console:

$ python -m pip install requests
Copy the code

To follow the code examples in this tutorial, make sure you are using Python3.8.1 and requests2.24.0 or higher.

Call your first API using Python

Without further ado – it’s time for your first API call! For the first example, you will invoke a popular API to generate random user data.

Throughout the tutorial, you’ll see the new API introduced in the alert block, as shown below. This is a convenient way to scroll through later and quickly find all the new apis you know.

**Random User Generator API: ** This is a great tool for generating Random User data. You can use it to generate any number of random users and related data, as well as specify gender, nationality, and many other filters that are useful when testing applications or, in this case, apis.

When you start using the Random User Generator API, the only thing you need to know is which URL to invoke it from. For this example, the URL to use is randomUser. me/ API /, which you can use in… API calls:

>>>

>>> import requests
>>> requests.get("https://randomuser.me/api/")
<Response [200]>
Copy the code

In this small example, you import the Requests library and then retrieve (or retrieve) the data from the RANDOM user generator API in the URL. But you don’t actually see any data returned. You get a Response [200], which in API terms means everything is fine.

If you want to see the actual data, you can use.text from the returned Response object:

>>>

>>> import requests
>>> response = requests.get("https://randomuser.me/api/")
>>> response.text
'{"results":[{"gender":"female",
"name":{"title":"Ms","first":"Isobel","last":"Wang"}...'
Copy the code

That’s it! That’s the basics of API consumption. You managed to get your first random user from the random user generator API using Python and the Requests library.

Endpoints and Resources

As you saw above, the first thing you need to know about using an API is the API URL, often referred to as the base URL. The basic URL structure is no different from the URL you use to browse Google, YouTube, or Facebook, even though it usually contains words

API. It’s not mandatory, just more rules of thumb.

For example, here are the basic urls of some well-known API players:

  • api.twitter.com

  • api.github.com

  • api.stripe.com

As you can see, all of the above content is at https://api and includes the remaining official domains, such as.twitter.com or.github.com. There is no specific standard for how an API base URL should look, but it is common for it to mimic this structure.

If you try to open any of the above links, you’ll notice that most of them return an error or ask for credentials. This is because apis sometimes require authentication steps to use them. You will learn more about this later in the tutorial.

**TheDogAPI: ** This API is interesting, but also a good example, it is a good API and well documented. With it, you can get different dog breeds and some images, but if you sign up, you can also vote for your favorite dog.

Next, using the just introduced TheDogAPI, you’ll try to make a basic request to see how it differs from the Random User Generator API you tried above:

>>>

>>> import requests
>>> response = requests.get("https://api.thedogapi.com/")
>>> response.text
'{"message":"The Dog API"}'
Copy the code

In this case, when you invoke the base URL, you receive a generic message with the TheDog API. This is because you are calling the base URL, which is usually used for very basic information about the API, rather than real data.

Calling the base URL alone isn’t very interesting, but that’s where the endpoints come in handy. An endpoint is to specify which URL is part of the resource that you want to retrieve. Documented apis often contain API references, which are useful for understanding the exact endpoints and resources of an API and how to use them.

You can check out the official documentation for more information on how to use TheDogAPI and the endpoints available. There, you will find a/Breeds endpoint, which you can use to obtain all available breed resources or objects.

If you scroll down, you’ll find the send test request section, where you’ll see a form like the following:

This is what you’ll see in a lot of API documentation: a quick way to test an API directly from the documentation page. In this case, you can click Send to quickly get the result of calling the endpoint. Wait, you just call the API, you don’t have to write any code for it.

Now, try the code locally using the breed endpoint and some API knowledge you already have:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds")
>>> response.text
'[{"weight":{"imperial":"6 - 13","metric":"3 - 6"},"height": ...}]'
Copy the code

Ok, you used the dog API to list your first breed!

If you’re a cat lover, don’t worry. There is also an API for you to use that has the same endpoint but a different base URL:

>>>

>>> response = requests.get("https://api.thecatapi.com/v1/breeds")
>>> response.text
'[{..."id":"abys","name":"Abyssinian"}]'
Copy the code

I bet you’re already thinking about different ways to use these apis to make cute side projects, and that’s what makes them great. Once you start using them, there’s nothing stopping you from turning your hobby or passion into a fun little project.

As you go forward, one thing you need to understand is the difference between the endpoints http:// and https://. In short, HTTPS is an encrypted version of HTTP that makes all traffic between clients and servers more secure. When using public apis, you should definitely avoid sending any private or sensitive information to http:// endpoints, and use only those apis that provide secure https:// base URLS.

For more information on the importance of sticking with HTTPS when browsing online, see Exploring HTTPS with Python.

In the next section, you’ll take a closer look at the major components of an API call.

Request and response

As you read briefly above, all interactions between the client (in this case, your Python console) and the API are divided into requests and responses:

  • The request contains relevant data about your API request invocation, such as the base URL, endpoint, method used, criteria, and so on.

  • The response contains relevant data returned by the server, including data or content, status codes, and headers.

Using TheDogAPI again, you can delve into the exact contents of the Request and Response objects:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds") >>> response <Response [200]> >>> response.request <PreparedRequest [GET]> >>> request = response.request >>> request.url 'https://api.thedogapi.com/v1/breeds' >>> request.path_url '/v1/breeds' >>> request.method 'GET' >>> request.headers {'User-Agent': 'python-requests/2.24.0', 'accept-encoding ': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'} >>> response <Response [200]> >>> response.text '[{"weight":{"imperial":"6 - 13","metric":"3 - 6"}, "Height" : {" imperial ":" 9-11.5 ", "metric" : "23-29}", "id" : 1, "name" : "Affenpinscher." ...}]' >>> response.status_code 200 >>> response.headers {'Cache-Control': 'post-check=0, pre-check=0', 'Content-Encoding': 'gzip', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Sat, 25 Jul 2020 17:23:53 GMT'... }Copy the code

The above example shows you some of the most important properties available for Request and Response objects.

You’ll learn more about some of these properties in this tutorial, but if you want to dig deeper, you can check out Mozilla’s HTTP message documentation for a more in-depth explanation of each property.

The status code

The status code is one of the most important pieces of information to look for in any API response. They tell you if your request was successful, if data is missing, if credentials are missing, and so on.

Over time, you can identify different status codes without help. But for now, here are some of the most common status codes you’ll find:

You saw 200 OK in the example you executed earlier, and you can even identify 404 Not Found by browsing the web.

** Fun Fact: ** Companies tend to use 404 error pages for personal jokes or just for fun, as in the following example:

  • MantraLabs

  • Gymbox

  • Pixar

  • Slack

In the API world, however, developers have limited room to respond to this fun. But they make up for it in other places, such as HTTP headers. You’ll see some examples soon!

You can use.status_code and to check the status.reason of the response. The Requests library also prints the status code in the representation of the Response object:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds")
>>> response
<Response [200]>
>>> response.status_code
200
>>> response.reason
'OK'
Copy the code

The above request returns 200, so you can consider it a successful request. But now look at the failed request/Breedz that fires when you include a spelling error in an endpoint:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breedz")
>>> response
<Response [404]>
>>> response.status_code
404
>>> response.reason
'Not Found'
Copy the code

As you can see, the/Breedz endpoint does not exist, so the API returns a 404Not Found status code.

You can use these status codes to quickly see if your request needs to be changed, or if you should check the document again for any misspellings or missing parts.

HTTP headers

HTTP headers are used to define parameters that manage requests and responses:

There are many other headers you can find when examining the request or response. If you’re curious about the specific use of each, check out Mozilla’s list of extensions.

To check the headers of the response, you can use Response.headers:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds/1") >>> response.headers {'Content-Encoding': 'gzip', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Sat, 25 Jul 2020 19:52:07 GMT'... }Copy the code

To do the same for the request header, you can use Response.request. headers because Request is a property of the response object:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds/1") >>> response.request.headers {'User-Agent': 'python-requests/2.24.0', 'accept-encoding ': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}Copy the code

In this case, you did not define any specific headers when you made the request, so the default header is returned.

Custom title

Another standard you may encounter when working with the API is the use of custom headers. These usually start with an X-, but are not required. API developers typically use custom headers to send or request additional custom information to clients.

** Interesting Facts: ** Some companies go out of their way to be interesting and innovative by using HTTP headers in a way they shouldn’t, such as solicitating job applications.

You can define headers using dictionaries, and you can send them with your request. get().

For example, suppose you want to send some Request ids to the API server, and you know you can use x-request-id:

>>>

>>> headers = {"X-Request-Id": "<my-request-id>"} >>> response = requests.get("https://example.org", Headers =headers) >>> response.request. Headers {' user-agent ': 'python-requests/2.24.0', 'accept-encoding ': headers=headers >>> Response.request. Headers {' user-agent ': 'python-requests/2.24.0',' accept-encoding ': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'X-Request-Id': '<my-request-id>'}Copy the code

If you scroll through the Request. headers dictionary, you’ll find some other headers at the end of the X-Request-id, which are the default headers for any API request. Responses can have many useful headers, but one of the most important is content-Type, which defines the Type of Content returned in the response.

Content-type Today, most apis use JSON as the default Content Type, but you may want to use apis that return XML or other media types, such as images or videos. In this case, the content type will be different.

If you go back to one of the previous examples using TheDogAPI and try to examine the Content-Type header, you’ll notice how it is defined as Application /json:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds/1")
>>> response.headers.get("Content-Type")
'application/json; charset=utf-8'
Copy the code

In addition to a specific type of content (application/ JSON in this case), the header may return a specific encoding of the response content.

**PlaceGOAT API: ** This is a very silly API that returns goat images of different sizes that you can use as placeholder images in your website.

For example, if you try to get an image of a goat from the PlaceGOAT API, you’ll notice that instead of application/ JSON, the content type is defined as Image/JPEG:

>>>

>>> response = requests.get("http://placegoat.com/200/200")
>>> response
<Response [200]>
>>> response.headers.get("Content-Type")
'image/jpeg'
Copy the code

In this case, the Content-Type header declaration returns a JPEG image. You’ll learn how to view this in the next section.

The Content-Type header is very important to know how to handle the response and how to handle its Content. There are hundreds of other acceptable content types, including audio, video, fonts, and more.

Reply content

As you just learned, the Type of Content you find in the API response will vary depending on the Content-Type header. To read the Response Content correctly based on the different Content-Type headers, the Requests package comes with several different Response attributes that you can use to manipulate the Response data:

  • .text returns the response content in Unicode format.

  • .content Returns the response content in bytes.

You have already used the property above.text. But for certain types of data, such as images and other non-text data, it is often better to use.content, even if it returns a very similar result.

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds/1") >>> response.headers.get("Content-Type") 'application/json; charset=utf-8' >>> response.content b'{"weight":{"imperial":"6 - 13","metric":"3 - 6"}... 'Copy the code

As you can see,.content is the same as the.text. Used earlier, but by looking at the content-Type header of the response, you can see that the content is

application/json; A JSON object. For this type of content, the Requests library contains a specific.json() method that you can use to instantly convert API byte responses into Python data structures:

>>>

>>> response = requests.get("https://api.thedogapi.com/v1/breeds/1") >>> response.headers.get("Content-Type") 'application/json; charset=utf-8' >>> response.json() {'weight': {'imperial': '6 - 13', 'metric': '3 - 6'}, 'height': {'imperial': '9-11.5 ', 'metric':' 23-29 '}... } >>> response.json()["name"] 'Affenpinscher'Copy the code

As you can see, after executing Response.json (), you get a dictionary that you can use just like any other dictionary in Python.

Now, review a recent example running with the PlaceGOAT API and try to get the same goat image and see its contents:

>>>

>>> response = requests.get("http://placegoat.com/200/200") >>> response <Response [200]> >>> response.headers.get("Content-Type") 'image/jpeg' >>> response.content b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01\x00H\... 'Copy the code

In this case, because you are requesting an image,.content is not very helpful. In fact, it’s almost impossible to understand. However, you know that this is a JPEG image, so you can try to store it in a file and see what happens:

>>>

>>> response = requests.get("http://placegoat.com/200/200")
>>> response
<Response [200]>
>>> response.headers.get("Content-Type")
'image/jpeg'
>>> file = open("goat.jpeg", "wb")
>>> file.write(response.content)
>>> file.close()
Copy the code

Now, if you open the working folder, you’ll find a goate.jpeg file that is a random image of the goat you just got using the API. Isn’t that amazing?

HTTP method

When calling an API, there are several different methods, also known as verbs, that can be used to specify the action to be performed. For example, if you want to GET some data, you use the method GET, and if you want to create some data, you use the method POST.

You usually stick with GET requests when consuming data purely using apis, but here are the most common methods and their typical use cases:

These four methods are often referred to as CRUD operations because they let you reate, [EAD, u PDATE, and d elete resources.

** Note: ** There is an additional PATCH method that is also associated with CRUD operations, but it is slightly less than the four methods above. It is used to make partial modifications rather than completely replace the resource PUT.

You can read more about the differences between putPatches and learn about their different requirements.

If you’re curious about the rest of the HTTP methods, or if you just want to learn more about the ones already mentioned, check out Mozilla’s documentation.

So far, you’ve only used.get() to get data, but you can also use the Requests package for all other HTTP methods:

>>>

>>> requests.post("https://api.thedogapi.com/v1/breeds/1") >>> requests.get("https://api.thedogapi.com/v1/breeds/1") >>>  requests.put("https://api.thedogapi.com/v1/breeds/1") >>> requests.delete("https://api.thedogapi.com/v1/breeds/1")Copy the code

If you try these on the console, you’ll notice that most of them return the 405Method Not Allowed status code. This is because not all terminals will allow POST, PUT, or DELETE methods. Especially when you use public apis to read data, you’ll find that most apis only allow GET requests, because you’re not allowed to create or change existing data.

Query parameters

Sometimes, when you call an API, you get a lot of unneeded or unwanted data. For example, when you call the /breeds endpoint of TheDogAPI, you get a lot of information about a given breed. But in some cases, you might just want to extract some information about a given variety. This is where query parameters come in!

You may have seen or used query parameters while browsing online. For example, when watching a YouTube video, you have a similar

www.youtube.com/watch?v=aL5… Putting v= in the URL is what you call the query parameter. It usually comes after the base URL and the endpoint.

To add query parameters to a given URL, you must? Add a question mark () before the first query parameter. If you want to have multiple query parameters in your request, you can use ampersand (&) to separate them.

The same YouTube url has multiple query parameters like this: www.youtube.com/watch?v=aL5…

In the API world, query parameters serve as filters that you can send along with API requests to further narrow the response. For example, going back to the Random UserGenerator API, you know how to generate Random users:

>>>

>>> requests.get("https://randomuser.me/api/").json()
{'results': [{'gender': 'male', 'name':
{'title': 'Mr', 'first': 'Silvijn', 'last': 'Van Bekkum'},
'location': {'street': {'number': 2480, 'name': 'Hooijengastrjitte'},
'city': 'Terherne', 'state': 'Drenthe',
'country': 'Netherlands', 'postcode': 59904...}
Copy the code

However, suppose you specifically want to generate only random female users. According to the documentation, you can use the query parameter gender= :

>>>

>>> requests.get("https://randomuser.me/api/? gender=female").json() {'results': [{'gender': 'female', 'name': {'title': 'Mrs', 'first': 'Marjoleine', 'last': 'Van Huffelen'}, 'location': {'street': {'number': 8993, 'name': 'De Teebus'}, 'city': 'West-Terschelling', 'state': 'Limburg', 'country': 'Netherlands', 'postcode': 24241...}Copy the code

That’s great! Now suppose you only want to generate female users from Germany. Looking at the documentation again, you’ll find the section about Nationality, and you can use the query parameter NAT = :

>>>

>>> requests.get("https://randomuser.me/api/? gender=female&nat=de").json() {'results': [{'gender': 'female', 'name': {'title': 'Ms', 'first': 'Marita', 'last': 'Hertwig'}, "location" : {" street ": {' number: 1430, 'name' : 'and then to decide Waldstra e},' city ':' Velden ', 'the state: 'Rheinland-Pfalz', 'country': 'Germany', 'postcode': 30737...}Copy the code

With query parameters, you can begin to get more specific data from the API, making the entire experience more relevant to your needs.

To avoid reconstructing the URL over and over again, you can use this params property to send a dictionary of all query parameters to attach to the URL:

>>>

>>> query_params = {"gender": "female", "nat": "de"}
>>> requests.get("https://randomuser.me/api/", params=query_params).json()
{'results': [{'gender': 'female', 'name':
{'title': 'Ms', 'first': 'Janet', 'last': 'Weyer'},
'location': {'street': {'number': 2582, 'name': 'Meisenweg'},
'city': 'Garding', 'state': 'Mecklenburg-Vorpommern',
'country': 'Germany', 'postcode': 56953...}
Copy the code

You can apply the above to any other API you like. If you return to TheDogAPI, there is a method in the documentation that lets you filter the breed endpoints to return only breeds that match a particular name. For example, if you want to find labrador breeds, then you can implement q with query parameters:

>>>

>>> query_params = {"q": "labradoodle"}
>>> endpoint = "https://api.thedogapi.com/v1/breeds/search"
>>> requests.get(endpoint, params=query_params).json()
[{'weight': {'imperial': '45 - 100', 'metric': '20 - 45'},
'height': {'imperial': '14 - 24', 'metric': '36 - 61'},
'id': 148, 'name': 'Labradoodle', 'breed_group': 'Mixed'...}]
Copy the code

Do you have it! By sending q the query parameter Labradoodle with value, you can filter all varieties that match that particular value.

** Tip: ** When you reuse the same endpoint, it is best practice to define it as a variable at the top of your code. This makes your life easier as you interact with the API over and over again.

With the help of query parameters, you can further narrow down the request and specify exactly what you are looking for. Most oF the apis you can find on the web have some kind of query parameters that you can use to filter data. Remember to check the documentation and API reference to find them.

Learn advanced API concepts

Now that you have a good understanding of the basics of using apis in Python, there are some more advanced topics worth exploring, even briefly, such as authentication, paging, and rate limiting.

validation

API authentication is probably the most complex topic covered in this tutorial. Although many public apis are free and fully public, there are many more apis available behind some form of authentication. There are many apis that require authentication, but here are some good examples:

  • GitHub API

  • The twitter API

  • Instagram API

Authentication methods range from straightforward (such as those using API keys or basic authentication) to more complex and secure techniques (such as OAuth).

Typically, calling the API with no credentials or with incorrect credentials returns 401Unauthorized or 403 Forbidden status codes.

API keys

The most common level of authentication is the API key. These keys are used to identify you as an API user or customer and track your use of the API. The API key is typically sent as a request header or query parameter.

**NASA apis: ** One of the coolest collections of public apis is one provided by NASA. You can find the API to get astronomical images of the day or EPIC images.

For this example, you will use NASA’s MarsRover Photo API and get the Photo taken on July 1, 2020. For testing purposes, you can use the API key provided by DEMO_KEYNASA by default. Otherwise, you can quickly generate your own API by going to NASA’s main API page and clicking Start.

You can add the API key to your request by appending the API_key = query parameter:

>>>

>>> endpoint = "https://api.nasa.gov/mars-photos/api/v1/rovers/curiosity/photos"
>>> # Replace DEMO_KEY below with your own key if you generated one.
>>> api_key = "DEMO_KEY"
>>> query_params = {"api_key": api_key, "earth_date": "2020-07-01"}
>>> response = requests.get(endpoint, params=query_params)
>>> response
<Response [200]>
Copy the code

So far so good. You manage to make an authenticated request to NASA’s API and get a 200 OK response.

Now look at the Response object and try to extract some images from it:

>>>

>>> response.json() {'photos': [{'id': 754118, 'sol': 2809, 'camera': {'id': 20, 'name': 'FHAZ', 'rover_id': 5, 'full_name': 'Front Hazard Avoidance Camera'}, 'img_src': 'https://mars.nasa.gov/msl-raw-images/...JPG', 'earth_date': '2020-07-01', 'rover': {'id': 5, 'name': 'Curiosity', 'landing_date': '2012-08-06', 'launch_date': '2011-11-26', 'status': 'active'}}, ... } >>> photos = response.json()["photos"] >>> print(f"Found {len(photos)} photos") Found 12 photos >>> photos[4]["img_src"] 'https://mars.nasa.gov/msl-raw-images/proj/msl/redops/ods/surface/sol/02809/opgs/edr/rcam/RRB_646869036EDR_F0810628RHAZ0 0337M_.JPG'Copy the code

Using.json() to convert the response to a Python dictionary, and then photos gets the fields from the response, you can iterate through all the Photo objects and even get the image URL for a particular Photo. If you open the url in your browser, then you will see Mars taken by a Mars rover with the following picture:

For this example, you select a specific earth_date(2020-07-01) from the response dictionary (), and then select a specific photo 4. Before proceeding, try changing the date or getting the photo from a different camera to see how it changes the final result.

Introduction to request:

Another very common standard in API authentication is OAuth. In this tutorial, you will only learn the basics of OAuth, because this is a very broad topic.

Even if you don’t know it’s part of OAuth, you’ve probably seen and used the OAuth process many times. Whenever an application or platform has login or continue options, this is the starting point of the OAuth process:

Here’s a step-by-step breakdown of what happens when you click Continue using Facebook:

1. The Spotify app will require the Facebook API to initiate the authentication process. To do this, the Spotify application sends its application ID (client_id) and URL (redirect_URI) to redirect the user upon success or error.

2. You will be redirected to the Facebook site and asked to log in using your credentials. The Spotify application will not see or be able to access these credentials. This is the most important benefit of OAuth.

3. Facebook will show you all the data requested by the Spotify app from your profile and will ask you to accept or refuse to share that data.

4. If you accept Spotify access to your data, you will be redirected back to the Spotify app that you are logged in to.

When you perform Step 4, Facebook will provide Spotify with a special credential (access_token) that can be reused to get your information. This particular Facebook login token is valid for 60 days, but other applications may have a different validity period. If you’re curious, Facebook has a Settings page where you can see which apps have received your Facebook access token.

Now, from a more technical point of view, here’s what you need to know when using the API with OAuth:

  • You need to create an application with ID (app_id or client_id) and secret (app_secret or client_secret).

  • You need a redirect URL (redirect_URI) that the API will use to send you information.

  • You get a code as a result of authentication that you need to exchange for an access token.

There are some variations above, but in general, most OAuth processes have steps similar to these.

** Tip: ** When you are just testing and need some kind of redirect URL to get. Code, you can use a service named httpbin.

More specifically, you can use it httpbin.org/anything as a heavy… URL, because it just outputs whatever it gets as input. You can test it yourself by navigating to the URL.

Next, you’ll delve into an example using the GitHub API!

OAuth: A practical example

As you saw above, the first thing you need to do is create an application. You can refer to the GitHub documentation for detailed step-by-step instructions on how to do this. The only thing to remember is to mention httpbin.org/anything above… Used for the authorization callback URL field.

**GitHub API: ** You can use the GitHub API for many different use cases, such as getting a list of repositories that you belong to, getting a list of followers that you have, and so on.

Once the application is created, copy and paste the Client_ID and Client_Secret redirect urls of your choice into a Python file named github. Py:

import requests # REPLACE the following variables with your Client ID and Client Secret CLIENT_ID = "<REPLACE_WITH_CLIENT_ID>" CLIENT_SECRET = "<REPLACE_WITH_CLIENT_SECRET>" # REPLACE the following variable with what you  added in the # "Authorization callback URL" field REDIRECT_URI = "<REPLACE_WITH_REDIRECT_URI>"Copy the code

Now that you have all the important variables in place, you need to be able to create a link to redirect users to their GitHub accounts, as described in the GitHub documentation:

def create_oauth_link():
    params = {
        "client_id": CLIENT_ID,
        "redirect_uri": REDIRECT_URI,
        "scope": "user",
        "response_type": "code",
    }

    endpoint = "https://github.com/login/oauth/authorize"
    response = requests.get(endpoint, params=params)
    url = response.url
    return url
Copy the code

In this code, you first define the required parameters expected by the API, then use the Requests package and.get(). When you make a request to the /login/ oAuth/Authorize endpoint, the API automatically redirects you to the GitHub web site. In this case, you want the URL to take parameters from the response. This parameter contains the exact URL that GitHub redirects you to.

The next step in the authorization process is to exchange the code you obtain for access tokens. Again, you can create a method for it by following the steps in the GitHub documentation:

def exchange_code_for_access_token(code=None):
    params = {
        "client_id": CLIENT_ID,
        "client_secret": CLIENT_SECRET,
        "redirect_uri": REDIRECT_URI,
        "code": code,
    }

    headers = {"Accept": "application/json"}
    endpoint = "https://github.com/login/oauth/access_token"
    response = requests.post(endpoint, params=params, headers=headers).json()
    return response["access_token"]
Copy the code

Here, you POST a request to exchange the code to access the token. In this request, you must send your CLIENT_SECRETandcode so that GitHub can verify that this particular code was originally generated by your application. Only then will the GitHub API generate a valid access token and return it to you.

Now you can add the following to your file and try to run it:

link = create_oauth_link()
print(f"Follow the link to start the authentication with GitHub: {link}")
code = input("GitHub code: ")
access_token = exchange_code_for_access_token(code)
print(f"Exchanged code {code} with access token: {access_token}")
Copy the code

If all goes according to plan, you should be rewarded with a valid access token that you can use to invoke the GitHub API to impersonate an authenticated user.

Now try adding the following code to get your User profile using the User API and print out your name, username, and number of private repositories:

def print_user_info(access_token=None):
    headers = {"Authorization": f"token {access_token}"}
    endpoint = "https://api.github.com/user"
    response = requests.get(endpoint, headers=headers).json()
    name = response["name"]
    username = response["login"]
    private_repos_count = response["total_private_repos"]
    print(
        f"{name} ({username}) | private repositories: {private_repos_count}"
    )
Copy the code

Now that you have a valid access token, you need to send it in all API requests using the Authorization header. The response to your request will be a Python dictionary containing all user information. From this dictionary, you want to get the fields name, login and total_private_repos. You can also print the Response variable to view other available fields.

Well, it should be! The only thing left to do is to put them together and try it out:

1import requests 2 3# REPLACE the following variables with your Client ID and Client Secret 4CLIENT_ID = "<REPLACE_WITH_CLIENT_ID>" 5CLIENT_SECRET = "<REPLACE_WITH_CLIENT_SECRET>" 6 7# REPLACE the following variable with what  you added in 8# the "Authorization callback URL" field 9REDIRECT_URI = "<REPLACE_WITH_REDIRECT_URI>" 10 11def create_oauth_link(): 12 params = { 13 "client_id": CLIENT_ID, 14 "redirect_uri": REDIRECT_URI, 15 "scope": "user", 16 "response_type": "code", 17 } 18 endpoint = "https://github.com/login/oauth/authorize" 19 response = requests.get(endpoint, params=params) 20 url = response.url 21 return url 22 23def exchange_code_for_access_token(code=None): 24 params = { 25 "client_id": CLIENT_ID, 26 "client_secret": CLIENT_SECRET, 27 "redirect_uri": REDIRECT_URI, 28 "code": code, 29 } 30 headers = {"Accept": "application/json"} 31 endpoint = "https://github.com/login/oauth/access_token" 32 response = requests.post(endpoint, params=params, headers=headers).json() 33 return response["access_token"] 34 35def print_user_info(access_token=None): 36 headers = {"Authorization": f"token {access_token}"} 37 endpoint = "https://api.github.com/user" 38 response = requests.get(endpoint, headers=headers).json() 39 name = response["name"] 40 username = response["login"] 41 private_repos_count = response["total_private_repos"] 42 print( 43 f"{name} ({username}) | private repositories: {private_repos_count}" 44 ) 45 46link = create_oauth_link() 47print(f"Follow the link to start the authentication with GitHub: {link}") 48code = input("GitHub code: ") 49access_token = exchange_code_for_access_token(code) 50print(f"Exchanged code {code} with access token: {access_token}") 51print_user_info(access_token=access_token)Copy the code

The following happens when the above code is run:

1. A link is generated asking you to go to the GitHub page for authentication.

2. After following the link and logging in with your GitHub credentials, you will be redirected to the callback URL you defined, which contains

One of the fields in the code query parameter:

1. After pasting the code in your console, you can exchange the code for reusable access tokens.

2. Your user information is obtained using this access token. Print your name, username, and private repository count.

If you follow these steps, you should get an end result similar to this:

$ John Doe (johndoe) | number of private repositories: 42
Copy the code

There are many steps to follow, but it’s important that you take the time to really understand each step. Most apis that use OAuth will share many of the same behaviors, so understanding the process fully when you read data from the API will unlock a lot of potential.

Feel free to refine the example and add more features, such as getting your public and star-tagged repositories or traversing your followers to identify the most popular repositories.

There are many excellent resources on the web about OAuth, and if using the API behind OAuth is what you really need, I recommend that you do more research specifically on that topic. Here are some good places to start:

  • What exactly is OAuth?

  • OAuth 2 simplified version

  • OAuth 2.0 authorization framework

From an API consumption perspective, knowing OAuth will certainly come in handy when you interact with public apis. Most apis adopt OAuth as their authentication standard, and for good reason.

paging

Sending large amounts of data back and forth between clients and servers comes at a price: bandwidth. To ensure that the server can handle a large number of requests, apis often use paging.

In very simple terms, paging is the act of breaking up large amounts of data into smaller parts. For example, whenever you go to the problems page in Stack Overflow, you’ll see something like this at the bottom:

You probably recognize this from many other sites, and the concept is roughly the same from site to site. In the case of apis, this is usually handled with the help of query parameters, which are as follows:

1. A page, which page attributes you are currently requesting to be defined

2. Size defines the size attribute for each page

The exact query parameter names can vary widely from API developer to API developer, but the concepts are the same. Some API players may also use HTTP headers or JSON responses to return the current paging filter.

Using the GitHub API again, you can find the event endpoint in the document that contains the paging query parameter. This parameter per_page= defines the number of items to return, and page= allows you to paginate multiple results. Here’s how to use these parameters:

>>>

>>> response = requests.get("https://api.github.com/events?per_page=1&page=0")
>>> response.json()[0]["id"]
'14345572615'
>>> response = requests.get("https://api.github.com/events?per_page=1&page=1")
>>> response.json()[0]["id"]
'14345572808'
>>> response = requests.get("https://api.github.com/events?per_page=1&page=2")
>>> response.json()[0]["id"]
'14345572100'
Copy the code

With the first URL, you can only get one event. But with the page= query parameter, you can continue to page the results, ensuring that you can get all the events without overloading the API.

The speed limit

Given that apis are public and can be used by anyone, malicious people often try to abuse them. To prevent such attacks, you can use a technique called rate limiting, which limits the number of requests a user can make in a given time frame.

If you regularly exceed defined rate limits, some apis may actually block your IP or API key. Be careful not to exceed the limits set by the API developer. Otherwise, you may need to wait a while before calling the API again.

For the example below, you will use the GitHub API and/Events endpoint again. According to its documentation, GitHub allows about 60 unauthenticated requests per hour. If you exceed this value, you’ll get a 403 status code and won’t be able to make any more API calls for quite some time.

** Warning: ** Running the next piece of code does prevent you from calling GitHub for a while, so make sure you don’t need access to GitHub’s API for a while before running it.

For demonstration purposes, you’ll purposefully try to exceed GitHub’s rate limit and see what happens. In the following code, you will request the data until you get the status code 200 OK in addition:

>>>

>>> endpoint = "https://api.github.com/events" >>> for i in range(100): >>> response = requests.get(endpoint) >>> print(f"{i} - {response.status_code}") >>> if response.status_code ! = 200: >>> break 0 - 200 1 - 200 2 - 200 3 - 200 4 - 200 5 - 200 ... 55 - 200 56 - 200 57 - 403 >>> response <Response [403]> >>> response.json() {'message': "API rate limit exceeded for <ip-address>.", 'documentation_url': 'https://developer.github.com/v3/#rate-limiting'}Copy the code

You have it: After about 60 requests, the API stops returning a 200 OK response and a 403 Forbidden response notifying you that the API rate limit has been exceeded.

Some apis, such as GitHub’s, might even include additional information in the header about the current rate limit and the number of requests remaining. These can be very helpful for you to avoid exceeding defined limits. Take a look at the latest Response.headers and see if you can find those specific rate-limiting headers.

Using the API with Python: a real-world example

Now that you know all the theory and have tried out some apis, you can reinforce this knowledge with some more practical examples. You can modify the examples below to customize them for your own purposes.

You can learn these examples by downloading the source code provided at the following link:

Search and get popular GIFs

How do I create a script to get the top three popular GIFs from GIPHY? To do this, you need to create an application and get the API key from GIPHY. You can find instructions by expanding the box below or check out GIPHY’s Quick Start documentation.

Create a GIPHY application to show hide

Once you have the API key, you can start writing some code to use the API. However, sometimes you want to run some tests before implementing a lot of code. I know, I know. The problem is that some apis actually give you tools to get API data directly from documents or their dashboards.

In this particular case, GIPHY provides you with an API Explorer that allows you to start using the API after you create the application without having to write a line of code.

Some of the other apis will provide you with a resource manager in the document itself, which is what TheDogAPI does at the bottom of each API reference page.

Anyway, you can always use code to use the API, and that’s what you’re going to do here. Get the API key from the dashboard, and you can start using the GIPHY API by replacing the value of the API_KEY variable below:

 1import requests
 2
 3# Replace the following with the API key generated.
 4API_KEY = "API_KEY"
 5endpoint = "https://api.giphy.com/v1/gifs/trending"
 6
 7params = {"api_key": API_KEY, "limit": 3, "rating": "g"}
 8response = requests.get(ENDPOINT, params=params).json()
 9for gif in response["data"]:
10    title = gif["title"]
11    trending_date = gif["trending_datetime"]
12    url = gif["url"]
13    print(f"{title} | {trending_date} | {url}")
Copy the code

At line 4 and line 5 at the top of the file, you define the API and GIPHYAPI for your API_KEY, which must be set to the endpoint because they don’t change as often as the other parts.

In line 7, use what you learned in the query parameters section to define params and add your own API key. You also include some other filters: limit gets 3 results and rating gets only the appropriate content.

Finally, after you get the response, you iterate through the result on line 9. For each GIF, you print its title, date, and URL in line 13.

Running this code in the console outputs a somewhat structured list of GIFs:

Excited Schitts Creek GIF by CBC | 2020-11-28 20:45:14 | https://giphy.com/gifs/cbc-schittscreek-schitts-creek-SiGg4zSmwmbafTYwpj
Saved By The Bell Shrug GIF by PeacockTV | 2020-11-28 20:30:15 | https://giphy.com/gifs/peacocktv-saved-by-the-bell-bayside-high-school-dZRjehRpivtJsNUxW9
Schitts Creek Thank You GIF by CBC | 2020-11-28 20:15:07 | https://giphy.com/gifs/cbc-funny-comedy-26n79l9afmfm1POjC
Copy the code

Now, suppose you want to make a script that allows you to search for a particular word and get the first GIPHY match for that word. The different endpoints and slight variations of the code above can do this quickly:

import requests

# Replace the following with the API key generated.
API_KEY = "API_KEY"
endpoint = "https://api.giphy.com/v1/gifs/search"

search_term = "shrug"
params = {"api_key": API_KEY, "limit": 1, "q": search_term, "rating": "g"}
response = requests.get(endpoint, params=params).json()
for gif in response["data"]:
    title = gif["title"]
    url = gif["url"]
    print(f"{title} | {url}")
Copy the code

Do you have it! Now you can modify this script to your liking and generate GIFs on demand. Try getting GIFs from your favorite shows or movies, add shortcuts to your terminal to get the most popular GIFs on demand, or integrate with another API in your favorite messaging system (WhatsApp, Slack, etc.). Then start sending GIFs to your friends and colleagues!

Obtain confirmed COVID-19 cases for each country/region

Although you may be tired of this thing by now, there is a free API that contains the latest global COVID-19 data. This API doesn’t require authentication, so it’s easy to get some data right away. The free version you’ll use below has rate limits and some data limits, but is sufficient for small use cases.

For this example, you get the total number of confirmed cases as of the previous day. Again I chose Germany at random, but you can choose any country/region you like:

 1import requests
 2from datetime import date, timedelta
 3
 4today = date.today()
 5yesterday = today - timedelta(days=1)
 6country = "germany"
 7endpoint = f"https://api.covid19api.com/country/{country}/status/confirmed"
 8params = {"from": str(yesterday), "to": str(today)}
 9
10response = requests.get(endpoint, params=params).json()
11total_confirmed = 0
12for day in response:
13    cases = day.get("Cases", 0)
14    total_confirmed += cases
15
16print(f"Total Confirmed Covid-19 cases in {country}: {total_confirmed}")
Copy the code

In lines 1 and 2, you import the necessary modules. In this case, you have to import the Date and Timedelta objects to get today’s and yesterday’s dates.

In lines 6 through 8, you define the query parameters for the country/region SLUG, endpoint, and API requests to use.

The response is a list of days, and for each day, you have a case field containing the total number of confirmed Cases for that date. In line 11, you create a variable to hold the total number of confirmed cases, and then in line 14, you iterate over all the days and add them up.

Printing the final results will show the total number of confirmed cases in the selected country/region:

Total Confirmed Covid-19 cases in germany: 1038649
Copy the code

In this example, you are looking at the total number of confirmed cases for the entire country. However, you can also try to view the documentation and get data for a specific city. Why not make it more thorough and get some other data, such as the number of recovered cases?

Search Google Books

If you’re passionate about books, you probably need a way to quickly search for specific books. You might even want to connect it to your local library’s search to see if a given book can be found using the book’s ISBN.

For this example, you will use the Google Books API and a public volume endpoint to perform a simple book search.

Here’s moby Dick’s simple code for finding words in the entire directory:

1import requests
 2
 3endpoint = "https://www.googleapis.com/books/v1/volumes"
 4query = "moby dick"
 5
 6params = {"q": query, "maxResults": 3}
 7response = requests.get(endpoint, params=params).json()
 8for book in response["items"]:
 9    volume = book["volumeInfo"]
10    title = volume["title"]
11    published = volume["publishedDate"]
12    description = volume["description"]
13    print(f"{title} ({published}) | {description}")
Copy the code

This code example is very similar to the code examples you’ve seen before. Starting with lines 3 and 4, define important variables, such as endpoints, in this case queries.

After making the API request, start iterating through the results at line 8. Then, at line 13, print the most interesting information for each book that matches the initial query:

Moby-Dick (2016-04-12) | "Call me Ishmael." So begins the famous opening...
Moby Dick (1892) | A literary classic that wasn't recognized for its...
Moby Dick; Or, The Whale (1983-08-16) | The story of Captain Ahab's...
Copy the code

You can book to print variables inside the loop to see what other fields you have available. Here are some things that might help further improve this code:

  • industryIdentifiers

  • AverageRating and ratingsCount

  • imageLinks

An interesting challenge with this API is to use your OAuth knowledge and create your own bookshelf application to record all the books you read or want to read. Then you can even connect it to your favorite bookstore or library to quickly find books near you from your wish list. This is just one idea — I’m sure you can think of more.

conclusion

There are a million other things you can learn about the API: different headers, different content types, different authentication techniques, and so on. However, the concepts and techniques you learn in this tutorial will allow you to practice with any API you like and use Python for any API usage needs you may have.

Click to follow, the first time to learn about Huawei cloud fresh technology ~