This article is participating in Python Theme Month. See the link to the event for more details

About grequests

We all know that requests are in the hands of the gods, but they are serial, which makes them less efficient in concurrent scenarios due to blocking. He once thought that applications were designed for humans, but when people needed to send requests asynchronously to make them more efficient, he released the new grequests library (just over 100 lines of code).

Grequests builds on gEvents and requests. It works just as well as if you were using GEvent alone to send requests, but it’s much easier to do with Grequests. Should say not, K god cow!

Walk into grequests

Brothers who have used Requests will be familiar with Grequests, as you’ll notice that the use of quests is very similar.

The installation

pip install grequests
Copy the code

example

Use grequests asynchronous request Baidu

import grequests,time

url = 'https://www.baidu.com'
req_list = [
    grequests.get(url),
    grequests.get(url),
    grequests.get(url),
]

start_time=time.time()
res_list = grequests.map(req_list)
end_time=time.time()
print(res_list,end_time-start_time)
Copy the code

Use requests for serial requests

import  requests,time

url = 'https://www.baidu.com'
start_time=time.time()
res1 = requests.get(url)
res2 = requests.get(url)
res3 = requests.get(url)
end_time=time.time()

print([res1,res2,res3],end_time-start_time)
Copy the code

Comparison and analysis

Based on the results, you can see that Grequests and requests have exactly the same functionality, but a rough estimate of grequests’ performance over requests is more than 186%. The greater the concurrency, the greater the upgrade.

About grequests. The map ()

Greqles.map () receives an AsyncRequest list object, which the map method converts into a task queue that gEvent can then execute asynchronously. A response list is returned after all tasks are completed.

Advanced usage

Custom exception handling

import grequests,time

# Custom error handler that accepts request and Exception as parameters
def err_handler(request,exception) :
    print('An exception has occurred with the following information:',exception)

url = 'https://www.not-ok.com'
req_list = [
    grequests.get(url),
    grequests.get(url),
    grequests.get(url),
]

start_time=time.time()

# Note that err_handler is here
res_list = grequests.map(req_list,exception_handler=err_handler)
end_time=time.time()
print(res_list,end_time-start_time)
Copy the code

Other USES

Through the source code we know:

That’s all for today, thanks for reading, see you next time.