This is the 19th day of my participation in the Genwen Challenge

I heard that you are in your favorite star wallpaper? Such as Li Yifeng, Wang Yibo, Yi Yangqianxi, Wang Yuan, Wang Junkai, Li Xian and so on. Download these star hd wallpapers **

Recently map panic, nothing to do to climb li Yifeng hd pictures, as mobile phone wallpaper is also a good choice. No more nonsense, the following effect:

Isn’t it handsome?

The steps to crawl are also very simple:

1. Get the target url. 2. The browser captures the packet and obtains the specific URL of the picture. 3. Write code in Python to request urls with the Requests module and download images to a local folder.

This article is through the object-oriented method, the code is more stable, as many images as you want to download as many images.

1. Analyze target websites

First of all, in order to crawl the image, the first step is definitely to analyze the target url, in order to get the URL of the image. Open the console of the browser, capture the package of the web page, and find that the picture on the web page is loaded by waterfall flow, so it is sliding down, and the url pointed to by the lower arrow is obtained, then whether this is what we need.

To verify that the site has the data we need, open Preview and find that this is what we need, and the data is presented to us in JSON format.

It’s much easier to grab the urls we need, and then request those urls via the Requests module.

2. Write code

Let’s go ahead and import the modules we need first

import requests
import time
import random
from urllib.request import urlretrieve
import os
Copy the code

Let’s start knocking on the code, through the object oriented way, first write down the specific framework

class Wangyibo(object) :
    def __init__(self,name) :
        self.headers = {
            'User-Agent': 'the Mozilla / 5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'
        }
        self.name = name
 
 
if __name__ == "__main__":
    # Celebrity pictures
    name="Li Yifeng"
    Wangyibo(name).run()

Copy the code

In order not to be discovered by the website we are a crawler and not a browser, so we need to include the request header.

def __init__(self) :
  self.headers = {
  'User-Agent':'the Mozilla / 5.0 (Windows NT 10.0; Win64; X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36'
  }
Copy the code

Now to get on track, send a request to the url, temporarily climbing nine pages

def run(self) :
        for i in range(0.200.24):
            url = 'https://www.duitang.com/napi/blog/list/by_search/?kw='+self.name+'&type=feed&include_fields=top_comments%2Cis_root%2Csource_link%2Citem%2Cbuyable%2Croot_id%2Cstatus%2Clike_count%2Clike_ id%2Csender%2Calbum%2Creply_count%2Cfavorite_blog_id&_type=&start={}'.format(
                i)
            self.request(url)
Copy the code

Make a request to the url below to download the image locally

def download_pic(self, href) :
        # check whether the folder exists in the current directory, if not create a folder
        if not os.path.exists(self.name):
            os.mkdir(self.name)
 
 
        name = os.path.split(href)[1]
        urlretrieve(href, '/'+self.name+'/ {}'.format(name))
        print("=================={} Download completed ===================".format(name))
        time.sleep(random.random())
Copy the code

    def request(self, url) :
        response = requests.get(url, headers=self.headers)
        time.sleep(random.uniform(0.1))
        lists = response.json()['data'] ['object_list']
        for list in lists:
            pic_url = list['photo'] ['path']
            self.download_pic(pic_url)  # pic_URL is the url of the image
Copy the code

In order to prevent crawling too fast, IP may be blocked, so every crawling image should rest for a certain time to ensure the safety and stability of the crawler. Finally run the whole program, after a while, Wang Yibo’s handsome photos are all downloaded to the local.

3, conclusion

1, in order to facilitate everyone to learn, I put the complete source of this article to upload, need to get through the public of the same name: star wallpaper

2, the whole program code is relatively very simple, interested partners can try their own.

if __name__ == "__main__":
    # Celebrity pictures
    name="Li Yifeng"
    Wangyibo(name).run()
Copy the code

3, here to change your favorite star name can be, his/her HD wallpaper is for you.