preface

Poverty limits the my imagination, so long from university into the society, from the beginning of the cheerful, feel what is difficult to earn money, to the present applauded the dozen face, very true, now that rent a house, in the crowded city, want to have a belongs to own space, small make up today to crawl HOME LINK secondary data, It can save money and avoid a lot of time to see a house, understand the second-hand data in advance, not into the pit, not into the pit.

I. Find the location of data:

Open the official website of Lianjia, enter the second-hand housing page, select a city, you can see the total number of housing resources and housing listing data of the city.

Two, determine the data storage location:

Some websites’ data is stored in HTML, while some have API interfaces, and even some are encrypted in JS. Fortunately, linkedhome’s housing data is stored in HTML:



Obtain HTML data:

Get HTML data for each page in requests

# url crawl, crawl by default in nanjing HOME LINK property information url = '{} / https://nj.lianjia.com/ershoufang/pg'. The format (page) # request url resp = requests. Get (url, headers=headers, timeout=10)Copy the code

Parse HTML to extract useful data:

BeautifulSoup parses the HTML and extracts the relevant useful data

soup = BeautifulSoup(resp.content, Sellistcontent = soup. Select ('.sellListContent Li.logClickData ') # Loop for sell in sellListContent: Self. select('div. Title a')[0].string HouseInfo = list(self.select ('div.houseInfo')[0].stripped_strings Info = houseInfo [0]. The split type (' | ') # house house_type = info [1]. The strip () # size area = info [2]. The strip () toward toward = # room Info [3]. Strip () # toptoping = info[4]. Strip () # toptoping = info[3] '.join(list(self. select(' div.positioninfo ')[0].stripped_strings) Join (list(self. select(' div.totalprice ')[0].stripped_strings) list(sell.select('div.unitPrice')[0].stripped_strings)[0]Copy the code

I hope it will be helpful to those friends who are confused because of renting a house in another country. Learning Python can not only help us with our work, but also facilitate our life.

Click to get a study resource (● ‘psa’ ●) Blue ♥ Python answer, to meet problems don’t panic, come here ~ basic learning materials, data analysis, crawl video, to here ~\