Believe the zodiac we are not strange, everyone born time has close relationship with the zodiac, babies born in each equal amounts of time, just as this constellation, this with our Chinese zodiac is quite similar to that of each of the 12 constellation with the myth of color, each have different character represents a constellation and luck, So today’s little project is to climb the horoscope.

1. Web crawler

import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt
import matplotlib.style as psl
psl.use('ggplot')

import warnings

No warning
warnings.filterwarnings('ignore') 
print('Library import complete! ')
Copy the code

Library import complete!

2, historical constellation data collection – website production

Start by importing the crawled constellation name and date name, which can be used for web url composition.

xz_lst = ['leo'.'libra'.'scorpio'.'aquarius'.'taurus'.'gemini'.'cancer'.'virgo'.'sagittarius'.'capricorn'.'pisces'.'aries']

date_lst = ['20190101'.'20190102'.'20190103'.'20190104'.'20190105'.'20190106'.'20190107'.'20190108'.'20190109'.'20190110'.'20190111'.'20190112'.'20190113'.'20190114'.'20190115'.'20190116'.'20190117'.'20190118'.'20190119'.'20190120'.'20190121'.'20190122'.'20190123'.'20190124'.'20190125'.'20190126'.'20190127'.'20190128'.'20190129'.'20190130'.'20190131'.'20190201'.'20190202'.'20190203'.'20190204'.'20190205'.'20190206'.'20190207'.'20190208'.'20190209'.'20190210'.'20190211'.'20190212'.'20190213'.'20190214'.'20190215'.'20190216'.'20190217'.'20190218'.'20190219'.'20190220'.'20190221'.'20190222'.'20190223'.'20190224'.'20190225'.'20190226'.'20190227'.'20190228'.'20190301'.'20190302'.'20190303'.'20190304'.'20190305'.'20190306'.'20190307'.'20190308'.'20190309'.'20190310'.'20190311'.'20190312'.'20190313'.'20190314'.'20190315'.'20190316'.'20190317'.'20190318'.'20190319'.'20190320'.'20190321'.'20190322'.'20190323'.'20190324'.'20190325']
Copy the code

3. Url synthesis

url_lst = []
for i in xz_lst:
    for j in date_lst:
        # Synthesizes the url from the two lists in [2]
        url_lst.append('http://www.xzw.com/fortune/' + i + '/' + j + '.html')  
# url_lst
Copy the code

4. Web page data collection

From each site, select keywords. In this step, it is necessary to analyze the source code of the web page, find the analysis characteristics between several groups of features, and then use the split function to split the text and choose the appropriate keywords.

def get_data(ui) :
    ri = requests.get(url = ui)   # Visit the page
    soupi = BeautifulSoup(ri.text, 'lxml')  # Parse page
    dic = {}
    dic['title'] = soupi.find('h4').text
        # Get title information
    infor1 = soupi.find('div',class_="c_main").find('ul').find_all('li')
    dic[infor1[0].text] = infor1[0].find('em') ['style'].split(':') [1].split('p') [0]
    dic[infor1[1].text] = infor1[1].find('em') ['style'].split(':') [1].split('p') [0]
    dic[infor1[2].text] = infor1[2].find('em') ['style'].split(':') [1].split('p') [0]
    dic[infor1[3].text] = infor1[3].find('em') ['style'].split(':') [1].split('p') [0]
    dic[infor1[4].find('label').text] = infor1[4].text.split('：') [1]
    dic[infor1[5].find('label').text] = infor1[5].text.split('：') [1]
    dic[infor1[6].find('label').text] = infor1[6].text.split('：') [1]
    dic[infor1[7].find('label').text] = infor1[7].text.split('：') [1]
    dic[infor1[8].find('label').text] = infor1[8].text.split('：') [1]
    dic[infor1[9].find('label').text] = infor1[9].text.split('：') [1]
        Obtain information such as horoscope
    infor2 = soupi.find('div',class_="c_cont").find_all('p')
    dic['Comprehensive Horoscope text'] = infor2[0].find('span').text
    dic['Love Horoscope Words'] = infor2[1].find('span').text
    dic['Career and Academic writing'] = infor2[2].find('span').text
    dic['Fortune and Fortune Words'] = infor2[3].find('span').text
    dic['Health Horoscope'] = infor2[4].find('span').text
        Get horoscope text information
    return dic
print('Function is built! ')
Copy the code

The function is built!

data_lst = []
n = 0
for u in url_lst:
    try:
        # Batch data collection
        data_lst.append(get_data(u))
        n += 1
        if n%100= =0 :
            print('% I data collected successfully' % n)
    except:
        continue
Convert the data to Pandas formatDf_result = pd.DataFrame(data_lst) Collection is successful100A piece of data is collected successfully200A piece of data is collected successfully300A piece of data is collected successfully400A piece of data is collected successfully500A piece of data is collected successfully600A piece of data is collected successfully700A piece of data is collected successfully800A piece of data is collected successfully900A piece of data is collected successfully1000The data df_result. Head ()Copy the code

5. Renderings:

The History of Aries

In an ancient and distant country, the king and queen divorced because of their disharmony and took another woman. Unfortunately, the new queen was born with jealousy and could not bear the king’s love for the two children left by his ex-wife, so an evil plot gradually took shape in her mind. When spring came, it was the season of sowing and planting, and the new queen sent the fried wheat to the unsuspecting farmers throughout the country. Of course, no matter how much you water and fertilize the ripe wheat, it will not sprout. The farmer, who was in the dark, was puzzled. At this time, the new queen spread rumors about the wheat, saying that the wheat could not sprout because the country was cursed, and that the curse was all because of the evil thoughts of the prince and princess, which caused the wrath of the heaven and caused the god to punish the country. Honest and simple personality of the farmers a listen, god! That’s good! How terrible it would be that all the people in the land would fall into poverty and hunger because of the wicked prince and princess! Soon, men and women, young and old, all in all parts of the country demanded that the king must put the prince and princess to death, so that the curse would be lifted, the wrath of heaven would be appeased, the people’s hard work would be fruitful, and the country would return to its former stability and wealth. The king was reluctant to give up, but in order to appease the public anger, he reluctantly agreed to the demand of the people. He was ready to put the princess and the prince to death in order to win the trust of the people. When the news reached the mother of the prince and princess, she was both frightened and frightened, and she hurried to the great god Zeus for help. Zeus, of course, knew that it was the queen who was responsible, so he agreed to help. On the day of execution, a ram with golden hair suddenly appeared in the sky and rescued the princes. On the way to fly across the sea, this ram was not careful and let his sister fall into the sea and died. Later, Zeus rewarded this brave but careless ram by hoisting him up in the sky, where he is now known as Aries

That’s all for now, thanks for watching. I’m a programmer who likes to share more Python content.