Use Python to crawl all kinds of fund data and dynamic display

preface

Last year, I got in touch with the fund and felt the fragrance of the fund (really sweet). These days are also the days when we started trading after the Chinese New Year. Today, I climb the data of “Egg roll Fund” and show the ups and downs of the fund through the Visual way of Pyecharts GIF.

This paper will focus on the three points to climb the data and visually display the data in the GIF:

Nearly a month up or down the top 10
Fund each stage rise and fall
The net value of nearly 30 trading days

Data acquisition

The data source

Data source for this article: egg Roll Foundation

https://danjuanapp.com/
Copy the code

I read this article before maotaixiang technology! Python analysis of how profitable liquor funds are! Have learned how to crawl data in “egg roll fund” (Ajax asynchronous interaction), can not go to see, the article has detailed steps!!

The data analysis

The next data to be retrieved involves five categories (five funds)

Equity fund
Hybrid fund
Bond fund
Index fund
QDII fund

Analyzing the rules of Ajax asynchronous interactive links through packet capture:

Type is the code name for each of the five funds

Order_by is the order corresponding to the recent rise and fall of the fund

'Nearly a week':'1w'
'Last Month':'1m'
'Last Three months':'3m'
'Last June':'6m'
'Nearly 1 year':'1y'
'Last 2 years':'2y'
'Last 3 years':'3y'
'Last 5 years':'5y'
Copy the code

Page is the corresponding number of pages, starting on page 1

Note: “egg roll fund” this website has no anti climb!! Request does not require cookies!!

Ok, now that you’re clear, you can start to crawl the data!

Data visualization

Due to ** “egg roll fund” this website **** no anti climb!! **, so data crawling and visual analysis together (directly after crawling data visualization!) \

Analysis 1: The top 10 in recent months

The crawler code

### Fund type
dict_type={"Stock type": 1,"Hybrid": 3."Bond type": 2."Exponential": 5,"QDII type"11} :# # #
dict_time={'Nearly a week':'1w'.'Last Month':'1m'.'Last Three months':'3m'.'Last June':'6m'.'Nearly 1 year':'1y'.'Last 2 years':'2y'.'Last 3 years':'3y'.'Last 5 years':'5y'}


for key in dict_type:
    url = "https://danjuanapp.com/djapi/v3/filter/fund?type="+str(dict_type[key])+"&order_by=1w&size=10&page=1"
    res = requests.get(url, headers=headers)
    res.encoding = 'utf-8'
    s = json.loads(res.text)
    s = s['data'] ['items']
    name = []
    value = []
    for i in range(0,len(s)):
        print(s[i]['fd_name'] +":"+s[i]['yield'])
        name.append(s[i]['fd_name'])
        value.append(s[i]['yield'])
    ### Start drawing
    pie(name, value, str(key)+"Fund ups and downs"."["+str(key)+"] Fund in the last month up or down top 10")
Copy the code

Pie chart visualization code

# # # the pie chart
def pie(name,value,picname,tips) :
    c = (
        Pie()
            .add(
            ""[list(z) for z in zip(name, value)],
            The first item in the array is the abscissa and the second item is the ordinate
            When set to percentage, the first item is relative to the container width and the second item is relative to the container height
            center=["35%"."50%"],
        )
            .set_colors(["blue"."green"."yellow"."red"."pink"."orange"."purple"])  # set color
            .set_global_opts(
            title_opts=opts.TitleOpts(title=""+str(tips)),
            legend_opts=opts.LegendOpts(type_="scroll", pos_left="70%", orient="vertical"),  # Adjust the legend position
        )
            .set_series_opts(label_opts=opts.LabelOpts(formatter="{b}: {c}"))
            .render(str(picname)+".html"))Copy the code

Here, the pie chart visualization code is encapsulated into a function that is simply called to draw the pie charts of the five funds

### Start drawing
pie(name, value, str(key)+"Fund ups and downs"."["+str(key)+"] Fund in the last month up or down top 10")
Copy the code

1. Equity funds

2. Hybrid funds

Bond funds

Index funds

5. The QDII funds

Analysis of the

The figure above is the selection of five categories of funds in nearly a month of the highest rise and fall of the top 10 funds for drawing.

Similarly, a week, three months, or a year can be drawn using this code, just by changing the parameter order_BY

'Nearly a week':'1w'
'Last Month':'1m'
'Last Three months':'3m'
'Last June':'6m'
'Nearly 1 year':'1y'
'Last 2 years':'2y'
'Last 3 years':'3y'
'Last 5 years':'5y'
Copy the code

Analysis 2: fund each stage rise and fall

The above analysis can be clear about the five categories of funds in nearly a month of the highest ranking, the following from the ranking of the first selected fund (five categories of each selected the first) respectively show the fund in each stage of the rise and fall situation \

Stage situation:

'Nearly a week':'1w'
'Last Month':'1m'
'Last Three months':'3m'
'Last June':'6m'
'Nearly 1 year':'1y'
'Last 2 years':'2y'
'Last 3 years':'3y'
'Last 5 years':'5y'
Copy the code

The crawler code

#### analysis 2: fund each stage rise and fall
def analysis2() :
    name =['Last 1 week'.'Last January'.'Last March'.'Last 6 months'.'Nearly 1 year'.'Last 3 years'.'Last 5 years']
    ## Five categories of funds
    dict_value={}


    for key in dict_type:
        #### obtain the first ranking fund code
        url = "https://danjuanapp.com/djapi/v3/filter/fund?type="+str(dict_type[key])+"&order_by=1w&size=10&page=1"
        res = requests.get(url, headers=headers)
        res.encoding = 'utf-8'
        s = json.loads(res.text)
        ### take first place
        fd_code = s['data'] ['items'] [0] ['fd_code']


        #### access to the first ranked fund at all stages
        fu_url = "https://danjuanapp.com/djapi/fund/derived/"+str(fd_code)
        res = requests.get(fu_url, headers=headers)
        res.encoding = 'utf-8'
        s = json.loads(res.text)
        data = s['data']


        valuess=[]


        #### to prevent the maximum time of the fund is less than 1 year, 2 years, 5 years of the situation error, fill with 0
        # # almost 1 week
        try:
            valuess.append(data['nav_grl1w'])
        except:
            valuess.append(0)
        # # nearly 1 month
        try:
            valuess.append(data['nav_grl1m'])
        except:
            valuess.append(0)
        # # for nearly three months
        try:
            valuess.append(data['nav_grl3m'])
        except:
            valuess.append(0)
        # # nearly six months
        try:
            valuess.append(data['nav_grl6m'])
        except:
            valuess.append(0)
        # # almost 1 year
        try:
            valuess.append(data['nav_grl1y'])
        except:
            valuess.append(0)
        # # for nearly three years
        try:
            valuess.append(data['nav_grl3y'])
        except:
            valuess.append(0)
        # # almost five years
        try:
            valuess.append(data['nav_grl5y'])
        except:
            valuess.append(0)
        ### add to collection
        dict_value[key]=valuess
    bars(name,dict_value)
Copy the code

Visual code

# # # bar charts
def bars(name,dict_values) :


    # chain call
    c = (
        Bar(
            init_opts=opts.InitOpts(  # Initial configuration item
                theme=ThemeType.MACARONS,
                animation_opts=opts.AnimationOpts(
                    animation_delay=1000, animation_easing="cubicOut"  # Initial animation delay and easing effect
                ))
        )
            .add_xaxis(xaxis_data=name)  # x
            .add_yaxis(series_name="Stock type", yaxis_data=dict_values['Stock type'])  # y
            .add_yaxis(series_name="Hybrid", yaxis_data=dict_values['Hybrid'])  # y
            .add_yaxis(series_name="Bond type", yaxis_data=dict_values['Bond type'])  # y
            .add_yaxis(series_name="Exponential", yaxis_data=dict_values['Exponential'])  # y
            .add_yaxis(series_name="QDII type", yaxis_data=dict_values['the QDII type'])  # y
            .set_global_opts(
            title_opts=opts.TitleOpts(title='Up or down', subtitle='Painted by Yunchen Li'.# Configure and position the title
                                      title_textstyle_opts=opts.TextStyleOpts(
                                          font_family='SimHei', font_size=25, font_weight='bold', color='red',
                                      ), pos_left="90%", pos_top="10",
                                      ),
            xaxis_opts=opts.AxisOpts(name='stage', axislabel_opts=opts.LabelOpts(rotate=45)),
            Set x name and Label rotate to eliminate the use of too long Label names
            yaxis_opts=opts.AxisOpts(name='up or down point'),


        )
            .render("All stages of the fund up and down.html"))Copy the code

Analysis of the

From the above GIF can be clear about the five types of fund the first fund of each stage of the rise and fall.

Some funds do not reach the maximum time of 3 years or 5 years, here use padding 0 processing.

Analysis 3: The net value of the last 30 trading days

Similarly, in the above analysis, it can be clear about the ranking of the highest ups and downs of these five types of funds in the past month. Below, the first fund is selected from the ranking (the first in each of the five categories) to show the net value of the fund in the past 30 trading days.

The crawler code

#### analysis 3: the net value of nearly 30 trading days
def analysis3() :for key inDict_type: #### get the first ranking fund code URL = "https://danjuanapp.com/djapi/v3/filter/fund?type=" + str(
            dict_type[key]) + "&order_by=1w&size=10&page=1"
        res = requests.get(url, headers=headers)
        res.encoding = 'utf-8'
        s = json.loads(res.text)
        ### take first place
        fd_code = s['data'] ['items'] [0] ['fd_code']


        #### obtain the first ranking fund nearly 30 trading days of net value
        fu_url = "https://danjuanapp.com/djapi/fund/nav/history/"+str(fd_code)+"? size=30&page=1"
        res = requests.get(fu_url, headers=headers)
        res.encoding = 'utf-8'
        s = json.loads(res.text)
        data = s['data'] ['items']
        name=[]
        value= []for k in range(0,len(data)):
            name.append(data[k]['date'])
            value.append(data[k]['nav'])


        silder(name, value,key)
Copy the code

Visual code

# # # stretch figure
def silder(name,value,tips):
    c = (
        Bar(init_opts=opts.InitOpts(theme=ThemeType.DARK))
            .add_xaxis(xaxis_data=name)
            .add_yaxis(tips, yaxis_data=value)
            .set_global_opts(
            title_opts=opts.TitleOpts(title=str(tips)+"Net worth for the last 30 trading days"),
            datazoom_opts=[opts.DataZoomOpts(), opts.DataZoomOpts(type_="inside")],
        )
            .render(str(tips)+"Net worth for the last 30 trading days.html"))Copy the code

1. The stock type

2. The hybrid

3. The bond type

4. The index type

5. Type the QDII

Analysis of the

From the above GIF can be clear about the five types of fund the first fund of nearly 30 trading days of net value.

conclusion

The above is to crawl the fund data and visually display the fund’s ups and downs through the Pyecharts GIF.

Climb the data around these three points and visually display the data in the GIF:

Nearly a month up or down the top 10
Fund each stage rise and fall
The net value of nearly 30 trading days

Top 10 Best Popular Python Libraries of 2020 \

2020 Python Chinese Community Top 10 Articles \

5 minutes to quickly master the Python timed task framework \

Special recommendation \

Click below to read the article and join the community

Use Python to crawl all kinds of fund data and dynamic display

The data source

The data analysis

Analysis 1: The top 10 in recent months

The crawler code

Pie chart visualization code

Analysis of the

Analysis 2: fund each stage rise and fall

The crawler code

Visual code

Analysis of the

Analysis 3: The net value of the last 30 trading days

The crawler code

Visual code

Analysis of the

Related Posts

How is Redis’ expiration policy implemented?

Time slice, context, scheduling algorithm and other knowledge points

The language of actual combat: API server (2) run the process booklet | free education