preface

Always feel a lot of recent earthquake news, just want to climb a wave of earthquake-related data to see.

The development tools

Python version: 3.6.4
Related modules:

Scrapy module;

Jieba module;

Pyecharts module;

Folium module;

Wordcloud module;

Fake_useragent module;

And some modules that come with Python.

Environment set up

Install Python and add it to the environment variables. PIP installs the required related modules.

Data crawl

First create a new project:

Then open the China Earthquake Network:

http://news.ceic.ac.cn/index.html
Copy the code

You can use the query function of the website to query the seismic data of the specified time and latitude and longitude range:

OK, then start writing code ~

First define the data we want to crawl in the kitems.py file:

class EarthquakeItem(scrapy.Item):
Copy the code

The spiders create the main.py function to write our main program, where we climb all the recorded earthquake data on the site since January 1, 2000 (in fact, after running the code we find that the site only has earthquake data T_T from 2012) :

'''main'''
Copy the code

OK, that’s done, just run the following command under the first EARTHQUAKE folder:

scrapy crawl earthquake -o eqdata.json -t json
Copy the code

The crawl data will be stored in the eqdata.json file.

I share Python data crawler cases every day. In the next video, I will share Python to capture and analyze weather data and China earthquake network data visualization

All done Complete source code see personal Profile or private letter to get related files.