It’s probably due to Python’s popularity in recent years, plus the fact that we’re often seeing people using Python to make web crawlers to grab data. Therefore, some students have such a misunderstanding, want to grab data from the network must learn Python, must write code.
First, let’s talk about the way to get data: one is to use existing tools, we only need to know how to use the tools to get data, we don’t need to care about how the tools are implemented. For example, if we are on the shore and we want to go to some island in the sea, and there is a boat on the shore, our first thought is to go there by boat, not to build a boat to go there.
The second is to do some custom chemical tools according to the needs of the scene, which requires some programming foundation. For example, we’re still going to an island in the ocean and we want a ton of goods delivered to the island in 30 minutes.
Therefore, if you simply want to obtain data in the early stage, and have no other requirements, you should choose existing tools first.
It’s probably due to Python’s popularity in recent years, plus the fact that we’re often seeing people using Python to make web crawlers to grab data. Therefore, some students have such a misunderstanding, want to grab data from the network must learn Python, must write code.
Today we introduce several tools that can quickly access data on the Web.
1.Microsoft Excel
You read that right, it’s Excel, one of the three musketeers of Office. Excel is a powerful tool, and being able to grab data is one of its features. I take earphone as the keyword, grab jingdong commodity list.
2. Locomotive collector
Locomotive is an old brand in the crawler industry, and is currently the Internet data capture, processing, analysis and mining software with the largest number of users. Its advantage is the collection of unlimited web pages, unlimited content, or distributed collection, the efficiency will be higher. The disadvantage is that the user is not very friendly to the white, there is a certain threshold of knowledge (such as webpage knowledge, HTTP protocol knowledge), but also need to spend some time familiar with the tool operation.
Because of the learning barrier, the upper limit of data collection is very high once the tool is mastered. Students with time and energy can go to toss.
Official website: www.locoy.com/
Octopus collector
Octopus collector is a very suitable for novice collectors. It is easy to use, so you can get started in a few minutes. Octopus provides some common crawl site templates, using templates can quickly crawl data. If you want to crawl sites without templates, the official website also provides very detailed graphic tutorials and video tutorials.
Octopus is based on the browser kernel to achieve visual data capture, so there is a lag, slow data acquisition characteristics. But this flaw does not obscure the advantages, it can basically meet the novice in a short time to grab data in the scene, such as paging query, Ajax dynamic loading of data.
Website: www.bazhuayu.com/
4. Soking GooSeeker set
Set search is also an easy to use visual data collection tool. It can also crawl dynamic web pages, it can also crawl data from mobile web sites, and it can also crawl data floating on an index chart. Aggregator crawls data in the form of a browser plug-in. Although it has the above mentioned points, but there are also disadvantages, can not multithreaded data collection, browser lag is inevitable.
Website: www.gooseeker.com/
5.Scrapinghub
If you want to crawl foreign sites, consider Scrapinghub. Scrapinghub is a cloud crawler platform based on Python’s Scrapy framework. Scrapehub is the most sophisticated and powerful web scraping platform on the market, providing data scraping solutions.
Address: scrapinghub.com/
6.WebScraper
WebScraper is an excellent foreign browser plug-in. It is also a visual tool for beginners to grab data. We simply set up some fetching rules and let the browser do the rest.