Chrome is one of the most commonly used tools for Python crawlers on PC.
Given the power of The Chrome browser, there are many powerful plug-ins in the Chrome Web App Store that can help us to crawl data quickly and efficiently.
Today’s recommended 6 Chrome plug-ins can greatly improve our crawler efficiency.
EditThisCookie
EditThisCookie is a Cookie manager that makes it easy to add, delete, edit, search, lock and block Cookies.
You can save the Cookies after login to the local first, and use cookielib library to directly climb the data after login.
Avoid packet capture and simulated login, help us quickly crawler.
Web Scraper
The Web Scraper is a free, free-for-all crawler with no programming foundation.
The operation is simple, only need the mouse click and simple configuration, can quickly crawl the data on the Web side. It supports complex website structures, data support text, links, data blocks, pull-down load data blocks, and other data types.
In addition, you can export the crawled data to a CSV file.
Xpath Helper
Xpath Helper is a structured web element selector that supports list and single-node data fetching,
It can quickly locate web elements.
Compared to Beautiful Soup, Xpath web element lookup performance is better; Xpath is easier to write than regular expressions.
Once you write an Xpath, the number of matches and their locations are displayed in real time so that you can determine if the statement was written correctly.
Toggle JavaScript
The Toggle JavaScript plug-in can be used to detect which elements of the current web page are dynamically loaded through AJAX.
It can be used to quickly switch between allowing JS loading and disabling JS loading.
User-Agent Switcher for Chrome
User-agent Switcher for Chrome allows you to easily modify the User-Agent of the browser.
Can simulate different browsers, clients, including Android, IOS to simulate requests.
For some special websites, switching user-Agent can be more convenient for data crawling.
JSON-handle
Json-handle is a powerful Chrome extension for parsing JSON data.
It presents JSON documents in a simple, clear tree style that can be edited in real time.
For scenarios with a large amount of data, you can perform partial selection analysis.
This is the end of sharing Chrome plugins. Partners have a supplementary welcome message oh! More Python knowledge will continue to be shared!