Recently discovered a Postman god operation, with the help of Postman import request function, less than a minute, and a line of code without writing to achieve a crawler with login state, this article is a summary of the comprehensive use of various tools on the Web.

Without further ado, I’ll use the Nuggets’ boiling point as an example to grab all the trending posts.

1. Analyze the request and obtain the API address

As shown in the image below, open Chrome’s F12 data interface to analyze the boiling point of gold nuggets. Right-click the API address => Copy => Copy as cURL(bash). Step 1 ends. Simple.

2. Play the Postman artifact

Import the request as shown below.

After the request was imported, He clicked the Send button to test it and found that the data returned was ok.

To witness the magic moment, click on Code as shown in the picture below, and the various Code requests are written for you, because I am a Java programming ape, here I use OkHttp for example, copy the OkHttp request Code here. If you’re on the front end, you can choose JavaScript. This is the end of the front end, don’t look down, the things behind the big front end of the students all understand.

3. Install the GsonFormat plug-in of IDEA

In order to write a crawler without writing a line of code, the JavaBean class is generated in reverse from JSON. Install the GsonFormat plug-in and restart IDEA to take effect.

Copy the Json result as shown below.

Create a New Maven project and create a new class to store Json results. Call it JueJin.

Paste in the Json result that you copied in Postman, click OK, and then click OK in the next popover to generate the JueJin class.

Next, write a main method that pastes the OkHttp request code we copied above.

4. Maven imports jar packages

Here import the JAR packages for OkHttp and Gson in the POM.xml file. You can solve the problem of not finding a package in main.

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>4.7.2</version>
</dependency>
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.5</version>
</dependency>
Copy the code

Write two lines of code

I haven’t even written a line of code up here, and I’ve already got the boiling point data. Next, let’s use the JueJin class generated by GsonFormat to extract the nuggets separately. Add the jSON-to-bean code below. If you get a row of friends, make sure you increase the one in the argument.

The results are shown below:

6. The ending

Remember to pay attention to the crawler etiquette when you crawl, so as not to crash the site too quickly. In the meantime, if other sites require logins, you can log in yourself by copying Postman’s request code. After login, the crawler’s behavior represents your behavior, which is easily monitored by the background, and cherished.

Welcome to pay attention to my public number “donggudonggua”, their own wechat search, willing to take the bait. Ha ha.