preface

This article crawler source has been by
GitHub
https://github.com/2335119327/PythonSpiderIncluded (
Connotation of more crawlers not included in this blog post, interested partners can have a look) and will be updated later. Welcome
Star.

bloggerThe perennial wandering in the cattle guest by the area, summed up the byte, Ali, Baidu, Tencent, Meituan and so on the high frequency of the big factory exam, but today, I teach you how to face by the climb, if you can help the small friends, trouble a lot of support, grateful disrespect!!

This climb to Java surface as an example, learn the small partners can according to the law to climb Niuke any surface


teaching

Enter the Java interface area and open the console to refresh the request

Can be found, send the URL in the browser, get the response content is no surface, so where does the surface data come from?? Don’t worry, so many requests we continue to see!

If I scroll down, I can see the request with JSON, and my experience tells me that this is the request

Copy the URL, we go to the browser to request the URL, we can find that we get the face of the data

However, since it is in JSON format, we can copy it to the online JSON parsing tool to view it, as shown below

You can see that the DiscussSposts under Data saves all the posts, namely the interview information

But unlike what I’ve seen before, the JSON string does not directly store the URL of the post detail page, but we can provide the pattern of discovery through the access path

You can see that the access path has 675866, which corresponds to the postId in the JSON string, and the parameters behind it can be omitted


tip

Presumably the single page is certainly not able to meet the small partners, so if you carry out a number of pages to climb, don’t worry, I will summarize the rules for you, also hope that the small partners can click three even oh!!

The following figure shows the C++ area of the surface by the JSON string, I should not need to teach it

The complete code

Trouble to pay attention to the public number of small partners, background reply == climb the interview question == can get the complete source 😁

Follow-up public number will also only need to publish high-quality blog posts, not allow crawlers to miss Oh! 🤣

The results show


The last

I am aCode pipi shrimp, a love of sharing knowledge of mantis shrimp lovers, in the future will continue to update the beneficial blog, look forward to your attention!!

Creation is not easy, if this blog post is helpful to you, I hope you can == a key three even oh! Thanks for your support. See you next time ~~~

== Share outline ==

Big factory interview topic column






Java from entry to the grave learning route directory index






Open source crawler example tutorial directory index

More exciting content to share, please clickHello World (low low ◡)

This article crawler source has been by
GitHub
https://github.com/2335119327/PythonSpiderIncluded (
Connotation of more crawlers not included in this blog post, interested partners can have a look) and will be updated later. Welcome
Star.