Background:

Project idea:

The idea of the project is relatively simple, divided into the following steps.

  1. Analysis of digging gold pamphlet site, we click on a chapter each time, will find the URL behind a string of changes, so suspect that this string of characters is the ID of each chapter, so the first thing to get is the whole volume of all chapter ID;
  2. After searching for the above ideas, we found the XHR requestget? uidThis request, look at the response, sure enough there is a string of yessectionID;
  3. Once you have all the chapter ids, you need to look for the API that gets the chapter content. In this case, you can go to the Developer tools Filter and search for section (Intuition)getSection? uidSectionID = sectionID; sectionID = sectionID; sectionID = sectionID;
  4. Once all the apis are found, you can construct parameters and crawl whatever you want.

Github README

GitHub README directly to take over, write more detailed, suitable for small white start.

Load your nuggets into your Kindle

  1. Download the nuggets booklet and save it in MarkDown format
  2. Download the nuggets booklet and save it in MOBI format

Note: to convert to MOBI, useThis project, only support level 1 directory and level 2 directory, resulting in the current MOBI format is messy, the pictures inside are not processed, if you have a good tool or code to transfer MOBI, be sure to issue to tell me, or directly contribute.

Method of use

  • Obtain client_ID, UID, and token information from the nuggets directory and fill it in./config/config.json
  • go run main.go
  • For friend not Go environment, has been compiled in the tool folder Linux&Mac version of the executable file,. / juejinxiaoceToolForLinux or. / juejinxiaoceToolForMac can
  • Waiting for your booklet to come out 🍺

Small white operation steps

  1. Go to the Nuggets website and open up one of your brochures, as shown below:

  1. Windows press F12,Mac use the shortcut keys Conmand + Shift + I to enter the developer tools, as shown below:

  1. Select XHR (which stands for filtering asynchronous Ajax requests), refresh the page, and find get? Uid = XXX, circled in red. Click on the get? In the part of uid= XXX, you can see the request and return information of the API, which is actually the UID, ID,token and so on we need, as shown in the following figure:

  1. You can click On Response to see the message returned by the API. Golang is also the key to constructing the structure. See the code for details.
  2. Using this API we can get an array of sectionids, each corresponding to a section in the article, searched in filtergetSection, the API is to get the content of the specified sectionId;

The URL used above is in config/config.json, which can be used directly.

The above is the basic operation, after getting it done, the execution of go run main. Go wait for your booklet to come out 🍺.

You can see that both MarkDown and MOBI format books have been downloaded.

  • Project address: github.com/hantmac/jue…
  • In fact, this MOBI format is not particularly good, if you have any good method, please share!
  • If Golang doesn’t have a good package, there will be a Python version later. Python has a lot of good tools.

Finally, this blog is only to share technology, the author is not easy to create a booklet, we do not spread after downloading.