Where there are e-commerce, there are channels, and I, as a channel business, can profit from the intermediate price difference. Geek Time provides the coder community with excellent content about Internet technology at a very reasonable price.

Therefore, Shanyue created a cashback platform today, which is convenient for everyone to buy from it and saves the intermediate price difference: “If you buy from the platform, you can add my wechat cashback intermediate price difference”, so that you can buy a geek time column in a cheaper way.

Today, let’s talk about the feeling and technology stack of doing this platform

❝

The effect looks like this: geek.shanyue.tech

❞

Technology stack considerations

There are three parts about the implementation of technology: crawler, development, deployment.

The technical requirements are: rapid deployment (CICD), low network latency (domestic network), SEO friendly

The original idea was: next. Js, Serverless and Github Actions. But two of them went wrong:

  1. next.js: Better SEO, but more complex deployment, because it starts with a server (which is fine later)
  2. serverless: Using Tencent cloudserverlessDeployment, however, my domain name in Tencent cloud did not record, give up

The final technology stack is Gatsby and Vercel, which can meet the problems of rapid deployment and SEO friendliness, except for the domestic network. But now that I’m using Vercel, I’ll consider switching back to its next-js cousin. Of course, the same could be true of future technology stacks

  1. next.js+ Personal server deployment: Better SEO, deployment is expected to be unsatisfactory
  2. gatsby + alioss + cdn: A lot of preliminary work, especially with CDN and HTTPS
  3. next.js + tencent serverless: It takes 20 days to register in Tencent cloud

Oh yeah, and crawlers, you can use a new library I recently wrote github.com/shfshanyue/… To control the concurrency of hundreds of requests

const { map, sleep } = require('@shanyue/promise-utils')

const items = await map(courses, async course => {
  const sku = course.column_sku
  const intro = await getIntro(sku)
 await sleep(100)  return intro }, {  concurrency: 3 }) Copy the code

About feeling

Today, it took half an hour from crawler writing to deployment, including half an hour for crawler writing, one hour for React, three or four hours for CSS, and half an hour for deployment. Think about how hard it is to live at the front.

I also found an interesting status code when I wrote the crawler: 451(UnavailableFor Legal Reasons), which means the website is not accessible due to Legal problems. If you are also interested, you can try the following request:

curl curl -vvv 'https://time.geekbang.org/serv/v1/my/data' \
  -X 'POST' \
  --compressed
Copy the code

Pay attention to my

❝

This article is collected in GitHub mountain moon blog: shfshanyue/blog, including my problems encountered in practical work, thinking about business and learning in the direction of the full stack

  • Front-end engineering series
  • Node advanced series

❞

Welcome to pay attention to the public number [full stack growth road], regular push Node original and full stack growth article

Welcome to the road to full stack growth

This article is formatted using MDNICE