This is the 18th day of my participation in the August Challenge.

preface

Play the Nuggets series and add some interesting features to the Nuggets, such as:

  • The Nuggets column is being generated as a PDF file
  • [Playing nuggets] Nuggets signing author, force parameter analysis, data

One day, I entered my personal homepage and found that I had already given others thousands of likes. Oh my God. As a new person, support each other once, also calculate normal, my these 💗, all gave who, pursue true like, oneself start to realize a bar.

Results demonstrate

The list of details is clickable to the article page

The source address

Source code address: JJMyStarAndC

Back-end services are written in NodeJS, and you need to install the corresponding installation package.

Implementation approach

Possible ideas:

  1. Chrome plugin + Monkey script
  2. Static pages + nginx proxy
  3. Static page + service forwarding
  4. .

The idea of this paper is static page + NodeJS custom service. For fun, Server Sent Events (SSE for short) is added. The Server side unilateral push, whose function is to unilaterally push the data to the customer side after the Server side gets it.

IO socket. IO is jQuery in sockets. Socket. IO is jQuery in sockets.

Better implementation: Express + HTTP-proxy-Middleware can be passed through

Implementation details

Data acquisition

It’s all about data, no data, no pie, no ideals.

For Star retrieval, one interface is sufficient!

Request address: api.juejin.cn/interact_ap… Request method: POST Request Parameters: Does it have a hidden limit parameter option? You can try a wave!

 {
    cursor: `${cursor}`.// Start query location
    item_type: 2.// Give a thumbs-up to the article
    sort_type: 2.// sort with near guide
    user_id: uid, / / user id
  }
Copy the code

Return result:

{
   has_more: true.data: [{
       author_user_info: {user_id: "3465271329953806"./ / user ID
           user_name: Nezha / / user name
       }, 
       article_info: {
           article_id: "6996484371305725965"  / / article ID
           title: "K8s Study two"  // Title of the article}}}]Copy the code

The cursor parameter is very important when requesting the cursor, indicating the position of the pointer and where to continue to move forward. The important has_more field in the result indicates that there is no data yet, if there is, continue to fetch.

I remember that some interfaces can pass the limit parameter as 1000, also can, later made improvements, worthy of a good wave.

Its loop gets the core code:

 while (res.has_more) {
    data.cursor = `${cursor}`
    
    res = (await axios.default.post(url, data, {
      headers
    })).data;
    
    cursor += 10;
    await delay(undefined.16).run();  // Pause 16ms on purpose
  }
Copy the code

As we said, this is the server getting the data, not pushing it to the front end.

Data push

The NODEJS side uses SSE, which is also very simple, so I don’t use the tripartite library here.

  1. Request sends a UID. The uid and rid parameters indicate the user ID and the RID indicates the requestId
  2. getStarsStart query
  3. That’s what SSE is all aboutContent-TypewithConnection

    'Content-Type': 'text/event-stream'Type declarations

    'Connection': 'keep-alive'Indicates that the connection is not closed
  4. Since this is also a request, we use the event center to send events to requests that write back data
app.get('/sseStream'.function (request, response) {
  response.writeHead(200, {
    'Content-Type': 'text/event-stream'.'Cache-Control': 'no-cache'.'Connection': 'keep-alive'
  });

  const { uid, rid } = request.query;
  console.log("uid:", uid);

  // Query the user's likes
  getStars(uid, rid);

  // Don't use this on the production line
  eventsCenter.removeAllListeners("ssePush");
  // Event center
  eventsCenter.on("ssePush".function (event, data) {
    // console.log("push message to clients");
    response.write("event: " + String(event) + "\n" + "data: " + JSON.stringify(data) + "\n\n");

  });
});
Copy the code

So where is the event sent out, that’s where the data fetch is sent out, so I’m going to fill in the code, and there are two additional events messageTotal, messageEnd, one is the total message, and one is the end of the request message.

async function getStars(uid, rid) {
  let cursor = 0
  const data = {
    cursor: `${cursor}`.item_type: 2.sort_type: 2.user_id: uid,
  }
  let res = {
    has_more: true
  };

  while (res.has_more) {
    data.cursor = `${cursor}`

    res = (await axios.default.post(url, data, {
      headers
    })).data;

    console.log("res:", data, res)

    eventsCenter.emit("ssePush"."messageTotal", {
      uid,
      rid,
      count: res.count
    });
    eventsCenter.emit("ssePush"."message", {
      uid,
      rid,
      datas: (res.data || []).map(d= > ({
        user_id: d.author_user_info.user_id,
        user_name: d.author_user_info.user_name,
        title: d.article_info.title
      }))
    });
    cursor += 10;
    await delay(undefined.16).run();
  }
  eventsCenter.emit("ssePush"."messageEnd", {
    uid,
    rid
  })
}
Copy the code

Front-end data acquisition

The foreground corresponds to the listening event, simple as that.

    const source = new EventSource(`/sseStream? uid=${uid}&rid=${rid}`);

    // New data received
    source.addEventListener('message'.function (e) {
        let data = JSON.parse(e.data)
        // Not the required data
        if(data.uid ! = uid || data.rid ! = rid) {return; } listArr.push(... data.datas);// console.log("listArr:", listArr);
        renderList(listArr);
        gotStarsEl.innerHTML = listArr.length;

    }, false)

    // Total data message received
    source.addEventListener('messageTotal'.function (e) {
        let data = JSON.parse(e.data)

        // Not the required data
        if(data.uid ! = uid || data.rid ! = rid) {return;
        }
        totalStarsEl.innerHTML = data.count;
    }, false)

    // Complete the statistics
    source.addEventListener('messageEnd'.function (e) {
        let data = JSON.parse(e.data)
        console.log("meesage", data);
    }, false)
Copy the code

Statistics and Grouping

  1. Statistics: The user ID is used as the key. If the user ID does not exist, the user ID is created. If the user ID exists, the user base is changed.
  2. Map keys to array, then sort.

It’s that simple!

    const statObj = list.reduce((obj, cur) = > {
        if (hasOwnProperty.call(obj, cur.user_id)) {
            obj[cur.user_id].count += 1;
            obj[cur.user_id].items.push(cur);
        } else {
            obj[cur.user_id] = {
                items: [cur],
                count: 1. cur }; }return obj;
    }, {});

    / / group
    const groupList = Object
        .keys(statObj)
        .map(k= > statObj[k])  / / group
        .sort((a, b) = > a.count > b.count ? -1 : 1);  / / sorting
Copy the code

For more implementation details, go to the source code.

Write in the last

3-5 minutes, 500-1000 words, gain, but not tired, if you feel good, your praise and comment is the biggest motivation for me to move forward.

Technical group please come here. Or add my wechat Dirge-Cloud and learn together.