Where there are people, there are rivers and lakes, and where there are rivers and lakes, there are disputes. A few days ago, after why Is Deno Now Replacing Node.js in its entirety, there was “Is Deno Dead? Perhaps this is the lake. Back to the title “Deno in hand, I have the world”, in fact, Deno is Charj, NodeJS, Java and other nouns would be the same, after all, it is only the title, maybe the real difference is the pleasure of using it. For me, can bring me the pleasure of Copy, is very good!
origin
git cloneA fierce operation such as tiger, a look, as if the realization of these several projects are almost the same, also consulted the giant to determine the realization is the same, but why to open 4 REPO? Maybe that’s what it isSingle responsibility principle. And then, I wondered if I could put it together and add some other top trending lists to make ittrending in oneIsn’t that a great idea? Hahaha. Next, I’m going to go after Toutiao – “The hot search wins the world, bring the hot search list! “, three times out of two to get the top trending search (despite the need to enter a captcha). Finally, resource consolidation, but readme. md is broken, so singleton is good.
Get today’s trending list instantly ➡️ trending- in-One
Analyze the
On how to achieve the summary of hot search, if you do not look at the source, I can only think of the relevant hot search list interface to obtain, but other people’s interface and how to call you at will? By reading the big guy’s source code, I saw how to get the corresponding title link by matching the DOM node with the re, I saw how to annotate the contents of the package and replace it, I saw how JavaScript handles duplicate data, I saw how to build Deno apps with Github Actions… It’s a small project, but it’s all-encompassing, and the way the author mixes and manipulates the techniques is amazing (PS: it seems true to me).
In fact, regardless of language and platform, to achieve hot search summary list, nothing more than three steps: ① access to data ② processing data ③ output data. Here is a detailed explanation of how these three steps work from Deno’s perspective:
To get the data
When we meet a demand, we may start from its essence, for example, to achieve hot search summary, first of all, we need the hot search data of each major platform, now the hot search list of daily headlines, zhihu hot video, Zhihu hot topic, knowledge hot search, Weibo hot search and so on, how to obtain? The usual method is to press F12 to see the Network, if you have to try the packet capture tool. Fortunately, “predecessors plant trees and future generations enjoy the shade”, so we have the interface of various platforms, and it is easy to obtain data.
Top hot list: [https://is-lq.snssdk.com/api/suggest_words/?business_id=10016](https://is-lq.snssdk.com/api/suggest_words/?business_id= 10016) the microblogging hot search: [https://s.weibo.com/top/summary] (https://s.weibo.com/top/summary) on zhihu hot topic: [https://www.zhihu.com/api/v3/feed/topstory/hot-lists/total?limit=100](https://www.zhihu.com/api/v3/feed/topstory/hot-li STS /total? Limit =100) [https://www.zhihu.com/api/v3/feed/topstory/hot-lists/zvideo?limit=100](https://www.zhihu.com/api/v3/feed/topstory/hot-l Ists /zvideo? Limit =100) [https://www.zhihu.com/api/v4/search/top_search](https://www.zhihu.com/api/v4/search/top_search)Copy the codeOf course, the above interface is not permanent, so it can only be used for as long as it can be used. If you are not satisfied with this, you can also try Twitter, Medium, etc. (PS: good people live a safe life). /(.+)< /a>/g Can you go to the comment section for more advice?
In the case of Toutiao, getting data seems to be super easy, but ask yourself “await, fetch, TypeScript” what do I know?
const response = await fetch(
"https://is-lq.snssdk.com/api/suggest_words/?business_id=10016",);if(! response.ok) {console.error(response.statusText);
Deno.exit(-1);
}
const result: ToutiaoTopSearch = await response.json();
const words = result.data[0].words;
Copy the codeProcess the data
Eventually our numbers could look something like this — see top trending trending 2020-12-01
Careful friends may have seen the comments in the.md file, is this comment played a key role, graft? Making wedding clothes for others? When updating data, we can skillfully use annotations as a reference, dynamically replace the content between the beginning and end of annotations, and use Deno standard library and some built-in APIS to complete file operations, such as the creation and update of Toutiao hot search data:
export function createTuotiaoList(words: ToutiaoWord[]) :string {
return ` <! -- BEGIN TOUTIAO --> <! -- Last updatedThe ${Date()} -->
${
words.map((x) => ` 1. [${x.word}] (${x.url}) `)
.join("\n")}<! -- END TOUTIAO -->`;
}
export async function createReadme4Toutiao(
words: ToutiaoWord[],
) :Promise<string> {
const readme = await Deno.readTextFile("./README.md");
return readme.replace(
/ <! -- BEGIN TOUTIAO -->[\W\w]*<! -- END TOUTIAO -->/,
createTuotiaoList(words),
);
}
Copy the codeIn addition, when comparing new and old data, I used the commonly used array iteration method and ** object.entries ()**, I felt that MDN also had to brush more, completely unskilled, especially when string together:
/** merge two hot searches and delete */
export function mergeWords4Toutiao(words: ToutiaoWord[], another: ToutiaoWord[],) :ToutiaoWord[] {
const obj: Record<string.string> = {};
for (const w of words.concat(another)) {
obj[w.url] = w.word;
}
return Object.entries(obj).map(([url, word]) = > ({
url,
word,
}));
}
Copy the codeThe output data
In fact, when processing data, we have basically completed the preparation of data output. Since only.md files and.json files are output in this project, we can directly call the previous data processing method, which is simple and crude:
export async function toutiaoSearch() {
// Save the raw data
await Deno.writeTextFile(fullPath, JSON.stringify(wordsAll));
/ / update the README. Md
const readme = await createReadme4Toutiao(wordsAll);
await Deno.writeTextFile("./README.md", readme);
/ / update the archives
const archiveText = createArchive4Toutiao(wordsAll, yyyyMMdd);
const archivePath = join("archives/toutiao-search".`${yyyyMMdd}.md`);
await Deno.writeTextFile(archivePath, archiveText);
}
Copy the codeWe can do a lot more than just output data as files, sending hot search data (or whatever data you can get your hands on) to email, SMS, pin bots… Can not help but think of Lao Shi often said “paid shit” necessary, think every morning the first bubble can enjoy the hot list of hot, it seems to have a kind of “domestic affairs, national affairs and world affairs, everything concerned about the illusion, sitting open a hot search said” I have read “……
Github Action
If Serverless is a good thing, thenGithub ActionHow do you escape the law of true fragrance?Automate, customize, and execute software development workflows in the GitHub Actions repository. You can discover, create, and share actions to perform any job you like (including CI/CD) and merge the actions into a fully customized workflow.The layman is that you can do whatever you want, such as build Deno applications on a regular basis. In order to update our hotflows summary every hour, we needed to add a **. Github /workflows** folder to the Github repositoryworkflowThen create ci.yml and Schedule. yml to tell Github Action what to do, so that she is a good little secretary, because she will tell you the detailed result of each workflow execution, and even if the build fails, there will be a nice email. As for what should be written in YML, please refer to the official documentation and the classic case of Deno community. Of course, github’s advice is available in the Action TAB of the warehouse if you want to get started. Show your sword, boy!
The end of the
The footsteps of exploration never stop, not too me, as if in a small dark room, no matter how I go can not go out, more than 10 years, still in situ step step. When I was a child, I dreamed of growing up a family of brothers to realize the illusion of family business, but I lost my cohesion at an early age; To the middle school, hate in the heart of the fragile was the teacher in charge of the lost key middle school entrance ticket fell to give up self; Fortunately, some universities read, but wasted time in Internet cafes, playground, lovers slope, back hill, the end of everything…… To this moment and the future, after all, to pay for the past decadence and decadence. Alas, old age, but also how to toss? In this paragraph, I would like to advise the young people who have no mining and no power in the court not to waste their life, at least Deno is really sweet!
Full project address: github.com/hu-qi/trend…