Editor’s note:High availability architecture shares and disseminates articles that are typically significant in the architecture field, in this paper, bySangShiLongIn the highly available architecture groupTo share. Please quote from the HA Architecture public account “ArchNotes”.

Sang Shilong, CTO of Tianjin Empty String Technology, author of open source project Moajs, and advocator of Node.js technology. He has worked in Sina and Netqin, and has been responsible for front-end, back-end, data analysis and mobile terminal, chief architect, technical director, and full-stack technology practitioner. At present, I mainly focus on technical architecture and team echelon construction.

“JavaScript is the most widely used language in the world, and no one else uses it more, including backend developers.” – stackoverflow

The global status of Node.js



While Node.js is not popular in China, according to StackOverflow’s 2016 Developer survey, Node.js, full stack, and javascripts are leading in a number of areas, including full stack and back end.



(http://stackoverflow.com/research/developer-survey-2016).

The back-end distribution



(http://stackoverflow.com/research/developer-survey-2016).

Node.js comes with two inherent features:

  • event-driven

  • non-blocking I/O

In the past, the emphasis on asynchronous features, to today’s asynchronous is not an obvious advantage. So anything but performance is a disease?

Callback Hell problem

So far it has been well solved. Promise/generator/async will be covered later.

2. Package management

NPM is already the largest package manager in the open source world, with lots of modules (256,000).

Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

We used to like to talk about asynchronism, but now we boast about node.js’s powerful ecology.

Why node.js?

Kongxian technology is a SaaS service based on cloud storage, providing services to small and medium-sized sellers. The core system is purchase, sale and storage, order pool and WMS.

Let’s take a look at our bottleneck

  • People (Tianjin is difficult to recruit people). Node.js is not available, many of them are transferred from Java, the front end is not easy to find, many of them are also transferred from Java, we basically build the team from zero

  • Development speed. Startups build rockets in five minutes, you know that. So it was really important for us to get the development up to speed.

  • Stability. How to ensure system availability and stability in the absence of professional operation and maintenance personnel?

Which brings me to what I think are the benefits of Node.js

  • Also not optimized, better performance than most languages. Even when optimized, it is simpler than other languages, such as Java.

  • There is plenty of choice and architectural balance.

  • The truth is not enough, Java patch.

Node.js gives us plenty of tools to choose from

You can even use a variety of compilers such as coffee, typescript, Babel (ES), etc. For a team starting from zero, it can be process-oriented and then progressively more difficult as the team matures.

Provide good foundation and package management tools

  • Test related TDD/BDD test coverage

  • Normalized standard, various Lints, hints

  • Build relevant gulp, Grunt, Webpack, lots of plug-ins

  • Generator Yo, etc.

  • The package management tool NPM is simple enough to use

All of these things are the foundation of large-scale software, and Node.js does this very well

Scene-specific quickness

Many people combine MEAN (e.g. Mean.io). The advantage of this is that if you are familiar with it, the development speed is very fast indeed, but it is so difficult that few people can master it. Metetor blurs the server side and the client side, is a typical application of isomorphism and is very efficient for real-time scenarios. This kind of thing is fast in a specific scene, generally not easy to use, the difficulty of tuning is very difficult, if someone can cover, it is very efficient in the early stage.

Summary requirements: can be simple, can be difficult; Can be fast, can be slow; Large software can be developed

What if the above does not satisfy? This is where architectural balance comes in.

Balance architecture

Just do what’s right for each of us in the architecture, and we’ll be comfortable with node.js’s strengths and weaknesses and do a good architectural balance.



1, can do at the language level, that language level

  • There are a large number of modules on NPM (256,000 + currently)

  • Build your own wheels (simple syntax NPM = fast)

  • Using Node. Js (nan) https://github.com/nodejs/nan in their packaging C/C + + wheels

From above, most of the requirements can be met

2. If you can’t do it at the language level, do it at the architecture level

  • Business boundaries, module splitting, service orientation

  • MQ, RPC, cache

  • Operation and maintenance, monitoring and automation

First of all, architecture is not directly related to Node.js. For example, Rabbitmq has good Node module support. Thrift, Grpc, Tchannel support in RPC is good. We use senecajs, Redis, Ioredis and other software, after doing HA is the same.

3. If the architecture level cannot be solved…

Use the right stuff for the right scene. There are a lot of things that Node.js is not good at, and it’s not in the framework. What to do? If this is not true enough, Java complement (strictly speaking, should be called other languages complement)

  • Like complex Excel generation

  • For example, apNS push (Go is also good, but no one can maintain it except me)

However, node.js support is available for mature libraries in Java or other languages that can be used as stand-alone services. Avoid spending too much time building the wheel and affecting the development schedule.

4. Analysis of the pros and cons of Node.js

  • Performance efficiency, also not optimized, is better than most languages.

  • Development efficiency, Node.js itself is relatively simple, development efficiency is relatively high. A complete ecosystem, such as tests, tools, and a large number of NPM modules.

  • Lack of big kill like Rails, scaffolding, ORM is too weak.

Node.js Web development frameworks, such as Express and Koa, are simple, small and exquisite, but not fully integrated enough. The existing MEAN, Yo, and sails are always unsatisfying in some way

Team Node.js usage status

When choosing Node.js, we need to: solidify the project structure; Limit the ORM; Custom scaffolding.

Since Node.js already provides the following features, you can complete a scaffold in 30 minutes.

  • Cli command module, write very easy

  • Based on theJavaScript The template engine (known as the 30)

What do we do with Node.js?

  • API service

  • Frontend (moa-frontend)

  • SDK (OAuth Provider)

  • Assist in developing CLI tools

The current progress

  • Using 0.10.38, develop Moajs framework, Express/MongoDB

  • Pm2 deployment, front and back end separation, SLB load of Ali Cloud, Alinode monitoring

  • Moa-api, MOA-frontend, MOA-H5

  • Use Redis cache, Rabbitmq, senaca as RPC

Some aspects that are under construction

  • Use Kong as the API gateway

  • Consul does service discovery and configuration

  • Upper ELK is processed as log analysis

  • Use Docker Compose as your local development environment

  • Online docker

Technology stack updates are planned, including Nodejs 4.x (expected in June; Koa (Generator/CO); Es6 / ES7 (Babel).

4. X has a great improvement in memory and performance. New language features, asynchronous flow and syntax need to be learned, so there is no rush to upgrade until the talent team is complete.

The current approach is to take baby steps, learning one new technology at a time; In addition to form echelon, you can prepare for new things; Make good use of NPM to realize 3: modularization, minimization and servitization

Why choose MEAN architecture

MEAN architecture



MEAN is currently the most fashionable full stackJavaScript Architecture. The scheme is aJavaScript Platform for modern Web development framework,It is the first letter combination of the MongoDB Express AngularJS Node.js framework. It stands for a complete set of development tools like traditional LAMP.

From my point of view

  • MySQL is replaced with MongoDB, the most RDBMS of NoSQL, which has both development and performance advantages (see Bi’s post on the High Availability Architecture Group: MongoDB 2015 Review: The New Milestone WiredTiger Storage Engine).

  • Angular was the age of IoC, bidirectional binding, directives, and so on.

  • Node.js offers a complete ecosystem and toolchain, and it has almost everything you need, thanks to NPM. In the early days, Node.js was several blocks ahead of PHP.

  • As a Demonstration project for Node.js, Express is very streamlined and a suitable Web framework

Why did I choose MEAN architecture?

  • Mature, stable, simple, and we can cover problems, so we chose Node.js.

  • Grasp the trend, the future prospects of Node.js is very promising, especially the unified front and back end, full stack direction.

  • Architecture can shield potential risks, not put all eggs in one basket, not see the wood for the trees, and use other languages reasonably, as long as every feature is presented as a service, it doesn’t matter what language it is written in.

  • The cost of recruitment is relatively high, the technology stack is new, easy to attract talent.

One of the most important things is having someone to cover when there’s a problem, and that’s the most important thing in the early days.

Node.js latest Web technology stack

https://cnodejs.org/topic/55651bf07d4c64752effb4b1

Node.js asynchronous process

Asynchronous flow control

The evolution of JavaScript flow control is divided into the following five parts:

  • The callback function Callbacks

  • Asynchronous JavaScript

  • Promise/A + specification

  • Generators/ yield (ES6)

  • Async/ await ( es7 )



  • All versions currently support the Promise/A + specification

  • Node.js 4.0 currently supports Generators/ yield

  • Async/await in ES7 is not currently supported, but can be implemented via Babel

On the whole, the asynchronous process control is better solved.

The Promise of node.js’s latest technology stack

https://cnodejs.org/topic/560dbc826a1ed28204a1e7de

Rapid development practices

Business boundary optimization

Startups have a lot of variability, a lot of systems to build, and it’s very difficult to keep the boundaries of the business systems, and we actually took a lot of detours.

Static API theory



When the requirements and UE are determined, static API will be written, so that APP, H5 and the front end can use static API to complete the function, and the back end can also use static API as the standard to achieve the overall efficiency is relatively high.

API best practices

http://developer.github.com/v3/ (restful) strictly

The Twitter API (readable and relatively traditional) is similar to the one we use, and the convention structure is similar.

res.api is an express middleware for render json api , it convention over api format like this :

     data : {},

    status: {

        code : x,

MSG: ‘some message’

Client API development summary

https://cnodejs.org/topic/552b3b9382388cec50cf6d95

Contract structure

Similar to the directory structure in Java development, this layer of layers, appropriately according to Express/Koa add middleware, routing, and other directories to facilitate development.



Use NPM modularization

  • Private private modules using NPMJS (current practice)

  • Using NPM’s native module development approach (very fast to test and deploy)

  • Setting up NPM Private Server (TODO)

Writing a generator

In Web development, Moajs generators are written, similar to Rails

moag order name:string password:string

Other development, such as iOS development model verification is very boring, so I wrote a JSON2objC command line tool, read JSON, generate OC code, can save a lot of time

The Moajs frame is separated from the front and rear ends

  • The front-end (earth-sized – frontend https://github.com/moajs/moa-frontend).

  • Public uses Nginx as the reverse proxy

  • Others use Express Jade to simplify code (Ajax and back-end interaction)

  • The back-end (earth-sized – API) https://github.com/moajs/moa-api

The MOA generator, the scaffolding generator mentioned above.

Moa-frontend stack: Express/Jade/bootstrap, bootstrap-table/jQuery/gulp/Nginx

Moa-api Technology Stack:

  • base2(mirco kernel)  https://github.com/base-n/base2-core

  • mongoose  https://github.com/Automattic/mongoose

  • bluebird  https://github.com/petkaantonov/bluebird

  • res.api https://github.com/moajs/res.api

Features

  • Automatic route loading, user management, user authentication using JsonWebToken

  • Support MongoDB configuration, integration of MongooseDAO, quickly write DAO interface such as CRUD

  • Supports Migrate and Mocha tests

  • Res.api is integrated by default for easy write interfaces

  • Integrate Supervisor, code changes, automatic reloading, gulp automatically monitors file changes, run tests

  • Gulp routes Generates the route description

  • Use log4JS to record logs

From the development effect, or very fast, very stable.

Evolution of the Moajs framework

https://cnodejs.org/topic/567e2388aacb6923221de469

Full stack or all rotten?

Node.js related tools

  • grunt/gulp/fis/webpack

  • bower/spm/npm

  • tdd/bdd cucumber/mocha

  • standard

  • babel/typescript/coffee

Front-end development has four phases

  • Html/ CSS/JS (Basic)

  • JQuery,jQuery-ui, Extjs (once popular)

  • Backbone (MVC), Angularjs, Vuejs

  • React componentization (future trend), Vuejs

Combining the best of Angular and React, Vuejs should be the next big thing.

Development of Hybrid

Hybrid development refers to the development of cross-browser applications using H5 technology and the final package of HTML5 / JS/CSS into APK and IPA. It can also be uploaded to the app store for mobile devices to install. Its biggest benefit is that it can be developed with H5 once and installed on multiple platforms.

The future will be JavaScript (Node.js as the back end, traditional Web and H5 using JavaScript, smarter tools like gulp, simpler writing like Coffeescript, etc.). The H5 is all the rage (faster network speeds, increased hardware memory).

cross-platform

C/S architecture to B/S architecture, this is mostly clear, not to say more.

Mobile terminal shell, in the browser, the page generated each mobile terminal APP file.



PC side shell, the same is a continuation of the browser, but this time the page generated for each PC platform executable file.

  • Node – its is renamed (NW. Js https://github.com/nwjs/nw.js).

  • Electron https://github.com/atom/electron – Build cross platform desktop apps with web technologies

Popular editor at the momentAll based on Electron packing:

  • Atom  https://github.com/atom/atom

  • vscode  https://github.com/Microsoft/vscode



Componentization: uniform usage

React the emergence of the most influential was the emergence of JSX, which solved the long-standing problem of componentalization:

  • We go back and forthJavaScript Still can’t fix

  • We try OO, like extJS

  • We’ll end up with an intermediate format JSX

React is only a view level, not enough to apply, so there is Redux. Core concepts: Actions, Reducers and Store, which is simply state control, and then combined with packaging and shell, into app or executable file. Cordova is used for iOS and Android, and Electron is used for PC.

  • React component definition

  • Control state transitions between components (Redux)

  • Pack or shell (Cordova or Electron)

This part actually componentizes the front end, so can we use this idea to componentize the mobile end?

react-native

https://github.com/facebook/react-native)

A framework for building native apps with React. 

http://facebook.github.io/react-native/

Simply put, React syntax is used to componentize the iOS or Android SDK. They’re all telling us, you can just play with these components in the future, you don’t need to know what a sophisticated SDK is.

Current popular gameplay

Medisis a beautiful, easy-to-use Redis management application built on the modern web with Electron, React, And redux. It’s powered by many awesome node. js modules, especially ioredis and ssh2.

https://github.com/luin/medis



Technical point

  • Use the Node.js module

  • Build with Webpack

  • Use React Redux (control logic)

  • Pack using Electron shell

Honey, do you see the future?

How to full stack?

Node tools, front-end 4 stages, hybrid, a variety of cross-platform, the current is to introduce the Node full stack of various possibilities, the following talk about how to achieve Node full stack?

Full stack core, the back end will not UI (interface related), the front end will not DB (business related), as long as get through these two points, the rest is relatively easy.

1. Transfer from the back end

Back-end people are familiar with databases, no matter MongoDB, Mysql or Postgres, but have a weak understanding of the front-end and are familiar with basic Html, Css and template engines.

4 stages step by step, build and tools fly together, front-end development 4 stages, my feeling is in order, step by step.

  • Html/Css/JavaScript (Basic)

  • jThe Query,jQuery-ui, Extjs (once popular)

  • Backbone, Angularjs and Vuejs

  • React (Future Trend), Vuejs

Combining the best of Angular and React, Vuejs should be the next big thing

2. Turn from the front end

From front to back, APIS are very easy to learn. Frameworks like Express and Koa can be learned by most people in a week. The most difficult thing is to understand the DB and ER models, or to put it bluntly, to understand the business requirements

Let’s think about what the typical front-end skills are.

  • Html

  • Css (Compatible with Browsers)

  • JavaScript Will point (probably more will pointjThe Query)

  • PS cut figure

  • Firebug and Chrome Debuger are not popular

  • I’ve used a few frames, but most people just know how to use them

  • General English

  • Svn/Git is a little better

So what are the challenges for them to get deeper into the front end?

  • Basics: OO, design patterns, commands, shells, builds, etc

  • Understanding of programming ideas (MVC, IoC, protocol, etc.)

  • To distinguish the concept

  • Peripheral acceptance, such as H5 and Hybird

  • Catch up, how to learn new things

These are all pain points. So a better way is:

  • Play with front-end utility classes like NPM and GULP (still front-end at this point)

  • Use Node for front and back separation (at this point, the front end)

  • Express, Koa, etc

  • Jade, EJS and other template engines

  • Nginx

  • How to play the “back-end” asynchronous processes processing promise/es6 (generator | yield)/es7 (async | await)

  • Play [back-end] MongoDB, Mysql corresponding Node module

From our experience, this is more reliable. https://github.com/moajs/moa-frontend is the most simple separation before and after the end, there is no any related to DB.

Technology stack

  • Express

  • Jade

  • The bootstrap, bootstrap – table

  • jQuery

  • gulp

  • Nginx

The general front end is very easy to learn, basically two weeks have been very skilled, my plan is six months later, let them contact [asynchronous process processing] and [database] related content, learn back-end code, can be full stack

3. Transfer from mobile

Mobile terminal: native development, hybrid development. Native development is oc/ Swift for iOS, Java or Scala for Android, etc. Even with occasional webView embedded, the chance to play JavaScript is very rare. The best way to move to full stack is to start with Cordova (formerly phoneGap) and do hybrid development. Just pay attention to the WWW directory H5 can be relatively simple. If H5 is not enough, write a cordova plugin that allows JavaScript to invoke functionality in the native S DK. The CLI of Cordova can be installed through NPM, learn the good method of NPM, learn the gulp build tool.

Once you get into the H5 pit, it’s actually very easy to do.

  • Then H5, Zeptojs, iScroll, FastClick, etc

  • Then wechat commonly used, such as weui, VUX (vue Weui), jMUI (React Weui)

  • And then you can play with frames, likejQuery Mobile, Sencha Touch

  • Then you can play with the advanced ionicframework (Angularjs, cordova).

  • Then the front end of 4 stages, in turn play strange upgrade

  • Then the Node. Js

This is basically a summary of the path I’ve taken since 2010, when I was writing IOS and working on PhoneGap (0.9.3 at the time).

Looking into the future of Node.js technology

Node.js may be a sex dream,

It could be an opportunity for change;

We believe it’s a transformational opportunity,

Wait and see!

Attached: Node.js 2015 development history

Q1 First Quarter

  • IO. Js 1.0.0 release

  • Joyent launched the Node.js Foundation

  • Joyent, IBM, Microsoft, PayPal, Fidelity, SAP and The Linux Foundation Join Forces to Support Node.js Community With Neutral and Open Governance

  • Proposed settlement between IO. Js and Node.js

Q2 Second Quarter

  • NPM supports private modules

  • Node project leader TJ Fontaine is stepping down from his core role and leaving Joyent

  • A changing of the guard in Nodeland

  • Node.js and io.js are merged under the Node Foundation

Q3 Third Quarter

Q4 Fourth quarter

  • Node V4.2.0, first long Term support release (LTS)

  • Apigee, RisingStack and Yahoo join the Node.js Foundation

  • Node Interactive

  • The first annual Node.js conference by the Node.js Foundation

Version of the emperor? It started at V0.10.35 last year

  • V1.0.0 (io.js) was released on January 14, 2015.

  • 2. X (IO. Js)

  • 3. X (IO. Js)

  • In September 2015, the Node.js Foundation has released the first version of Node.js v4.0 combined with IO

  • Node.js V4.2.0 will be the first long-term LTS support release in October 2015

  • Released to 4.2.4&& 5.4.0 by the end of the year

Current (March 20, 2016) 2 versions

  • V4.4.0 LTS (Long Term Support version)

  • V5.9.0 Stable



On the whole, it’s stable.

  • The Establishment of the Node Foundation will enable Node.js to be better supported by the open source community in the future.

  • The LTS release means the API is stable.

  • Fast release version, a lot of people joke about this, in fact, from another perspective, it is also a reflection of active community, but if you really look at CHANGELOG, in fact, are small improvements, and is the corner of the improvement, that is to say node.js core (core) has been very stable, can be large-scale

    Use.

(refer to link: http://i5ting.github.io/history-of-node-js/)

Node.js enterprise memorabilia

Node.js’s big business

Nearform 2014 (Why has Node.js become the technology of choice for enterprises? http://www.nearform.com/nodecrunch/node-js-becoming-go-technology-enterprise/).

In 2015 IBM (extend cloud services business by acquiring StrongLoop) http://www-03.ibm.com/press/us/en/pressrelease/47577.wss

Founding members of the Node.js Foundation include Joyent, IBM, Paypal, Microsoft, Fidelity, and the Linux Foundation.

For enterprise-level development, Node.js is sufficient and excellent in terms of performance, security, stability, etc.

Kongxian technology is a SaaS service based on cloud storage, providing services to small and medium-sized sellers. The core system is purchase, sale and storage, order pool and WMS. So far there’s been no problem,

es && babel

In 2015, ECMA International Congress announced the official approval of ECMA-262 version 6, also known as ECMAScript 2015 (formerly known as ECMAScript 6, ES6).

Babel (http://babeljs.io/) is already in extensive use as an ES compiler, modules are great, and some people write compilers in other languages with Babel. Es6 features were added in Node.js after 0.12 and are not currently supported in ES7. So to use es’s advanced features in Node.js, Babel is required to compile them. This is the standard Node is chasing.

January 22, 2016, (Microsoft request Node. Js support ChakraCore https://github.com/nodejs/node/pull/4765).

In the future, Node.js will not only be based on the Chrome V8 kernel, but also support many other browser kernels, which is very good for ecology, efficiency and more.

CAI wei brother of chakra benchmark contrast (https://github.com/DavidCai1993/ES6-benchmark), the basic conclusion is V8 ES5 > chakra ES6 > chakra ES5 > V8 ES6

Q & A

1. Have you considered other languages besides Node.js for the full stack?

SangShiLong:Yes, swift and Lua are possible in the future. Swift has a big syntax and performance advantage, and Lua has a chance with OpenResty, but not as big as Swift.Things like WebAssembly don’t look so good.

2. Ask Mr. Sang: Did you refer to the STATIC API in the concurrent development process just now? And if so, who writes it? Are you already working as a single person from the front end to the back end?

SangShiLong: Didn’t do a document that is currently static API, so now is directly provide json and part (json – https://github.com/typicode/json-server) server, is responsible for the backend development leader in writing, He’s about a week ahead of normal development. At present, not one person writes all the front and back ends, the team was established shortly, tianjin Node.js will not have many, so the front and back ends are separated. However, moa-frontend enables the frontend to learn back-end knowledge such as Express, and when appropriate, provides opportunities to transfer front-end knowledge to back-end knowledge.

3. Your company mentioned static API in the development collaboration. Is there any good tool you can recommend?

Sang Shilong: NIn the ode. Js (json – server https://github.com/typicode/json-server) better

I would love to write a generator for various requests around a static API. Once the API comes out, the documentation and HTTP request code for each platform will be generated, and I can also pressure test the official API, but I don’t have the energy to write it yet.

4. Do hybrid apps have performance problems on mobile terminals? Is there any optimization experience you can share?

SangShiLong:Lightweight enough, less big framework, good front-end optimization. Note the difference between Touch and click, such as the Tap gesture in Fastclick or Zeptojs. Chrome Profile (CSS3 Animation). Use Weinre real machine test. Reference: H5 practice http://mp.weixin.qq.com/s?__biz=MzAxMTU0NTc4Nw==&mid=222892082&idx=1&sn=ba1cdb42b43fbec08e4328c5080774e5#rd (I).

5. If it’s all stacked, what’s the current division of labor in your team?

SangShiLong:Our team still tends to specialize, with very small service granularity to allow for job rotation and to prepare for open code like Google. However, there are many situations where mobile commandos are needed (especially in the startup era), which can be combined at random, and the full stack provides remote with additional convenience.

6. H5 on the phone with iScroll pit more ah, especially Samsung opened the hardware acceleration of the render page, Mulberry teacher how to view?

Sang Shilong: You can try the H5 virtualization of Taobao system, ghostway once spoke at the AS conference, we are not able to do so deep level optimization at present.

7. Does Node.js have enough performance and precision to do business calculation

SangShiLong:You’re not asking node.js, but the database node.js operates on.Performance – consuming calculations can be architecturally balanced, if delayed, MQ can do it. If it’s non-latency, you can write the corresponding service in another language, and you don’t have to have Node.js. In our current scenario, there is no computational bottleneck.

8. In terms of API return format, why don’t we just flatten the status and put code and message out? Is there any benefit to this setting?

SangShiLong:More semantically clear. The only json returned is data and status. If status.code! = 0, I’ll just take MSG, if it’s 0, it’s not a very good design to process data, but the structure is clear, for developers, it’s more acceptable.

Finally, Miss Sang would like to add the most important sentence, don’t forget to tell each other.

Vacant string technology is looking for full stack related architect, senior programmer, language no limit, coordinate tianjin, email: [email protected]

This article is planned by Li Qingfeng, editor Wang Jie, and reviewer Tim Yang. We would like to discuss more NodeJS full stack development, please pay attention to the public account to get into the group opportunity. Please indicate that the reprint is from the wechat official account of “ArchNotes” and contains the following QR code.

Highly available architecture

Change the way the Internet is built



Long press the QR code to subscribe to the ha Architecture official account