The current situation of The Japanese Internet industry

I had heard about the bizarre state of the Japanese Internet industry before I went to Japan, and I have experienced a lot since I came here. In general, there are the following aspects:

  1. General technology: Japan’s computer theory research is actually very good, but the industrial application of the general, a little unambitious feeling. To what extent do you want to be unambitious? For example, Yahoo Japan, Japan’s largest web portal, uses Google’s API for its search engine.

  2. Backward development system: Because Japan’s manufacturing industry is very developed and has achieved great success, many industrial products are achieved through layers of outsourcing. OES, simply put. Through this model, they have achieved great success in the traditional manufacturing industry, so they do not hesitate to follow this model for the emerging Internet industry — layers of outsourcing.

  3. Most of the workers are half-baked: most are untrained, mostly transferred from other industries. People of all ages have it.

The above causes many problems, which I will elaborate on in the following examples.

When I first joined the club

The company I work for is ト, which is an information consulting company. It has many branches, basically covering all aspects of Japanese life. Real estate, beauty, hotel reservations, used cars, hiring, and more. You can imagine the situation where Domestic dianping, Homelink, 58.com, Ctrip and other information providers are all in one company. After the author enters the job, the specific job calls the branch company of estate information platform that mentions in the title to work. The group I entered was responsible for the development and maintenance of mobile terminal websites.

Technology stack

  • Front-end technology stack: PHP(Zendframework)+ template engine Smarty+ jquery-like library developed by myself. The whole environment is deployed on AWS.

  • Backend technology stack: The backend architecture is not clear because it is outsourced, and development and maintenance are done by outsourcing companies. Yes, you are right. The biggest real estate information platform in Japan has outsourced the most core data.

So what can your own company do? So that leaves the front end. So let’s take a look at what they did on this front end.

The big front end

For in-house development, that is, self-development (in fact, about half of the self-development here are outsourcing staff who have been in the company for years), only PHP and the front end are left. So some friends will ask, isn’t that there is no back-end? Isn’t there PHP here? Those of you who are early adopters probably know how websites were built before AngularJS, ReactJS, VueJS and other front-end frameworks. In the early years, template engines were at the back end. After the backend gets the necessary data, it draws the basic usable HTML according to the template and sends it to the front end. After the front end gets the drawn HTML, it just does some animations or AJAX asynchronous requests and so on. Here PHP is doing the template drawing and accessing the API for processing data. In this framework, PHP can be categorized as a big front-end.

Problems faced

As the largest real estate information platform in Japan (with only 400,000 daily active users), there must be problems with such a poor front-end architecture. So what’s the problem?

Technical problems

  • JS/CSS layer has no framework, DOM operation is directly manual operation, and the library like Jquery developed by ourselves also has some bugs that are difficult to check from time to time, with repeated codes and global functions flying all over the sky.

  • Template engine in PHP layer, zendframework as the framework, so there are more or less rules, by page as a unit of template code, but there are a lot of duplicate code between pages, resulting in maintenance difficulties.

  • In general, it’s chaos.

Organizational issues

As for why so messy, very organizational composition has a lot to do with.

Japanese Internet companies are a breeding ground for product managers. Our company is a large enterprise in which many programmers are members. But in most Internet companies, the programmers are contractors. Only the product manager is a full member. The result is an organizational structure in which the programmer is completely subordinate to the product manager. Even the programmers who are members are not very vocal. How about, some Japanese, want to be a product manager why don’t you come to Japan?

This organizational structure leads to the scrambling code for the KPI (s) ば schoschol ゃ code. Product managers can’t read code and don’t care about code quality, let alone senior executives. In this case, coupled with the quality of its programmers themselves is not in place, the code written can be imagined, anyway, most of the outsourcing programmers are finished a quarter will leave.

Say many are tears, the environment is such, I a small programmer can not change, I can do is in the group to promote the advanced concept technology.

The original solution

If your company happens to be in one of these very backward architecture situations and you are looking for a way out, I hope this article will give you a little inspiration.

When I came on board, AngularJS was already weak, ReactJS was already well known, and VueJS was just starting to take off. The front end is like the Spring and Autumn period and the Warring States period.

The preparatory work

The first instinct, of course, is to import a new framework. Of course, before importing the front-end framework, you need to prepare for importing the framework. Such as extracting JavaScript from the template, extracting reusable components from the template, and so on.

Import webpack

Then import the WebPack into the project in the form of an entry for each page in interface units. Import the JavaScript referenced by each page in entry. This way all JavaScript is packaged in Webpack. One of the problems with the webPack multi-page packaging process is that it is repackaged. Multiple pages tend to import some of the same packages. Here I solved this problem by using code split, originally the purpose of code split was to load on demand and avoid large packages. I’ve made all packages asynchronous with code split so that all packages are loaded on demand and can be reused easily.

Refactoring:

After the refactoring:

Because importing WebPack allows you to do a lot of interesting things. For example, I built a storybook-like tool in-house, but unlike Storybook, I can run PHP templates. Of course, it is also possible to write JavaScript in ES6.

Also imported esLint, postCSS, various advanced tools. Greatly improve the work efficiency and code quality.

Choose the framework

Most importantly, after importing WebPack, import the front-end framework.

However, after various comparisons, VueJS was selected as the framework for import. There are several reasons

  • VueJS is easy to get started, because the team’s internal and external contractors are all halfway out of their careers, including those who have turned to music, auction and sales. Yes, you read that right, there are female programmers in their 40s, single mothers in their 30s, and children in their 20s who graduated from college. But there is a common feature, is several fixed technology dry no less than 5 years. The cost of time and error is very high for them to learn and become familiar with new techniques. So if you can use existing technology, that’s great. With VueJS, HTML/CSS/JavaScript is easy to learn. I set up the Webpack environment for them, so they don’t have to learn it directly.

  • It can be partially imported, specifically mounted on HTML nodes that have already been drawn, meaning that you can still bind VueJS instances to PHP drawn HTML.

Everything seemed to be going well, but there were a lot of problems with the actual import.

I can’t get past it

SEO is very, very important to our site, and at that time Google officially introduced mobile Index First policy, which means that search will be done from mobile sites.

PHP’s template engine is old, but SEO friendly, and is in excellent condition after years of modification and operation. The company takes this area very seriously. As you all know, although VueJS theoretically solved the SEO problem through SSR, after all, it is a new technology, and for big companies, and Japanese companies, it is very, very cautious. Then have a friend will say, can experiment on a small scale. Let me talk about the organizational structure first

Company for the development of the mobile terminal of the web site is more than one group, the development of the group is to distinguish according to the market field, such as the land market, is led by the market group of soil, the land in the field of web development is led by the land market group, the development of new functions are initiated land group, and then by the land development team the following programmer. Generally speaking, the site is the same site, according to the market group, each group has its own program development. But for site-wide maintenance and updates, there’s a site-wide development team, and I’m on that site-wide development team. However, organizationally, the site-wide group has no control over other market groups, and the control is in the hands of the product manager of the market group. There are five or six small groups throughout the site. There are organizational barriers, if the level is not enough, it is very difficult to promote the whole station.

Thoughts on refactoring

We can’t change the situation, we have to find another way. Let’s go back to where we started. What problem am I trying to solve?

It’s not about SEO, it’s not about whether PHP needs to exist, it’s about making development more efficient.

Why is current development inefficient?

  • Template code is organized by interface, and common code between interfaces is copy-pasted. There is too much duplicate code, and changes are jumping around.
  • JS/CSS is more chaotic than Template, they exist in different places, there is no unified partition of files or folders according to the interface or function mode, often in order to find a CSS, JS everywhere, there is a lot of overlap between these CSS, JS.

So the problem is to organize the relationship between Template/JS/CSS. How to organize it?

Componentization is good medicine

Whether it’s AngularJS, ReactJS or VueJS, they offer cool features like data binding and virtual DOM, but we don’t have much use for traditional large CMS sites. However, one of their most important concepts is very applicable to our site: Web Component.

So the question is, how do you componentize without a framework?

webpack-component-loader

The specific solution, as I’ve detailed in another article,

How do you componentize without a large front-end framework

To sum up: with webpack-loader, you put templates /JS/CSS together in component units during development and move them to where they belong during packaging.

conclusion

After the import of my loader, componentalization has been realized, but the existing technology stack has not been changed, and SEO has not been affected, and the efficiency has been greatly improved. And the cost of moving to VueJS, ReactJS, or a new framework (as long as it fits the componentization idea) is not particularly high, given the componentization trend everyone is pushing.

PS

In the comments, many friends asked me how I came to work in Japan. I did not go to university in Japan, but came to work in Japan directly after I graduated from China. In fact, there are many channels to come over, especially IT talents, this urgent shortage. Interested friends can follow my Zhihu weibo account, welcome to consult.

The author zhihu

The author weibo