WebIDE has finally landed inside the cloud music team. At present, there is no problem to cover the normal business development of the front end on the function, and also stepped on a lot of pits during the period. Therefore, I write this article to make a reference for wanting to land WebIDE in the team. Of course, some very detailed problems, an article is not good to cover, this paper is more of a design scheme and landing direction reference.

The significance of WebIDE

In fact, at the beginning of launching WebIDE project, many people, such as superiors, colleagues and department leaders, would question the project: what is the significance? What business value does it bring?

To tell the truth, this question, including now at the beginning I also feel difficult to answer, novice more convenient? Telecommuting? More secure code? How long do developers spend writing code? Private plug-in domain?

But that doesn’t seem to be a particularly painful point in business development. Of course, there is some business value in the business of applets, FAAs, or cloud services, after all, some of these services themselves depend on WebIDE. But excluding these business scenarios, for most ordinary business teams, WebIDE’s meaning and business value is frankly not that great.

But measuring the value of a project requires more than just thinking from a business perspective. Other values of WebIDE that I have personally combed through at the team, r&d efficiency, and company level (all after doing similar things) :

  1. Invigorate the team and increase the retention rate. As a less popular technology, it can more or less stimulate team thinking and increase team vitality. And as a technology that could be a front-end infrastructure in the future, it helps to retain new people. Infrastructure is not perfect, then the new recruit, the heart is bound to ridicule, loss is taken for granted.
  2. Switching projects is more convenient. For the front end team of Multirepo, I believe many students often switch between 3 or 4 projects, starting and closing a project is not easy, not to mention the computer power, memory is not a small load. Webide transfers this part of the load to the remote container, saving the local load and the inconvenience of opening each startup (finder -> terminal -> run dev -> open vscode), to be honest, I feel it is more convenient to develop. After all, when you start a project, you open a web page, and when you close a project, you don’t have to worry about progress not being saved.
  3. Unified team configuration. Unified plug-in, IDE configuration, more smooth code transfer
  4. telecommuting Under the epidemic, not to mention, you can write code without proper equipment, which greatly improves the convenience
  5. The code is more secure. With a few restrictions, you can prevent code from being copied, and at the company level, it’s easier to measure programmer hours (PS: I won’t include this feature, the IDE market for programmers is not very useful for measuring programmer value).
  6. Demo is easy to write. Open source has many public tools, but often requires proprietary packages, which external open source projects can do little about
  7. Integration of R&D platform, close r&d link. With a large team, various platforms emerge in endlessly, and the development process is often accompanied by the switch of multiple platforms. IDE embedded in the web page, while integrating various RESEARCH and development platforms, to avoid the tedious and inconvenience caused by repeated platform switching
  8. Private plug-ins. Of course there are ways to avoid team plug-in publishing in the common domain, but setting up a private plug-in market directly is much more intuitive.

In my opinion, discussing the meaning of a technology that has not yet been implemented is not necessarily a meaningful discussion. How meaningful WebIDE is depends more on how big a team can, or wants to, do it. So I won’t go into this article. Interested in IDE? The job is done.

Comparison of mainstream products and technical solutions in the industry

Although WebIDE itself has not yet been popular, there are already products in major cloud manufacturers and major domestic manufacturers. It is not a relatively new technology, and its implementation scheme is also basically clear and mature. Here, a few are selected for a brief introduction.

Vscode genre

More popular solutions, on behalf of coder, code-server, VSonline.

This class of IDE products, based on vscode for the web side of some modifications, while fitting github, to achieve personal projects webide development.

However, the disadvantages are all the same: fees are charged and it is difficult for teams to access (team login check, GitLab access), and code security is difficult to guarantee

Theia

Theia is a popular solution. Some features reuse vscode, but are designed to be extended, so private deployment and customization are extremely convenient.

Currently available products are:

Gitpod, Ali SYSTEM IDE (developed kaitian is not based on theia), Huawei Cloud IDE, Google Cloud IDE.

In terms of expansion capacity, it is far superior to VScode, and has the advantages of full open source, high community activity, low cost of access and transformation, and no charge

The IDE I landed in the team was also based on this framework.

stackblitz

Stackblitz is a handy IDE for writing demos. I thought it was in the same school as CodesandBox, but it introduced the concept of Web Container, which provides a pure front-end Node environment with no containerization solution, which is very valuable.

At present, the solution is not open source, the current speculation is more serverice Worker request interception, WebAssembly provides a Node environment, so as to achieve no remote machine, all run in the browser an IDE.

Overall, the advantage of its technical solution is that it does not consume remote resources, but the disadvantage is that it is not open source, and the node environment is simulated after all, which may be missing at some level of each system (guess).

other

There are a few other options:

  1. Codesandbox based on front-end construction, this online analysis of the article more, not here, its disadvantage is that there is no complete command line, running webpack-based front-end page is ok, Node code can not run, more is used to write a UI demo tool.
  2. Kaitian, a pure IDE developed by Alibaba, tries to replace a pure IDE developed by THEIA. It is not open source at present
  3. Coding team webide, coding, has been opened source, currently unknown

The overall contrast

  1. Vscode genre: pay, access is difficult, not easy to extend
  2. Theia genre: popular open source solutions in the industry, mature landing products, expansion is very easy
  3. Stackblitz: Unique technology solution, pure front-end WebIDE, but currently there is no solution except writing demo fast, team access and extension
  4. Codesandbox: front-end build, more for writing demos
  5. Kaitian, Codeing and other self-developed, not open source, the details are not clear

Access Scheme Selection

For the WebIDE that the team needs, I personally think it is necessary to meet the daily development requirements. So easy to expand, must be at the top of the list, this point is theia. Open source, free charge, good stability and active community are also strengths of THEia. In the long term, theIA is a great solution with low access cost, good scalability, active community and convenient maintenance. I landed in the cloud music team is the final adoption of this program.

How to land quickly within the team

Having said so much, this chapter is actually the beginning of the ground.

WebIDE as a mature technology, in recent years only gradually appeared in more occasions, which is also a number of technical reasons. At present, the major WebIDE schemes are basically container-based schemes, which require the team to have relatively perfect container-based management and dynamic deployment capabilities.

So before landing, the ability to container deployment is a must for the team, of course, using external commercialization is also possible.

In addition, some basic infrastructure needs to be improved, such as gitLab, member management, front-end and back-end project deployment and other infrastructure products.

Only with the above capabilities can the requirements of fast landing be met.

The overall structure

The author believes that the landing of WebIDE can give full play to the iterative way of Internet small step iteration and fast online. First a car, then a motorcycle, then a car. Improve the functionality step by step.

In my opinion, the basic capabilities required for a WebIDE for bikes are:

  1. Access the team member management system or OA
  2. Team GitLab access
  3. The ability to install private packages
  4. IDE management, new, close, delete and other basic functions
  5. Support front-end project running

In order to meet the requirements of the above capabilities, the author initially (and later only added some independent projects) divided the project into four projects:

  1. Ide-studio: the front end manages the back end, responsible for the creation and closure of Ide, and can expand the plug-in market and other functions in the later stage
  2. Ide- Studio – Backend: a pure NodeJS backend, which manages the interfaces of the backend and connects to the team member management system. It can be used to expand the functions of private and common plug-ins and cooperate with containerized deployment platforms.
  3. Ide-extension: For developing theIA extensions, customizing team-specific features such as access to the team member management system, logging active time, integration with the team’s internal development platform, etc
  4. Ide-docker: Used to store Ide docker-related files. Since theia itself is built for extension, you only need a package.json to generate IDE dependencies, so separate out the docker image build

The ground plan

Theia container builds

Theia itself has a container startup scheme and has provided a Dockerfile file, which can be directly used for deployment. The author’s team uses modifications based on TheIA-Docker.

The official dockerfile is as follows, and I have directly added my own comments to it:

Select the underlying node dependency of the container
ARG NODE_VERSION=12.18.3
FROM node:${NODE_VERSION}-alpine
Add the necessary system dependencies required by Theia Install
RUN apk add --no-cache make pkgconfig gcc g++ python libx11-dev libxkbfile-dev libsecret-dev
ARG version=latest
WORKDIR /home/theia
ADD $version.package.json ./package.json
# GITHUB_TOKEN specifies the GITHUB_TOKEN when the container is built.
ARG GITHUB_TOKEN
Install and clean up the global cache
RUN yarn --pure-lockfile && \
    NODE_OPTIONS="--max_old_space_size=4096" yarn theia build && \
    yarn theia download:plugins && \
    yarn --production && \
    yarn autoclean --init && \
    echo *.ts >> .yarnclean && \
    echo *.ts.map >> .yarnclean && \
    echo *.spec.* >> .yarnclean && \
    yarn autoclean --force && \
    yarn cache clean

# Re-choose the underlying system dependencies to keep the container system minimalist, removing the system dependencies required during theia installation, thereby reducing mirroring references
FROM node:${NODE_VERSION}-alpine
# See : https://github.com/theia-ide/theia-apps/issues/34
RUNaddgroup theia && \ adduser -G theia -s /bin/sh -D theia;
RUNchmod g+rw /home && \ mkdir -p /home/project && \ chown -R theia:theia /home/theia && \ chown -R theia:theia /home/project;
Add git bash and other base system dependencies
RUN apk add --no-cache git openssh bash libsecret
ENV HOME /home/theia
WORKDIR /home/theia
COPY --from=0 --chown=theia:theia /home/theia /home/theia
EXPOSE 3000
ENV SHELL=/bin/bash \ THEIA_DEFAULT_PLUGINS=local-dir:/home/theia/plugins
ENV USE_LOCAL_GIT true
USER theia
The # run theia service runs on port 3000 by default
ENTRYPOINT [ "node"."/home/theia/src-gen/backend/main.js"."/home/project"."- the hostname = 0.0.0.0" ]
Copy the code

Generally speaking, this is a relatively standard front-end engineering operation file, and it is not difficult to understand docker without basic search.

However, a few of the actual potholes need to be noted:

  1. Github_token must be specified; otherwise, some NPM packages cannot be downloaded. Use your own github_token. See the official documentation for the generation method
  2. Note that shutting down agents (such as Charles, etc.) on your own computer while building an image can also cause NPM package installation problems

Of course, this is the official mirror of Theia, and when you actually implement the bike version of WebIDE, you need to do the following:

  1. Inject git public and private keys. You are advised to use the private key of a single repository to minimize permissions
  2. Automatic clone object code
  3. The NPM private domain is injected so that no further configuration is required by the consumer
  4. Execute user-defined automation scripts

These operations can be written as a Node script and executed at the end together with the startup of the Theia service.

In the actual landing and use of the process, but also encountered some practical problems:

  1. How to obtain the git public and private key? Here it is really difficult to obtain the git public and private key in IDE container, there are authentication and other problems, so the author inject the relevant variables into the container environment variables when the container is created (at this time there is user identity information), and the container directly take the data of environment variables to build the public and private key
  2. As shown in dockerfile, theia created a workspace for the sake of container security, and the user’s identity is theia. However, some dependencies in the system are not enough, such as GCC, g++ and other parts of the NPM package installation need system dependencies. There are also no necessary system dependencies such as Curl. Therefore, you can grant root permission to the user
  3. You can integrate oh-my-zsh into the command line. You can use bash for Linux by default without git, highlighting, etc

In order to solve the above problems, the author added the following commands to the original dockerfile

# Add system dependencies so that the NPM installation can proceed smoothly
RUN apk add --no-cache make pkgconfig gcc g++ python curl

Install ZSH and beautify the command line
RUN apk add --no-cache git openssh bash zsh libsecret-dev

Add root so that users can add their own global system dependencies
RUN chmod 4755 /bin/busybox
RUN echo 'root:danger' | chpasswd

Modify the final startup command
ENTRYPOINT ["/bin/sh"."/home/cloud-theia/start.sh"]

In start.sh, execute the git, NPM global configuration script, and start theia's own service
Copy the code

At this point, the function of the container is basically complete. You can just build one locally and run it.

IDE management page setup

This part is more conventional front end page, here we should be familiar with, using some background templates to build.

The main functions are:

  1. IDE management: add delete check change
  2. Plug-in management: private/public plug-in, plug-in upload and management
  3. Basic team member login/logout

The plugin management can use OpenVSX recommended by Theia itself, which is fully open source and also recommended for private deployment.

At that time, because the IDE management system wanted to directly switch seamlessly with the plug-in management system (lazy), I just forwarded the interface directly to OpenVSX through node background, so as to obtain its data, and realized the public plug-in acquisition/download simply and quickly.

IDE background service setup

The author builds the background of IDE based on egg.js, which mainly carries the following capabilities:

  1. IDE management, more conventional background interface

  2. Plug-in management, self-storage of private plug-ins, public plug-ins for forwarding to OpenVSX

  3. · Handled the interaction of netease Cloud container deployment, domain name forwarding and other services (netease Xiaozhou, personalization and other services are not bad when used)

  4. Handles some of the services needed for plug-in installation/search within the IDE container

  5. Basic member login/logout

Overall, the code is basically the same as normal Node service. Needless to

Sensics extension

This part should be difficult for most students who have not contacted theIA. While theia itself provides very convenient scalability, this convenience is based on your understanding of how theia is built.

The official website is not much, you can see this article, run up the problem is not big, the rest of the use of the view of the source code.

However, there are not many places that need to be expanded to access my own team. The author has only changed the following items so far:

  1. Team member login/logout (pure front-end extension)
  2. Team logo, member information display, related help link jump

This section does not even cover Theia’s JSONRPC, back-end extension capabilities. But the bike version is enough.

In the last picture, I only scratched the surface of my ability to expand:

In the actual use process, the author also changes the following environment variables:

  1. VSX_REGISTRY_URL is a plugin source modification for the plugin market, which by default points to OpenVSX. To implement a private plugin, change it directly to your own private market and implement the associated interface
  2. THEIA_WEBVIEW_EXTERNAL_ENDPOINT, the web version of the webview, the default is pointing to the uuid minibrowser. {uuid}. The minibrowser. Uuid. Minibrowser. {the hostname}, Because in the actual landing scenario, HTTPS universal domain name certificate cannot be used when the domain name is written in this way, and one MUST be applied for by an IDE. Webview uses service worker, so it cannot work without HTTPS, so the author directly changes it to hostname. See this article for reasons of interest
  3. THEIA_MINI_BROWSER_HOST_PATTERN, as above, is also changed to hostname to complete the preview of the image file

other

Document station construction, template construction and other conventional operations, no longer described

conclusion

Overall, with all of this in place, you have a webIDE that can be used by your team.

At present, the webIDE of the author’s team has been able to meet the daily development and use, and will be further expanded in the future to serve as the bearing point of each RESEARCH and development platform to solve the problem of repeated switching between different platforms in the current development process.

Of course, in the actual landing, there will be a lot of trivial details, we can communicate with ha.

The last

Webide as a mature technology, the team landed the whole process, the author thinks that looking back, there is no particularly big pit and technical difficulties.

Originally, I prepared the topic: how to implement webIDE in the front end of the team, but later I thought, this is a bit of a mislead, although I did not write a few lines of front-end related code, but I need to reserve some other knowledge, such as container, Node, operation and maintenance knowledge.

Finally, after setting up a full-fledged WebIDE, I developed my own business on WebIDE, but most of the team remained conservative. We will continue to improve the development experience and get more and more members to adopt this new IDE with the old technology.