I recently shared my experience with the latest version of GitLab, GitLab 14 and the China release: Polar Fox.
However, as we all know, after V10, GitLab has increased its functionality and gradually adjusted its focus to become a one-stop platform. As products tend to be oriented towards companies and organizations, it has become increasingly dependent on server resources, which can run smoothly from the initial resources of around 1GB of memory. To the point that it now requires at least six to seven GIGABytes of memory to run smoothly.
It becomes a challenge for developers and small teams to use it with relative restraint and light weight. So this article will try to configure GitLab so that it can provide services with relatively low resource consumption.
Writing in the front
If you pursue absolute resource usage, only hope to have a lightweight code warehouse, for project management related functions don’t mind, today, GitLab regardless of how to optimize warehouse capabilities to meet the other focus on code project, recommend you to use the lightweight “Gitea” program, have a few articles have to mention how to install before deployment, As well as the use of CI, believe that smart you, a few minutes to run up the set of services.
But if you want a Project management experience similar to GitHub’s and have private deployment requirements, GitLab is the place to go.
To test the installation, we use the method mentioned in the previous article to quickly initialize the container environment. (Code repository github.com/soulteary/l…
curl -o- https://raw.githubusercontent.com/soulteary/linux-scripts/main/docker-with-mirror.sh | bash
curl -o- https://raw.githubusercontent.com/soulteary/linux-scripts/main/docker-compose.sh | bash
Copy the codeBefore tuning, let’s take a look at how well the application performs when launched with the default configuration.
Observe the applications started with the default configuration
After installing the Docker environment, you can use the following configuration to start the application without any application Settings:
version: "3"
services:
gitlab:
image: Gitlab/gitlab - ce: 14.0.5 - ce. 0
container_name: gitlab
hostname: gitlab.soulteary.com
ports:
- "8080:80"
Copy the codeSave the above content as docker-compose. Yml, then start the app with docker-compose up-d.
Wait for a moment, after the application Web interface is normal service, use Docker STATS to check the initial resource consumption:
CONTAINER ID NAME CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O PIDS 4AF8EAD479B2 gitlab 5.82% 3.197GiB / 7.774GiB 41.13% 10.3kB / 33.5kB 135MB / 224MB 416Copy the codeYou can see that when GitLab comes up, it eats up to 3 GB of memory when it’s not in use, and if you keep watching, you can see that the CPU usage is bouncing around from 5% to 20%.
Enter the container and look at the process tree to see something like this.
Wrapper ─ ┬ ─ gitaly ─ ┬ ─ 2 * [ruby ─ ─ ─ 38 * [{ruby}]] │ └ 15 * [{gitaly}] ├ ─ ─ gitlab - CTL ─ ─ ─ omnibus - CTL ─ ─ ─ sh ─ ─ ─ xargs ─ ─ ─ tail │ ├─ ch.pdf ─┬─ ch.pdf ─┬─ ch.pdf │ ├─ ch.PDF │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf ├ ─ runsv ─ ┬ ─ redis server - ─ ─ ─ 4 * [{redis - server}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitaly - wrapper ─ ─ ─ 8 * [{gitaly - wrapper}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ postgres ─ ─ ─ 17 * (postgres) │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ bundle ─ ┬ ─ 2 * [bundle ─ ─ ─ 14 * [{bundle}]] │ │ ├ ─ 4 * [bundle ─ ─ ─ 13 * [{bundle}]] │ │ └ ─ 4 * [{bundle}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ ruby ─ ┬ ─ bundle ─ ─ ─ 72 * [{bundle}] │ │ └ ─ │ {ruby} └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitlab - workhors ─ ─ ─ 11 * [{gitlab - workhors}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ nginx ─ ─ ─ 9 * / nginx │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitlab - exporter ─ ─ ─ 4 * [{gitlab - exporter}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ redis_exporter ─ ─ ─ 10 * [{redis_exporter}] │ └ ─ svlogd │ ├─ Prometheus ──12*[{Prometheus}] │ ├─ Prometheus ──12*[{Prometheus}] │ ├─ Prometheus ──10*[{Prometheus}] │ ├─ Prometheus ├─runsv─┬─postgres_export── 9*[{postgres_export}] │ ├─runsv─┬─ grafana-server── 12*[{grafana-server}] ├─ svlogdCopy the codeAdjust the configuration and try to run light
After looking at the initial performance of the program in the default configuration, let’s look at the process tree in the container after adjusting the configuration.
Wrapper ─ ┬ ─ gitaly ─ ┬ ─ 2 * [ruby ─ ─ ─ 38 * [{ruby}]] │ └ 12 * [{gitaly}] ├ ─ ─ gitlab - CTL ─ ─ ─ omnibus - CTL ─ ─ ─ sh ─ ─ ─ xargs ─ ─ ─ tail │ ├─ ch.pdf ─┬─ ch.pdf ─┬─ ch.pdf │ ├─ ch.PDF │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf │ ├─ ch.pdf ├ ─ runsv ─ ┬ ─ redis server - ─ ─ ─ 4 * [{redis - server}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitaly - wrapper ─ ─ ─ 7 * [{gitaly - wrapper}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ postgres ─ ─ ─ 11 * (postgres) │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ bundle ─ ─ ─ 9 * [{bundle}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ ruby ─ ┬ ─ bundle ─ ─ ─ 18 * [{bundle}] │ │ └ ─ {ruby} │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitlab - workhors ─ ─ ─ 10 x │ [{gitlab - workhors}] └ ─ svlogd └ ─ runsv ─ ┬ ─ nginx ─ ─ ─ 9 * / nginx └ ─ svlogdCopy the codeYou can see how much the tree has been reduced. As above, after waiting for the application Web interface to be properly served, use Docker Stats to view the initial resource consumption:
CONTAINER ID NAME CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O PIDS 9A14FB16A437 gitlab 1.21% 1.967gib / 7.774gib 25.30% 19MB / 345kB 254kB / 178MB 200Copy the codeIf you continue to observe, you can see that the CPU resources of the application hover between 1 and 3%, and the memory is stable at the edge of 2GB. Because the heavier services are stripped away, the program’s resource consumption remains very stable after a few hours of running.
At this point, GitLab retains functions such as code management, project management, Wiki documentation, an online IDE, and various integration capabilities. Before I give you the configuration, let’s take a look at what features have been cut.
Disable unnecessary services
It is suggested to choose the treatment according to your own situation, and I will describe it in terms of how to save resources.
By default, GitLab provides a package repository, container repository, and dependency management, which can be replaced by distribution or Nexus in the article I shared earlier on Building a Simple and reliable Container Repository with Containers. The former is more lightweight and efficient, while the latter is more professional. You can turn these functions off in GitLab by setting some configuration items.
# Turn off the container repository
gitlab_rails['gitlab_default_projects_features_container_registry'] = false
gitlab_rails['registry_enabled'] = false
registry['enable'] = false
registry_nginx['enable'] = false
Package repository, dependency management
gitlab_rails['packages_enabled'] = false
gitlab_rails['dependency_proxy_enabled'] = false
Copy the codeThe GitHub Pages service provides a good way to document and Demo open source projects, as does GitLab, but in private deployment scenarios, CI can be used in conjunction with other more efficient tools to do this. For example, Hugo (Golang)/MdBook (Rust). Inside GitLab, Pages is a relatively complex service, with a wide selection of configuration options, if you flick through the Pages. You can also adjust the configuration to turn it off.
# GitLab Pages
gitlab_pages['enable'] = false
pages_nginx['enable'] = false
Copy the codeAs mentioned in the previous article, GitLab 14 has so many enhancements for monitoring that, after the application is initialized, a project is created specifically to monitor the health of the GitLab ontology. For personal use scenarios, you can disable monitoring and performance benchmarking functions to ensure data security because there is no complex load.
# Turn off monitoring and performance benchmarking
prometheus_monitoring['enable'] = false
alertmanager['enable'] = false
node_exporter['enable'] = false
redis_exporter['enable'] = false
postgres_exporter['enable'] = false
pgbouncer_exporter['enable'] = false
gitlab_exporter['enable'] = false
grafana['enable'] = false
sidekiq['metrics_enabled'] = false
Copy the codeIn addition, you can disable application performance analysis and reporting.
# Usage Statistics
gitlab_rails['usage_ping_enabled'] = false
gitlab_rails['sentry_enabled'] = false
grafana['reporting_enabled'] = false
Copy the codeFor individual scenarios, KAS and Terraform can be turned off if you do not have hybrid/public cloud scenarios, or if you do not need to use GitLab for CD management. While Kerberos and Sentinel appear in the document to be functions of enterprise edition software, the configuration is adjusted here to explicitly declare that the functions are disabled in order to save resources. While Mattermost is a good chat app, it can also be turned off in single-player scenarios, or in familiar IM scenarios.
# GitLab KAS
gitlab_kas['enable'] = false
gitlab_rails['gitlab_kas_enabled'] = false
# Terraform
gitlab_rails['terraform_state_enabled'] = false
# Kerberos documentation says EE only, but the default value is true
gitlab_rails['kerberos_enabled'] = false
# Sentinel
sentinel['enable'] = false
# Mattermost
mattermost['enable'] = false
mattermost_nginx['enable'] = false
Copy the codeBy default, GitLab creates a number of Puma processes to provide Web services capabilities. We can adjust and set it appropriately, just enough. In addition, the sideKIQ that manages scheduling can also reduce concurrency, avoiding unnecessary waste of resources. As for Gitaly, it is not recommended to adjust it after the actual measurement. On the one hand, the saving of resources is very limited; on the other hand, if the operating number of Gitaly is excessively limited, the use experience will be directly affected. For details, see below. (In the case of cluster deployment, we even need to independently deploy the Gitaly service to ensure the experience)
# Disable PUMA cluster mode
puma['worker_processes'] = 0
puma['min_threads'] = 1
puma['max_threads'] = 2
# Reduce the number of daemons concurrent
sidekiq['max_concurrency'] = 5
Copy the codeIn addition, you can turn off the email-related features if you don’t need them.
Turn off email related functions
gitlab_rails['smtp_enable'] = false
gitlab_rails['gitlab_email_enabled'] = false
gitlab_rails['incoming_email_enabled'] = false
Copy the codeFinally, remember from the previous article that GitLab CPU usage fluctuates a lot? If you’re willing to replace it with a lightweight Drone, consider turning off the default CI feature to keep CPU consumption down to very low values.
gitlab_ci['gitlab_ci_all_broken_builds'] = false
gitlab_ci['gitlab_ci_add_pusher'] = false
Copy the codeComplete configuration
Merge the above contents and update them to the configuration file. The complete configuration is as follows:
version: "3"
services:
gitlab:
restart: always
image: Gitlab/gitlab - ce: 14.0.5 - ce. 0
container_name: gitlab
hostname: gitlab.soulteary.com
ports:
- "8080:80"
- "2222:22"
volumes:
- ./config:/etc/gitlab
- ./data:/var/opt/gitlab
environment:
TZ: Asia/Shanghai
GITLAB_OMNIBUS_CONFIG: | external_url 'http://gitlab.soulteary.com' gitlab_rails['time_zone'] = 'Asia/Shanghai'
Turn off email related functions
gitlab_rails['smtp_enable'] = false
gitlab_rails['gitlab_email_enabled'] = false
gitlab_rails['incoming_email_enabled'] = false
# Terraform
gitlab_rails['terraform_state_enabled'] = false
# Usage Statistics
gitlab_rails['usage_ping_enabled'] = false
gitlab_rails['sentry_enabled'] = false
grafana['reporting_enabled'] = false
# Turn off the container repository
gitlab_rails['gitlab_default_projects_features_container_registry'] = false
gitlab_rails['registry_enabled'] = false
registry['enable'] = false
registry_nginx['enable'] = false
# package warehouse
gitlab_rails['packages_enabled'] = false
gitlab_rails['dependency_proxy_enabled'] = false
# GitLab KAS
gitlab_kas['enable'] = false
gitlab_rails['gitlab_kas_enabled'] = false
# Mattermost
mattermost['enable'] = false
mattermost_nginx['enable'] = false
# Kerberos
gitlab_rails['kerberos_enabled'] = false
sentinel['enable'] = false
# GitLab Pages
gitlab_pages['enable'] = false
pages_nginx['enable'] = false
# Disable PUMA cluster mode
puma['worker_processes'] = 0
puma['min_threads'] = 1
puma['max_threads'] = 2
# Reduce the number of daemons concurrent
sidekiq['max_concurrency'] = 5
gitlab_ci['gitlab_ci_all_broken_builds'] = false
gitlab_ci['gitlab_ci_add_pusher'] = false
# Turn off monitoring
prometheus_monitoring['enable'] = false
alertmanager['enable'] = false
node_exporter['enable'] = false
redis_exporter['enable'] = false
postgres_exporter['enable'] = false
pgbouncer_exporter['enable'] = false
gitlab_exporter['enable'] = false
grafana['enable'] = false
sidekiq['metrics_enabled'] = false
Copy the codeAgain, save the above contents as docker-compose. Yml and start with docker-compose up -d. At this point, you can have a “codebank service” with project management, warehouse storage, and a good experience with an online editor at a relatively light cost.
If you need to use HTTPS, you can refer to the previous article how to Configure GitLab to Use HTTPS with Traefik for configuration adjustment. If you need to back up and restore your data, read the GitLab Concise Maintenance Guide (V2020.05).
Hide unwanted features in the interface
In the official GitLab community, users have mentioned the following question:
You need an option to close the Security and Operations TAB on the screen
User A: The Operations option takes up too much space, but not all projects need it. For example, we have some projects that only use Issues and Wiki functionality without code. User B: “Security & Compliance” is a paid option, and as a free user, it is of no use to me.
This problem is still open and has not been replied by the official, but the solution is very simple. First copy the menu-related code from the running container to the host:
docker cp gitlab:/opt/gitlab/embedded/service/gitlab-rails/lib/sidebars/projects/menus menus
Copy the codeOpen a random menu file, such as menus/monitor_menu.rb:
# frozen_string_literal: true
module Sidebars
module Projects
module Menus
class MonitorMenu< : :Sidebars::Menu
override :configure_menu_items
def configure_menu_items
return false unlesscontext.project.feature_available? (:operations. context.current_user) add_item(metrics_dashboard_menu_item) add_item(logs_menu_item) add_item(tracing_menu_item) add_item(error_tracking_menu_item) add_item(alert_management_menu_item) add_item(incidents_menu_item) add_item(serverless_menu_item) add_item(terraform_menu_item) add_item(kubernetes_menu_item) add_item(environments_menu_item) add_item(feature_flags_menu_item) add_item(product_analytics_menu_item)true
end.Copy the codeDef configure_menu_items = false; def configure_menu_items = false;
# frozen_string_literal: true
module Sidebars
module Projects
module Menus
class MonitorMenu< : :Sidebars::Menu
override :configure_menu_items
def configure_menu_items
false
end.Copy the codeGitLab will ignore the initialization of the menu program, and you will no longer see similar buttons in the interface. Of course, remember to map the changed content to the container, or re-encapsulate an image that belongs to you.
.
volumes:
- ./config:/etc/gitlab
- ./data:/var/opt/gitlab
- ./menus:/opt/gitlab/embedded/service/gitlab-rails/lib/sidebars/projects/menus
.
Copy the codeNot recommended: Gitaly
As mentioned earlier, configuration adjustments to the Gitaly service are not recommended because the service has a few logical issues with the acquisition and determination of environment variables.
gitaly['ruby_num_workers'] = 3
Copy the codeEven if we only configure the number of workers, no concurrent number, and no cgroups limit, we will get an error message similar to the following, and the whole application will restart all the time, but normal service cannot be provided.
gitlab | time="2021-07-14T06:49:31Z" level=info msg="Starting GitalyversionGitaly, Version 14.0.5 "gitlab | time =" 2021-07-14 T06: securely Z "level = warning MSG =" git path not configured. Using the default path resolution" resolvedPath=/opt/gitlab/embedded/bin/git gitlab | time="2021-07-14T06:49:31Z" level=fatal msg="load config: config_path \"/var/opt/gitlab/gitaly/config.toml\": cgroups mountpoint cannot be empty" gitlab | {"gitaly":4639,"level":"warning","msg":"forwarding Signal "and" signal ": 17," time ":" the 2021-07-14 T06: securely. The 415 z ", "wrapper" : 4633} gitlab | {" error ":" OS: process already finished","gitaly":4639,"level":"error","msg":"can't forward the Signal "and" signal ": 17," time ":" the 2021-07-14 T06: securely. 415 z ", "wrapper" : 4633}Copy the codeIf you look at the resource usage and you see that the resource consumption is very close, there is not much point in fiddling with the service.
CONTAINER ID NAME CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O PIDS 526B770574AF gitlab 0.94% 1.985GiB / 7.774GiB 25.53% 1.52kB / 0B 0B / 5.27MB 238Copy the codeIn addition, the documents on the official website, GitLab default configuration template, there are multiple conflicts and errors for the data and default values of this service, and there are undocumented configurations, which are in a “black box state”. Configuring this service is not recommended.
other
Remember earlier when I said that “no matter how optimized GitLab is to this day, it’s hard to match other projects that focus on repository functionality”?
I shared how to use the lower version of GitLab in an earlier article, “Container Mode using a Lighter Version of GitLab,” and the process tree for GitLab in that article looks like this.
Wrapper ─ ┬ ─ gitlab - CTL ─ ─ ─ omnibus - CTL ─ ─ ─ sh ─ ─ ─ xargs ─ ─ ─ tail └ ─ runsvdir ─ ┬ ─ runsv ─ ┬ ─ SSHD │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ postgres ─ ─ ─ 8 * (postgres) │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ bundle ─ ┬ ─ bundle ─ ─ ─ 7 * [{bundle}] │ │ └ ─ 3 * [{bundle}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ bundle ─ ─ ─ 17 * [{bundle}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ gitlab - workhors ─ ─ ─ 18 * [{gitlab - workhors}] │ └ ─ svlogd ├ ─ runsv ─ ┬ ─ nginx ─ ─ ─ 2 * / nginx │ └ ─ svlogd └ ─ runsv ─ ┬ ─ gitlab - logrotat ─ ─ ─ sleep └ ─ svlogdCopy the codeWhen you compare GitLab to the default configuration in this article and to the modified GitLab configuration, you’ll see that the GitLab service is bloated. This is the essence of what we often hear about GitLab as “heavier”.
In a context where products are increasingly targeted at B-end users and there is pressure to make money, performance is always the last thing to consider when compared to development efficiency.
The last
This article is a practical attempt to the discussion in the group before. After a lot of work, the personal use scenario is still more recommended to use Gitea, without the project management function, without the built-in online Web IDE function, can quickly have a stable and lightweight warehouse service.
For team usage scenarios, GitLab is still worth investing some hardware resources into further use. However, whether to use GitLab for in-depth project management or to change the working mode of the team in combination with GitLab may be relatively difficult to be implemented. The authorities may have noticed, too, which is why there are more best practice sharing and training courses.
Perhaps GitLab V12 / V13 is a safer bet for the short term.
–EOF
If you think the content is still practical, welcome to share with your friends, thank you here.
We have a little toss group, which gathered hundreds of people like to toss.
Without advertising, we would talk about hardware and software, HomeLab, programming issues, and occasionally share information about technical salons in the group.
Like to toss friends welcome to scan code to add friends. (Add friends, please note the real name, indicate the source and purpose, otherwise it will not be approved)
All that stuff about being in a group
This article is licensed under a “CC BY 4.0” license. You are welcome to reprint or modify this article, but the source must be noted. 4.0 International (CC BY 4.0)
By Su Yang
Creation time: July 14, 2021 statistical word count: 8518 words reading time: 18 minutes to read this article links: soulteary.com/2021/07/14/…