Editor’s note

Today, Battery Ventures, the legendary Silicon Valley venture capital firm that has invested in Google, Oracle, Salesforce, Linkedin, and more, published its own research and thoughts on hot open source projects and communities on TechCrunch, It also ranks the 40 most popular open source projects from several different perspectives

Many of today’s newest and hottest enterprise-oriented technologies are free “open source” technologies at their core. As a result, many large companies, from financial giants to retailers to service companies, have built their businesses around new, community-based technologies that stand in stark contrast to the IT practices of the past.

But how will corporate customers and investors evaluate these open source projects? How do they tell which projects (often with strange names like Ansible, Vagrant, Gradle) generate the most user usage trends? Which are the most sought after by software developers and which have the most market share potential?

These questions are especially difficult to answer because most open source companies are still private and don’t have to disclose key user and financial data. (That’s changing, though, as Cloudera recently announced plans to go public, a move that has brought more attention to open source.)

That’s why we decided to create a new, exhaustive index to track popular open source software projects and gain some insight into the companies using these open source technologies. We introduced this Index, called the Battery BOSS Index, which we’ve spent months putting together publicly available information. We hope to update the index quarterly, and it will become more accurate as more open source companies use these projects to go public.

The index contains 40 open source projects, selected from the list of open source projects on Github and Datamation. The top 25 are in the table below, and the full list can be found on our website.

We focus on enterprise-level IT-related areas such as IT operations, including technology-driven operations and service provisioning systems; Data and analytics, including artificial intelligence, machine learning and database-related tools; DevOps includes keeping an eye on the latest “container” technology trends, which help developers develop quickly in a given environment.

THE BATTERY OPEN-SOURCE SOFTWARE INDEX
ranking The project name Comprehensive project scoring field Related to the company Source code reading
1 Linux 100.00 IT Operations Red Hat, Ubuntu
2 Git 31.10 DevOps GitHub, GitLab
3 MySQL 25.23 Data & Analytics Oracle
4 Node.js 22.75 DevOps NodeSource, Rising Stack
5 Docker 22.61 DevOps Docker
6 Hadoop 16.19 Data & Analytics Cloudera, Hortonworks
7 Elasticsearch 15.72 Data & Analytics Elastic
8 Spark 14.99 Data & Analytics Databricks
9 MongoDB 14.68 Data & Analytics MongoDB
10 Selenium 12.81 DevOps Sauce Labs, BrowserStack
11 NPM 12.31 DevOps NPM
12 Redis 11.61 Data & Analytics Redis Labs
13 Tomcat 11.04 IT Operations NA
14 Jenkins 10.47 DevOps CloudBees
15 Vagrant 8.15 IT Operations HashiCorp
16 Postgres 8.02 Data & Analytics EnterpriseDB
17 Gradle 7.68 DevOps Gradle
18 Nginx 7.57 IT Operations Nginx
19 Ansible 7.42 IT Operations Ansible
20 Kafka 7.22 Data & Analytics Confluent
21 GitLab 6.42 DevOps GitLab
22 Hbase 6.41 Data & Analytics Cloudera, Hortonworks
23 Chef 6.37 IT Operations Chef*
24 TensorFlow 5.97 Data & Analytics Google
25 Cassandra 5.74 Data & Analytics DataStax

There are some well-known names on the list, including the ones that spawned some of the biggest companies: Linux, which spawned Red Hat; MySQL, a spin-off of the company of the same name that was later acquired by Sun Microsystems for $1 billion in 2008 (and is now owned by Oracle); And Hadoop, which spawned Cloudera and Hortonworks.

However, some relatively unknown names, such as Selenium, also ranked highly, indicating that there is a lot of grassroots innovation in the open source community and that many are spawning new companies. However, our research also shows that having a large audience for an open source project does not necessarily translate into a commercially viable company.

We ranked these projects according to four points:

  • Public interest, as measured by Google search activity
  • User activity, measured by the number of times a project is mentioned on Stack Overflow, a community of technical discussion forums
  • Career impact, measured by the number of jobs that mention these open source projects on job sites Indeed and Simply Hired
  • Impact in the open source community, measured by the project’s impact on Github. Specifically, we tracked the number of projects that were “branched”; The number of stars on Github; And the number of observations, updated to February 9, 2017.

Because some projects did extremely well at one point, or extremely poorly – for example, one project had the best Google search numbers but the number of jobs was not great – we removed the best and worst individual scores for each project. It’s called a “cut-tail mean,” and it’s similar to gymnastics scoring at the Olympics. (We don’t have east German judges on our team, but we have to be careful.)

Editor’s note: At the 1988 Seoul Olympics, the USA gymnastics team was beaten by the Olympic gymnastsThe unfair treatment of the east German referee

Even so, there’s always room for improvement. Some measures of adoption and popularity, like number of downloads, are obviously hard to measure, and we certainly don’t have all the latest and hottest tools. But as we update our data quarterly, we should be able to capture the latest industry leaders. So we hope to gradually get input from the open source community to improve our index. If you have a better idea for the index, please email us at [email protected]

Here are some important conclusions based on our research.

Linux, Git, and MySQL dominate

It should come as no surprise that the top open source project on our index is Linux, which was first released in 1991 and has since become one of the most widely adopted open source projects. It was commercialized by several companies, including Red Hat, one of the few publicly traded open source companies, as well as Ubuntu and SUSE.

Our no. 2 Git inspired two companies, Github and Gitlab. This extremely popular open source project is a “version control system” for tracking changes and coordinating efforts among software developers.

In third place is MySQL, an open source technology developed in 1995. MySQL currently helps companies like Google, Facebook, and Twitter flatten the Web. Note, however, that some “NoSQL” projects also rank high.

These NoSQL technologies include MongoDB (# 9) and Redis (# 12), which is being commercialized by Redis Labs; Cassandra is no. 25 and is backed by DataStax, a database company; Elasticsearch is number 7 and is currently being commercialized by Elastic.

MongoDB raised its most recent funding round in late 2015 — the company is now valued at around $1.5 billion — and is now competing with giants like Oracle, IBM, and Microsoft in the database space. In general, these NoSQL vendors are growing independently of each other, rather than merging into one large system. This also bodes well for a more fragmented data infrastructure space in the future, with several NoSQL powerhouses becoming public companies in the future.

Big data drives open source

As many organisations struggle to manage vast amounts of structured and non-institutional data — generated by everything from security software to tweets to networked sensors in factories — they increasingly need new data management and storage systems. This trend is also reflected in our Index. Fifteen of the 40 projects help with databases and data processing.

Hadoop, mentioned earlier, is one such project. Another is Spark, which is being commercialized by Databricks, at Number 8.

Some other items worth mentioning
Other names to know

Docker, the darling of container technology, which helps software develop faster and more efficiently, ranked fifth in our index. Docker has been seen by many as a possible alternative to giant VMware, and its easy and inexpensive use in the open source community has accelerated its adoption.

Docker is also competing with the likes of Google’s Kubernetes at Number 33, and Mesos, the orcheation layer in software development.

Another hot open source trend is “continuous integration and delivery”, the ability to continuously integrate developed code with other platforms. Tools in this area include Jenkins, currently being commercialized by CloudBee, and TravisCI. Other devOPs-related technologies include Maven (No. 30) and Artifactory, a fast-growing binary management software that is being commercialized by JFrog.

Road to commercialization

As we mentioned earlier, having a large user base, while critical for eventual commercial growth, does not necessarily guarantee that open source projects will turn out to be good commercial projects. Good business projects require more work and innovation, especially new business models; Take advantage of complex open source licenses; Adapt traditional enterprise sales times to fit open source projects, as we discussed on TechCrunch last year.

In addition, we find that the business success of IT enterprises based on open source projects increases if they can provide multiple open source technologies and combine them into a technology stack. Elasticsearch (# 7), Kibana (# 36), and Logstash (# 29) have an “ELK” stack.

InfluxData, a time series database company, also has a TICK technology stack, representing Telegraf, InfluxDB, Chronograh, and Kapacitor. A final example is HashiCorp, a DevOps company that has also commercialized many open source projects, including Vagrant (No. 15) and Vault (No. 40) on our list. Software developers like to pick their favorite components from these technology stacks, which fits into the “best of both worlds” mentality of software development today.

These open source projects are no longer your father’s Sun or Oracle. What is clear, however, is that ciOs of multinational companies, including some of those mentioned in our list, rely on these open source technologies to run their technology architectures, and these projects are here to stay. Indeed, at our open source summit last year, IT executives from Goldman sachs to CapitalOne expressed their “open source first” attitude. As open source companies go public over the next few years, we’ll get more insight into the performance of these leading projects and their impact on the industry.