Revelation! The mysterious organization behind open source software

The Flink community will share the “Inside ASF” series, which provides an overview of how ASF works, then explains in detail how to contribute to specific Apache projects, become a Committer, PMC on a project, How to choose multiple Apache projects to contribute to multiple fields and become ASF Member, etc., hope to help you really understand open source, open source participation.

Every time talking about open source, open source community, we have to mention Apache Software Foundation (ASF), as the world’s largest open source organization, how does it run and manage?

My open Source journey

I joined Ali in 2011 and experienced several organizational structure adjustments, including OPlog, Arirang, cloud transcoding, document conversion and other products. In October 2016, the team was lucky enough to have access to the development of Blink and began to understand the Apache Flink community. From participating in community development at the early stage to gradually leading the development of specific modules, to responsible for the construction of Apache Flink Python API(PyFlink). ASF Member, PMC Member of @ApacheFlink and a Committer for @ApacheFlink, @ApacheBeam, @ApacheiOTDB My open source journey is as follows:

Why did you write the ASF Series?

Why write the “Inside THE ASF Series” article? The core consideration is to find that most people in China have heard of ASF, and our country is one of the most open source users in the world. The chart below shows the download ranking of ASF software in different countries around the world, as revealed in the ASF 2019 annual report:

China has become the most downloaded region for Apache Software Foundation projects. However, there are very few people who really understand and participate in ASF contributions and become ASF members. This phenomenon is not that we are unable to participate in ASF contributions, let alone become ASF members, but that we have few channels to know ABOUT ASF, and there may be a language barrier more or less. Compared with other Western countries in the world, the number of people participating in ASF in China is very few! Here are some numbers:

ASF has 7,000 + code contributors, but the number of Chinese engineers contributing to ASF projects is only one thousand, less than one seventh.
There are about 350 ASF projects, but there are only 19 ASF projects initiated by China, and less than 5% of them have become top projects.
The ASF incubator has more than 200 mentors, but there are no more than five active Chinese mentors.
ASF holds ApacheCon every year in the United States, Europe and other places, but so far, ASF has not held an ApacheCon in China.
Latest statistics (2020.04.05) There are 765 ASF members in the world, 40 of whom are Chinese, 22 of whom are in China.

As the world’s largest Apache open source consumer, ASF has a wide user base in China. In recent years, major Domestic Internet companies have gradually participated in the construction of Apache open source. It is necessary for us to think about how to give Chinese engineers more opportunities to know ABOUT ASF and participate in ASF contribution. There are more opportunities to become a Committer, PMC, or even ASF Member on Apache open Source projects.

Therefore, “Into THE ASF Series” is to share my understanding of ASF and experience of participating in ASF contribution with everyone, vigorously promote the development of ASF in China, and make every effort to let Chinese people occupy a place in the FIELD of ASF open source ~ ~

What will be in The “Into ASF” series

In the ASF Series, we will start with ASF organization structure, ASF company and ASF project governance scheme to get a macro understanding of how ASF operates. Then, I will introduce the details in multiple chapters, such as how to contribute to specific Apache projects, how to become a Committer of a project, how to become a PMC member of a project, and how to select multiple Apache projects to contribute in multiple fields. How to become ASF Member and so on.

The following will share with you the organizational structure and governance of ASF.

What is ASF?

I thought, “What is ASF?” “Is one of the simplest questions and one of the hardest to answer. Simple because you can describe it from any Angle, like a blind man touching an elephant. And it’s hard because it’s hard to define in one sentence, “What is ASF?” . So my approach to this kind of problem is to describe the appearance first, and then explore the details. Just like when we answer what BMW is, we say, “Don’t touch me.” What is Audi? We would say “four circles”. So what is ASF? I mean, ASF is this beautiful feather down here;)

Knowing the Logo is probably the best place to start appreciating something new. So the next second you’re going to have a whole bunch of questions coming up? How old is she? What for? How does it work? What does it have to do with me? Let’s take our time…

Be the same age as Ali

The Apache Software Foundation (ASF) is a non-profit public charitable organization registered in the United States, founded in 1999. So what is the ultimate purpose of the establishment? The core purpose of ASF is as follows:

Provide the foundation for open collaborative software development projects by providing hardware, communication tools, and business infrastructure.
Establish a separate legal entity to which any company or individual can make donations and be able to ensure that those donations will be used for the public good.
Provide individual volunteers with the means to avoid legal action against foundation programs.
Protect the Apache brand used in its software products from abuse by other organizations.

Therefore, ASF is a public welfare organization with a good vision and great mission. ASF and Alibaba were both founded in 1999, and heroes think alike in terms of altruism. Alibaba takes “making it easy to do business” as its mission, while ASF is committed to building an open source software ecosystem, enabling software technology to be shared around the world, which is the incomparable charm of ASF!

By its roots

ASF was not created out of thin air. The ASF’s predecessor was the Apache Team, which maintained an HTTPD Web server written by NCSA. From 1995 to 1999, Apache HTTPD Web services, developed under the leadership of the Apache Team, became the market leader and continues to this day, accounting for 65% of the market. More and more people used the service that year, but the original developers lost interest in the project and moved on to other projects, leaving users unsupported.

So Brian Behlendorf created a mailing list on his computer for those users to collaborate on fixing, maintaining, and improving the software. Starting with less than a dozen software engineers sharing code patches via email in February 1995, and later with the Apache web site, which hosted many associated projects (e.g. Mod_perl, PHP, Java Apache), the problems mounted. For example, there is a growing need for a more coordinated and structured organization to address the potential for individuals to be attacked by the law. So in June 1999, ASF was registered! 🙂

Growing pains

As the software product matures, more users respond with more questions, more suggestions for software enhancements, and more people are involved in code fixes. Problems arose. The original founders could not handle all the problems, Review all the code patches, Merge all the submissions, so what to do?

When Apache members decided that a developer had “earned” the right to be a member of the community, they granted the developer direct access to the code repository, known as Committer, increasing the team’s power and ability to develop projects. This leads to more effective maintenance and development of the team. Of course, this is only the tip of the iceberg. ASF governance is not only the development of Committer, but also the systematic introduction in the following sections.

The ASF governance

At present, ASF has more than 140 cross-technology projects, involving more than 350 specific projects, more than 7,000 code contributors, covering more than 230 countries around the world, to ensure its healthy development must have a simple and efficient organizational structure and governance means. Then ASF governance will involve the governance of the ASF organization itself and ASF governance of the projects hosted by Apache web sites.

Corporate governance

In general, the governance of ASF is quite simple: members elect the board of directors; The Board appoints various officers and creates PMCs; PMCs report regularly to the board, and most other managers report to the CEO, who in turn reports to the board.

Organizational structure

ASF corporate governance should first have a sound organizational structure and clear division of roles and responsibilities. As shown below:

The organizational structure above is structured around the role of a “board of directors” with a Chair and a Vice Char. The company has president, vice president, secretary, financial controller, legal counsel and other positions. Each role has its own role to play. As an article for getting to know ASF for the first time, we can simply understand the following macro content:

ASF is governed by a board of directors composed of members.
Existing members (members) nominate and elect new members on a regular basis, nominating and electing 9 directors to the Board each year.
The board appoints business members and assigns responsibility for specific policies or business areas to each member. For example, the Vice President of the Legal Affairs Committee is responsible for setting the legal policy for ASF and all Apache programs and working with company consultants.
The board appoints executive officers, including the President, secretary, and Chief Financial Officer, who are responsible for ASF’s specific areas of operations.
Most officers report directly to the PRESIDENT on a monthly basis, who in turn reports to the board of directors on the overall operation of the ASF.
The Vice President of Infrastructure reports to the President and provides macro oversight of the operations of the infrastructure team. Within the team, the paid infrastructure manager role helps manage day-to-day operations, with paid staff prioritizing work and maintaining services used in all Apache projects.

So how do these characters come about? ASF also needs a system of election and appointment to ensure the healthy operation of the whole organization.

Election and appointment

The Board is responsible for creating and updating the Project Management Committee (PMCs). The board approves only reasonable build requests from the incubator (graduating as TLPs) or requests from the PMCs themselves (adding or removing PMC members). In each case, the changes recommended to the board have already been voted on by the relevant PMC. As shown below:

The Board of directors (the Board)
The Board of Directors consists of nine directors responsible for the management and supervision of various ASF matters. This includes people, assets (money, intellectual property, trademarks, and the infrastructure needed to support the project) and allocating resources to the project.
Project Management Committee (PMC)
The project management Committee is established by resolution of the board of directors and is responsible for the management of a specific project. Each PMC is composed of at least one ASF officer and appoints one member as president. The chair in each PMC is the eyes and ears of the ASF in each project and is critical to the ASF. The PMC is, of course, responsible for developing new PMC members and committers for development projects.
All kinds of VP
ASF has this VP. Most of their officers report directly to the PRESIDENT on a monthly basis, and the president reports to the board on a monthly basis on the overall operation of the ASF.

Perhaps these contents for just understand ASF or just joined the ASF open source construction of students simply understand, because it seems to be very far away. (In quotes because I think the thought is far away, not really that far). Next, let’s look at something closely related to specific project contributors – ASF project governance.

Project governance

Each project is managed independently by the project’s PMC, which follows some core principles set by all officers for all projects, in an Apache manner. As shown below:

In the figure above, I have sketched the outline of project governance using the Apache Flink project as an example. Each PMC is governed by the ASF Board, and each PMC is responsible for setting the technical direction of its own project.

The PMC reports directly to the board quarterly. The board oversees the PMC to ensure its healthy development, smooth management of the community, and compliance with Apache principles. The technical direction of the project is specified by the PMC, and the board does not provide technical governance for the project.
The chair of each PMC is the vice president of the program and therefore an ASF officer. The chairman’s primary responsibility is to ensure that project reports are comprehensive and submitted to the board.
The PMC votes on software releases. Appropriate release governance by the PMC ensures that all source code releases are ASF official.
PMCs nominates and elects new committers for projects. The PMCs is also responsible for nominating and voting on new PMC members, and the PMCs then reports to the Board of Directors on proposed changes.
The Apache Incubator is a special (I) PMC: its job is to mentor the newly established Podling community to help them learn the Apache Way. After each Podling votes for their software release, IPMC members will also monitor and vote on the release process for this Podling to ensure that Podling gradually grows into a qualified Apache project.
According to the policy, only individuals may act as members, committers or PMC members or administrators. This is a way for ASF and Apache projects to maintain their independence.

And each project has different contributing roles, such as: Users, Contributors, Committers, new PMC Members, and so on. The responsibilities of different roles are as follows:

Users – The general public who use the project and contribute to the project by providing feedback to developers in the form of bug reports and feature suggestions.
Contributors — that’s developers who contribute to projects in the form of writing code or writing documentation. Developers can share the community in a variety of ways, such as participant mailing list discussions, submitting code patches, submitting documents, and so on.
Committers – Committers are special contributors who are developers with write permissions to the repository.
PMC Member – A Member of the Project Management Committee (PMC), elected by Committers who have demonstrated outstanding performance in the development of the project. They have access to the code repository, voting rights on community-related issues, and the right and responsibility to nominate new PMC members and committers.

Status of ASF Board

This year’s ASF annual Member meeting was held from March 31 to February 2. The election for the ASF board of Directors was quite fierce, with 16 ASF members participating in the election. As a result, seven of last year’s nine board members were replaced, with Craig Russell and Shane Curcuru re-elected. The current membership is as follows:

Shane Curcuru (Re-elected Director)
Bertrand Delacretaz (former Director)
Roy Fielding (Former Director)
Niclas Hedhman (New Director)
Justin Mclean (New Director)
Craig Russell (Re-elected Director)
Sam Ruby (Former Director)
Patricia Shanahan (New Director)
Sander Striker (Former Director)

2020 New ASF Member

As mentioned in the election and appointment section above, the annual membership meeting of ASF not only elects 9 board members, but also nominates and votes for new ASF members.

Congratulations to them (and my own luck, of course)! According to the latest statistics (2020.04.06), there are 765 ASF members in the world, among which more than 20 are in China.

The ASF picture

A Logo is easy to remember, and a picture is much easier to remember than text 🙂

The ASF pyramid describes the group characteristics of different roles. The higher the group is, the smaller the group is, but the greater the responsibility for ASF is; the lower the group is, the smaller the responsibility for ASF is, and the easier it is to participate! Remember “What is ASF?” Start with feather Logo and participate in ASF contribution from users.

The ASF sentence

Pictures can represent appearances, words can represent meanings, and the opening feather represents ASF, but what about this beautiful feather? ASF is a public welfare organization established in 1999, managed by a complete organizational (board) structure, with the mission of sharing software (140 fields) and technology globally (covering 230 countries).

summary

This is the first part of “Stepping into ASF” series. I try to experience “Feynman technique” on myself. I introduce the ASF organization structure, ASF corporate governance and project governance from an overview, aiming to give students who want to or have just participated in ASF contribution a preliminary understanding of ASF. Finally, the present ASF board members and the new Chinese ASF members in 2020 are introduced. At the same time, to make it easier for you to remember, I will end this article with a picture and a sentence.

At the same time, “Into ASF Series” is different from my previous “Apache Flink Ramble Series”. ASF discussion is more non-technical sharing, which is easy to turn into boring sharing without focus, content and harvest, which is the last thing I want to see. So please send me some comments and suggestions, good or bad, at the beginning of this series so that we can grow together! Thank you.

References:

[1] www.apache.org/foundation/…