Like it and see. Make it a habit.

This article has been collected on github-java3c, which contains my series of articles, interview questions, self-study materials, architect videos, e-books, etc.

First, the origin of creation

Hello everyone, MY name is Llami. Today, I would like to share with you 10 top projects of Alibaba open source, which are essential tools for Java development.

The 10 projects are all from Alibaba’s open source projects on Github.

They are all java3C projects that I have used in my work or studied before. I have also written articles about individual projects, but I have never organized them together. Over the weekend, I spent about half a day sorting them out for everyone to read and study.

Alibaba’s Github address

Ii. Project Introduction

1, arthas

So far, github has 27.4k stars and 6k forks, making it alibaba’s most popular open source project.

Arthas is an open source Java diagnostic tool for Alibaba that developers love.

Arthas can help you when you are stuck with a problem like the following:

  1. From which JAR is this class loaded? Why are all kinds of class-related exceptions reported?
  2. Why didn’t the code I changed execute? Did I not commit? Got the branch wrong?
  3. If you encounter a problem, you cannot debug it online. Can you only re-publish it by logging?
  4. There is a problem with a user’s data processing online, but it cannot be debugged online, and it cannot be reproduced offline!
  5. Is there a global view of the health of the system?
  6. Is there any way to monitor the real-time health of the JVM?
  7. How to quickly locate application hot spots, generate flame map?
  8. How do I find an instance of a class directly from the JVM?

Arthas provides an administrative interface for troubleshooting problems online, requiring no restart. Dynamically trace Java code; Monitor JVM status in real time.

Arthas supports JDK 6+, Linux/Mac/Windows, command line interaction, and rich Tab auto-completion to further locate and diagnose problems.

  • Making: github.com/alibaba/art…
  • Documents: arthas.aliyun.com/doc/

More than 120 companies have registered to use:

2, p3c

P3c is a Java protocol checking plug-in developed by Alibaba, which can be inherited into IDEA and Eclipse.

I believe that many students engaged in Java development have known Alibaba produced “Java Development Manual”, mainly sorted out the internal use of Ali Java development specifications.

P3c plug-in realizes 53 rules in the development manual, most of which are based on PMD implementation. Among them, 4 rules are based on IDEA implementation, and real-time detection function is realized based on IDEA Inspection. Some rules implement the Quick Fix feature.

At present, there are two modes of plug-in detection: real-time detection and manual trigger.

Overview of main functions:

3, druid

Druid is a JDBC component library. It is the same as C3P0 and DBCP. It contains database connection pools and SQL Parser components.

Druid connection pool

Druid connection pool is alibaba’s open source database connection pool project. Druid connection pooling is built for monitoring, with powerful built-in monitoring features that do not affect performance. Powerful, can prevent SQL injection, built-in Loging can diagnose Hack application behavior.

  • Github project address github.com/alibaba/dru…
  • Document github.com/alibaba/dru…
  • Download repo1.maven.org/maven2/com/…
  • Monitoring the DEMO at http://120.26.192.168/druid/index.html

Competing goods contrast

4, fastjson

Fastjson is Alibaba’s open source JSON parsing library, which can parse JSON-formatted strings and support serialization of Java beans to JSON strings, as well as deserialization from JSON strings to Javabeans.

The advantages of fastjson

1. Fast speed

Fastjson is fast compared to other JSON libraries, and since fastjson’s 1.1.x release in 2011, its performance has never been surpassed by other JSON libraries implemented in Java.

2, widely used

Fastjson is widely used in Alibaba, deployed on tens of thousands of servers, and is widely accepted in the industry. In 2012, it was selected as one of the most popular domestic open source software by Open Source China.

3, the test is complete

Fastjson has a large number of Testcases, with over 3321 testcases in 1.2.11. Regression testing is performed for each release to ensure quality stability.

4. Easy to use

The Fastjson API is very concise.

String text = JSON.toJSONString(obj); / / the serialization
VO vo = JSON.parseObject("{... }", VO.class); // deserialize
Copy the code

5, easyexcel

EasyExcel is a Java – based simple, save – memory Excel reading and writing open source project. Support for reading and writing mega – line Excel while saving as much memory as possible.

Java parsing, Excel generation frameworks are well known Apache POI, JXL. Poi has a SET of SAX mode API, which can solve some problems of memory overflow to a certain extent. However, POI still has some defects. For example, the decompression and storage after decompression of 07 Version Excel are completed in memory, which still consumes a lot of memory. Easyexcel rewrite POI 07 version of Excel parsing, a 3M Excel with POI SAX parsing still needs about 100M memory, switch to EasyExcel can be reduced to a few M, and no matter how big Excel memory overflow; Version 03 relies on POI’S SAX mode, and makes the encapsulation of model transformation in the upper layer, which makes the user more simple and convenient.

For example: 75M of Excel (46W rows and 25 columns) can be read from 64M of memory in 1 minute

There is also a speed mode that is faster, but with a memory footprint of a little over 100 megabytes.

6, canal

MySQL Binlog incremental subscription & consumption component

Introduction to the

The main purpose is to provide incremental data subscription and consumption based on MySQL database incremental log resolution.

Logging based incremental subscription and consumption services include

  • Database mirroring
  • Real-time Database backup
  • Index building and real-time maintenance (split heterogeneous index, inverted index, etc.)
  • Service Cache Refresh
  • Incremental data processing with business logic

Current canal supports source MySQL versions including 5.1.x, 5.5.x, 5.6.x, 5.7.x, 8.0.x

The working principle of

Principle of MySQL master/slave replication

  • MySQL master writes data changes to binary log (binary log events)
  • MySQL slave copy master binary log events to its relay log
  • MySQL slave replays events in the relay log to reflect data changes to its own data
Working principle of CANAL
  • Canal emulated the interaction protocol of the MySQL slave, disguised itself as the MySQL slave, and sent the dump protocol to the MySQL master
  • MySQL master receives dump request and starts pushing binary log to slave
  • Canal parses binary log objects (originally byte streams)

7. Spring Cloud Alibaba

Spring Cloud Alibaba is committed to providing a one-stop solution for microservice development. This project contains the necessary components for developing distributed application microservices that developers can easily use to develop distributed application services through the Spring Cloud programming model.

Relying on Spring Cloud Alibaba, you only need to add some annotations and a little configuration, you can plug Spring Cloud application into Ali Micro service solution, and quickly build distributed application system through Ali middleware.

The main function

  • Service traffic limiting degradation: By default, WebServlet, WebFlux, OpenFeign, RestTemplate, Spring Cloud Gateway, Zuul, Dubbo, and RocketMQ traffic limiting degrade are supported. You can modify traffic limiting degradation rules in real time through the console at runtime and view the Metrics of traffic limiting degradation.
  • Service Registration and Discovery: It ADAPTS to the Spring Cloud Service Registration and Discovery standard and integrates Ribbon support by default.
  • Distributed configuration management: Supports external configuration in distributed systems, and automatically refreshes configuration changes.
  • Message driven capabilities: Build message driven capabilities for microservice applications based on Spring Cloud Stream.
  • Distributed Transactions: Solve distributed transaction problems efficiently and without intrusion to the business using the @GlobalTransactional annotation.
  • Ali Cloud object storage: Ali Cloud provides massive, secure, low-cost and highly reliable cloud storage services. Supports storage and access of any type of data in any application, anytime, anywhere.
  • Distributed task scheduling: provides second-level, precise, highly reliable, and highly available scheduled (Cron expression based) task scheduling services. It also provides a distributed task execution model, such as grid tasks. Grid tasks allow quantum tasks to be evenly distributed to all workers (schedulerx-client) for execution.
  • Ali Cloud SMS service: global SMS service, friendly, efficient and intelligent interconnection communication capabilities, help enterprises quickly build customer access channels.

component

  • Sentinel: Take traffic as the entry point to protect the stability of services from multiple dimensions, such as flow control, fuse downgrading and system load protection.
  • Nacos: A dynamic service discovery, configuration management, and service management platform that makes it easier to build cloud-native applications.
  • RocketMQ: An open source distributed messaging system that provides low-latency, highly reliable message publishing and subscription services based on highly available distributed clustering technology.
  • Dubbo: Apache Dubbo™ is a high-performance Java RPC framework.
  • Seata: Alibaba open source product, an easy-to-use high-performance microservices distributed transaction solution.
  • Alibaba Cloud OSS: Alibaba Cloud Object Storage Service (OSS) is a massive, secure, low-cost and highly reliable Cloud Storage Service provided by Alibaba Cloud. You can store and access any type of data in any application, anytime, anywhere.
  • Alibaba Cloud SchedulerX: a distributed task scheduling product developed by Alibaba Middleware team, which provides second-level, accurate, highly reliable and highly available timed (Cron expression based) task scheduling service.
  • Alibaba Cloud SMS: global SMS service, friendly, efficient and intelligent interconnected communication capabilities, to help enterprises quickly build customer access channels.

8 nacos.

Nacos profile

Nacos is the service infrastructure for building modern “service” centric application architectures (e.g., microservices paradigm, cloud native paradigm).

Nacos supports the discovery, configuration, and management of almost all major types of “services” : Kubernetes Service, gRPC & Dubbo RPC Service, Spring Cloud RESTful Service

Nacos features

  • Service discovery and service health monitoring

    Nacos supports DNs-based and RPC-based service discovery. After a Service provider registers a Service using a native SDK, OpenAPI, or a separate Agent TODO, Service consumers can use DNS TODO or HTTP&API to find and discover services.

    Nacos provides real-time health checks on services to prevent requests from being sent to unhealthy hosts or service instances. Nacos supports health checks at the transport layer (PING or TCP) and the application layer (e.g. HTTP, MySQL, user-defined). Nacos provides two health check modes: Agent report mode and server active check mode for complex cloud environments and network topologies, such as VPCS and edge networks. Nacos also provides a unified health check dashboard to help you manage service availability and traffic based on health status.

  • Dynamically configured service

    Dynamically configured services allow you to manage application configuration and service configuration for all environments in a centralized, external, and dynamic manner.

    Dynamic configuration eliminates the need to redeploy applications and services when configuration changes, making configuration management more efficient and agile.

    Centralized configuration management makes it easier to implement stateless services and make it easier for services to scale flexibly on demand.

    Nacos provides an easy-to-use UI (sample console Demo) to help you manage the configuration of all your services and applications. Nacos also provides a number of out-of-the-box configuration management features including configuration version tracking, Canary publishing, one-click rollback configuration, and client configuration update status tracking to help you more securely manage configuration changes and reduce the risks associated with configuration changes in a production environment.

  • Dynamic DNS Service

    The dynamic DNS service supports weighted routing, enabling you to implement load balancing at the middle layer, flexible routing policies, traffic control, and simple DNS resolution services on the data center Intranet. Dynamic DNS services also make it easier for you to implement DNS protocol-based service discovery to help eliminate the risk of coupling to vendor-proprietary service discovery apis.

    Nacos provides some simple DNS APIs TODO to help you manage your service’s associated domain name and available IP:PORT list.

  • Services and their metadata management

    Nacos enables you to manage all services and metadata in your data center from a microservices platform construction perspective, including managing service descriptions, life cycles, static dependency analysis of services, health of services, traffic management of services, routing and security policies, SLA of services, and most importantly metrics statistics.

Nacos map

  • Feature diagram: The feature demands of the problem domain to be solved will be comprehensively introduced from functional features and non-functional features
  • Architecture big picture: Quick access to the world of Nacos through clear architecture
  • Business Picture: The business scenarios that can be supported with the current features, and their best practices
  • Ecological picture: Systematically combs the relationship between Nacos and mainstream technology ecology
  • Advantages: Show the core competitiveness of Nacos
  • Strategic picture: The macro advantages of Nacos should be explained from strategic to tactical level

Nacos ecological figure

9, Sentinel

Sentinel introduces

With the popularity of microservices, stability between services becomes increasingly important. Sentinel is a flow control component for distributed service architecture. It mainly takes traffic as the entry point to help you guarantee the stability of micro services from multiple dimensions such as flow control, fusing degradation and system adaptive protection.

Basic concepts of Sentinel

resources

Resources are the key concept of Sentinel. It can be anything in a Java application, for example, a service provided by the application, or another application invoked by the application, or even a piece of code. In the rest of the documentation, we will describe code blocks in terms of resources.

Any code defined through the Sentinel API is a resource that can be protected by Sentinel. In most cases, resources can be identified using method signatures, urls, and even service names as resource names.

The rules

The rules set around the real-time status of resources can include flow control rules, fuse degrade rules, and system protection rules. All rules can be dynamically adjusted in real time.

Sentinel function and design concept

Flow control

Flow control is a common concept in network transmission, which is used to adjust the sending data of network packets. However, from the perspective of system stability, there are also a lot of considerations on the speed of processing requests. Requests arriving at any time are often random and uncontrollable, and the processing capacity of the system is limited. We need to control the flow according to the processing capacity of the system. Sentinel, as a dispatcher, can adjust random requests to fit as needed, as shown in the following figure:

Flow control has the following angles:

  • Resource invocation relationship, such as resource invocation link, resource and resource relationship;
  • Performance metrics, such as QPS, thread pool, system load, etc.
  • Control effects, such as direct current limiting, cold start, queuing, etc.

Sentinel is designed to give you the freedom to choose the Angle of control and combine it flexibly to achieve the desired effect.

Fusing the drop

In addition to flow control, reducing unstable resources in the call link is one of Sentinel’s missions. Because of the complexity of the invocation relationship, if a resource in the invocation link is unstable, requests will eventually pile up. This problem is the same one described in Hystrix.

Sentinel and Hystrix share the same principle: when a resource in the call link is unstable, such as timeout, and the proportion of exceptions increases, the call to this resource is restricted and the request is allowed to fail quickly to avoid affecting other resources, resulting in an avalanche effect.

Sentinel and Hystrix have taken radically different approaches to their limitations.

Hystrix isolates dependencies (in our sense resources) by means of thread pools. The benefit of doing this is to achieve the most complete isolation between resources. The disadvantage is that in addition to the increased cost of thread switching, there is also a need to allocate thread pool sizes for each resource up front.

Sentinel approaches this problem in two ways:

  • Limit by the number of concurrent threads

Unlike the resource pool isolation approach, Sentinel reduces the impact of unstable resources on other resources by limiting the number of concurrent threads for resources. Not only is there no thread switching wastage, you don’t need to pre-allocate the size of the thread pool. When a resource is unstable, such as a long response time, the direct effect on the resource is a gradual accumulation of threads. When the number of threads accumulates to a certain number on a particular resource, new requests for that resource are rejected. The stacked thread completes its task before continuing to receive requests.

  • Degrade resources by response time

In addition to controlling the number of concurrent threads, Sentinel can quickly degrade unstable resources through response time. If the response time of a dependent resource is too long, all access to the resource is denied until the specified time window expires.

System load protection

Sentinel also provides adaptive protection at the system dimension. Avalanche prevention is an important part of system protection. When system load is high, if you continue to let requests in, the system may crash and fail to respond. In a clustered environment, the network load balancer will forward the traffic that should be carried by this machine to other machines. If other machines are also in an edge state, the increased traffic will cause that machine to crash and the cluster to become unavailable.

Sentinel provides a protection mechanism to balance incoming traffic with the load of the system and ensure that the system can handle the most requests within its capacity.

Sentinel working mechanism

  • Provide adaptive or display apis for mainstream frameworks to define resources that need to be protected, and provide facilities to perform real-time statistics on resources and call link analysis.
  • Traffic is controlled based on preset rules and real-time resource statistics. Sentinel also provides an open interface for defining and changing rules.
  • Sentinel provides a real-time monitoring system that allows you to quickly understand the current status of your system.

The overall framework

10, COLA

COLA stands for Clean Object-oriented and Layered Architecture.

So far, COLA has grown toCOLA 4.0.

COLA is divided into two parts, COLA architecture and COLA components.

Good application architectures follow some common patterns, whether hexagonal architecture, onion ring architecture, clean architecture, or COLA architecture, all advocate business as the core, decouple external dependencies, and separate business complexity from technical complexity.

In order to quickly create apps that meet COLA architecture, we provide two archetypes under cola-Archetypes:

  1. cola-archetype-service: used to create pure back-end servicesarchetype.
  2. cola-archetype-web: used to createadapterIntegrated with back-end serviceswebapplicationarchetype.

Run the following command:

MVN archetype: the generate \ - DgroupId = com alibaba. Cola. The demo. The web \ - DartifactId = demo - web \ - Dversion = 1.0.0 - the SNAPSHOT \ -Dpackage=com.alibaba.demo \ -DarchetypeArtifactId=cola-framework-archetype-web \ -DarchetypeGroupId=com.alibaba.cola \ - DarchetypeVersion = 4.0.1Copy the code

If the command is executed successfully, the following application code structure is displayed:

The core responsibility of the COLA architecture is to define a good application architecture and provide best practices for the best application architecture. Through exploration, we found that a good hierarchical structure, a good definition of package structure, can help us to govern the chaotic business applications.

Write in the last

Although it is not technical work to collect the information of these projects, it really took a lot of time to sort it out. I hope you can have an impression when you come up with technical solutions/technology selection or interview in the future, and maybe help you solve a problem.

If you like, please transfer praise, the author tuo Light all for you to see!