About the Apache Pulsar

Apache Pulsar is the top project of Apache Software Foundation. It is the next generation cloud native distributed message flow platform, integrating message, storage and lightweight functional computing. It adopts the architecture design of computing and storage separation, supports multi-tenant, persistent storage, and multi-room cross-region data replication. It has strong consistency, high throughput, low latency, and high scalability. GitHub address: github.com/apache/puls… This month’s report is compiled by the StreamNative team.

Introduction:

December 2020 Pulsar Community Monthly report is here! First of all, a belated New Year’s greetings for 2021!

This month the Pulsar community refined and updated many of the product details, all over the place. Thanks to the following community members for supporting Pulsar this month and keeping Apache Pulsar alive! (In no particular order, see if you made the list 🏆🏆) :

@Jeames00, @CongboBO184, @renkai, @reswqa , @lhotari, @codelipenghui, @Jerrypeng, @eolivelli, @WolfStudy, @Nodece, @Erobot, @315157973, @gaoran10, @zymap, @eolivelli, @reswqa , @Robertindie, @Gjiangtao, @Erobot, @Mkozioro, @Massakam, @k2la, @renkai, @Jennifer88Huang, @tuteng, @ZSH0139, @huanli-Meng, @Aahahmed – Se, @Sijie, @Sijia-W, @Rathipry, @hangc0276, @bewaremypower, @Jianyun80 23, @Merlimat, @aloyszhang, @Marvincai, @DockerZhang, @rdhabalia, @Flowchartsman, @Aloyszhang, @Jerrypeng, @Massakam, @Sijie, @Jiaz Hai, @Cimura, @zzzming, @Codelipenghui, @rdhabalia, @Zymap, @saosir, @Massakam, @DockerZhang, @shoothzj, @eolivelli, @gaoran10, @mil Os-matijasevic, @K2LA, @deonvDV, @Wangjialing218, @Astifter, @yufan022

Product dynamic

Function

Function: Key-based batch processing is supported in Go Function Builder. Github.com/apache/puls…

Function: Adds a timeout to the openTable() call that represents Function state. Github.com/apache/puls…

Function: provides an interface for Function worker services. Github.com/apache/puls…

Function: Allows state modification in Function without blocking Function execution. Github.com/apache/puls…

Function: Make the schema information of source topic available for downstream sinks. Github.com/apache/puls…

Function: Added the –retain-key-ordering flag to the Pulsar Functions. Github.com/apache/puls…

Function: Support to get Go and Python Function by configuring the URL. Github.com/apache/puls…

Function Worker: Split Function Worker and client authentication logic. Github.com/apache/puls…

Package management

Package Management: Adds the Package management service to the Pulsar startup process. Github.com/apache/puls…

Package Management: Package management service supports BookKeeper storage. Github.com/apache/puls…

Tiered storage

Tiered Storage: When you initialize offloader, you initialize it only once. Github.com/apache/puls…

Test

Test: PulsAR-Perf supports multiple consumer subscriptions. Github.com/apache/puls…

Java Client

Java Client: The Java Client supports setting a reader subscription name. Github.com/apache/puls…

Java Client: Implements producer memory limits. Github.com/apache/puls…

Java Client: Supports the Creation of WaitForExclusive by Producer. Github.com/apache/puls…

Schema

Schema: Add default values for attributes in SchemaInfoBuilder. Github.com/apache/puls…

KoP

KoP: Statistics can be updated using specified statistics. Github.com/apache/puls…

Websocket

Websocket: Supports the deliveryAt and deliverAfter properties of Websocket Producer. Github.com/apache/puls…

WebSocket: Returns a status code based on the type of PulsarClientException. Github.com/apache/puls…

C++

C++ Client: adds the consumer configuration for reader to decrypt encrypted messages. Github.com/apache/puls…

C++ Client: the C++ Client supports adding setters to set internal subscription name Settings for readers. Github.com/apache/puls…

C++: optimize batch message cache allocation. Github.com/apache/puls…

C++ : correct subscription API parameter names. Github.com/apache/puls…

Broker

Broker: Supports configuring the maximum number of subscriptions per topic at the topic level. Github.com/apache/puls…

Broker: Exports the Prometheus indicators of messageTTL. Github.com/apache/puls…

Broker: A manager action that implements package management. Github.com/apache/puls…

Broker: Introduce lightweight Broker metadata. Github.com/apache/puls…

Broker: Add a REST API for package management services. Github.com/apache/puls…

Broker: Add package management service to Pulsar startup process. Github.com/apache/puls…

Broker: Change the getWebServiceUrl method to asynchronous. Github.com/apache/puls…

Broker: Supports client package management commands. Github.com/apache/puls…

Broker: Introduce the continuous offset of Pulsar. Github.com/apache/puls…

Broker: Add the updateRates method to the KoP to collect the publish rate. Github.com/apache/puls…

Broker: Supports setting the Netty Acceptor thread pool size. Github.com/apache/puls…

Broker: Supports viewing Broker entry metadata. Github.com/apache/puls…

Broker: Supports limiting the maximum number of topics per namespace. Github.com/apache/puls…

Broker: Exposes statistics on disconnected deleted messages. Github.com/apache/puls…

Broker: Add the beforeSendMessage method to intercept an entry before sending it to the consumer. Github.com/apache/puls…

Broker: Supports setting the maximum number of subscriptions per topic at the namespace level. Github.com/apache/puls…

Broker: Improved error handling when brokers do not trust client certificates. Github.com/apache/puls…

Broker: Enables namespace isolation policy updates to take effect. Github.com/apache/puls…

Broker: for SubscriptionBusyException add if branch. Github.com/apache/puls…

Broker: Delete duplicate Broker Prometheus indicator types. Github.com/apache/puls…

Broker: Added the original Prometheus indicator provider. Github.com/apache/puls…

Broker: Pulsar-admin supports automatic certificate refresh. Github.com/apache/puls…

other

The continuous offset of Pulsar is introduced. Github.com/apache/puls…

Perf: pulsar-perf supports loading WebSocket service urls from conf files. Github.com/apache/puls…

SQL: Add retry for SQL tests. Github.com/apache/puls…

Bug fix

Broker

Broker: Use new threads to delete non-persistent subscriptions to avoid deadlocks when inactive subscriptions are deleted. Github.com/apache/puls…

Broker: Supports deleting all topics of a given namespace when deleting it. Github.com/apache/puls…

Broker: Delete non-persistent permanent subscriptions to topics that are deleted from different threads to avoid deadlocks when inactive subscriptions are deleted. Github.com/apache/puls…

Broker: Fixed performance issues caused by invalid logging configuration. Github.com/apache/puls…

Broker: Check that the subscription list is copied when messages expire to avoid deadlocks. Github.com/apache/puls…

Broker: Fixed an issue where empty topics could not obtain lastMessageId due to message retention. Github.com/apache/puls…

Broker: repair PersistentStickyKeyDispatcherMultipleConsumers of NPE. Github.com/apache/puls…

Broker: Clean up topics that cannot be unloaded from the cache. Github.com/apache/puls…

Broker: Update the Maven artifact version. Github.com/apache/puls…

Broker: Resolved metadata compatibility issues. Github.com/apache/puls…

Broker: Fixed issue where subscription distribution rates would not work after uninstalling a topic without distribution rate limits. Github.com/apache/puls…

Broker: Ensures that rack information is dynamically updated. Github.com/apache/puls…

Broker: Fix formatting errors in items in Pulsar Broker. Github.com/apache/puls…

Broker: Fixed issue where DelayedDelivery had a default value at the Broker level. Github.com/apache/puls…

Function

Function: Fixed single quotes added to user configuration. Github.com/apache/puls…

Function: Fixed panic triggered by Go Function discarding messages. Github.com/apache/puls…

C++ Client: fix race conditions in BlockingQueue. Github.com/apache/puls…

Schema: GenericJsonReader converts an empty value to the string “null”. Github.com/apache/puls…

Client

Java Client: Fixed an NPE issue that occurred when the Pulsar Client received an acknowledgement message from a producer publication that had been closed. Github.com/apache/puls…

Java Client: Add more information when sending a TimeoutException for troubleshooting. Github.com/apache/puls…

Java Shade Client: Add encryption integration tests. Github.com/apache/puls…

Pulsar Client: Fixed an issue where authParams parameters with encrypted fields (*****) were displayed in logs. Github.com/apache/puls…

Client: Fixes unavailable Hash range conditions. Github.com/apache/puls…

Admin

Admin: repair AdminApiTest2. TestMaxSubPerTopicApi Flaky test. Github.com/apache/puls…

Pulsar-managed-ledger-admin: fix deleting multiple ledgers. Github.com/apache/puls…

Pulsar CI

Pulsar CI: Disable Maven’s HTTP connection pool by passing -dhttp. keepAlive = false -dmaven.wagon. Http. pool = false in the MAVEN_OPTS environment variable. Github.com/apache/puls…

other

Pulsar IO: Fixes Pulsar -io. Yaml filename and sourceConfigClass. Github.com/apache/puls…

Schema: Fixed an issue where custom Avro Schema was not available for consumers. Github.com/apache/puls…

Pulsar Build: Use the absolute pathfinder license and checkstyle plugin configuration. Github.com/apache/puls…

Pulsar Package: Fixed an issue where packages could not be uploaded. Github.com/apache/puls…

Common: Fixed an exception raised when viewing compressed messages (Airlift does not support read-only buffers). Github.com/apache/puls…

Transaction: Fix Transaction message order errors and deduplicate errors. Github.com/apache/puls…

Proxy: Returns the correct Authz and Auth errors from the Proxy server to the client. Github.com/apache/puls…

Community dynamic

  • Pulsar User Survey 2020

Click “Read the original article” or scan the QR code below to participate in the Pulsar 2020 user survey and fill out the questionnaire for a chance to receive the new Pulsar peripheral!

  • Tgip-cn 028: Design and implementation of Apache Pulsar delay queue

  • Video Review of Pulsar Summit Asia 2020

  • ApacheCon 2020 – Pulsar/ BookKeeper Special

Technology of dry

  • Impact of Apache Pulsar 2.7 release on the development of Infinitic

  • Cloud-Native Apache Pulsar 2.7 Supports Transactions and Azure Blob Storage Offloader

  • Work-Bench Snapshot: Augmenting Streaming and Batch Processing Workflows

  • How Apache Pulsar is Helping Iterable Scale its Customer Engagement Platform

  • Using Apache Pulsar With Kotlin — Gilles Barbier

  • New features in Pulsar 2.7.0

  • Pulsar Flink Connector is new in 2.7.0

That’s the Pulse tour for December 2020. Apache Pulsar is growing fast, thanks for your support!

Apache Pulsar community encourages you to actively participate in the open source community, whether it is documentation, code, translation, or technical blog, you are welcome to actively participate and become Pulsar Contributor as soon as possible.

If you’re not familiar with the Pulsar Contribute process, check out this tutorial to familiarize yourself with how to Contribute to Pulsar via GitHub: How to Contribute to Pulsar for beginners | How to Contribute to Pulsar for Non-technical people.