Hello, everyone, I am for the majority of programmers brothers worry broken heart xiaobian, every day to recommend a small tool/source, full of your favorites, every day to share a small skill, let you easily save development efficiency, do not work overtime do not stay up late do not lose hair, is my goal!

Do you ever worry about these problems

  • You, the architect, want to solve the unique problem of database primary keys, especially in distributed systems with multiple databases.
  • You want this primary key to use the least amount of storage, index faster, and Select, Insert, and Update faster.
  • You need to consider primary key values that can be used directly and reflect the business timing of a table.
  • If such a primary key is too Long to exceed the front-end JS Number maximum and you have to convert Long to String, you can get a little frustrated.
  • Even if the Guid is self-increasing, it takes up a lot of space, which is not what you want.
  • You may have more than 50 application instances and each concurrent request may reach 10W/s.
  • You want the system to last more than 100 years.

In order to solve this problem, today’s editor recommends a new snowflake drift algorithm – IdGenerator, which makes ID shorter and faster generation. The key is to shorten the ID length and have a very high instantaneous concurrent processing capacity (50W/0.1s). Top optimization, super performance (shorter bits, faster), new SnowFlake algorithm, support C#/Java/Go/PHP and other languages.

Open source licenses

Use the MIT Open Source license

The link address

[idG] get git address

Traditional algorithm problem

  • The generated ID is too long.
  • Not enough instantaneous concurrency.
  • Failed to resolve the time callback problem.
  • Postcompletion generation of preorder ids is not supported.
  • Depends on the external storage system.

Features of the new algorithm

  • Integer numbers, monotonically increasing over time (not necessarily continuous), shorter in length, will not exceed the maximum js Number type in 50 years. (Default WorkerId is 6 bits and autoincrement is 6 bits)
  • Faster than traditional snowflake algorithm, 2-5 times, 0.1 seconds can generate 500,000. (i7 laptop, default algorithm configuration 6bit+6bit)
  • Time callback processing is supported. For example, if the server time is dialed back by 1 second, the algorithm can automatically adapt to the unique ID of the critical time.
  • Manual insertion of new ids is supported. When the service needs to generate new ids in the historical time, the reserved capacity of this algorithm can generate 5000 ids per second.
  • Can send out notification events during drift. Let the caller know exactly the algorithm drift record, Log concurrent call amount.
  • Does not rely on any external caches or databases. (But the WorkerId must be externally specified)

The performance data

(Parameter: 10-bit auto-increment sequence, 1000 drift maximum)

The effect

  • The maximum Number of js Number types is 9007199254740992. This algorithm can take 70 years to reach the JS Number Max value while maintaining the concurrency performance (5W+/0.01s) and the maximum Number of workerids (6 bits).
  • When increasing the WorkerId bit to 8 bits (256 nodes), the JS Number Max value is reached in 15 years.
  • Extreme performance: 500W/1s.
  • All test data were calculated based on generation 8 low voltage I7.

What is “I”

  • This algorithm is a class library, it is based on the net standard2.0 base library, does not rely on any third party components.
  • This algorithm does not depend on any external data system (except to be assigned a WorkerId).

Scope of application

  • Small, medium, and large projects that require globally unique ids (not GUids).
  • Distributed projects.
  • You don’t want to convert longs to strings for front-end projects. (If the front end supports BigInt, the type can not be changed)

How do I handle time callback

  • When system time rollback occurs, the algorithm uses the reserved ordinal number of the past sequence to generate a new ID.
  • The default is 100 per second (speed adjustable).
  • The ID generated by callback can be set to the back or the front by default.
  • Allow time to dial back to the preset cardinality of this algorithm (parameter is adjustable).

How long can

  • In the default configuration, the ID is available for 71000 years without repetition.
  • When 1024 working nodes are supported, the ID can be used for 4480 years.
  • When 4096 working nodes are supported, the ID is available for 1120 years without repetition.
  • All the above working nodes have instantaneous processing speed of 50W/0.1s.

At the end

This issue is to share here, I am xiaobian South wind blowing, focus on sharing interesting, novel, practical open source projects and developer tools, learning resources! I hope to learn and communicate with you together. Welcome to follow my official account ** [Github navigation station] **.