This is the 17th day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021
A distributed ids
UUID
General-purpose unique identifiers, long in length, with very few repetitions and errors, are irregular and not suitable for indexing.
UUID.randomUUID();
Copy the codeThe database
A separate database creates a table whose ID is set to auto-increment, which emulates auto-increment when an ID is needed. To get it, use last_insert_id(). More than one database, but also need to connect, performance and stability problems
Snowflakes algorithm
The generated ID is of type Long, with 1 bit for symbol bit, 41 bits for timestamp, 10 bits for machine ID, and 12 bits for serial number.
public class SnowflakeSequence implements Sequence {
/** * Start time (2021-11-17 19:44:8) */
private final long twepoch = 1637149448L;
/** * The number of bits occupied by the machine ID */
private final long workerIdBits = 5L;
/** * The number of digits in the data id */
private final long datacenterIdBits = 5L;
/** * The maximum machine ID supported, resulting in 31 (this shift algorithm can quickly calculate the largest decimal number represented by several binary digits) */
private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
/** * The maximum data id supported. The result is 31 */
private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
/** * The number of digits in the sequence */
private final long sequenceBits = 12L;
/** * Move the machine ID 12 bits to the left */
private final long workerIdShift = sequenceBits;
/** * Data id is moved 17 bits to the left (12+5) */
private final long datacenterIdShift = sequenceBits + workerIdBits;
/** ** The time warp moves 22 bits to the left (5+5+12) */
private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
/** * Generate the mask of the sequence, where 4095 (0b111111111111= 0xFFf =4095) */
private final long sequenceMask = -1L ^ (-1L << sequenceBits);
/** * Working machine ID(0~31) */
private long workerId;
/**
* 数据中心ID(0~31)
*/
private long datacenterId;
/** * milliseconds sequence (0~4095) */
private long sequence = 0L;
/** * The last time the ID was generated */
private long lastTimestamp = -1L;
@Override
public synchronized long nextValue(a) throws SeqException {
long timestamp = timeGen();
// If the current time is less than the last timestamp generated by the ID, an exception should be thrown when the system clock is rolled back
if (timestamp < lastTimestamp) {
throw new SeqException("[Snowflakesequence-nextValue] The current time is less than the time when the serial number was generated last time. The time is rolled back. Please confirm the setting of the server time.");
}
// If they are generated at the same time, the sequence is performed in milliseconds
if (lastTimestamp == timestamp) {
sequence = (sequence + 1) & sequenceMask;
// Sequence overflow in milliseconds
if (sequence == 0) {
// block until the next millisecond to get a new timestamptimestamp = tilNextMillis(lastTimestamp); }}else {
// The timestamp changes and the sequence is reset in milliseconds
sequence = 0L;
}
// The last time the ID was generated
lastTimestamp = timestamp;
// Shift and put together by or to form a 64-bit ID
return ((timestamp - twepoch) << timestampLeftShift) | (datacenterId << datacenterIdShift)
| (workerId << workerIdShift) | sequence;
}
/** * blocks to the next millisecond until a new timestamp * is obtained@paramLastTimestamp Specifies the time when the ID was last generated *@returnCurrent timestamp */
private long tilNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
/** * returns the current time in milliseconds *@returnCurrent time (ms) */
private long timeGen(a) {
return System.currentTimeMillis();
}
public void setWorkerId(long workerId) {
if (workerId > maxWorkerId) {
throw new SeqException("[snowflakesequence-setWorkerId] The workerId cannot be greater than 31.");
}
this.workerId = workerId;
}
public void setDatacenterId(long datacenterId) {
if (datacenterId > maxDatacenterId) {
throw new SeqException("[Snowflakesequence-setdatacenterId] datacenterId cannot be greater than 31.");
}
this.datacenterId = datacenterId;
}
/** * The next generation number (with format) *@return
* @throws SeqException
*/
@Override
public String nextNo(a) throws SeqException {
returnString.valueOf(nextValue()); }}Copy the codeRedis
Run the incr command
Distributed scheduling -Elastic-job
The difference between scheduled tasks and message queues
In common
- Asynchronous processing
- The application of decoupling
- Traffic peak clipping
The difference between
Timed tasks are time driven and tend to batch, whereas MQ is event driven and tend to process item by item.
Elastic-Job
Based on quartz secondary development, it consists of elastic-Job-Lite and Elastic-Job-Cloud. Positioning is a lightweight decentralized solution. Zk-based storage and notifications.
public class Job implements SimpleJob{
public void execute(ShardingContext context){
// Scheduled task services}}public static void main(String[] args){
ZookeeperConfiguration zk = new ZookeeperConfiguration("localhost:2181"."namespace");
CoordinatorRegistryCenter cr = new CoordinatorRegistryCenter(zk);
cr.init();
JobCoreConfiguration conf = JobCoreConfiguration.newBuilder("name"."*/2 * * * *?".1).build();
SimpleJobConfiguration simple = new SimpleJobConfiguration(conf, Job.class.getName());
JobSchedule jobSchedule = new JobScheduler(cr, LiteJobConfiguration.newBuilder(simple).build());
jobSchedule.init();
}
Copy the codeshard
public class Job implements SimpleJob{
public void execute(ShardingContext context){
intitem = shardingContext.getShardingItem(); String param = shardingContext.getShardingParameter(); }}public static void main(String[] args){
ZookeeperConfiguration zk = new ZookeeperConfiguration("localhost:2181"."namespace");
CoordinatorRegistryCenter cr = new CoordinatorRegistryCenter(zk);
cr.init();
JobCoreConfiguration conf = JobCoreConfiguration.newBuilder("name"."*/2 * * * *?".3).shardingItemParameters("0=red,1=blue").build();
SimpleJobConfiguration simple = new SimpleJobConfiguration(conf, Job.class.getName());
JobSchedule jobSchedule = new JobScheduler(cr, LiteJobConfiguration.newBuilder(simple).build());
jobSchedule.init();
}
Copy the codeMainly lies in the code JobCoreConfiguration. NewBuilder (” name “, “2 * * * * * /?” , 3). ShardingItemParameters (” 0 = red, 1 = blue “). The build (). Call at job String param = shardingContext. GetShardingParameter (); . Every time a elastice-job project is started, sharding is calculated to ZK by default.
Sesson Shared
Solution to session consistency
-
Nginx IP_HASH policy
The session is sticky. The advantage is that the configuration is simple and the application is not invaded. The disadvantages are that session loss, single point of failure, and single point of load may be too high.
-
Session replication
Session replication by modifying configuration files between multiple Tomcats. this was an early solution and is no longer recommended. The advantage is that the server does not invade applications, can be horizontally expanded, can adapt to various load balancing, restart or downtime will not cause session loss. Disadvantages are low performance, high memory consumption, can not store too much data, high latency.
-
Session Shared, centralized storage
Session storage in Redis has the advantage of being able to adapt to various load balancing, and will not cause session loss due to restart or downtime. It has strong expansion capability and is suitable for large clusters, but has the disadvantage of invading applications.