Why plumecache?

When you use caching in your daily projects, whether you choose Redis, memcached, or any other type of cache, you need to do some wrapping to make it work. Plumecache is a summary of my personal experience with caches, aiming to solve some of the problems and pain points of using caches on a daily basis. I would like to share it with you and welcome your star and Issue Gitee addresses

Introduction to the

Plumecache is a common distributed cache encapsulation tool library, learning related client API to reduce costs, improve work efficiency, the current support reids (standalone, sentinel, cluster), memcached

What problem does Plumecache solve?

  • 1. Used to replace cache tool classes or glue codes such as CacheService, CacheHelper, CacheHandler, and CacheUtils
  • 2. Support multiple cache instance management, solve the problem of both want and want
  • 3. Configure the cache key prefix and cache version
  • 4. Support cache serialization unified processing, do not need to write serialization in each call place, support customization
  • 5. Support large cache compression and customization
  • 6. AOP extensions that support custom interceptors to handle parameters, return values, and exceptions

Quick to use

General access

  • 1. Maven (due to domain name problems, there is no upload central repository), you can download the source code to local or upload to the private server, or download jar first
<dependency>
    <groupId>org.plume</groupId>
    <artifactId>plumecache-core</artifactId>
    <version>1.0.0</version>
</dependency>
Copy the code
  • 2. The configuration
plumecache:
  instances:
    - type: redis
      endpoint: 127.0. 01.: 6379
      prefix: order
Copy the code
  • Use 3.
@Test
public void test(a){
        CacheService cacheService=CacheServiceFactory.getInstance();
        cacheService.set("name"."anson.yin");
        System.out.println(cacheService.get("name"));
}
Copy the code

Spring boot access

  • 1. Maven (due to domain name problems, there is no upload central repository), you can download the source code to local or upload to the private server, or download jar first

<dependency>
    <groupId>org.plume</groupId>
    <artifactId>plumecache-spring-boot-starter</artifactId>
    <version>1.0.0</version>
</dependency>
Copy the code
  • 2. The configuration
plumecache:
  instances:
    - type: redis
      endpoint: 127.0. 01.: 6379
      prefix: order
Copy the code
  • Use 3.
@Autowired
private CacheService cacheService;

@RequestMapping(value = "/hello")
public String hello(a){
        cacheService.set("name"."anson.yin");
        return"hello,".concat(cacheService.get("name"));
}
Copy the code

Functional specifications

configuration

plumecache:
  instances:
    - type: redis # required
      name: redis # Optional. The default value is the same as type
      endpoint: 127.0. 01.: 6379  # required
      prefix: order # optional, like "prefix@key"
      serializer: org.plumecache.samples.FastjsonCacheSerializer  Gson serialization is used by default
      compressor: org.plumecache.samples.NoneCacheCompressor  # Optional, specify the compression class. Gzip is used by default
      exclude-interceptors: SlowCacheInterceptor,LogCacheInterceptor # optional to exclude unwanted interceptors
Copy the code

Multiple instances

configuration

plumecache:
  instances:
    - type: redis # required
      name: redis # Optional. The default value is the same as type
      endpoint: 127.0. 01.: 6379  # required
      prefix: order # optional, like "prefix@key"
      serializer: org.plumecache.samples.FastjsonCacheSerializer  Gson serialization is used by default
      compressor: org.plumecache.samples.NoneCacheCompressor  # Optional, specify the compression class. Gzip is used by default
      exclude-interceptors: SlowCacheInterceptor,LogCacheInterceptor # optional to exclude unwanted interceptors
    - type: memcached
      endpoint: 127.0. 01.: 11211
    - type: memcached
      name: memcached2
      endpoint: 127.0. 01.: 11222
    - type: rediscluster
      endpoint: 127.0. 01.: 6379127.00 0.1:6389127.00 0.1:6399
Copy the code

instructions

  • 0. Four cache instances are configured above
  • 1. The default cache instance is the first configuration
  • 2. Multiple instances of the same type need to be distinguished by different names

Code sample

@Test
public void testMultipleInstances(a) {
        / / and CacheService redis = CacheServiceFactory. GetInstance (" redis ")
        CacheService redis = CacheServiceFactory.getInstance();
        redis.set("instance"."redis");
        System.out.println(redis.get("instance"));

        CacheService memcached = CacheServiceFactory.getInstance("memcached");
        memcached.set("instance"."memcached");
        System.out.println(memcached.get("instance"));
}
Copy the code

serialization

  • 1. Gson is used as serialization by default
  • 2. To customize serialization, implement the CacheSerializer interface and configure the serialization class

The following is an example implementation of FastjsonCacheSerializer

public class FastjsonCacheSerializer implements CacheSerializer {

    @Override
    public <T> String serialize(T value) {
        return JSON.toJSONString(value);
    }

    @Override
    public <T> T deserialize(String value, Class<T> clazz) {
        returnJSON.parseObject(value, clazz); }}Copy the code

Cache compressed

  • 1. Add the @CachecomPress annotation to the entity class
@Data
@CacheCompress
public class User {
    private String name;
    private Integer age;
    private String address;
}
Copy the code
  • 2. The invocation method remains unchanged
    @Test
    public void testCompress(a) {
        User user = new User();
        user.setAge(100);
        user.setName("zhangsanfeng");
        user.setAddress("zhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfen gzhangsanfeng");

        cacheService.set("user", user);
        User cacheUser = cacheService.get("user", User.class);
        
        System.out.println(cacheUser);
    }
Copy the code
127.0.0.1:6379 > get order @ user "\x04>]H4sIAAAAAAAAAKtWSkxJKUotLlayUqrKSMxLL07MS0vNSx8otpKOUmJ6qpKVoYGBjlJeYm4qmruU\nagHJDU9isgAAAA=="Copy the code
  • 3. Customize the compressor

Gzip compression is used by default, and you can customize the implementation. For details, see GzipCachecompressor.java

Cached version

  • 1. Add the @cacheVersion annotation to the entity class
@Data
@ CacheVersion (" 2.0 ")
public class User {
    private String name;
    private Integer age;
    private String address;
}
Copy the code
  • 2. The invocation method remains unchanged
    @Test
    public void testVersion(a) {
        User user = new User();
        user.setAge(100);
        user.setName("zhangsanfeng");
        user.setAddress("address");

        cacheService.set("user", user);
        User cacheUser = cacheService.get("user", User.class);

        System.out.println(cacheUser);
    }
Copy the code
127.0.0.1:6379 > get order @ [email protected] "\ x04 > 5 {\" name \ ": \" zhangsanfeng \ ", \ "age \" : 100, \ "address \" : \ "address \"}"Copy the code

The interceptor

Interceptors are facets of caching operations, and the custom interceptor operations are as follows

  • 1. Implement CacheInterceptor or inherit BaseCacheInterceptor, for example
@Slf4j
public class SlowCacheInterceptor extends BaseCacheInterceptor {

    @Override
    public boolean preHandle(CacheService target, Method method, Object[] args, Map<String, Object> context) {
        Instant begin = Instant.now();
        context.put("SlowCacheInstantBegin", begin);
        return true;
    }

    @Override
    public void postHandle(CacheService target, Method method, Object[] args, Map<String, Object> context, Object result) {
        Instant begin = (Instant) context.get("SlowCacheInstantBegin");

        if (Duration.between(begin, Instant.now()).toMillis() > 500) {
            log.warn("[SlowCacheInterceptor]slow cache, method:{},args:{},result:{},cost:{}"
                    , method.getDeclaringClass().getName() + "." + method.getName()
                    , JSON.toJSONString(args)
                    , JSON.toJSONString(result)
                    , Duration.between(begin, Instant.now()).toMillis());

            //do something others}}}Copy the code
  • 2. Add the SPI configuration

Path: the resources/services/com. Plumecache. Core. The interceptor. CacheInterceptor

org.plumecache.samples.SlowCacheInterceptor
Copy the code

Or refer to the Core package implementation

  • LogCacheInterceptor.java
  • VersionCacheInterceptor.java
  • PrefixCacheInterceptor.java

Management interface

This takes effect when Spring Boot Stater is a Web project

  • 1. List Displays the instance list
curl -X GET localhost:8080/plumecache/list
Copy the code
[{"properties":{"name":"memcached","type":"memcached","endpoint":"127.0.0.1:11211","prefix":null,"serializer":null,"compressor":null,"excludeInterceptors":null},"statistics":{"cmd_touch":"0","moves_to_cold":"5","incr_hits":"1","get_flushed":"0","evictions":"0","touch_hits":"0","expired_unfetched":"0","pid":"1","time_in_listen_disabled_us":"0","response_obj_bytes":"65536","cas_badval":"0","cmd_flush":"0","total_items":"6","read_buf_oom":"0","round_robin_fallback":"0","slab_reassign_rescues":"0","log_watcher_skipped":"0","cas_hits":"0","accepting_conns":"1","auth_errors":"0","slab_reassign_evictions_nomem":"0","log_watcher_sent":"0","reserved_fds":"20","slab_reassign_running":"0","response_obj_oom":"0","crawler_items_checked":"12","direct_reclaims":"0","conn_yields":"0","slab_reassign_busy_deletes":"0","version":"1.6.10","read_buf_count":"8","listen_disabled_num":"0","slab_global_page_pool":"0","get_misses":"0","hash_is_expanding":"0","touch_misses":"0","get_expired":"0","auth_cmds":"0","cas_misses":"0","delete_misses":"0","cmd_meta":"0","get_hits":"4","slab_reassign_inline_reclaim":"0","malloc_fails":"0","delete_hits":"0","log_worker_written":"0","read_buf_bytes_free":"49152","lru_bumps_dropped":"0","curr_connections":"2","bytes_written":"4650","slab_reassign_busy_items":"0","hash_bytes":"524288","libevent":"2.1.8-stable","read_buf_bytes":"131072","lrutail_reflocked":"0","crawler_reclaimed":"0","decr_hits":"0","limit_maxbytes":"67108864","max_connections":"1024","decr_misses":"0","lru_crawler_running":"0","reclaimed":"0","rejected_connections":"0","cmd_get":"4","hash_power_level":"16","curr_items":"3","threads":"4","cmd_set":"5","bytes_read":"360","slab_reassign_chunk_rescues":"0","lru_crawler_starts":"27","uptime":"22253","log_worker_dropped":"0","unexpected_napi_ids":"0","total_connections":"9","evicted_active":"0","incr_misses":"1","connection_structures":"4","bytes":"221","lru_maintainer_juggles":"30247","evicted_unfetched":"0","rusage_system":"12.600134","time":"1638171110","slabs_moved":"0","moves_within_lru":"0","pointer_size":"64","moves_to_warm":"0","rusage_user":"8.090528","response_obj_count":"1"}},{"properties":{"name":"redis","type":"redis","endpoint":"127.0.0.1:6379","prefix":"order","serializer":"org.plumecache.samples.FastjsonCacheSerializer","compressor":"com.plumecache.core.compressor.NoneCacheCompressor","excludeInterceptors":["SlowCacheInterceptor","LogCacheInterceptor"]},"statistics":{"io_threaded_reads_processed":"0","tracking_clients":"0","uptime_in_seconds":"22224","cluster_connections":"0","current_cow_size":"0","maxmemory_human":"0B","aof_last_cow_size":"0","master_replid2":"0000000000000000000000000000000000000000","mem_replication_backlog":"0","aof_rewrite_scheduled":"0","total_net_input_bytes":"19507","rss_overhead_ratio":"1.58","hz":"10","current_cow_size_age":"0","redis_build_id":"69ab6eec4665acbc","aof_last_bgrewrite_status":"ok","multiplexing_api":"epoll","client_recent_max_output_buffer":"0","allocator_resident":"5181440","mem_fragmentation_bytes":"6549960","repl_backlog_first_byte_offset":"0","tracking_total_prefixes":"0","redis_mode":"standalone","cmdstat_get":"calls=10,usec=342,usec_per_call=34.20,rejected_calls=0,failed_calls=0","redis_git_dirty":"0","allocator_rss_bytes":"3031040","repl_backlog_histlen":"0","io_threads_active":"0","rss_overhead_bytes":"2990080","total_system_memory":"2082197504","loading":"0","evicted_keys":"0","maxclients":"10000","cmdstat_set":"calls=10,usec=1941,usec_per_call=194.10,rejected_calls=0,failed_calls=0","cluster_enabled":"0","redis_version":"6.2.5","repl_backlog_active":"0","mem_aof_buffer":"0","allocator_frag_bytes":"447160","io_threaded_writes_processed":"0","instantaneous_ops_per_sec":"0","used_memory_human":"1.59M","cmdstat_incr":"calls=1,usec=68,usec_per_call=68.00,rejected_calls=0,failed_calls=0","total_error_replies":"0","role":"master","maxmemory":"0","used_memory_lua":"37888","rdb_current_bgsave_time_sec":"-1","used_memory_startup":"809880","used_cpu_sys_main_thread":"146.683765","lazyfree_pending_objects":"0","used_memory_dataset_perc":"38.70%","allocator_frag_ratio":"1.26","arch_bits":"64","used_cpu_user_main_thread":"38.482678","mem_clients_normal":"512400","expired_time_cap_reached_count":"0","unexpected_error_replies":"0","mem_fragmentation_ratio":"5.04","aof_last_rewrite_time_sec":"-1","master_replid":"9c1aa9fc501b889e35727910956569cac1a2fda1","aof_rewrite_in_progress":"0","lru_clock":"10781132","maxmemory_policy":"noeviction","run_id":"d05aa7e4384c6ccd09ee83e919c9e9edcd2d8450","latest_fork_usec":"439","tracking_total_items":"0","total_commands_processed":"1349","expired_keys":"0","used_memory":"1664360","module_fork_in_progress":"0","dump_payload_sanitizations":"0","mem_clients_slaves":"0","keyspace_misses":"3","server_time_usec":"1638171084987972","executable":"/data/redis-server","lazyfreed_objects":"0","db0":"keys=232,expires=0,avg_ttl=0","used_memory_peak_human":"4.53M","keyspace_hits":"7","rdb_last_cow_size":"520192","used_memory_overhead":"1333640","active_defrag_hits":"0","tcp_port":"6379","uptime_in_days":"0","used_memory_peak_perc":"35.03%","current_save_keys_processed":"0","blocked_clients":"0","total_reads_processed":"1900","expire_cycle_cpu_milliseconds":"7066","sync_partial_err":"0","used_memory_scripts_human":"0B","aof_current_rewrite_time_sec":"-1","aof_enabled":"0","process_supervised":"no","cmdstat_info":"calls=2,usec=559,usec_per_call=279.50,rejected_calls=0,failed_calls=0","master_repl_offset":"0","used_memory_dataset":"330720","used_cpu_user":"38.497928","rdb_last_bgsave_status":"ok","tracking_total_keys":"0","cmdstat_ping":"calls=1325,usec=19580,usec_per_call=14.78,rejected_calls=0,failed_calls=0","atomicvar_api":"c11-builtin","allocator_rss_ratio":"2.41","client_recent_max_input_buffer":"16","clients_in_timeout_table":"0","aof_last_write_status":"ok","mem_allocator":"jemalloc-5.1.0","cmdstat_incrby":"calls=1,usec=21,usec_per_call=21.00,rejected_calls=0,failed_calls=0","used_memory_scripts":"0","used_memory_peak":"4751720","process_id":"1","master_failover_state":"no-failover","used_cpu_sys":"146.736539","repl_backlog_size":"1048576","connected_slaves":"0","current_save_keys_total":"0","gcc_version":"8.3.0","total_system_memory_human":"1.94G","sync_full":"0","connected_clients":"25","module_fork_last_cow_size":"0","total_writes_processed":"1349","allocator_active":"2150400","total_net_output_bytes":"18709","pubsub_channels":"0","current_fork_perc":"0.00","active_defrag_key_hits":"0","rdb_changes_since_last_save":"0","instantaneous_input_kbps":"0.00","configured_hz":"10","used_memory_rss_human":"7.79M","expired_stale_perc":"0.00","active_defrag_misses":"0","used_cpu_sys_children":"0.029570","number_of_cached_scripts":"0","sync_partial_ok":"0","used_memory_lua_human":"37.00K","rdb_last_save_time":"1638169845","pubsub_patterns":"0","slave_expires_tracked_keys":"0","redis_git_sha1":"00000000","used_memory_rss":"8171520","rdb_last_bgsave_time_sec":"0","os":"Linux 5.10.47-linuxkit x86_64","mem_not_counted_for_evict":"0","active_defrag_running":"0","rejected_connections":"0","total_forks":"3","active_defrag_key_misses":"0","allocator_allocated":"1703240","instantaneous_output_kbps":"0.00","second_repl_offset":"-1","rdb_bgsave_in_progress":"0","used_cpu_user_children":"0.012939","total_connections_received":"575","migrate_cached_sockets":"0"}}]
Copy the code
  • 2. Execute The cache operation
curl localhost:8080/plumecache/execute \
-X POST \
-H "Content-Type: application/json" \
-d '{"command":"get","key":"name"}'
Copy the code
anson.yin
Copy the code

Interface specification

Refer to the CacheService. Java

RoadMap

Version 1.0.0

  • 1. Support spring Boot project access, complete initialization –done based on Spring Boot Starter and Enable as well as conditional annotations
  • 2. Support the intervention of Spring project to provide the initial chemical tool class –done
  • 3. Supports the key prefix done
  • 4. Multiple cache instances are supported: done
  • 5. Supports the get, set, delete, exists, expire, and TTL commands –done
  • 6. Support Dashboard API (instance list, instance parameters, command execution)
  • 7. Support custom interceptor –done
  • 8. Support custom serialization –done
  • 9. Return the done object (RedissonClient, MemcachedClient)
  • 10. Implement hget,hset,hgetall,lock –done
  • 11. Support cache compression (large value compression) –done
  • 12. Support versioning –done

Version 2.0.0

  • 0. Cache configuration parameters (timeout, poolsize idlecount, attempts)
  • 1. Support for in-thread caching (threadLocal based)
  • 2. Support field serialization (based on ObjectOutputStream)

The project structure