RxCache
RxCache is a Local Cache that supports Java and Android. Currently, it supports heap memory, off-heap memory, and disk cache.
Github address: github.com/fengzhizi71…
Off-heap memory
Objects can be stored in heap memory, off-heap memory, disk cache, or even distributed cache.
In Java, the opposite of out-of-heap memory is heap memory. Heap memory follows the JVM’s memory management mechanism, while off-heap memory is not subject to this restriction and is managed by the operating system.
There is a distinct difference, or opposite, between off-heap memory and heap memory.
Off-heap memory is more suitable for:
- Store objects with long life cycles
- It can be shared between processes, reducing object duplication between JVMS and making split deployment of JVMS easier.
- Local caching, reducing the response time of disk caching or distributed caching.
The off-heap memory used in RxCache
First, create a DirectBufferConverter that converts objects to and from ByteBuffers and objects to and from byte arrays. Bytebuffer.alloctedirect (capability) is used to allocate out-of-heap memory. Cleaner is a self-defined class for releasing DirectByteBuffer. The specific code can be viewed: github.com/fengzhizi71…
public abstract class DirectBufferConverter<V> {
public void dispose(ByteBuffer direct) {
Cleaner.clean(direct);
}
public ByteBuffer to(V from) {
if(from == null) return null;
byte[] bytes = toBytes(from);
ByteBuffer.wrap(bytes);
ByteBuffer bf = ByteBuffer.allocateDirect(bytes.length);
bf.put(bytes);
bf.flip();
return bf;
}
abstract public byte[] toBytes(V value);
abstract public V toObject(byte[] value);
public V from(ByteBuffer to) {
if(to == null) return null;
byte[] bs = new byte[to.capacity()];
to.get(bs);
to.flip();
returntoObject(bs); }}Copy the codeNext, define a ConcurrentDirectHashMap<K, V> to implement the Map interface. It is a generic that supports converting V to ByteBuffer, stored in ConcurrentDirectHashMap’s map.
public abstract class ConcurrentDirectHashMap<K.V> implements Map<K.V> {
final private Map<K, ByteBuffer> map;
private final DirectBufferConverter<V> converter = new DirectBufferConverter<V>() {
@Override
public byte[] toBytes(V value) {
return convertObjectToBytes(value);
}
@Override
public V toObject(byte[] value) {
returnconvertBytesToObject(value); }}; ConcurrentDirectHashMap() { map =new ConcurrentHashMap<>();
}
ConcurrentDirectHashMap(Map<K, V> m) {
map = new ConcurrentHashMap<>();
for(Entry<K, V> entry : m.entrySet()) { K key = entry.getKey(); ByteBuffer val = converter.to(entry.getValue()); map.put(key, val); }}protected abstract byte[] convertObjectToBytes(V value);
protected abstract V convertBytesToObject(byte[] value);
@Override
public int size(a) {
return map.size();
}
@Override
public boolean isEmpty(a) {
return map.isEmpty();
}
@Override
public boolean containsKey(Object key) {
return map.containsKey(key);
}
@Override
public V get(Object key) {
final ByteBuffer byteBuffer = map.get(key);
return converter.from(byteBuffer);
}
@Override
public V put(K key, V value) {
final ByteBuffer byteBuffer = map.put(key, converter.to(value));
converter.dispose(byteBuffer);
return converter.from(byteBuffer);
}
@Override
public V remove(Object key) {
final ByteBuffer byteBuffer = map.remove(key);
final V value = converter.from(byteBuffer);
converter.dispose(byteBuffer);
return value;
}
@Override
public void putAll(Map<? extends K, ? extends V> m) {
for(Entry<? extends K, ? extends V> entry : m.entrySet()) { ByteBuffer byteBuffer = converter.to(entry.getValue()); map.put(entry.getKey(), byteBuffer); }}@Override
public void clear(a) {
final Set<K> keys = map.keySet();
for(K key : keys) { map.remove(key); }}@Override
public Set<K> keySet(a) {
return map.keySet();
}
@Override
public Collection<V> values(a) {
Collection<V> values = new ArrayList<>();
for (ByteBuffer byteBuffer : map.values())
{
V value = converter.from(byteBuffer);
values.add(value);
}
return values;
}
@Override
public Set<Entry<K, V>> entrySet() {
Set<Entry<K, V>> entries = new HashSet<>();
for (Entry<K, ByteBuffer> entry : map.entrySet()) {
K key = entry.getKey();
V value = converter.from(entry.getValue());
entries.add(new Entry<K, V>() {
@Override
public K getKey(a) {
return key;
}
@Override
public V getValue(a) {
return value;
}
@Override
public V setValue(V v) {
return null; }}); }return entries;
}
@Override
public boolean containsValue(Object value) {
for (ByteBuffer v : map.values()) {
if (v.equals(value)) {
return true; }}return false; }}Copy the codeCreate ConcurrentStringObjectDirectHashMap, it is the K type String, V is arbitrary Object. Among them, serialization and deserialization use byteKit mentioned in Common Java Byte Encapsulation.
public class ConcurrentStringObjectDirectHashMap extends ConcurrentDirectHashMap<String.Object> {
@Override
protected byte[] convertObjectToBytes(Object value) {
return Bytes.serialize(value);
}
@Override
protected Object convertBytesToObject(byte[] value) {
returnBytes.deserialize(value); }}Copy the codeMemory level cache is realized based on FIFO and off-heap Memory.
public class DirectBufferMemoryImpl extends AbstractMemoryImpl {
private ConcurrentStringObjectDirectHashMap cache;
private List<String> keys;
public DirectBufferMemoryImpl(long maxSize) {
super(maxSize);
cache = new ConcurrentStringObjectDirectHashMap();
this.keys = new LinkedList<>();
}
@Override
public <T> Record<T> getIfPresent(String key) {
T result = null;
if(expireTimeMap.get(key)! =null) {
if (expireTimeMap.get(key)<0) { // Cached data never expires
result = (T) cache.get(key);
} else {
if (timestampMap.get(key) + expireTimeMap.get(key) > System.currentTimeMillis()) { // The cached data has not expired
result = (T) cache.get(key);
} else { // The cached data has expiredevict(key); }}}returnresult ! =null ? new Record<>(Source.MEMORY,key, result, timestampMap.get(key),expireTimeMap.get(key)) : null;
}
@Override
public <T> void put(String key, T value) {
put(key,value, Constant.NEVER_EXPIRE);
}
@Override
public <T> void put(String key, T value, long expireTime) {
if (keySet().size()<maxSize) { // There is space in the cache
saveValue(key,value,expireTime);
} else { // The cache space is insufficient, and one needs to be deleted
if (containsKey(key)) {
keys.remove(key);
saveValue(key,value,expireTime);
} else {
String oldKey = keys.get(0); // The earliest cached key
evict(oldKey); // Remove the first cached data FIFO algorithmsaveValue(key,value,expireTime); }}}private <T> void saveValue(String key, T value, long expireTime) {
cache.put(key,value);
timestampMap.put(key,System.currentTimeMillis());
expireTimeMap.put(key,expireTime);
keys.add(key);
}
@Override
public Set<String> keySet(a) {
return cache.keySet();
}
@Override
public boolean containsKey(String key) {
return cache.containsKey(key);
}
@Override
public void evict(String key) {
cache.remove(key);
timestampMap.remove(key);
expireTimeMap.remove(key);
keys.remove(key);
}
@Override
public void evictAll(a) { cache.clear(); timestampMap.clear(); expireTimeMap.clear(); keys.clear(); }}Copy the codeAt this point, you have completed the encapsulation that exists in and out of the heap in the RxCache. In fact, there are many caching frameworks that support off-heap memory, such as Ehcache and MapDB. RxCache currently has MapDB modules.
conclusion
RxCache is a Local Cache that has been used in our project and in my own crawler framework, NetDiscovery. In the future, it will continue to be used as a mature component in other corporate and personal projects.
RxCache series
- ReentrantReadWriteLock Read/write lock and its use in RxCache
- Retrofit style RxCache and its various cache replacement algorithms
- RxCache integrates the Persistence layer frameworks of greenDAO and Room of Android
- Build a simple responsive Local Cache for Java and Android
Java and Android technology stack: update and push original technical articles every week, welcome to scan the qr code of the public account below and pay attention to, looking forward to growing and progress with you together.
