We have found a File based Cache with small memory and fast speed

A: background

1. Tell a story

18 years in the making of pure memory project encountered such a problem in the process, because some core data are floating in memory, so additional valuable memory space for us, but it happened in the project need to cache some data, such as need to drill down the points on the report, based on considerations of performance, do not want to use separate cache middleware, For example, redis and mongodb, after all, still need to use network IO, but it is not practical to put it directly in the native memory. Is there a solution to balance between native cache and cache server? Disk cache is used to read and write more DISK I/OS than network I/OS, let alone SSDS.

Two: Find solutions

1. Retrieve dead simple

With disk cache as a general direction, you can search github to see if there is a similar middleware. To be honest, there are quite a few Java middleware, such as the famous Guava, EhCache, not only has the simple operation of cache, but also provides a variety of statistics, which refresh the perception of cache. Ehcache is an in-heap, out of heap, disk, distributed cache, and so on. It is not practical to use C# to call Java.

2. Use SQLite as the disk cache

Since the open source community has nothing good to offer, it seems that it has to be wrapped up in its own package. It is not possible to use a simple SQLite as a native diskcahe, such as EhCache.

class DiskCache { private static readonly string dbFile = $@"{Environment.CurrentDirectory}\mysqlite1.db"; private static readonly string connectionString = $@"Data Source={dbFile}; Version=3"; Private static Timer = new Timer((arg) => {}, null, 1000, 1000 * 60); private static Timer = new Timer(arg) => {}, null, 1000, 1000 * 60); static DiskCache() { if (! File.Exists(dbFile)) { var schema = @"CREATE TABLE Cache ( cachekey VARCHAR (1000) PRIMARY KEY NOT NULL, cachevalue TEXT NOT NULL, created DATE NOT NULL, expried DATE NOT NULL );" ; using (SQLiteConnection connection = new SQLiteConnection(connectionString)) { connection.Execute(schema); } } } public static void Set<T>(string key, T value, int expiredMinutes) { using (SQLiteConnection connection = new SQLiteConnection(connectionString)) { var sql = $"delete from Cache where cachekey =@key;" + $"insert into Cache(cachekey,cachevalue,created,expried) values (@cachekey,@cachevalue,@created,@expried)"; connection.Execute(sql, new { key = key, cachekey = key, cachevalue = Newtonsoft.Json.JsonConvert.SerializeObject(value), created = DateTime.Now, expried = DateTime.Now.AddMinutes(expiredMinutes) }); } } public static T Get<T>(string key) { using (SQLiteConnection connection = new SQLiteConnection(connectionString)) { var sql = $"select cachevalue from Cache where cachekey=@cachekey and expried > @expried"; var query = connection.QueryFirstOrDefault(sql, new { cachekey = key, expried = DateTime.Now }); var json = JsonConvert.DeserializeObject<T>(query.cachevalue); return json; }}}Copy the code

Here are two caveats:

• Because it is to do cache, so the creation of the database and table through the program automation, whether the database exists to determine whether the file file can be.
• The problem with expired data, because I have expried fields, this point can learn the GC idea, using Timer to clean up periodically in the background.

With that in mind, now that the atomized cache is in place, try the basic Get/Set method.

This solution saves my precious memory, and the speed is a balance between Networkio and Native. It is a good solution.

Three: AspNetcore EasyCaching

EasyCaching @ Catcher is garden Wong works [www.cnblogs.com/catcher1994. Look under the provides a lot of kinds of the provider, the diagram below:

I’m sure there will be more providers to come, such as Leveldb and Cassandra. Let’s see how this works.

1. Installation and use

Can be installed on nuget search EasyCaching. SQLite, followed by using document: EasyCaching. Readthedocs. IO/en/latest/S… The diagram below:

In the document, dependency injection is used, and my program is a console mode back-end service, and there is no ServiceCollection. Try to simulate it first.

static void Main(string[] args) { IServiceCollection services = new ServiceCollection(); services.AddEasyCaching(option => { option.UseSQLite(c => { c.DBConfig = new SQLiteDBOptions { FileName = "demo.db", CacheMode = SqliteCacheMode.Default, OpenMode = SqliteOpenMode.ReadWriteCreate, }; }, "m1"); }); IServiceProvider serviceProvider = services.BuildServiceProvider(); var factory = serviceProvider.GetService<IEasyCachingProviderFactory>(); var cache = factory.GetCachingProvider("m1"); cache.Set("user", "hello world!" , TimeSpan.FromSeconds(20)); var info = cache.Get<string>("user"); Console.WriteLine(info); }Copy the code

Open demo.db with SQLiteStudio to see how the data is rendered:

As you can see, the framework has a name field that is used to isolate multiple caches, but there are three areas that need to be optimized.

• Not every program needs to use dependency injection, and it would be nice to provide a more convenient way to initialize.
• Looking at the source code, there is no business logic to periodically delete expired data.
• You are advised to provide cache statistics, such as the hit count and last hit time of a key.

Four:

There are more and more open source projects in recent years, and the community is good, which deserves praise.