Persist objects in Redis key and value databases

This is the first day of my participation in Gwen Challenge.

Persist objects in Redis key and value databases

Abstract — This article introduces a method for persisting objects in Redis to provide a key-value database. Patterns focus on consistency of objects under conditions of competition and failure. The proposed pattern is useful for cloud storage implementations requiring speed and scalability. The recent “NoSQL movement” has brought a lot of attention to key-value databases or stores (starting now on KVS). The need for cloud Web applications to quickly request and use relational databases (from now on RDBMSS) can sometimes be a major bottleneck in architectures 3. However, this does not mean that RDBMSS should be avoided. The problem is that they are used as building blocks for each problem of the generic. Simplified KVS is fast and offers very little. This results in a significant shift from the data persistence layer to developer responsibilities. NoSQL is easy to understand as the anti-SQL, but it should be understood as “more than SQL.” KVS is not a suitable replacement for relational databases, but provides a better way to meet non-functional requirements, especially in cloud architectures. These databases or stores cannot offer ACID-style transactions. However, object persistence needs to be consistent. This can be achieved through good design patterns and component levels implemented in the architecture. In this article, the sample pattern shows how to store objects in KVS and to some extent compare them to RDBMS persistence. The rest of this article is organized as follows: First, some background on cloud application architecture and the key concepts of KVS are presented. Some comparisons of KVS for RDBMS have also been done. Then the introduction object gives persistence in KVS. After these main principles in terms of object persistence, the different KVS implementations are briefly compared and then the key concepts of Redis KVS are explained. A Redis – specific object implementation persistence is illustrated with a pseudo-code example. The implementation in the last section is evaluated and examples are shown with rough performance tests using ActiveRedis (Ruby on Rails Object Persistence) library.

Second, a RCHITECTURAL motivation for Web applications needs to process requests as quickly as possible. Most requests consist mainly of reads from the database. The Brewers CAP theorem says that in a distributed environment, such as the cloud, we have three requirements – comments: consistency, availability and fault tolerance of partitions, but we can only have two of these three. What this means is that the system needs to operate consistently with data, it must be available to users, and it must store data in multiple locations. To satisfy all three, we can choose two and work around the third requirement. Instead of writing our application to meet ACID requirements-ment (atomicity, consistency, persistence, and isolation), we write our architecture ((mostly available, soft state, and ultimately consistent) using the BASE method. This means that we do our best to keep the data available, but for example, if certain network links fail, we still provide services to the data that we can access. Changes to data are made to ensure final execution, for example by deferring writing operations to job queues. Due to the nature of the network, requests are independent and have some time between them, this approach can be very successful. Removing ACID at the schema level also gives us the opportunity to remove it at the database level as well. This is where the KVS option gets interesting. Typically most of the operations are read and extend Web application means to expand the read.

Relational database RDBMS must meet ACID requirement – data when modified. This makes it difficult to partition data across different physical locations because the data needs to be all written to OK before the location can be returned. And write operations often become bottlenecks because of disk IO. RDBMSS also have fixed schemas, which can lead to bloated table designs with a large number of unused columns, for example. The design of KVS is very simple. Can save a value with a unique key. This value can be loaded later with the same key. This is often called hash addressing and was described by R.Morris in 1968. KV does not actively support relational or full-text queries, but can better match certain situations. Examples of this are storing operational data, metadata, logging, message queuing, caching, and providing pre-computed data. One way to implement KVS is to store the key in memory and persist the data set to be archived asynchronously. When the KVS server is restarted, all data is loaded into memory from this file. RDBMSS have distinct advantages over KVS: they are well-tested, have good management tools, and are programmatically available patterns. On the other hand, key values are fairly new, each of them has a different approach and is only good at it in a very narrow segment sometimes the distinction between RDBMS and KVS is almost nonexistent. For example, by indexing primary keys when using MySQL, it can be performed very well and can be more easily implemented than partitioning if needed with current KVS implementations. There are also actions like sorting at the application level to make the data fit that faster. There is no single technology for scalable systems. Such systems must combine different technologies and architectural designs. Developers familiar with RDBMSS and the fact that as mature systems they are easier to understand, there may be enough reason not to consider the KVS option. On the other hand, scalability requirements may force consideration.

KVS objects that traditionally persist in RDBMSS are relational mappers (ORMs). Although KVS implementations differ, there are some common requirements for persistence. An object is an instance language of A class implemented in the host. Since objects and methods are structured in code, entities themselves are defined by their unique identifiers, and attribute values. Simple attribute types such as strings, integers, Booleans, and floating-point numbers are suitable for storing in KVS. A collection of objects can be defined as a unique identifier. From these identifiers we can get all the database attributes needed to create these objects. Can use the following key storage object attribute and value to the ClassName: Identifier: AttributeName ⇒ values. Example: Car with two properties as a single key Car: 34: Colour flicker green car, speed flicker 120 This method may have problems as described in the section below. B. Consistency of objects Concurrency of reads and writes makes the previous approach problematic because KVS does not provide the same type of transport operations we are used to in the RDBMS world. If this is not considered, it can lead to potential inconsistencies. For example, a concurrent update operation can modify the same object during a property read operation. When the object is returned to the application, it can have some old values and some new values. In addition, when an object is deleted, it is possible for execution to be terminated, such as a power failure and only some properties to be deleted. C. Adapter motivation between KVS requires object persistence of the adapter and application, even if the operation is trivial. Using adapters between applications and KVS helps avoid race conditions and other programming errors. It is also possible to implement its interface using the component KVS adapter designed for some ORMs. D. Not all instances of a schemaless data model have exactly the same properties as others. Some may have properties that exist only in a few of all objects. Schemaless data models are useful in the development phase, but also interesting in production. For example, administrators can add attributes dynamically for some instances, and applications can be designed to show that only those attributes exist in the instance. A schemaless approach does not necessarily equate to chaos supporting an architecture if it is designed. A simple example evaluation section is provided in. V. Key – Value store KVS has different implementations, most of which some are not fully mature. Cloud services have defined a new set of problems, and amazon, like Facebook, is tackling them in its own way with distributed implementations like Cassandra and Dynamo. These distributed large-scale KVS have proven to be effective for them all well and good, but for smaller (not very large) Web applications they can be too heavy. We’ve got things that grant index data like RDBMSS, provide query languages and perspectives that are currently being studied. Smaller KVS implementations include examples such as CouchDB, MongoDB, and Redis. They’re all different and they’re suited to different types of tasks. For example, MongoDB provides a query-like relational database that is essentially a document database. All three support partitioning and replication levels of data. Most of the lab persistence adapters are available for all implementations.

Redis does not support complex queries or indexes, but does support data structures as values. Quite simply, it is also many people’s fastest KVS implementation of basic operations. Interesting combinations of speed and data structures can be used for simple object persistence. Data structures in Redis are often referred to as Redis data types. These include strings, lists, sets, sorted sets, and hashes. Redis provides these types of commands that can be modified. For example, List supports common list operations like push and pop. The entire data set is kept in memory, so it can be no more than the amount of physical RAM. Redis server writes entire data sets to disk at configurable intervals. This can also be configured so that the write always returns OK before each modification to disk. Master/slave replication is also available and the cluster is currently under development. Atomic commands and Redis transactions Every Redis command is executed atomically. For pre-sufficient INCR commands increase the integer value of the key atomically. Two separate commands that are not atomic combine them with Redis transactions. Transactions in Redis are actually queued commands executed in atomic order. If one command raises an error, all other commands in the queue are still processed. Redis starts queuing commands MULTI after commands and performs those executions in an atomic transaction. A. Sort sets and hash sort sets are similar to RDBMS indexes. The sorting set contains member fractions sorted by special integer values. For example, “ZADD Foo:all 10 10” stores the value 10 key Foo:all scores 10. Hashes are keys and key-value pairs within a suit that can store object properties. For example “HMSETCat:2: Attributes Color Black Age 20” add two key values to pair with the key Cat:2: Attributes.

In the following implementation, each persistent object has its own unique integer identifier. A key Foo: ID: Attributes containing object attributes based on this identifier is created (assuming the object is named Foo). This key stores the property as a Redis hash. When the object is created, a special sort set Foo:all is updated to contain the identifier. And the “main” set can tell whether the object is persisted or not. Because writes are different atomic operations, the concurrency of writes can lead to object inconsistencies.