• Top 5 Embedded Databases for JavaScript Applications
• By Fernando Doglio
• Translation from: The Gold Project
• This article is permalink: github.com/xitu/gold-m…
• Translator: No problem
• Proofreader: FinalWhy PassionPenguin

Top 5 embedded databases for JavaScript applications

We’re used to thinking of databases as one big storage platform, where we throw in all the data we need and retrieve it using some query language. Scaling these databases to keep the information consistent and fault-tolerant is a challenge in itself. So what happens when we need very little data?

What happens when RedShift, BigQuery, or even MySQL becomes too heavy for our minimal data storage needs? There’s actually another category of applications that might be appropriate. In fact, there are many options, so in this article I’ll cover the top five embedded databases for small data storage needs.

So what exactly is an embedded database?

When you read the word “embedded,” 90% of people think I’m talking about IOT or mobile devices, but I’m not.

At least that’s not the only case. Admittedly, these systems have very limited resources, making it difficult to configure and install most traditional database systems on them.

But there are other use cases for small databases that are embedded as part of a software product. For example, imagine searching through an IDE in a large code repository. IDE can embed an inverted index database that allows you to search by keyword and get quick references to relevant files. Or do a search on your favorite mail desktop client, which most likely also has an embedded database. All mail is stored and indexed, so you can get the information you want quickly and easily.

Another huge benefit of using an embedded database is that it does not require network communication to interact, which is a huge performance improvement over databases of various standards. In general development, you might put your database on your own server (or server cluster) so that its resource consumption doesn’t affect other components of your architecture, whereas with embedded databases, you might want them as close to the client as possible. This reduces communication delays between them and avoids dependence on communication channels (i.e., networks).

Embedded databases now come in many forms. From fast in-memory databases that use JSON files as their primary storage, to small, efficient relational databases that can be queried using an SQL-like language,

Let’s look at these five options first.

LowDB

Let’s start simple. LowDB is a small in-memory database that is a very basic solution, but addresses a very simple use case: the need to store and use JSON-like structures (i.e., documents) in a javascrt-based project.

One of the main benefits of LowDB is that it is designed to be used in JavaScript, which means it can be used on the back end, desktop, or anywhere else where you can run browser code.

You can use LowDB in Node.js, or it can be integrated into the Electron project to develop desktop clients, or it can run directly in browsers with js runtime integration.

The API provided by LowDB is also very simple and lightweight, and does not provide any search functionality out of the box. It is limited to loading the data from the JSON file into an array variable and leaving it up to you (the user) to find the data you want.

Consider the following example:

import { LowSync, JSONFileSync } from 'lowdb'

const title = "This is a test"
const adapter = new JSONFileSync('file.json')
const db = new LowSync(adapter)

db.read()  // Read the contents of the JSON file into memory
db.data ||= { posts: []}/ / the default value

db.data.posts.push({ title }) // Add data

db.write() // Save the data to a JSON file

// Anything like a lookup is up to the user

let record = db.data.posts.find( p= > p.title == "Hello world")

if(! record) {console.log("No data found!")}else {
  console.log("== Record found ==")
  console.log(record)
}
Copy the code

As you can see, the interesting thing is not the default behavior but that I’m using an interface called JSONFileSync. I can easily create an instance and use it, and that’s the real strength of this database,

LowDB is highly extensible and compatible with TypeScript, which provides schema-like behavior for data storage (that is, it does not allow you to add data that does not conform to the preset schema).

If you mix these two options, LowDB provides some interesting Promise-based apis for manipulating native JSON-like data.

LevelDB

LevelDB is an open source key-value pair database created by Google. It is a super-fast but very limited key-value storage scheme in which data is stored by key-sort, out of the box.

It has only three basic operations: Put, Get, and Delete – a very lightweight API, similar to LowDB.

More like LowDB, LevelDB is wrapped as no client-server, which means that no language can communicate with it. To use it, you must use the C/C++ library, and if you want server-side behavior, you need to wrap it yourself.

As with most of the cases covered in this article, the LevelDB functionality is very basic because it covers only one very simple but practical requirement: a data store close to the code, and fast access.

The database’s storage schema is centered around log-structured Merge Trees (LSM), which means that it is best used for large continuities rather than small, random write operations.

One of the major limitations of LevelDB is that once started, it gets a system-level lock, which means that only one process can interact with it at a time. Of course, you can use multiple threads within a process to parallelize certain operations. But that’s about it.

Interestingly, LevelDB is used by Chrome’s IndexedDB as a back-end database, and apparently Minecraft Bedrock uses it as some block and entity data stores (although they use slightly modified versions implemented by Google).

Raima Database Manager

I mentioned the Internet of Things, didn’t I? Raima is one of the fastest database managers and is optimized for running on IoT devices with limited system resources.

What do I mean by resource-constrained environments? Raima only needs 350KB of RAM to run, so I can minimize resource usage.

The main difference between this scenario and the previous ones is that it fully supports SQL. It provides a relational data model and allows queries in the SQL language.

Unlike LevelDB, it allows multi-process access to the database via client-server architecture (that is, it allows your database to be slightly further away from the source code than other databases). If you decide to make an embedded application close to the source code, you can also use multithreading to access multiple databases concurrently.

Raima’s flexibility allows you to move from the traditional client-server pattern to the most efficient (and, of course, limited) use case for a single client using a single in-memory database. But isn’t that a very effective use case for an embedded database? .

Its flexibility makes it a very versatile solution. Of course, each deployment pattern has its own benefits and limitations, but there are specific use cases that are best suited. So make sure you choose what works best for you, and then get the most out of it.

Apache Derby

If you’re looking for another small, SQL-like database, Apache Derby is probably the one for you.

Derby, which is written entirely in JAVA, claims to use only 3.5MB of memory, which may be a little unrealistic. After all, you can’t run or use the JVM without it installed on the main system.

However, if JVM is part of your use case, that’s great, and you can continue to consider using Derby, otherwise you might need more native solutions like LevelDb or Raima.

But as I said, if you’re already working on a JAVA project and need to integrate a small, reliable, SQL-based database, Derby is definitely a potential candidate.

It integrates with the JDBC driver, so no additional dependencies are required. It can work in an embedded mode in your JAVA application or as a separate server that interacts with multiple applications simultaneously (similar to Raima, but without as many variables).

Honestly, the biggest weakness of this project is its documentation. This may be a standard in the JAVA community, but it’s not user-friendly, and most of the official links point to a private merge page. Other solutions, most of which have a smooth documentation experience, help users get used to the product.

solidDB

Last, but definitely not least, solidDB covers a very interesting case of improving model persistence while providing an in-memory relational database. It claims to keep its two data-storage options in sync in real time.

Essentially like the previous ones, solidDB can be used through ODBC or JDBC, allowing JAVA or C applications to interact with it through SQL.

As with the previous ones, you can deploy it in several ways:

• Highly available mode. This involves multiple servers with duplicate data. Of course, this pattern doesn’t quite fit the use case we’re considering.
• Shared memory access. This is interesting because it not only stores the data in memory (such as the previous ones) but also allows multiple applications to use the memory (the shared memory portion). Of course, only applications within the same node can use the shared memory directly, but JDBC/ ODBC-based applications from external nodes can also access the same data. Convert shared memory to an in-memory database with external access.

Several large companies, such as Cisco, Alcatel, Nokia, or Siemens, claim to use the database for mission-critical operations because of its lightning-fast data access.

Given all its deployment options, extensive documentation, and highly demanding customer list, I think it is one of the most reliable, stable, and fast embedded databases in this article.

Embedded databases are designed to handle very specific usage scenarios, either by providing reliable and fast data storage with minimal latency, or by providing fast and secure data access. The solutions listed here approach these goals in different ways, which one is right for you depends on you and your environment.

Have you ever tried any of these solutions? Has there ever been a need for an embedded database in your project? Which one did you choose?

If you find any errors in the translation or other areas that need to be improved, you are welcome to revise and PR the translation in the Gold Translation program, and you can also get corresponding bonus points. The permanent link to this article at the beginning of this article is the MarkDown link to this article on GitHub.

Diggings translation project is a community for translating quality Internet technical articles from diggings English sharing articles. The content covers the fields of Android, iOS, front end, back end, blockchain, products, design, artificial intelligence and so on. For more high-quality translations, please keep paying attention to The Translation Project, official weibo and zhihu column.