Node JS various types of database common method encapsulation, heart process,

This is the process I went through to write the Access-DB method library, which I’ll document here

Birth of an idea

The development of small companies is not easy, cost, is to save as much as possible. It is not surprising that sometimes the work of more than one must be done by one. It was also in this situation that I gradually became full stack development.

At first, in order to save money, plus we only do small programs, we chose a third party cloud service provider as the background. At first, it was ok, but later, I found that the interface methods they provided were not as easy to use as I had imagined. Especially in the case of complex searches, the code is too complex to read and easy to modify. So I started thinking about how to simplify his search. FetchFindByCheck () ¶ fetchFindByCheck () ¶ fetchFindByCheck () ¶ fetchFindByCheck () ¶ fetchFindByCheck () ¶ fetchFindByCheck () ¶ fetchFindByCheck () ¶

fetchFindByCheck({
  tableName: app.table.active_sign,
  relationship: ['and'.'and'].way: ['in'.'compare'.'compare'].params: [{key: 'child_id'.value: childID },
    { key: 'active_id'.operator: '='.value: that.aid },
    { key: 'is_delete'.operator: '='.value: false},].limit: 30.page: 1.order_by: '-created_at'.expand: [],
}).then((res2) = > {
  let tempJoined = res2.data.objects
  console.log('tempJoin', tempJoined)
},(err) = >{})
Copy the code

But even so, I’ve encapsulated the simplest approach.

Minapp – fetch

As anyone who has used cloud services knows, there are many types of interfaces. For example, wechat cloud development will provide interface of small program side, interface of cloud function, and even Web interface. The cloud service provider we chose worked well at first, but later we had to use the Web due to business diversification. Moreover, the web interface provided by the cloud service provider is not the same as the call method above the small program, which leads to the same search, but also to write two different search methods. If you want to use his cloud functions, or other API, that’s a big headache.

What to do? In order to reduce the difficulty of development and maintenance, improve development efficiency. I came up with a bold idea: wouldn’t it be easy to unify all the interfaces into one? The idea is good, but the implementation is not so easy. When I first wrote the Web interface, I wanted to give up several times, feeling that it was really difficult to unify these different writing methods into one. Because not only do you want to make sure that you get the same result on different ends as much as possible, but you also want to make sure that the encapsulation method is as simple as possible through algorithms.

After some struggle, the method finally came out. At this time, I gave it a name, minapp-Fetch, which is also published on NPM, but it only supports the apis of the cloud service providers we used. It is written more succinctly as follows:

fetchFind(app.table.question, {
  p1: {
    way: 'compare'.data: ['status'.'>'.0]},p2: {
    way: 'compare'.data: ['is_admin_answered'.'='.false]},p3: {
    way: 'compare'.data: ['is_admin_read'.'='.false]},r: 'p1 && (p2 || p3)'.page: commen.page,
  limit: commen.limit,
  order_by: '-created_at'.expand: ['created_by']})Copy the code

Where p is each small condition, r is and, or of the small condition, page is page turning, limit is the number of pages per page, and order_by is order. Although it looks clearer at this point, it still has some shortcomings. First of all, the p condition is a little bit cumbersome; Second, the r rule does not support parenthesis nesting; Moreover, it does not support many methods.

Faced with the above problems, of course, is solved. After further efforts, minapp-fetch was written in its current form: p*: [‘ field ‘, ‘search method ‘,’ search content ‘]. The first parameter of the array is the field in the table, the second parameter is the search method, and the third parameter and the following parameter are the search content.

let rule = 'p3 || p7'

minapp.find(table, {
  p1: ['cat_label1'.'in', userChannels=[]],
  p2: ['cat_label2'.'stringLength'.23].// Search for a string of 23 characters
  p4: ['cat_label2'.'stringLength'.23.100].// Search for strings between 23 and 100
  p3: ['id'.'notIn', history],
  p7: ['status'.'='.20].p8: ['name'.'matches'./^foo/i].p15: ['geo_point'.'include'[15.15]], // Coordinates [longitude, latitude]
  p21: ['geo_point'.'withinCircle'[15.15], r], / / r unit km
  p23: ['geo_point'.'withinRegion'[15.15], max_r], //max_r, min_r unit is m
  p28: ['geo_point'.'within'[0.0], [6.0], [6.9], [0.9], [0.0]], // Set of coordinates, first point = last point, longitude/latitude
  p63: ['content'.'isNull'.true].p93: ['content'.'isExists'.false].r: isSelect ? rule : '(p1 && (p2 || p3)) && (p57 && p93)'.page: 1./ / the default value
  limit: 20./ / the default value
  orderBy: '-created_at'./ / the default value
  expand: []./ / the default value
  select: []./ / the default value
  withCount: false./ / the default value
})
Copy the code

As you can see, the find method above is pretty complicated, but it still feels pretty clean. You can even change the R rule at will to search for different situations. (Searches eventually follow rule R)

Access to the db

Later, due to some shortcomings of cloud service providers, plus our own business flexibility requirements, so we decided to write the background. Gradually, we began to abandon cloud service providers and develop our own. At this time, the problem also came, the connection of database is different, I am used to the writing method of Minapp-fetch, and then use the connection method of database alone, I feel really uncomfortable. And, different types of database, the method is not the same, maintenance and transformation, has become a big problem. As you can probably guess, yes, the connection to the database is also encapsulated!

Start with mongodb, the most popular non-relational database in Node.js, and start the journey of database method encapsulation. To be honest, the basic method of mongodb is very similar to the webAPI of previous cloud service providers, and it does not take too much effort to complete the packaging of common methods. But when we add a second database, mysql, the problem starts. Because mysql is an SQL statement that searches, it’s completely different. In other words, you have to algorithmically convert the encapsulated notation into SQL statements. Nothing complicated, you might think, just a simple string of components. That’s what I thought, but when I started working on it, I realized it was much more complicated than that. Those of you who know mysql should know that SQL statements can be renamed, linked table search, nested query, aggregate search, etc., and in order to make the search correct, often have to rename tables and fields, and then reference, etc., these problems make the conversion difficulty greatly improved. There may be times when you satisfy one search and cause problems with the other. But in order to develop more fun, more efficient; I’m going to go for it.

After a while of trying to get her act together, I sent her a new NPM package and gave her a new name: Access-DB. Now you can even reference multiple databases at the same time and use them in association:

import {mysql, mongodb, fastdb} from 'access-db'

async function exp() {
  let {data} = await mongodb.get('table11', id)
  
  await mysql.find('table2', {
    p0: ['num'.'=', data.num],
    r: 'p0'})}Copy the code

Because each database has its own characteristics, it is inevitable that there will be individual differences in encapsulation methods.

Fastdb appear

In the code above, you may have noticed that a new database, FastDB, has been introduced. Yes, it is a native JSON database built into Access-DB. It was also developed by myself after I found a similar local database. Its advantage is that it completely uses access-DB syntax to add, delete, change and check, at the same time, you do not need to install any additional database, can be used directly, very convenient. Disadvantages are also obvious, the form of storage is JSON file, there is no database password, such as unsafe, large amount of data, search efficiency is not so high.

conclusion

I hope that writing about my experiences will inspire or resonate with you. I am not a great god, and maybe you are not either, but as long as you do not stop thinking and have the courage to realize it, then you will certainly be able to make something meaningful!

Dedicate this method library to those who need it: the access-DB package access-DB document