This is the process I went through to write the Access-DB method library, which I will document here
The birth of an idea
The development of a small company is not easy. In terms of cost, save as much as you can. Sometimes what many people do turns to one person, it is not strange. It was in this situation that I slowly moved to full-stack development.
At first, in order to save money and we only do small programs, we chose a third-party cloud service provider as the background. It was good at first, but after a long time, I found that the interface methods they provided were not as easy to use as I had imagined. Especially in the case of complex lookups, the code is too complex to read and modify easily. So I started thinking about how to simplify his search. As you can see, fetchFindByCheck is not that simple to use, and it only supports the simplest AND, OR lookups. The code is not very readable or flexible.
fetchFindByCheck({
tableName: app.table.active_sign,
relationship: ['and', 'and'],
way: ['in', 'compare', 'compare'],
params: [
{ key: 'child_id', value: childID },
{ key: 'active_id', operator: '=', value: that.aid },
{ key: 'is_delete', operator: '=', value: false },
],
limit: 30,
page: 1,
order_by: '-created_at',
expand: [],
}).then((res2) => {
let tempJoined = res2.data.objects
console.log('tempJoin', tempJoined)
},(err)=>{})But even so, I’ve encapsulated a set of the simplest methods.
Minapp – fetch
As anyone who has used cloud services knows, there are many types of interfaces for cloud services. The WeChat cloud development, for example, provides interfaces to applets, interfaces to cloud functions, and even Web interfaces. We chose a cloud provider that worked well at first, but then, due to the diversification of our business, we were forced to use the Web. Moreover, the Web interface provided by the cloud service provider is not the same as the invocation method on the applet, which leads to writing two different search methods for the same search. If you have to use his cloud functions or other APIs, it’s a big headache.
What should I do? In order to reduce the difficulty of development and maintenance, improve the efficiency of development. I had a bold idea: if we could unify the interface of all the terminals, it would be very simple. The idea is good, but the implementation is not so easy. When I first wrote the interface on the Web side, I thought about giving it up several times, because it was a little hard to unify all these different ways of writing it. Because not only do you want to make sure that you get the same results on the different ends of the system as much as possible, but you also want to make sure that the encapsulation is as simple as possible through the algorithm.
After a while of struggle, the method finally came out, and at this point I gave it a name, minapp-fetch, and it’s now published on NPM, but it only supports the APIs of the cloud service providers that we used. It is written even more succinctly as follows:
fetchFind(app.table.question, {
p1: {
way: 'compare',
data: ['status', '>', 0]
},
p2: {
way: 'compare',
data: ['is_admin_answered', '=', false]
},
p3: {
way: 'compare',
data: ['is_admin_read', '=', false]
},
r: 'p1 && (p2 || p3)',
page: commen.page,
limit: commen.limit,
order_by: '-created_at',
expand: ['created_by']
})Where, p parameter is each small condition, r is the and, or of the small condition, page is page turning, limit is number of pages per page, order_by is sorting. Although it looks clearer now, it still has some shortcomings. First of all, the p condition, again, is a little bit cumbersome; Second, there is no support for nested parentheses in the R rule; Moreover, it does not support many methods.
In the face of the above problems, of course, is solved. P *: [‘ field ‘, ‘search method ‘,’ search content ‘]. The first argument of the array is the field in the table, the second argument is the search method, and the third argument after that is the search content.
let rule = 'p3 || p7' minapp.find(table, { p1: ['cat_label1', 'in', userChannels=[]], p2: ['cat_label2', 'stringLength', 23], ['cat_label2', 'stringLength', 23, 100], ['cat_label2', 'stringLength', 23, 100], ['id', 'notIn', history], p7: ['status', '=', 20], p8: ['name', 'matches', /^foo/i], p15: ['geo_point', 'include', [15, 15]], // coordinate points [longitude, latitude] p21: ['geo_point', 'withInCircle ', [15, 15], r], //r unit km p23: ['geo_point', 'withinRegion', [15, 15], MAX_R], // MAX_R, MIN_R unit is mp28: [' geo_point ', 'within' [0, 0], [6, 0], [6, 9], [0, 9], [0, 0]], / / coordinate point set, the first point = end point, warp/weft (longitude and latitude) about: ['content', 'isNull', true], p93: ['content', 'isExists', false], r: isSelect ? Rule: '(p1 && (p2 | | p3)) && (p57 && p93)', page: 1, / / the default limit: 20, / / the default orderBy: '- created_at', / / the default value expand: [], / / the default value of the select: [], / / the default withCount: false, / / the default})As you can see, the find method above is pretty complicated, but it still feels pretty clean. You can even change the R rule at will to search for different situations. (The search eventually follows the R rule.)
Access to the db
Later, due to the shortcomings of cloud service providers and our own requirements for business flexibility, we decided to write our own backend. Gradually, we started to abandon cloud service providers and develop our own. At this time, the problem also comes, the database connection is different, I used to MINAPP-FETCH writing method, then to use the database connection method alone, feel really not used to. And, different types of database, the method is not the same, maintenance and transformation, became a big problem. You must also guess, yes, is the connection of the database, also encapsulated!
Start with MongoDB, the most popular non-relational database in Node.js, and start the journey of database method encapsulation. To be honest, the basic approach of MongoDB is very similar to the WebAPI of the previous cloud service provider, and it doesn’t take much effort to encapsulate the common methods. The problem arises when a second database, MySQL, is added later. Because MySQL is a SQL statement to search, and completely different from the previous. In other words, you have to use an algorithm to translate the encapsulated notation into SQL statements. You might think, well, it’s not complicated, it’s just a simple composition of a string. That’s what I thought, but when I started working on it, it was much more complicated than that. For those of you who are familiar with MySQL, you should know that SQL statements can be renamed, linked tables can be searched, nested queries can be searched, aggregated queries can be searched, and in order to search correctly, you often have to rename tables and fields, re-reference, and so on. These problems make the conversion difficult greatly. There may be times when you satisfy one type of search, but the other type of search goes wrong. But in order to usually develop more cool, more efficient; I’m going to bite the bullet and make it happen.
After a while, she finally started to make some sense. I sent her a new NPM package and gave her a new name: Access-db. Now you can even reference multiple databases at the same time and associate them with each other:
import {mysql, mongodb, fastdb} from 'access-db'
async function exp() {
let {data} = await mongodb.get('table11', id)
await mysql.find('table2', {
p0: ['num', '=', data.num],
r: 'p0'
})
}Because each database has its own characteristics, so the encapsulation method, there will be individual differences, which is inevitable.
Fastdb appear
Those of you who are careful may have noticed that in the above code, a new database called FastDB is introduced. Yes, it is a native JSON database built into Access-DB. I developed it myself when I found out there was a similar local database. It has the advantage of using Access-DB syntax to add, delete, change and search, at the same time, do not need you to install any additional database, can be directly used, very convenient. Disadvantages are also obvious, the form of storage is a JSON file, no database password, these are not safe, the amount of data is large, the search efficiency is not so high.
conclusion
I hope that by documenting my experiences, I can inspire or resonate with you. I’m not a great person, and you probably aren’t either, but as long as you don’t stop thinking about it and have the courage to do it, you can do something meaningful!
Dedication of this method library to those who need it: Access-DB package Access-DB document