1. Overview of the Realtime DB

1.1 Introduction to the Realtime DB

Realtime DB is a database hosted in the cloud, where data is stored in JSON format and synchronized to each connected client in Realtime. It has the following characteristics:

  • Instead of using the usual HTTP request, the data synchronization mechanism is used. Any connected device receives updates in real time whenever data changes
  • Provides a flexible formula-based rule language that allows users to define data structures and when data can be read or written
  • NoSQL database based on MongoDB has different optimization direction and functional characteristics from relational database. The server-side API is designed to support only operations that can be performed quickly, so users need to think carefully about the data structure being stored.

1.2 Background and application scenarios of the Realtime DB

The development of cloud native APP based on BaaS requires a set of database middleware for information storage, real-time synchronization, atomic modification and offline cache, and remote modification of Serverless database to achieve the purpose of disconnecting from the server interface. In order to meet the above requirements, a middleware such as Realtime DB is designed and implemented. The application scenario is as follows:

  • Instant messaging
  • State synchronization
  • Real-time dynamic
  • Team collaboration
  • other

2. Realtime DB technology development history

• Local DB: Local data persistence Advantages: stable and reliable, Local persistence disadvantages: data cannot be shared among multiple ends, and the playability is low. • Server Interface: Data persistence is achieved through the communication mode of the Server Interface. Advantages: Data can be shared among multiple ends, and the cloud can be controlled. • Remote DB: Remote real-time database with persistent data in the cloud. Advantages: Multi-terminal data sharing, real-time synchronization, high playability, Serverless, and free customization of data structure on both sides. No server deployment and maintenance disadvantages: more dependent on network availability

3. Research status of Realtime DB

3.1 Industry Analysis

Firebase, a real-time back-end database startup acquired by Google in 2014, makes it easier for users to use Firebase in conjunction with Google’s cloud services. FireBase currently offers two client-accessible Cloud database solutions that support real-time data synchronization: Real-time Database and Cloud Firestore.

product The principle of advantages disadvantages
Real-time database Store the data as a large JSON tree • Simple data is easy to store

• Support online status detection		 • Complex, layered data is harder to organize on a large scale

• Weak query and sorting functions
Cloud Firestore Store data as a collection of documents • Simple data is easily stored in documents, much like JSON

• Complex, hierarchical data is easier to organize on a large scale by using subcollections in documents

• Less need for de-normalization and data flattening		 It is more complicated to use

3.2 Functional requirements and challenges

The initial planning of Realtime DB should meet the following requirements:

  • Add, delete, modify and check statement support
  • Support for real-time data synchronization between multiple terminals
  • Public cloud or private cloud implementation
  • Offline cache support
  • Automatic data expansion and expansion based on Serverless
  • Means of data security
  • Custom add, delete, check and change protocol, later extensible function

1. Selection of cloud database, realizing Serverless relational database VS non-relational database; realizing cloud database storage on the server based on the company’s self-developed Serverless cloud MongoDB

2. The multi-terminal real-time synchronization scheme adopts NoSQL database, and the data is stored in multi-fork tree (K-V) mode, that is, jsonization. There is no need to care about the problem of node data merging, and the server can overwrite the data based on the operation time

3. Data security measure Data nodes define read and write permissions, and the cloud cloud grants permissions [based on accounts], and generates permission configuration tables

4. Transaction support and atomic operation Custom remote database operation protocol, pre-reserved operator “. For later implementation of atomic operations and other instructions

5. Database real-time guarantee adopts long connection, self-developed long connection SDK-TapConnect based on MDP, to achieve stable transceiving and millisecond response

3.3 Solutions

The client architecture is divided into the following four layers

Data flattening simplifies JSON

Leaf node basic data types: Boolean String/Long/Double key value restrictions: must not contain: ‘/’ ‘. ‘ ‘#’ ‘$” ‘ ‘ ‘, the related operators, used to implement the atomic operations such as the value value restrictions: always be: ‘NaN’ Inf ‘, ‘-‘ Inf ‘

Client-server communication model

Multi-terminal synchronization scheme

The development of Realtime DB needs to be adapted to Android, iOS, Flutter and other platforms

Solution 1 Develop Realtime DB SDK for Android, iOS, flutter, etc. Advantages: Native level performance Disadvantages: high maintenance cost

Solution 2 develops Native version, provides SO, and uses it in each end. Advantages: Develop a set of code, which can be used in multiple ends. Disadvantages: Obvious performance problems, a layer of Native communication, and inconvenient to use, each end needs API adaptation

Solution 3 uses Kotlin Multiplatform framework for development advantages: a set of code, run multi-end, and the code through the framework compilation processing, automatic compilation of multi-end products, natural support of native level performance, using Kotlin, simple writing disadvantages: the current framework for alpha, there are unknown risks

Common Code: Use Kotlin to implement Common logic, and rely on the official existing multiplatform support library to implement network, IO, database and other operations

IOS Code: You can rely on the existing iOS native library through CocoaPods to call the principle: During the compilation of the framework, automatically generate the corresponding Kt class function according to the external interface of Objective-C/Swift, and provide external calls

Android Code: Can rely on the existing JAR/AAR library through Gradle, invoke principle: Kotlin and Java interoperability

If you have platform-differentiated Code that cannot be implemented directly in Common Code, you can implement the Expect interface class A in Common Code and the Actual class A in iOS Code and Android Code respectively

3.4 Future prospects of Realtime DB

The Realtime DB SDK is built into a middleware covering iOS, Android, Flutter, and even JS. The Kotlin Multiplatform Mobile framework is adopted to realize a set of code that runs everywhere, bringing continuous gains in the later stage.

4. To summarize

The Realtime DB allows secure access to the database directly from the client code. Thanks to this, it is possible to build rich collaborative applications where data is permanently retained on the client and real-time events continue to be triggered even when offline, giving end users a great real-time experience. Realtime DB is still in the research and development stage. For the deficiencies of scheme selection and implementation, you are welcome to communicate or put forward optimization opinions or suggestions.

Author’s brief introduction

Yijun OPPO application engineers focus on mobile platform development, real-time database, Severless, Quic and other technologies.

For more exciting content, please scan the code to follow [OPPO Internet Technology] public account