This article describes how to use Core Data with CloudKit to synchronize a common database to the local server and create a Core Data database mirror locally.

Three CloudKit databases

There are three databases in CloudKit:

Public database

Public databases hold data that developers want anyone to have access to. You cannot add a custom Zone to a public database. All data is stored in the default Zone. Whether users have an iCloud account or not, the data can be accessed through the app or the CloudKit Web service. The contents of the public database are visible in the CloudKit dashboard.

The data capacity of the public database counts toward the CloudKit storage quota of the application.

Private database

This is where iCloud users store their personal data, with apps that keep content they don’t want the public to see. Users can access the data only after logging into their iCloud account. By default, only users have access to their own private database (and can share some of it with other iCloud users). The user has all operation rights (create, view, change, delete) on the data. The data in the private database is not visible in the CloudKit dashboard and is completely confidential to developers.

Developers can create custom areas in private databases to make it easier to organize and manage data.

The data capacity of the private database is included in the user’s iCloud storage quota.

Shared database

The data that iCloud users see in a shared database is a projection of data that other iCloud users have shared with you, and that data is still stored in their own private databases. You don’t own the data and can only view and modify the content if you have the necessary permissions. The database is only available if you have logged in to your iCloud account.

For example, if you share a piece of data to a user, the data will still be stored in your private database, but the shared user can see the record in his shared database because of your authorization, and only according to the permissions you set.

You cannot customize a region in a shared database. The data is not visible in the CloudKit dashboard.

The capacity of the shared database is included in the CloudKit storage quota of the application.

Same noun, different meaning

In Core Data with CloudKit(2), we talked about how to synchronize your local database to your iCloud private database. Here we talk about how to synchronize your shared database to your local database. Although both articles are on the subject of synchronization, the underlying meaning and logic of synchronization are different.

Synchronizing local Data to a private database is still essentially a standard Core Data project, and developers from model design to code development are no different from [projects that support only local persistent databases]. CloudtKit only serves as a bridge to synchronize data to the user’s other devices. In most cases, developers can use managed objects without regard to the existence of private databases and CKRecords.

Synchronizing a public database to the local is an entirely different matter. Public database is the concept of network database. Standard logic for developers in CloudKit instrumentation platform to create Record Type, through the dashboard or client to add CKRecord to the public database, the client through access to the server to obtain network data records. Core Data with CloudKit makes it easy to use Core Data knowledge to do this. The data that is synchronized to the local server is the mirror of the public database on the server. The operation on the managed object data is indirectly completed on the CKRecord record on the server.

Authentication, discussed later, checks records or databases on the network side, even though the objects are managed objects or local persistent stores.

Public vs private databases

Let’s compare public and private databases on several dimensions.

authentication

Without regard to data sharing, the data in the private database is accessible only to the user (who is logged in to the iCloud account). The user, as the creator of the data, has all operation rights. The authentication rules for private databases are very simple:

In the iCloud Dashboard article, we introduced the concept of security roles. The system creates three preset roles for the public database: World, Authenticated, and Creator. In public databases, you need to consider factors such as whether a user has logged in to the iCloud account and whether the user is the creator of data records.

  • Every user can read the record (whether or not the account is logged in)
  • Each user who has logged in to an account can create a record
  • Users who have logged in can only modify or delete records created by themselves

In addition to the large amount of code required to determine permissions through the standard CloudKit API, the authentication time is also long (you need to access the server to get the results each time). Core Data with CloudKit backs up the metadata of CKRecord locally, which perfectly solves the authentication efficiency problem and provides a convenient API for developers to call.

We can use code like this to determine whether the user has permission to modify and delete the current ManagedObject:

let container = PersistenceController.shared.container

if container.canUpdateRecord(forManagedObjectWith:item.objectID) {
    // Modify or delete itME
}
Copy the code

The last two years, apple growing NSPersistentCloudKitContainer presence, add a lot of important methods for it. These methods can be used not only for public databases or managed objects within them, but also for other types of databases or data (private, local, shared, and so on).

  • CanUpdateRecord and canDeleteRecord

    Gets the permission to modify data. Returns true in either of the following cases:

    1. objectIDIs a temporary object identifier (meaning it has not yet been persisted).
    2. Persistent stores containing managed objects do not applyCloudKitLocal database not used for synchronization.
    3. Persistent storage manages private databases (users have full privileges on private databases)
    4. The persistent store manages the common database, and the user is the creator of the record, orCore DataManaged objects have not been updated toiCloudIn the.
    5. The persistent storage manages the shared database, and the user has the permission to change the data.

    In practicecanDeleteRecordThe results returned are not accurate, currently recommended to use onlycanUpdateRecord

    CanUpdateRecord returns false. This does not mean that you cannot delete data from local storage. It just means that you do not have permission to modify the network records associated with the managed object.

  • canModifyMangedObject(in:NSPersistentStore)

    Indicates whether specific persistent stores can be changed.

    Use this method to determine whether the user can write records to the CloudKit database. For example, when a user is not logged in to an iCloud account, they cannot write to the persistent storage that manages a public database.

    Similarly, canModifyManagedObjects returns false. This does not mean that you cannot write data to the local SQLite file, it just means that you do not have permission to modify the network storage corresponding to the persistent storage.

Since there is no concept of permissions for local data and persistent storage, it is very possible for a developer to write code that does not have permissions on the network side but still does the wrong thing locally. This can be dangerous in a project that synchronizes a common database with a shared database. If you modify or delete a Data record that is not authorized by the network side, the network side will reject your request. Core Data with CloudKit will stop all synchronization after being denied. Therefore, when writing a project to synchronize a common or shared database, you must ensure that you have the appropriate permissions before you operate on the data.

Synchronization mechanism

On the export side (synchronizing local data changes to the server), the performance is the same whether you synchronize a private database or a public database. Core Data with CloudKit synchronizes local Data changes to the server immediately. It’s an instant, one-way action.

In terms of import (synchronizing changes to network data locally), the mechanism for private and public databases is quite different.

In the Basics and CloudKit Dashboard articles, we have introduced the private database synchronization mechanism:

  • The client subscribes to the serverCKDatabaseSubscription
  • Server side customization in private databaseZonePushes silent remote alerts to the client after the content of
  • After receiving the notification, the client passesCKFetchRecordZoneChangesOperationRequest change data from the server
  • After comparing the tokens, the server synchronizes the changed data of the token update to the client

The whole process is back-and-forth, and the two sides work together to complete it.

Due to some technical limitations of public databases, the above mechanism cannot be applied to public database synchronization.

  • Public databases cannot be customizedZone
  • No customizationZoneYou can’t subscribeCKDatabaseSubscription
  • CKFetchrecordZoneChangesOperationUsing the proprietary technology of private databases, public databases can only be usedCKQureyOperation
  • The public database has no tombstone mechanism to record all user actions (deletes)

For the above reasons, Core Data with CloudKit can only poll for changes to obtain the change Data of the public database.

When the application is started or run every 30 minutes, NSPersistentCloudKitContainer changes through CKQurey operation to query the database and get the data. The import process is initiated by the client and responded by the server.

This synchronization mechanism will limit the applicable scenarios, and only data with low immediacy can be stored in a public database.

The data model

Due to different synchronization mechanisms, the following points should be considered when designing a data model for a common database:

  • The complexity of the

    Public databases using CKQureyOperation query the server since the last time change, its efficiency is far lower than the CKFetchRecordZoneChangesOperation. If you can control the number of entities and attributes of the ManagedObjectModel, fewer requests are required for query and the execution efficiency is higher. The model complexity of the common database should be reduced as much as possible if there is no special need.

  • tombstone

    After receiving a record deletion operation from the client, the private database immediately deletes the record from the server and saves the tombstone of the deletion operation. Other client devices through CKFetchRecordZoneChangesOperation access changes, private database will change record (including tombstones) and sent to the client. The client deletes the local data record according to the tombstone indication to ensure data consistency.

    The public database will also delete the record on the server immediately after receiving the record deletion operation. However, the public database does not have a tombstone mechanism, so when other clients query it for changes in data, the public database will only tell the client device about the changes of new or changed records, and cannot inform the client of the deletion operation. This means that we cannot pass the delete operation from one device to the other, and the local mirror of the two devices’ common database will be different.

    When we designed the common database data model, we avoided this discrepancy as much as possible by adding a tombstone-like attribute such as isDeleted.

// Set isDelete to true when deleting
if container.canUpdateRecord(forManagedObjectWith:item.objectID){
     item.isDeleted = true
  	 try! viewContext.save()
}
Copy the code

When the data is invoked, only records whose isDeleted is false are retrieved.

@FetchRequest(
        sortDescriptors: [NSSortDescriptor(keyPath: \Item.timestamp, ascending: true)],
        predicate: NSPredicate(format: "%K = false", #keyPath(Item.isDelete)),
        animation: .default
)
private var items: FetchedResults<Item>
Copy the code

Records aren’t actually deleted, they’re just blocked. The common database can transfer the record modification operation between devices, which ensures the data consistency between devices and also realizes the “deletion” of data. The “deleted” data still occupies space on the local server and the server. Therefore, you need to carefully select the time to clear the space occupied by the data.

Storage quota

Private database data is stored in users’ personal iCloud space, occupying the capacity quota of their personal space. If the user’s iCloud space is full, data will no longer be able to sync across the network. Users can solve this problem by cleaning up their personal space or choosing a larger space solution.

The data capacity of the public database is your application’s space quota. Apple provides a basic capacity limit for each cloudKit-enabled application, which is 10GB of Asset storage, 100MB of database, 2GB of data transfer per month, and 40 queries per second. Space, traffic, and requests will all increase as the number of active users of your app (used in 16 months) increases, up to 10 petabytes, 10 terabytes, and 200 terabytes per day.

Although the vast majority of applications do not exceed these limits, it is important for developers to use as little space as possible to improve data response efficiency.

Core Data with CloudKit synchronizes a common database by keeping a local image of the entire common database. Therefore, without good control over the amount of Data, the use of an application on a user’s device can be terrifying. The “delete” approach adopted above will further encroach on network and device space.

Developers should consider the timing of removing fake “deleted” data from the beginning of the project.

There is no guarantee that a purge will occur when all clients have synchronized the “delete” state, and it is acceptable to allow data inconsistencies between devices without affecting the application’s business logic.

Based on the average usage rate of the application, the developer can clear the data that the client “deleted” before a certain period of time. Although Core Data with CloudKit stores CKRecord metadata for managed objects locally, it does not provide an API for developers. In order to facilitate deletion, we can add a “delete” time attribute in the model to cooperate with the query during clearance.

Where public databases are used

Calling a public database with CloudKit and synchronizing a public database with Core Data with CloudKit have different technical characteristics and different priorities.

I personally recommend using Core Data with CloudKit to synchronize public databases in the following situations:

  • Read-only don’t write

    Such as providing templates, initial data, news alerts, etc.

    The creation, modification and deletion of the public database data are operated by the developer through the dashboard or specific application. The user’s application only reads the contents of the public database, and does not create or change them.

  • Only one record is processed

    The application creates only one piece of data associated with the user or device and updates only that piece of data.

    It is typically used to record the state or data associated with the device or the user (which can be associated). For example, a game’s high score leaderboard (which only keeps a user’s highest score).

  • Create only and do not modify

    Log class scenario. The user is responsible for creating the data and is not particularly dependent on the data itself. The application periodically clears the local stale data. Public database records are queried or backed up and periodically cleaned by CloudKit Web services or other specific applications.

When you consider using Core Data with CloudKit to synchronize data from public databases, you must carefully consider the pros and cons and choose the right application scenario.

Synchronizing public databases

This section covers Core Data with CloudKit (2) — synchronizing your local database to your iCloud private database and Core Data with CloudKit (3) — CloudKit dashboard. Please read both articles before continuing.

Project configuration

Configuring a public database in a project is almost exactly the same as configuring a private database.

  • In the projectTargettheSigning&CapabilitiesaddiCloud
  • chooseCloudKitAnd addContainer

If you are using only the public database in your project, you may not add itBackground ModetheRemote notificationsfunction

Using NSPersistentCloudKitContainer to create a local mirror

  • inXcode Data Model EditorCreate a newConfigurationAnd put the entities you want to make public (Entity) to this new configuration.
  • In yourCore Data StackFor example, template projectPersistenc.swift) Add the following code:
let publicURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("public.sqlite")
let publicDesc = NSPersistentStoreDescription(url: publicURL)
publicDesc.configuration = "public" / / the name of the Configuration
publicDesc.cloudKitContainerOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: "your.public.containerID")
publicDesc.cloudKitContainerOptions?.databaseScope = .public
Copy the code

The code is familiar? That’s right. In fact, synchronizing a public database requires only one more line of code than synchronizing a private database:

publicDesc.cloudKitContainerOptions?.databaseScope = .public
Copy the code

DatabaseScope is a new property added by Apple for cloudKitContainerOptions in 2020. The default value is.private. Therefore, you do not need to set this parameter when synchronizing private libraries.

Is this it?

Yes, here it is. Other configurations are the same as for synchronizing a private database. Add Descriptioin to persistentStoreDescriptions, configuration context, the necessary configuration Persistent History Tracking.

Configure the dashboard

Because NSPersistentCloudKitContainer access to public data (CKQurey) and access to private data (CKFetchRecordZoneChangesOperation) is different, We also need to modify the Schema on CloudKit instrumentation platform to ensure the normal operation of the program.

In the CloudKit dashboard, select Indexes and add two Indexes for each Record Type used for the public database:

At the time of writing this article, when I was building a demo project using Xcode 13 Beta5, I found that I needed to add one more index to properly synchronize the public database. If you are using Xcode 13, add an index Sortable to the dashboard.

other

Initialize the Schema

According to the above operation, proceed to inCloudKitWhen you add indexes to the dashboard, you will find that there is noneRecord TypeFor you to add indexes. This is because we did not initialize on the network database sideSchema.

There are two ways to initialize a Schema on the network side:

  • Create a managed object data and synchronize it to the server side

    After receiving data, if the server finds no corresponding Record Type, it will automatically create it

  • Using initializeCloudKitSchema

    InitializeCloudKitSchema allows you to initialize the Schema on the server side without creating the data. Add the following code to the Core Data Stack:

try! container.initializeCloudKitSchema(options: .printSchema)
Copy the code

After running the project, we can see the corresponding Record Type in the project on the dashboard.

This code needs to be executed only once, deleted or commented out after initialization.

InitializeCloudKitSchema can also be used in unit tests to verify that the Model meets the compatibility requirements of the synchronous Model.

let result = try! container.initializeCloudKitSchema(options: .dryRun)
Copy the code

Compliant with compatibility requirements result is true. DryRun means only checking locally and not actually initializing on the server side.

Multiple containers and configurations

As mentioned in previous articles, you can associate multiple CloudKit containers with a project, and one container can be used for multiple applications.

If your project uses both private and public databases and the containers are inconsistent, in addition to associating both containers in your project, you also need to set the correct ContainerID for the Description in your code.

let publicDesc = NSPersistentStoreDescription(url: publicURL)
publicDesc.configuration = "public"
publicDesc.cloudKitContainerOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: "public.container")
publicDesc.cloudKitContainerOptions?.databaseScope = .public

let privateDesc = NSPersistentStoreDescription(url: privateURL)
privateDesc.configuration = "private"
privateDesc.cloudKitContainerOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: "private.container")
Copy the code

A public database of NSPersistentStoreDescription URL with a private database URL must be different (that is, to create two sqlite files), the coordinator is unable to load the same URL many times.

let publicURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("public.sqlite")

let privateURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.appendingPathComponent("private.sqlite")
Copy the code

Xcode13 beta

Xcode 13 beta appears to have made undisclosed adjustments to the CloudKit module. When using Core Data with CloudKit in Xcode 13 Beta5, there are a lot of weird warnings. At this stage, it’s best to use Xcode 12 for this article’s testing.

conclusion

The implementation of local data synchronization to a private database is very similar to the implementation of synchronization to a public database in the code. Developers should not be fooled by this illusion. It is important to understand the nature of the synchronization mechanism so that they can better design the data model and plan the business logic.

I’ll move on to the next article in this series — Synchronizing shared databases — once Xcode 13 is stable.

This article originally appeared on my personal blog, Swift Notepad