preface
Persistent history tracking has been known about for a while, and I’ve skimped through the documents before without taking them too seriously. On the one hand, there is not much information about it, it is not easy to learn; On the other hand, there is no special power to use it.
In the planned 3 】 【 health notes, I consider to add widgets or other Extentsion App, I also intend to WWDC21 introduced NSCoreDataCoreSpotlightDelegate used on the new version of the App. To do this, you have to carefully understand how to use persistent history tracing.
What is Persistent History Tracking?
Use Persistent History Tracking to determine what has changed in the Store since the feature was enabled. — Official Apple document
In CoreData, if your data is stored in Sqlite (which is what most developers do) and persistent history tracking is enabled, you can call the database and register the application for notification whenever the data in the database has been changed (deleted, added, modified, etc.) Will receive a system alert that the database has changed.
Why use it
Currently, persistence history tracing can be applied in the following scenarios:
-
In App, the data changes generated by the App’s batch processing (BatchInsert, BatchUpdate, BatchDelete) business are merged into the current ViewContext.
Batch processing is directly through the coordinator (PersistentStoreCoordinator) to operate, because the operation is not through the context (ManagedObejctContext), therefore if does not do the special processing, The App does not timely reflect data changes caused by batch processing in the context of the current view. Before Persistent History Tracking, we had to use mergeChanegs, for example, to incorporate changes into the context after each batch operation. With Persistent History Tracking, we can unify all batch changes into a single code segment for merging.
-
In an App Group, when App and App Extension share a database file, changes made by one member in the database are reflected in the context of another member’s view in a timely manner.
Imagine a scenario where you have an App that aggregates web Clips and provides Safari Extentsion to save the appropriate Clips as you browse the web. After Safari Extension saves a Clip to the database, bring your App (which was already started and switched to the background while Safari saved the data) to the foreground. If the Clip list is showing, The latest Clip (added by Safari Extentsion) does not appear in the list. Once Persistent History Tracking is enabled, your App will be notified of changes to the database and will respond to them, and the user will see the new Clip in the list the first time.
-
When using PersistentCloudKitContainer will your CoreData database with Cloudkit data synchronization.
Persistent History Tracking is an important guarantee for data synchronization between CoreData and CloudKit. No developers set, when you use PersistentCloudKitContainer as container, CoreData had enabled for your database Persistent History Tracking function. However, unless you explicitly state that persistent history tracking is enabled in your code, your code will not be notified of any changes to the network synchronization and CoreData will quietly handle everything in the background.
-
When using NSCoreDataCoreSpotlightDelegate.
On this year’s WWDC2021, apple introduced a NSCoreDataCoreSpotlightDelegate, can be very convenient to data in CoreData Spotlight with integrated together. To use this feature, you must enable Persistent History Tracking for your database.
How Persistent History Tracking works
Persistent History Tracking is implemented using Sqlite triggers. It creates a trigger on the specified Entity that will record all data changes. This is why persistent history tracing can only be enabled on Sqlite.
A Transaction is recorded directly on Sqlite, so no matter how the Transaction was generated (with or without context), by the App or Extension, it can be recorded in detail.
All changes will be stored in your Sqlite database file. Apple has created several tables in Sqlite that record Transaction information.
Apple does not disclose the exact structure of these tables, but we can use the API provided by Persistent History Tracking to query and clear the data.
If you are interested, you can also take a look at the contents of these tables. Apple organizes the data very tightly. The ATRANSACTION is the transaction that has not been eliminated, the ATRANSACTIONSTRING is the author and contextName string, and the ACHANGE is the changed data, which is eventually converted to the corresponding ManagedObjectID.
Transactions are automatically recorded in the order in which they are generated. We can retrieve all changes that have occurred after a certain time. You can determine this point in time by using several expressions:
- Token-based
- Based on Timestamp
- Based on the Transaction itself
A basic process for Persistent History Tracking is as follows:
- In response to Persistent History Tracking NSPersistentStoreRemoteChange notice
- Check whether there are any transactions that need to be processed since the last time stamp was processed
- Merges the Transaction that needs to be processed into the current view context
- Records the timestamp of the last Transaction processed
- Delete the merged Transaction
App Groups
Before we move on to Persisten History Tracking, let’s take a look at App Groups.
Because Of Apple’s strict sandbox mechanism for apps, Extension has its own storage space for each App. They can only read the contents of their own sandbox file space. If we want to share data between different apps or between App and Extension, we can only exchange data through some third-party libraries before App Groups appeared.
To solve this problem, Apple launched its own solution, App Groups. App Group allows you to share data between different apps or between App&App Extension in two ways (must be the same developer account) :
- UserDefauls
- Group URL (storage space accessible to each member of the Group)
The vast majority of Persistent History Tracking applications occur when App groups are enabled. Therefore, it is necessary to know how to create App Grups, how to access Group shared UserDefaults, and how to read files in the Group URL.
Add the App to App Groups
In the project navigation bar, select the Target that you want to add to the Group, and click + in Signing&Capabilities to add the App Group function.
Select or create a group in App Groups
Group can only be added correctly if Team is set.
The App Group Container ID must be Group. At first, the reverse domain name is usually used later.
If you have a developer account, you can add App Groups under your App ID
Other apps or App Extensions are assigned to the same App Group in the same way.
Create UserDefaults that can be shared within a Group
public extension UserDefaults {
/// this parameter is used in userDefaults for app groups, where the contents set can be used by the members of the app group
static let appGroup = UserDefaults(suiteName: "group.com.fatbobman.healthnote")!
}
Copy the codeSuitName is the App Group Container ID you created earlier
In the App code in the Group, the UserDefaults data created using the following code will be shared by all members of the Group, and each member can read and write to it
let userDefaults = UserDefaults.appGroup
userDefaults.set("hello world", forKey: "shareString")
Copy the codeThe Group Container URL is obtained
let groupURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.fatbobman.healthnote")!
Copy the codeDoing things to this URL is exactly the same as doing things to urls in the App’s own sandbox. All members of the Group can read and write files in this folder.
The rest of the code assumes that the App is in an App Group and that data is shared through UserDefaults and the Container URL.
Enable persistent history tracing
Enable the Persistent History Tracking function is very simple, we only need to NSPersistentStoreDescription ` set.
Here is an example of enabling this in the CoreData template generated by Xcode: persistence.swift
init(inMemory: Bool = false) {
container = NSPersistentContainer(name: "PersistentTrackBlog")
if inMemory {
container.persistentStoreDescriptions.first!.url = URL(fileURLWithPath: "/dev/null")}// Add the following code:
let desc = container.persistentStoreDescriptions.first!
// If desc.url is not specified, the default URL is the Application Support directory of the current App
// FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask).first!
// Enable Persistent History Tracking on the Description
desc.setOption(true as NSNumber,
forKey: NSPersistentHistoryTrackingKey)
// Receive relevant remote notifications
desc.setOption(true as NSNumber,
forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)
// Description must be set before load, otherwise it will not work
container.loadPersistentStores(completionHandler: { (storeDescription, error) in
if let error = error as NSError? {
fatalError("Unresolved error \(error).\(error.userInfo)")}}}Copy the codeIf you create your own Description, something like this:
let defaultDesc: NSPersistentStoreDescription
let groupURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.fatbobman.healthnote")!
// The database is stored in the App Group Container and can be read by other apps or App Extension
defaultDesc.url = groupURL
defaultDesc.configuration = "Local"
defaultDesc.setOption(true as NSNumber, forKey: NSPersistentHistoryTrackingKey)
defaultDesc.setOption(true as NSNumber,
forKey: NSPersistentStoreRemoteChangeNotificationPostOptionKey)
container.persistentStoreDescriptions = [defaultDesc]
container.loadPersistentStores(completionHandler: { _, error in
if let error = error as NSError? {}})Copy the codePersistent History Tracking is set on the description, so if your CoreData uses multiple configurations, you can enable it only for the Configuration you need.
Responds to persistent store trace remote notifications
final class PersistentHistoryTrackingManager {
init(container: NSPersistentContainer.currentActor: AppActor) {
self.container = container
self.currentActor = currentActor
// Register the StoreRemoteChange response
NotificationCenter.default.publisher(
for: .NSPersistentStoreRemoteChange,
object: container.persistentStoreCoordinator
)
.subscribe(on: queue, options: nil)
.sink { _ in
// The content of the notification is meaningless and only serves as a reminder to be processed
self.processor()
}
.store(in: &cancellables)
}
var container: NSPersistentContainer
var currentActor: AppActor
let userDefaults = UserDefaults.appGroup
lazy var backgroundContext = { container.newBackgroundContext() }()
private var cancellables: Set<AnyCancellable> = []
private lazy var queue = {
DispatchQueue(label: "com.fatbobman.\ [self.currentActor.rawValue).processPersistentHistory")
}()
/// Handle persistent history
private func processor(a) {
// Operate in the correct context to avoid affecting the main thread
backgroundContext.performAndWait {
// Fetcher is used to fetch the transaction to be processed
guard let transactions = try? fetcher() else { return }
// merger merges transaction into the current view context
merger(transaction: transactions)
}
}
}
Copy the codeLet me explain the above code briefly.
We registered processor in response to a NSNotification. Name. NSPersistentStoreRemoteChange.
Every time an Entity with Persistent History Tracking enabled changes data in your database, the processor will be called. In the above code, we completely ignore the Notification because its content is meaningless, except that it tells us that the database has changed and needs to be processed by the processor, and that it is up to our code to determine what has changed and whether processing is necessary.
All data operations for Persistent History Tracking are carried out in the backgroundContext to avoid affecting the main thread.
The core of PersistentHistoryTrackingManager is we deal with Persistent History Tracking. In CoreDataStack (such as persistent.swift above), handle the Persistent History Tracking event by adding the following code to init
let persistentHistoryTrackingManager : PersistentHistoryTrackingManager
init(inMemory: Bool = false) {
.
// Mark the author name of the current context
container.viewContext.transactionAuthor = AppActor.mainApp.rawValue
persistentHistoryTrackingManager = PersistentHistoryTrackingManager(
container: container,
currentActor: AppActor.mainApp // The current member)}Copy the codeSince all members of the App Group can read and write to our database, we need to create an enumeration type to mark each member in order to better identify which Transaction was generated by that member.
enum AppActor:String.CaseIterable{
case mainApp // iOS App
case safariExtension //Safari Extension
}
Copy the codeCreate the tag for the member according to your needs.
Get the Transaction that needs to be processed
After received NSPersistentStoreRemoteChange news, the first thing we should to pick up the need to handle the Transaction. As mentioned in the previous working principle, the API provides us with three different approaches:
open class func fetchHistory(after date: Date) - >Self
open class func fetchHistory(after token: NSPersistentHistoryToken?). ->Self
open class func fetchHistory(after transaction: NSPersistentHistoryTransaction?). ->Self
Copy the codeGets the Transaction after the specified point in time that satisfies the criteria
I prefer to use Timestamp, also known as Date, for processing. There are two main reasons:
- When we use UserDefaults to store the last record,
DateIs a structure directly supported by UserDefaults and does not require conversion TimestampHas been recorded in Transaction (tableATRANSACTION) can be directly searched without conversion, and the Token needs to be calculated again
We can retrieve all Transaction information in the current SQLite database by using the following code:
NSPersistentHistoryChangeRequest.fetchHistory(after: .distantPast)
Copy the codeThis information includes transactions generated from any source, whether they are required or processed by the current App.
In the processing process above, we have described the need to filter out unnecessary information through timestamps and save the timestamp of the last Transaction processed. We keep this information in UserDefaults so that the members of the App Group can work on it together.
extension UserDefaults {
/// Get the latest timestamp from all app Actor's last timestamp
// delete only transactions before the latest timestamp, so that other appactors are guaranteed
/// All unprocessed transactions can be retrieved normally
/// Set a limit of 7 days. Even if some appActor is not used (did not create userDefauls)
/// Will also keep up to 7 days of transactions
/// -parameter appActors: specifies the role of the app, such as healthNote and widget
/// -returns: date (timestamp). If the return value is nil, all unprocessed transactions will be processed
func lastCommonTransactionTimestamp(in appActors: [AppActor]) -> Date? {
/ / seven days ago
let sevenDaysAgo = Date().addingTimeInterval(-604800)
let lasttimestamps = appActors
.compactMap {
lastHistoryTransactionTimestamp(for: $0)}// All actors have no set value
guard !lasttimestamps.isEmpty else {return nil}
let minTimestamp = lasttimestamps.min()!
// Check whether all actors are set
guard lasttimestamps.count ! = appActors.count else {
// Return the latest timestamp
return minTimestamp
}
// If you have not obtained all the actor values after 7 days, return 7 days, in case any actor is never set
if minTimestamp < sevenDaysAgo {
return sevenDaysAgo
}
else {
return nil}}/// Get the timestamp of the last transaction processed by the specified appActor
/// -parameter appActore: indicates the app role, such as HealthNote and Widget
/// -returns: date (timestamp). If the return value is nil, all unprocessed transactions will be processed
func lastHistoryTransactionTimestamp(for appActor: AppActor) -> Date? {
let key = "PersistentHistoryTracker.lastToken.\(appActor.rawValue)"
return object(forKey: key) as? Date
}
/// set the latest transaction timestamp for the specified appActor
/// - Parameters:
/// -appactor: app role, such as HealthNote,widget
/// -newdate: date (timestamp)
func updateLastHistoryTransactionTimestamp(for appActor: AppActor.to newDate: Date?). {
let key = "PersistentHistoryTracker.lastToken.\(appActor.rawValue)"
set(newDate, forKey: key)
}
}
Copy the codeBecause each App Group members will save their lastHistoryTransactionTimestamp, so in order to ensure the Transaction can be all members right after the merger, cleared off again, LastCommonTransactionTimestamp will return all the members of the latest timestamp. When lastCommonTransactionTimestamp cleared after the merger Transaction, will be used to.
With this in mind, the above code can be modified as follows:
let fromDate = userDefaults.lastHistoryTransactionTimestamp(for: currentActor) ?? Date.distantPast
NSPersistentHistoryChangeRequest.fetchHistory(after: fromDate)
Copy the codeWith the timestamp, we’ve filtered out a lot of transactions we don’t care about, but are they all we need for the rest of the transactions? The answer is no, there are at least two types of Transaction we don’t need to worry about:
-
A Transaction generated from the context of the current App itself
Typically, the App gives immediate feedback on its own changes in the data generated by the context. If the changes are already reflected in the view context (the main thread, ManagedObjectContext), then we can ignore these transactions. However, if the data is processed in batches, or in background Context, and is not merged into the view context, we still handle these transactions.
-
Transaction generated by the system
When you use the PersistentCloudKitContainer, for example, all the network synchronization data will produce a Transaction, the Transaction will be handled by CoreData, we don’t have to ignore.
Based on these two points, we can further narrow down the number of transactions we need to deal with. The final fetcher code looks like this:
extension PersistentHistoryTrackerManager {
enum Error: String.Swift.Error {
case historyTransactionConvertionFailed
}
// Retrieve the filtered Transaction
func fetcher(a) throws- > [NSPersistentHistoryTransaction] {
let fromDate = userDefaults.lastHistoryTransactionTimestamp(for: currentActor) ?? Date.distantPast
NSPersistentHistoryChangeRequest.fetchHistory(after: fromDate)
let historyFetchRequest = NSPersistentHistoryChangeRequest.fetchHistory(after: fromDate)
if let fetchRequest = NSPersistentHistoryTransaction.fetchRequest {
var predicates: [NSPredicate] = []
AppActor.allCases.forEach { appActor in
if appActor = = currentActor {
// This code assumes that in the App, even operations performed through the background are instantly merged into the ViewContext
// Therefore, for the current appActor, only transactions generated by a context named batchContext are processed
let perdicate = NSPredicate(format: "%K = %@ AND %K = %@",
#keyPath(NSPersistentHistoryTransaction.author),
appActor.rawValue,
#keyPath(NSPersistentHistoryTransaction.contextName),
"batchContext")
predicates.append(perdicate)
} else {
// All other transactions generated by appActor are processed
let perdicate = NSPredicate(format: "%K = %@",
#keyPath(NSPersistentHistoryTransaction.author),
appActor.rawValue)
predicates.append(perdicate)
}
}
let compoundPredicate = NSCompoundPredicate(type: .or, subpredicates: predicates)
fetchRequest.predicate = compoundPredicate
historyFetchRequest.fetchRequest = fetchRequest
}
guard let historyResult = try backgroundContext.execute(historyFetchRequest) as? NSPersistentHistoryResult.let history = historyResult.result as? [NSPersistentHistoryTransaction]
else {
throw Error.historyTransactionConvertionFailed
}
return history
}
}
Copy the codeIf your App is simple (such as PersistentCloudKitContainer) is not used, can do not need the finer the predicate of the process. Overall, even if more transactions are acquired than is needed, CoreData does not stress the system at merge time.
Because the fetcher is through NSPersistentHistoryTransaction. Author and NSPersistentHistoryTransaction. ContextName to further filter the Transaction, So in your code, explicitly mark the identity in the NS-managed object Context:
// Mark the author of the context in the code, for example
viewContext.transactionAuthor = AppActor.mainApp.rawValue
// For batch operations, mark name, for example
backgroundContext.name = "batchContext"
Copy the codeClearly marking Transaction information is a basic requirement for using Persistent History Tracking
Merge Transaction into the view context
With the transactions that need to be processed retrieved from the Fetcher, we need to merge those transactions into the view context.
The merge operation is simple, just save the last timestamp after the merge.
extension PersistentHistoryTrackerManager {
func merger(transaction: [NSPersistentHistoryTransaction]) {
let viewContext = container.viewContext
viewContext.perform {
transaction.forEach { transaction in
let userInfo = transaction.objectIDNotification().userInfo ?? [:]
NSManagedObjectContext.mergeChanges(fromRemoteContextSave: userInfo, into: [viewContext])
}
}
// Update the last transaction timestamp
guard let lastTimestamp = transaction.last?.timestamp else { return }
userDefaults.updateLastHistoryTransactionTimestamp(for: currentActor, to: lastTimestamp)
}
}
Copy the codeCan be selected according to their own habits combined code, the code below and above NSManagedObjectContext. MergeChanges are equivalent:
viewContext.perform {
transaction.forEach { transaction in
viewContext.mergeChanges(fromContextDidSave: transaction.objectIDNotification())
}
}
Copy the codeTransactions that have already occurred in the database, but are not yet reflected in the view context, will appear in your App UI immediately after the merge.
Clear the merged Transaction
All transactions are stored in Sqlite files, not only taking up space, but also affecting Sqlite access speed as more records are recorded. We need a clear cleanup strategy to remove transactions that have already been processed.
Open class func fetchHistory(after date: date) -> Self
open class func deleteHistory(before date: Date) - >Self
open class func deleteHistory(before token: NSPersistentHistoryToken?). ->Self
open class func deleteHistory(before transaction: NSPersistentHistoryTransaction?). ->Self
Copy the codeDelete any Transaction before the specified point in time that meets the criteria
The cleaning strategy can be coarse or very fine. For example, in apple’s official documents, a relatively coarse cleaning strategy is adopted:
let sevenDaysAgo = Date(timeIntervalSinceNow: TimeInterval(exactly: -604 _800)!)
let purgeHistoryRequest =
NSPersistentHistoryChangeRequest.deleteHistory(
before: sevenDaysAgo)
do {
try persistentContainer.backgroundContext.execute(purgeHistoryRequest)
} catch {
fatalError("Could not purge history: \(error)")}Copy the codeDelete all transactions that are seven days old, regardless of author. In fact, this seemingly crude strategy has virtually no problems in practice.
In this article, we’ll take a more nuanced look at the cleanup strategy, just as we did with Fetcher.
import CoreData
import Foundation
/// Delete the transaction that has already been processed
public struct PersistentHistoryCleaner {
/// NSPersistentCloudkitContainer
let container: NSPersistentContainer
/// app group userDefaults
let userDefault = UserDefaults.appGroup
/// all appActor
let appActors = AppActor.allCases
/// Remove persistent History transactions that have been processed
public func clean(a) {
guard let timestamp = userDefault.lastCommonTransactionTimestamp(in: appActors) else {
return
}
// Get the request for the transaction that can be deleted
let deleteHistoryRequest = NSPersistentHistoryChangeRequest.deleteHistory(before: timestamp)
Delete only transactions generated by members of the App Group
if let fetchRequest = NSPersistentHistoryTransaction.fetchRequest {
var predicates: [NSPredicate] = []
appActors.forEach { appActor in
// Clear transactions created by App Group members
let perdicate = NSPredicate(format: "%K = %@",
#keyPath(NSPersistentHistoryTransaction.author),
appActor.rawValue)
predicates.append(perdicate)
}
let compoundPredicate = NSCompoundPredicate(type: .or, subpredicates: predicates)
fetchRequest.predicate = compoundPredicate
deleteHistoryRequest.fetchRequest = fetchRequest
}
container.performBackgroundTask { context in
do {
try context.execute(deleteHistoryRequest)
// Reset all appActor timestamps
appActors.forEach { actor in
userDefault.updateLastHistoryTransactionTimestamp(for: actor, to: nil)}}catch {
print(error)
}
}
}
}
Copy the codeIn the I set up in the fetcher and cleaner so detailed predicate, because I myself is the use of Persistent in PersistentCloudKitContainer History Tracking function. Cloudkit synchronization will generate a large number of transactions, so you need to filter the objects you operate on more accurately.
CoreData will automatically process and clear all CloudKit synchronized transactions. However, if we accidentally delete any CloudKit Transaction that has not been processed by CoreData, we may cause a database synchronization error and CoreData will clear all current data. An attempt was made to reload data remotely.
So if you’re in
PersistentCloudKitContainer onWhen using Persistent History Tracking, be sure to clear only transactions generated by App Group members.If Persistent History Tracking is used only on PersistentContainer, both fetcher and Cleaner can be filtered less thoroughly.
After creating the PersistentHistoryCleaner, we can choose the time to invoke it according to our actual situation.
If PersistentContainer is used, try a more aggressive cleanup strategy. In PersistentHistoryTrackingManager add the following code:
private func processor(a) {
backgroundContext.performAndWait {
.
}
let cleaner = PersistentHistoryCleaner(container: container)
cleaner.clean()
}
Copy the codeSo after each response NSPersistentStoreRemoteChange notice, will try to remove the Transaction have merged.
But I personally recommend a less aggressive cleanup strategy.
@main
struct PersistentTrackBlogApp: App {
let persistenceController = PersistenceController.shared
@Environment(\.scenePhase) var scenePhase
var body: some Scene {
WindowGroup {
ContentView()
.environment(\.managedObjectContext, persistenceController.container.viewContext)
.onChange(of: scenePhase) { scenePhase in
switch scenePhase {
case .active:
break
case .background:
let clean = PersistentHistoryCleaner(container: persistenceController.container)
clean.clean()
case .inactive:
break
@unknown default:
break
}
}
}
}
}
Copy the codeFor example, when the app is back in the background, it is cleared.
conclusion
You can download the full code for this article at Github.
The following information is of vital importance to this paper:
-
Practical Core Data
This book by Donny Wals is one of my favorite CoreData books of recent times. There is a section on Persistent History Tracking. In addition, his Blog often contains articles about CoreData
-
SwiftLee
Avanderlee’s blog also has a number of great articles about CoreData, including Persistent History Tracking in CoreData. The structure of the code in this article is also affected by this.
Apple built Persistent History Tracking, allowing multiple members to share a single database and keep the UI up to date. Whether you’re building a set of applications, adding appropriate extensions to your App, or simply responding to data from batch operations uniformly, persistent history tracking is a great way to help.
Persistent History Tracking, while a bit of a system burden, is nothing compared to the convenience it brings. In practice, I felt almost no loss of performance.
This article originally appeared on my personal blog, Swift Notepad.