I promised to send a summary article of iOS13 wechat voice reminder of receipt to account, but I haven’t written it yet. On the one hand, it has not been online, on the other hand, it has been engaged in the red envelope project. Now both projects are successfully online, and I can finally stop to summarize the specific scheme and problems encountered in the development for everyone.
The background,
With the release of iOS13 at WWDC2019, this two-year-old summary of wechat iOS voice reminder development is no longer applicable. The specific reason is that In iOS13 (specifically compiled using XCode11) Apple no longer allows PushKit for non-voip phone scenarios. With iOS13, apple is more concerned than ever about user privacy and the device’s battery life, so PushKit’s capabilities have been rolled back. If you want to use PushKit, you need to access the CallKit interface. As a result, when receiving a Voip Push, a full-screen interface for receiving and making calls will be pulled up. Those who have published apps in China should know that pulling up this interface is not allowed by Party A. This article summarizes the voice broadcast migration scheme in iOS13 and some issues to be aware of. This feature has been added to wechat’s 7.0.10 version.
Ii. Technical scheme
Notification Service Extension
The new solution mainly utilizes the Notification Service Extension(NSE for short) introduced by apple in iOS10. When the payload of apns carries a value of 1, the NSE code is entered. The biggest difference from the Voip solution is that NSE cannot wake up the main application, nor can it access the file space of the main application, and can only handle the corresponding logic in the Extension process. In THE NSE, developers can change the content of the notification, use offline synthesis or download from the background to generate the content to be broadcast, and customize the notification ring tone to achieve the purpose of voice notification. The NSE solution is also the solution recommended by apple in WWDC2019 Session707.
UNNotificationSound
In NSE, you can assign the Sound property in UNNotificationContent to play a piece of custom audio when the notification pops up.
// The sound file to be played for the notification. The sound must be in the Library/Sounds folder of the app’s data container or the Library/Sounds folder of an app group data container. If the file is not found in a container, the system will look in the app’s bundle.
The document clearly describes the audio file storage path, and read priority:
- In the Library/Sounds folder in the main application
- Library/Sounds folder in the AppGroups shared directory
- In the main bundle
The customized ringtones can be in AIFF, WAV, and WAV formats. The ringtone length must be shorter than 30 seconds; otherwise, the system will play the default ringtone.
And because it is a notification ring, the sound is the default with mute switch, do not need to use the same as before to judge mute switch black magic (black magic in different models occasionally misjudgment).
AppGroups
Since we are customizing the ring tone in NSE, we cannot access the file paths 1 and 3. You can only store synthesized or downloaded audio files in your Library/Sounds folder under AppGroups, which requires Capablities, Way to through NSFileManager containerURLForSecurityApplicationGroupIdentifier: visit AppGroups root directory.
Speech synthesis
Wechat’s receiving-to-payment voice relies on our own powerful offline voice synthesis library. The payload of APNS carries the text content to be synthesized. After the WAV audio file is generated through the offline voice synthesis Library, the file is written to the Library/Sounds folder of AppGroups. Finally, change the UNNotificationSound property to make the notification broadcast a custom receipt-to-account speech.
If some small enterprises themselves do not have the ability to offline synthesis (see several powerful offline synthesis services on the market are required to charge), you can use online synthesis and then download through HTTP, Ifly.com and wechat have provided free services. The disadvantage of this scheme is that it depends on the background and the current network environment, which may lead to the problem of untimely news broadcast. If within 30 s and cannot be successful, now need in serviceExtensionTimeWillExpire method for processing, out of the best scheme is playing a default voice.
Problems encountered in the development process
Message playback queue
One problem with the NSE scheme is that when the client receives multiple broadcast notifications in a short period of time, the later notifications will override the earlier ones, resulting in incomplete broadcast of the earlier ones, which is quite confusing for merchants. Therefore, you need to add a message queue to add all notifications that need to be broadcast to the queue. After the previous message is played, the next message is played. The playback time of an audio file can be pushed by the background through the payload. If it is a synthesized WAV of its own, the playback time can be calculated by the value of (audio size – audio header)/(sampling frequency x sampling accuracy x number of channels).
Multithreading problem
Note that the NSE code logic is not executed on the main thread. Apple’s design is very reasonable. On the one hand, it avoids the problem of other application interfaces getting stuck in the foreground due to code design errors in NSE. On the other hand, the main project has been suspended or killed at this time, and should not be given the execution time of the main thread to NSE.
Therefore, when dealing with the message playback queue mentioned above, as well as the logic involving file reading and writing, we need to lock the corresponding code logic, otherwise there will be multithreading problems.
Message to heavy
Since the payment message has higher requirements on reachability and real time than the ordinary message, the dual channel was used to reduce the occasional message loss and delay of Voip in the original design. In previous Voip schemes, the client receives two Voip messages that are the same, and records the single number in the payload to cancel the message. In NSE, however, the client cannot do this voluntarily. The fundamental reason is that NSE is designed only to modify the NotificationContent, not to prevent notifications from popping up. This can be seen in the timeout handling documentation:
If your didReceive(_:withContentHandler:)method takes to long to execute its completion block, the system calls this method on a separate thread to give you one last chance to execute the block. Use this method to execute the block as quickly as possible. Doing so might mean providing some fallback content. For example, If your extension is still downloading an image file with the intent of attaching it to the notification’s content, You might update the notification’s alert text to indicate that an image is being downloaded. If you fail to execute the completion block from the didReceive(_:withContentHandler:) method in time, The System displays the Notification’s original content.
If you didn’t call a handler method within 30 s, and there is no implementation serviceExtensionTimeWillExpire method, then the system will help you to take the initiative to push back on to the client the original content.
The solution here is to have the background, double-channel triggered APNS messages with the same APns-collapsor-ID on the Requestheader, and the later notifications overwrite the previous ones. The problem is that even though the user sees a single message, the sound is played twice. This is done by recording the number of messages that have been played, and then replaying the number of repeated messages.
Third, summary
In retrospect, NSE was a much more elegant solution than Voip, and after the migration, the total amount of code was much less than Voip, so why didn’t we choose this solution? In fact, there are historical reasons for it: on the one hand, NSE is a new Extension that appeared after iOS10. When I made the first version of the scheme, iOS10 was just released, so I didn’t have a deep understanding of it. On the other hand, wechat did not have the ability to synthesize voice offline at that time, so it could only pull online synthesized voice through Cgi, and wechat Extension did not have the ability to request Cgi at that time. One of the best things about switching to NSE is that the voice broadcast and mute switch work perfectly together, and the annoying message delay problem has also been improved.
Iv. Relevant materials
Advances in App Background Execution – 2019
UNNotificationServiceExtension