Welcome to Tencent cloud community, get more Tencent mass technology practice dry goods oh ~

The author:
QQ Music Technology team

One, the introduction

When transmitting text with the server, the whole text may fail to be decoded because of different encoding format of a certain character, a byte missing, or an extra byte. In fact, if you can find this character and replace it with another character, then the whole text can be decoded with other characters, and the user may be able to guess what the correct character is on the UI, which is better than the user seeing a blank space.

Data that cannot be parsed with initWithData:encoding: is parsed byte by byte. A branch of the source code is as follows:

while(Retrieval did not exceed file length) {if(1 byte encoding) {/* Correct encoding, continue loop */}else if(2 bytes encoding) {CFStringRef CFSTR = CFStringCreateWithBytes(kCFAllocatorDefault, {byte1, byte2}, 2, kCFStringEncodingUTF8,false);
        if(CFSTR) {/* Correct encoding, continue loop */}else{/* Replace character */}}else if(3,4,5,6 bytes long decoding)... }Copy the code

Replace characters that cannot be parsed. The downside of this approach is that the CFStringCreateWithBytes method allocates a string in heap space, which can easily fragment if the data is too long.

There are two ways to solve this problem: allocate memory in stack space, or allocate a heap space that can be reused.

Second, the study of CFAllocatorRef

From the arguments provided by CFStringCreateWithBytes, the caller can specify the memory allocator. CFAllocatorRef alloc = CFAllocatorRef alloc The allocator to use to allocate memory for the new string. Pass NULL or kCFAllocatorDefault to use the current default The allocator. Let’s look at the data structure of this memory allocator and the differences between the six allocators provided by the system.

CFAllocatorRef data structure

typedef const struct CF_BRIDGED_TYPE(id) __CFAllocator * CFAllocatorRef;
struct __CFAllocator {
    CFRuntimeBase _base;
    CFAllocatorRef _allocator;
    CFAllocatorContext _context;
};
Copy the code

For iOS only, __CFAllocator has only three members. Where CFAllocatorContext _context is the core of the allocator, which can be used to customize the allocation and release callback function:

typedef void * (*CFAllocatorAllocateCallBack)(CFIndex allocSize, CFOptionFlags hint, void *info); typedef void (*CFAllocatorDeallocateCallBack)(void *ptr, void *info); typedef struct { ... CFAllocatorAllocateCallBack allocate; CFAllocatorDeallocateCallBack deallocate; . } CFAllocatorContext;Copy the code

When the system uses this allocator to allocate, release, and redistribute operations, it calls the corresponding callback function to execute them.

Next, look at the source code for a series of allocators provided by the system (consider iOS only).

  • KCFAllocatorMalloc: System allocation and release are essentially malloc(),realloc(),free().
static void * __CFAllocatorCPPMalloc(CFIndex allocSize, CFOptionFlags hint, void *info)
{returnmalloc(allocSize); } static void __CFAllocatorCPPFree(void *ptr, void *info) {free(ptr); }Copy the code

  • This allocator is the same as kCFAllocatorMalloc on iOS, but is different on Mac (malloc and malloc_zone_malloc).

  • KCFAllocatorNull: does nothing, returns NULL. The documentation is mainly used when memory should not actually be freed.

    static void *__CFAllocatorNullAllocate(CFIndex size, CFOptionFlags hint, void *info)
{ returnNULL; }Copy the code

  • KCFAllocatorUseContext: a fixed address that is used only when CFAllocatorCreate() creates the allocator. Indicates that the allocator is created using its own context->allocate method to allocate memory. Because the allocator is also a CF object.

    const CFAllocatorRef kCFAllocatorUseContext = (CFAllocatorRef)0x03ab;
Copy the code

  • KCFAllocatorDefault: this takes the system’s current default allocator, which needs to be combined with the other two apis. Solution: CFAllocatorGetDefault and CFAllocatorSetDefault methods. The system retains allocator twice so that when the default allocator is set, the previous default allocator will not be released. Isn’t there a memory leak here? Think to use with caution).

  • KCFAllocatorSystemDefault: this is the default dispatcher system level, if not call CFAllocatorSetDefault (), use CFAllocatorGetDefault allocator is the (). From the source, the current and kCFAllocatorMalloc no difference (maybe a long time ago because __CFAllocatorSystemAllocate not use malloc implementation. Later compatible, here the story has known welcome to inform)

Custom allocator

After looking at the allocators provided by the system, I found that they are allocating memory in the heap space, there is no suitable memory. It turns out that the system provides another API: CFAllocatorCreate. Consider a custom allocator that returns a fixed amount of memory for reuse when allocating memory.

void *customAlloc(CFIndex size, CFOptionFlags hint, void *info)
{
    return info;
}

void *customRealloc(void *ptr, CFIndex newsize, CFOptionFlags hint, void *info)
{
    NSLog(@"Warning: Memory reallocation occurred");
    returnNULL; // The system returns NULL if the callback is not written. Here's a simple sentencelog. } void customDealloc(void * PTR, void *info) {// The alloc address should be managed externally. } CFAllocatorRef customAllocator(void *address) {CFAllocatorRef allocator = NULL;if (NULL == allocator)
    {
        CFAllocatorContext context = {0, NULL, NULL, NULL, NULL, customAlloc, customRealloc, customDealloc, NULL};
        context.info = address;
        allocator = CFAllocatorCreate(kCFAllocatorSystemDefault, &context);
    }
    return allocator;
}

int main()
{
    char allocAddress[160] = {0};
    CFAllocatorRef allocator = customAllocator(allocAddress);

    CFStringRef cfstr = CFStringCreateWithBytes(allocator, tuple, 2, kCFStringEncodingUTF8, false);
    if(cfstr) { //CFRelease(cfstr); / / here don't release, here the memory allocated is allocAddress stack space, by the system it is good to their own recovery} CFAllocatorDeallocate (kCFAllocatorSystemDefault, (void *) allocator); }Copy the code

The trick here is to reuse the head of the memory using the context’s info. Why is the size of allocAddress 160 bytes? This size just takes the maximum length that CFStringRef needs. If your project needs to reference this method, consider how large the size should be. (Depending on the numBytes parameter value of CFStringCreateWithBytes(), there is some knowledge of byte alignment.)

The CFAllocatorRef created is also in heap space and needs to be freed as well. The system also provides a release API: CFAllocatorDeallocate. Note that the allocator of dealloc must be the same as that of CREATE. Otherwise, it cannot be released, causing memory leakage.

Four, conclusion

The custom allocator makes the allocation of memory operable to some extent. The application scenario in this paper is to return a fixed memory region for reuse when creating objects, avoiding the memory fragmentation problem caused by repeated creation and release. This operability will give you one more solution to your memory problems in the future.

The source code of CFBase was updated on September 11, 2015. The latest source code is also based on iOS9. When writing this kind of low-level code, we need to be extra careful. The author wrote this kind of low-level code because the allocator parameters of CFAllocatorCreate and CFAllocatorDeallocate are different, which leads to memory leakage and requires many tests. Greyscale strategy and switch control are required when publishing to the Internet.

As an extra tip, the default stack size for iOS threads is 512KB (this can get bigger with apple’s new OS and machine, so test it as much as you can). OrignalBytes was originally a temporary variable assigned to the stack, but was later assigned to the heap because the string was so long that the stack overran crash.

Refer to the link

1. github.com/opensource-…

2. gist.github.com/oleganza/78…

3. developer.apple.com/library/pre…

4. developer.apple.com/library/pre…

reading

IOS development of animation time using Skeleton Screen to enhance user perception experience

Has been authorized by the author tencent cloud community released, reproduced please indicate the article source The original link: https://cloud.tencent.com/community/article/383806