preface
The thing is, I studied webP coding on iOS and wrote an article about it. As mentioned in the previous article, webP coding on iOS will take up a lot of CPU resources (if the image pixel is too large, it can be up to 90%). As some logic in the actual project depends on a lot of calculation, the performance of mobile phone may be severely damaged when multi-threaded WebP coding is carried out simultaneously. There was a bit of research devoted to CPU usage, and since there was no good solution, this article is a simple document of the problem, not much practical stuff. Since WebP was launched by Google, it is already well supported at the bottom of Android. The end point of this article will be to compare the Android implementation.
Test the CPU usage of webP codes at both ends
I choose a 2400 × 1350 wallpaper, write a simple Demo of WebP coding on both ends, there is only one button on the interface, click to trigger WebP coding. Images of the test are as follows (Ghost Blade 😜) :
The Android part
- The demo code:
- CPU usage for coding once:
The iOS section
- The demo code:
- CPU usage for coding once:
hypothesized
I have proposed several possible hypotheses for the above test results.
- Does the WebPConfig configuration in the libwebp code differ due to partial differences?
- Android webP coding is encapsulated using Skia, is it because of Skia that there is a difference?
- Is Android webP coding useful for the GPU?
With the above doubts and relevant verification.
Configuration alignment for WebPConfig
Again, from the above mentioned SkWebpEncoder. CPP, you can see that WebPConfig is indeed modified before coding.
Combined with initialization of configuration, lossless indicates whether lossless encoding is lossless. The default is 0, which is lossy. Android apis are lossy by default. So just look at the code block inside if. For method, the default value is 4. In my understanding, it is a trade-off between encoding effect and efficiency. It represents whether certain methods need to be processed during the coding process to ensure the encoding effect. The following is libwebp’s explanation of method, indicating how many parameters will undergo processing, project source code. As the relevant image coding processing only stays at the application level, I cannot well analyze the specific differences among them. If you have this knowledge of friends can leave a message.
There is also a macro in the Skia source code above that defines SK_WEBP_ENCODER_USE_DEFAULT_METHOD to control whether the method is 3. There’s no way to know if it’s true, so I just set method to 3 in my iOS code to test. However, the effect of the test is not different from the default 4.
Skia
On Skia and its integrated image coding logic
Skia is a Google open source C++ two-dimensional graphics library that provides a variety of commonly used apis and runs on a variety of hardware and software platforms. . My understanding of Skia is that it is more of an adaptation platform, adapting different third libraries to a unified API for external use. For example, the compress method of Android Bitmap is used to call encode method SkImageEncoder. CPP (Android API 28)
bool SkEncodeImage(SkWStream* dst, const SkPixmap& src,
SkEncodedImageFormat format, int quality) {
#ifdef SK_USE_CG_ENCODER
(void)quality;
return SkEncodeImageWithCG(dst, src, format);
#elif SK_USE_WIC_ENCODER
return SkEncodeImageWithWIC(dst, src, format, quality);
#else
switch(format) {
case SkEncodedImageFormat::kJPEG: {
SkJpegEncoder::Options opts;
opts.fQuality = quality;
opts.fBlendBehavior = SkTransferFunctionBehavior::kIgnore;
return SkJpegEncoder::Encode(dst, src, opts);
}
case SkEncodedImageFormat::kPNG: {
SkPngEncoder::Options opts;
opts.fUnpremulBehavior = SkTransferFunctionBehavior::kIgnore;
return SkPngEncoder::Encode(dst, src, opts);
}
case SkEncodedImageFormat::kWEBP: {
SkWebpEncoder::Options opts;
opts.fCompression = SkWebpEncoder::Compression::kLossy;
opts.fQuality = quality;
opts.fUnpremulBehavior = SkTransferFunctionBehavior::kIgnore;
return SkWebpEncoder::Encode(dst, src, opts);
}
default:
return false;
}
#endif
}
Copy the codeAs you can see, SkEncodedImageFormat corresponds to the Bitmap.CompressFormat enumeration at the application layer. Different types of Encoder (JPG, PNG, webp) correspond to different lib (ps: JPG corresponds to libjpg, webp corresponds to libwebp… . As an aside, the Skia is designed to look a bit like an adapter, instrumentalized.
Verify webP coding using Skia on iOS
Because Skia does not have an iOS library available by default, consider using its source code and documentation for static library compilation. But in the actual operation found that the compilation will encounter a variety of problems at a time difficult to solve. So here is a reference to someone else has written the library, the original source code address and Skia static library. Use Skia on iOS
I’m just going to go straight to the code here, using Skia for Webp coding.
+ (NSData *)encodeWithSkia:(UIImage *)image quality:(float)q {
SkBitmap orig;
orig.setInfo(SkImageInfo::Make((int)image.size.width, (int)image.size.height, kRGBA_8888_SkColorType, kOpaque_SkAlphaType, nullptr));
orig.allocPixels();
CGColorSpaceRef colorSpace = CGImageGetColorSpace(image.CGImage);
CGFloat cols = image.size.width;
CGFloat rows = image.size.height;
size_t bytesPerRow = CGImageGetBytesPerRow(image.CGImage);
CGContextRef contextRef = CGBitmapContextCreate(orig.getPixels(), // Pointer to backing data
cols, // Width of bitmap
rows, // Height of bitmap
8.// Bits per component
bytesPerRow, // Bytes per row
colorSpace, // Colorspace
kCGImageAlphaNoneSkipLast |
kCGBitmapByteOrderDefault); // Bitmap info flags
CGContextDrawImage(contextRef, CGRectMake(0.0, cols, rows), image.CGImage);
CGContextRelease(contextRef);
auto webpData = SkEncodeBitmap(orig, SkEncodedImageFormat::kWEBP, 90);
SkData* skData = webpData.get();
NSData* webFinalData = [NSData dataWithBytes:skData->bytes() length:skData->size()];
return webFinalData;
}
Copy the codeUnfortunately, in actual tests there was no significant difference in CPU usage between using libwebp directly and skia on the iOS side. The data doesn’t seem to support that Skia is optimized for webP coding.
Webp Coding GPU acceleration?
I’ve always heard that Skia is a graphics engine, and the Internet describes it more in terms of graphics rendering. It’s not too much of a guess, will there be GPU acceleration on Android using Skia’s image coding? However, it can be found from the above research that Skia is more like a platform to integrate different third-party libraries for unification, while Skia itself does not seem to have anything to do with GPU acceleration, at least in the area of image coding.
The idea of GPU acceleration is to split discrete logic (such as traversal) that is repeated multiple times into small parallel computing blocks to improve performance. This may be a shallow understanding, but it can negate the possibility that the above webP coding can use gpu. Although the coded computation is a iterated process, there is no room for splitting the computation in terms of outer calls, unless libwebp itself supports gPU-accelerated capabilities.
The last
This article is just a record of my research on the high CPU usage of webP coding for iOS, and there is nothing substantial. It can be proven that both ends of webP coding are performed on the CPU. The low CPU occupancy of Android can only be guessed to be due to the limitation of WebP coding at the bottom of Android system. This guess is also due to the low CPU usage of Android, the relative coding time will also be longer (ps: this phenomenon may also be related to device configuration, performance). Finally, if there is a friend to understand the relevant knowledge, you can leave a message to discuss ~