Some time ago there was a demand for a real-time preview of the Flutter camera, using web API, which transmits Motion JPEG data format, and a few days ago there was a need to expand another brand of camera which transmits RGB data format.

Because the data of Motion JPEG is a complete JPEG Image, we only need to wrap the data and put it into image.memory (), while the RGB data is the primary color data of each pixel, which cannot be directly displayed, so it needs to be converted into a qualified Image encoding format.

According to the official image.memory () document

which can be encoded in any of the following supported image formats: JPEG, PNG, GIF, Animated GIF, WebP, Animated WebP, BMP, and WBMP

The following image formats are supported for encoding: JPEG, PNG, GIF, Animated GIF, WebP, Animated WebP, BMP, and WBMP

So we can convert it to BMP format for presentation. Don’t ask why BMP format, that’s all I know

Cut the crap. Look at the code

BMP image format structure can be divided into two parts, image header data and image body data

1. Create a new oneclassTo specify the image header data

class BMPHeader {
  int w;
  int h;

  late Uint8List _bmp;
  late int _headerSize;

  BMPHeader(this._width, this._height) : assert(_width & 3= =0) {
    _headerSize = 54;
    int fileLength = _headerSize + w * h * 3; // File length
    _bmp = new Uint8List(fileLength);
    ByteData bd = _bmp.buffer.asByteData();
    bd.setUint8(0.0x42);
    bd.setUint8(1.0x4d);
    bd.setUint32(2, fileLength, Endian.little); // File length
    bd.setUint32(10, _headerSize, Endian.little); // Picture data starts
    bd.setUint32(14.40, Endian.little); // The length of the header
    bd.setUint32(18, w, Endian.little); // The width of the image
    bd.setUint32(22, -h, Endian.little); // The height of the image
    bd.setUint32(26.1, Endian.little); // The target device indicates the number of bit planes
    bd.setUint32(28.24, Endian.little); // Color coding format
    bd.setUint32(34, _width * _height, Endian.little); // bitmap size
  }

  Uint8List appendBitmap(Uint8List bitmap) {
    int size = _width * _height * 3;
    assert(bitmap.length == size);
    _bmp.setRange(_headerSize, _headerSize + size, bitmap);
    return_bmp; }}Copy the code

2,RGBData import in, that is, the image body data, where the image display content

The obtained data includes not only the required data, but also the width and height. Pass the width and height into BMPHeader(w, h) to determine the image header. However, THE BMP format requires GBR data, so we need to convert it again and pass it in.

int w = rgbData.width;
int h = rgbData.heigh;
Uint8List rgbF = rgbData.frameData;
List<int> gbrf = [];
for(int i = 0; i<rgbF.length/3; i++) {
    gbrf.add(f[i*3+2]);
    gbrf.add(f[i*3+1]);
    gbrf.add(f[i*3]);
}

BMPHeader header = BMPHeader(w, h);
Uint8List bmp = header.appendBitmap(Uint8List.fromList(gbrf));
Copy the code

Well, we’ve seen how to solve this problem, and that’s it.

Bridge bean sack, know how, know why, problem solved, we need to know how to solve the problem.

Description of BMP format

There are already many descriptions of BMP format structure on the Internet. “DETAILED explanation of BMP Image Data Format”, “Analysis of different BMP bitmaps and palettes” have completely introduced how BMP is encoded, and we only need to understand some of them.

Each setUint is passed in the header with three values representing: position, value, and byte order

The first two are used to specify the value of each position, and the third value can be read by yourself if you are interested.

1. Length of the file header

According to the above detailed explanation, in fact, BMP files are divided into four parts:

  • File header: information such as the format and size of a file.
  • Information header: image data size, bit plane number, compression mode, color index and other information;
  • Palette: Optional. If you use an index to represent an image, a palette is a map of the indexes and their corresponding colors
  • Bitmap data: image data

The file header length is 14, the information header length is 40, the palette data is optional, and if there is no palette, it adds up to exactly 54.

2. Why is height negative

In the head of the image needs to specify the width and height of the picture, so not only specify the width and height, but also the direction of the image storage.

bd.setUint32(18, w, Endian.little); // The width of the image
bd.setUint32(22, -h, Endian.little); // The height of the image
Copy the code

While in BMP image rendering:

  • The first point is the lower left corner, and the first row is the most downward
  • Each line is scanned from left to right, line by line, up

So when we store the data, we need to store it from the bottom up, and render it backwards, so two negatives make a positive, and we get the right image

3. Color coding format

Specify the encoding format of the color to parse the bitmap data correctly. For 1, 4, 8, you need to specify the palette and the length of the header is 54 + the length of the palette. For 16, 24, 32, you don’t need a palette and the length of the header is 54.

bd.setUint32(28.24, Endian.little);
Copy the code

Let’s look at the description of each color value:

  • 1: A monochrome image. A palette of two colors, usually black and white
  • 4:16 color diagram
  • 8:25 6 color maps, usually referred to as grayscale images
  • 16:6 4K images, generally without a color palette, each two bytes in the image data represent a pixel, and five or six bits represent an RGB component
  • 24:16m true color images, generally without a color palette, each 3 bytes in the image data represents a pixel, each byte represents an RGB component
  • 32:4g true color, generally no color palette, each 4 bytes represents a pixel, compared to 24 bit true color, adds a transparency, namely RGBA mode

The color palette is set to 8

If we need a 256 color map, which is a grayscale map, we need to write to the palette when we set position 28 to 8

for (int rgb = 0; rgb < 256; rgb++) {
  int offset = _headerSize + rgb * 4;
  bd.setUint8(offset + 3.255); // A
  bd.setUint8(offset + 2, rgb); // R
  bd.setUint8(offset + 1, rgb); // G
  bd.setUint8(offset, rgb); // B
}
Copy the code

Here we have a loop that writes four lengths each time, 256 times, so it’s 1024, so we need the file length to be like this

int fileLength = _headerSize + w * h * 3 + 1024;
Copy the code

But the image is in grayscale, so it looks like this:

Set the color palette to 24

If you need a 16M true color map, place it28Provisions for24The three bytes become one pixel, but the BMP color format isBGRWhen we pass the image data directly in, it looks like this:

So what we need to do is convert RGB to GBR and pass it into BMP, which is very simple, is to group the data every three bytes, and exchange the first and third bits.

Uint8List rgbF = rgbData.frameData; // Raw RGB data
List<int> gbrf = []; // The converted GBR data
for(int i = 0; i<rgbF.length/3; i++) {
    gbrf.add(f[i*3+2]);
    gbrf.add(f[i*3+1]);
    gbrf.add(f[i*3]);
}
Copy the code

The effect will look like this:

Similarly, for other data formats, we need to set a different palette, change the headerSize based on the length of the palette, and not much else to say.

conclusion

This demand, in each link to card for a long time, or basic knowledge is not enough. What image format is supported from the image.memory of flutter? How to convert pictures to RGB? What image format should I use? What is the encoding structure of BMP pictures? Why is it coming out in the wrong color? How to set BMP palette? The shift operation when GBR transform and so on, all involve a lot of basic knowledge. But each requirement is also to fill in the basic knowledge, each requirement will be more and more smooth.