Original address: medium.com/flutter-com…

Suragch.medium.com/

Published: December 26, 2020 -25 minutes to read

If you can understand bytes, you can understand anything.

preface

I started working on this topic because I was studying how to communicate between Dart and PostgreSQL database servers. It turned out to be much lower than I had expected. I thought I’d write an essay to explain some of the new things I’ve learned. Well, after three full days of writing, that essay had turned into one of the most in-depth explanations I’d ever written on an already niche subject. Although the article is long, I don’t think you’ll get bored, and there’s a good chance you’ll learn a new thing or two about Dart, even if you’ve been using it for a while. I certainly did the same. As always, if you find anything wrong, please let me know. That’s how I learn. And try out your own code examples. That’s how you learn.

This is the latest version of Dart 2.10.

Bytes and bytes

Anyone who has read this article knows that a byte is eight bits.

00101010
Copy the code

This 8-bit byte has a value, in this case, 42, which is just an integer. Now combine this knowledge with the fact that all binary data is just a sequence of bytes, which means that any binary data can be represented as a list of integers in Dart.

List<int> data = [102.111.114.116.121.45.116.119.111.0];
Copy the code

These bytes may come from a file, a bitmap image, an MP3 recording, a memory dump, a network request, or the character code of a string. They’re all bytes.

Be more efficient

In Dart, the int type defaults to a 64-bit value. This is eight bytes. Here is the number 42, this time showing 64 bits for your reference.

0000000000000000000000000000000000000000000000000000000000101010
Copy the code

If you look closely, you may notice that many bits are not being used.

An int can store 9223372036854775807 values, but a maximum of one byte is 255. This is definitely a case of using a sledgehammer to break open a nut. Now think of a Megabyte file as a list of ints. Your problem is a million times bigger.

That’s what the Uint8List is for. This type is basically the same as List

, but for large lists, Uint8List is more efficient than List

. Uint8List is a list of integers whose values are only eight bits, or one byte. The U of the Uint8List means unsigned, so values range from 0 to 255. This is great for representing binary data!

Note: Negative numbers in binary

Have you ever wondered how to represent negative numbers in binary? Well, the way to do it is to make the leftmost seat 1 for a negative number, 0 for a positive number. For example, these 8-bit signed integers are all negative numbers because they all start with 1.

11111101
10000001
10101010
Copy the code

On the other hand, the following 8-bit signed integers are all positive because the leftmost bit is 0.

01111111
00000001
01010101
Copy the code

You might think that if 000001 is +1, then 10000001 should be -1. Otherwise you’re going to have two zeros. The 10000000 and 00000000 solutions are to use a system called “the complement of two”. If you’re interested, the video below explains it perfectly.

www.youtube.com/watch?v=mRv…

In Dart, you can already use the int type to represent signed values, both positive and negative. However, if you want to use a list of 8-bit signed integers, you can use type Int8List (note that there is no U). This allows values from -128 to 127. This isn’t particularly useful for most uses of representation byte data, so we’ll stick with the unsigned integers in the Uint8List.

willList<int>convertUint8List

Uint8List is part of dart: TypeD_Data, one of dart’s core libraries. To use it, add the following imports.

import 'dart:typed_data';
Copy the code

Now you can convert the previous List

List to Uint8List using the fromList method.

List<int> data = [102.111.114.116.121.45.116.119.111.0];
Uint8List bytes = Uint8List.fromList(data);
Copy the code

If any of the values in the List

List are outside the range of 0 to 255, they are wrapped. You can see this in the example below.

List<int> data = [3.256.2 -.2348738473];
Uint8List bytes = Uint8List.fromList(data); 
print(bytes); // [3, 0, 254, 169]
Copy the code

256 is one more than 255, so it becomes a 0, and the next number, minus 2, is less than 0, so it goes around to the top two positions, so it becomes 254. And I don’t know how many times 2348738473 was wrapped, but it eventually became 169.

If you don’t want this wrapping to happen, you can use the Uint8ClampedList instead. This will clamp all values greater than 255 to 255 and all values less than 0 to 0.

List<int> data = [3.256.2 -.2348738473];
Uint8ClampedList bytes = Uint8ClampedList.fromList(data);
print(bytes); // [3, 255, 0, 255]
Copy the code

This time the last three values are clamped.

Create a Uint8List

In the above method, you create a Uint8List by converting a List

. What if you just want to start with a list of bytes? You can do this by passing the length of the list to the constructor, like this.

final byteList = Uint8List(128);
Copy the code

This creates a fixed-length list where all 128 values are 0. Print byteList and you will see:

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]Copy the code

Count them at random. Make sure there are 128.

Modify a list of bytes

How to modify the Uint8List value? Just like modifying a normal List.

byteList[2] = 255;
Copy the code

Printing byteList again shows that the value of index 2 has changed.

[0, 0, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]Copy the code

The random-access nature of lists makes it quick and convenient to modify byte values at any index location.

Growable list

Your list above is fixed length. If you try to do something like this.

final byteList = Uint8List(128);
byteList.add(42);
Copy the code

You get the following exception.

Unsupported operation: Cannot add to a fixed-length list
Copy the code

If you want to do something like collect bytes from a stream, that’s not very convenient. To make a list of growable bytes, you need a BytesBuilder.

final bytesBuilder = BytesBuilder();
bytesBuilder.addByte(42);
bytesBuilder.add([0.5.255]);
Copy the code

You can add a single byte (type int) or a List of bytes (type List

).

You can use the toBytes method like this when you want to convert builder to Uint8List.

Uint8List byteList = bytesBuilder.toBytes();
print(byteList); // [42, 0, 5, 255]
Copy the code

There’s more than one way to look at bits

Bits and bytes are just a lot of zeros and ones. If you can’t explain them, they don’t mean much.

0011111100011101110011011101101110011111100110011011110000010101001110100010101010000111100101101111001111011100000101011001101111010101101000011101100101000111100000100100011111101011111110100111010011001100000000000111111010010010000000001111010001100000 
Copy the code

For the Uint8List, we interpret it as a number between 0 and 255 every 8 bits.

00111111 00011101 11001101 11011011 10011111 10011001 10111100 00010101 00111010 00101010 10000111 10010110 11110011 11011100 00010101 10011011 11010101 10100001 11011001 01000111 10000010 01000111 11101011 11111010 01110100 11001100 00000000 01111110 10010010 00000000 11110100 01100000 

Copy the code

You can reinterpret these same values as signed integers from -128 to 127. That’s what we’re doing in Int8List. But there are other ways to look at the data. For example, instead of using 8-bit blocks, you can think of data as a list of 16-bit blocks.

0011111100011101 1100110111011011 10011111100110011001 10111100000101 0011101000101010 1000011110010110 1111001111011100 0001010110011011 1101010110100001 1101100101000111 1000001001000111 1110101111111010 0111010011001100 0000000001111110 1001001000000000, 1111010001100000,Copy the code

Dart: The typeD_data library also has corresponding types. Use the Uint16List to handle unsigned integers from 0 to 65,535 or Int16List to handle signed integers from -32,768 to 32,767.

It didn’t stop there. You can also interpret the same data as a list of 32-bit values.

00111111000111011100110111011011 10011111100110011011110000010101 00111010001010101000011110010110 11110011110111000001010110011011 11010101101000011101100101000111 10000010010001111110101111111010 01110100110011000000000001111110, 10010010000000001111010001100000,Copy the code

Or 64-bit values.

0011111100011101110011011101101110011111100110011011110000010101 
0011101000101010100001111001011011110011110111000001010110011011 
1101010110100001110110010100011110000010010001111110101111111010 
0111010011001100000000000111111010010010000000001111010001100000
Copy the code

Even 128-bit numbers.

001111110001110111001101110110111001111110011001101111000001010100111010001010101000011110010110111100111101110000010101 10011011 110101011010000111011001010001111000001001000111111010111111101001110100110011000000000001111110100100100000000011110100 01100000Copy the code

Dart has all of these types.

  • Int32List
  • Uint32List
  • Int64List
  • Uint64List
  • Int32x4List(128)

For large amounts of data, using the specific List type you need is more efficient than using List

.

The byte view in Dart

Dart uses ByteBuffer to back up raw binary data. The types you saw in the previous section all implement a class called TypedData, which is just a generic way to view the data in ByteBuffer. This means that types like Uint8List, Int32List, and Uint64List are all just different ways to view the same data.

We will use the following four-byte list in the following example.

00000001 00000000 00000000 10000000
Copy the code

In decimal form, the list looks like this.

1, 0, 0, 128
Copy the code

Start by creating the list in Dart as before.

Uint8List bytes = Uint8List.fromList([1.0.0.128]);
Copy the code

As with any form of TypedData, you can get access to the underlying ByteBuffer of the Uint8List by accessing the Buffer property.

ByteBuffer byteBuffer = bytes.buffer;
Copy the code

Unsigned 16-bit view

Now that you have a byte buffer, you can do this by using as… Method to get a different byte view, this time asUint16List.

Uint16List sixteenBitList = byteBuffer.asUint16List();
Copy the code

What do you think the value would be before printing sixteenBitList to view the contents?

Think you know the answer? Ok, print the list.

print(sixteenBitList);
Copy the code

On my Mac, the results were as follows:

[1, 32768]
Copy the code

That’s weird. Because the original value was:

00000001 00000000 00000000 10000000 100 128Copy the code

I was expecting them to be grouped into 16-bit chunks, like this.

0000000100000000 0000000010000000
256              128
Copy the code

Instead, we got this.

1 32768 0000000000000001 1000000000000000Copy the code

Keep that in mind. Let’s examine the 32-bit view.

Unsigned 32-bit view

I’ll start with a list of bytes containing the decimal values 0, 1, 2, 3. So we can see if they’re in the same order. For clarity, here is the 8-bit binary form of the original list.

00000000 00000001 00000010 00000011 01 2 3Copy the code

Now run the following code.

Uint8List bytes = Uint8List.fromList([0, 1, 2, 3]);
ByteBuffer byteBuffer = bytes.buffer;
Uint32List thirtytwoBitList = byteBuffer.asUint32List();
print(thirtytwoBitList);
Copy the code

This time you have a 32-bit view in the underlying buffer from the original Uint8List. The print statement displays the value 50462976, which in 32-bit binary is.

00000011000000100000000100000000
Copy the code

Or if you add spacing (to help view parts).

00000011 00000010 00000001 00000000
Copy the code

This is the exact opposite of the original order! What’s going on?

Endianness

You don’t usually need to think about this kind of thing when you’re just happily building a Flutter application, but when you’re as low-key as we are today, you’re at war with the machine architecture.

Some machines arrange the bytes within a byte block (whether a 2-byte block, a 4-byte block, or an 8-byte block) in forward order. This is called great grace because the most important byte comes first. This is usually what we expect, because we read the numbers from left to right, starting with the largest part of the number.

Other machines, however, arrange the bytes of a chunk in reverse order. This is called little Endian. While this may seem counterintuitive, it actually works at the machine architecture level. The problem comes when big Ndean meets little Ndean. So that’s what we have up here.

Before I get into this, you might like to watch this video on Endianness. Good explanation (if you can get past the initial short story).

www.youtube.com/watch?v=_wk…

What are the causes of poor communication

When I run this code

Uint8List bytes = Uint8List.fromList([0.1.2.3]);
ByteBuffer byteBuffer = bytes.buffer;
Uint32List thirtytwoBitList = byteBuffer.asUint32List();
print(thirtytwoBitList);
Copy the code

Dart puts these four bytes in the list.

00000000 00000001 00000010 00000011
Copy the code

And handed them over to my computer. Dart knew that 000000 was first, 00000001 was second, 00000010 was third, and 00000011 was last. My computer knows this, too.

Dart then asks my computer to get a 32-bit partitioned view of the byte list from the byte buffer in my computer’s memory. My computer is very delighted, and return the: “0000000011000000100000000100000000”.

00000011000000100000000100000000
Copy the code

Well, it turns out my MacBook has a little Endian architecture. My computer still knows that 000000 is the first one, 000001 is the second, 000010 is the third, and 000011 is the last. However, when the print statement calls the toString method somewhere, it interprets the 32 bits as a single integer in great Endian format. I can’t say I blame it.

Check endianness in Dart

If you want to know what the Endian architecture of your system is, you can run the following code in Dart.

print(Endian.host == Endian.little);
Copy the code

If printed to true, then you also have a small Endian machine. Otherwise, it’s a Big Endian machine. But yours might be the Little Endine, since most personal computers now run on little Endine.

If you look at the source code for the Endian class, you’ll see that the way it checks the host architecture is very similar to what caused my initial surprise.

Class Endian {
  ...
  static final Endian host =
      (new ByteData.view(new Uint16List.fromList([1]).buffer))
        .getInt8(0) = =1Little big. }Copy the code

It represents 16 bits of 1, and then sees if the first eight bits are 1, and if so, it’s little Endian. And that’s because little Endian put the 16 bits in this order.

00000001, 00000000,Copy the code

And this is how the Grand Endyne was ordered.

00000000, 00000001,Copy the code

What do I care?

You might think you don’t need to worry about this because you’re not going to be so close to the machine hardware. However, Endianness appears in other situations as well. Basically any time you work with shared data larger than 8-bit chunks, you need to pay attention to endianness.

For example, if you watched the video I linked to above, you know that different file formats use different Endian encodings.

  • JPEG (large end)
  • GIF (small end)
  • PNG (big end)
  • BMP (small end)
  • Mpeg-4 (large end)
  • Network data (big end)
  • Utf-16 text file (big or small – watch this video)

So even if you know the endianness of the raw byte data you’re working on, how do you convey that to Dart so you don’t have the same misrepresentation problems you saw me have earlier?

Read on to find out.

Processing endianness in Dart

Once you’ve got your raw data as some sort of TypedData like Uint8List, you can use the ByteData class to perform random access read/write tasks on it. This way you don’t have to interact directly with BytesBuffer. ByteData also allows you to specify the endianness of the data you want interpreted.

Write the following lines of code.

Uint8List byteList = Uint8List.fromList([0.1.2.3]);
ByteData byteData = ByteData.sublistView(byteList);
int value = byteData.getUint16(0, Endian.big);
print(value);
Copy the code

That’s what it does.

  • You get it from a list of integersUint8ListType of data. It’s the same as what you saw before.
  • Then you useByteDataTo wrap this data. A view (or “sublist” view) is one way to view data in a buffer. The reason it’s called a sublist is that you don’t need to have a view of the entire buffer, because the buffer can be very large. You can view a smaller range of bytes in the buffer. But in our case,byteListThe buffer contains only these four bytes, so we need to look at the entire buffer.
  • After that, you get from the index0Start accessing 16 bits (or two bytes), where the index is incremented by one byte. If you choose1As an index, you will get the second and third bytes if you choose2As an index, you get the third and fourth bytes. In this case, anything higher than2An error is thrown because only four bytes are available.
  • Finally, you also tell Dart that you want to index from0The first two bytes are interpreted as greater Enddian. This is actuallyByteDataDefault value, so you don’t have to use this parameter.

Run this code and you’ll see that the output is 1, which makes sense because

00000000, 00000001,Copy the code

Interpreted in big-endian mode, the value is 1.

Do it again and replace endia.big with endia.little. Now when you run this code, you’ll see 256 because it interprets the second byte (000001) as before the first byte (00000000), so you get

0000000100000000
Copy the code

This is not the data you want, so change the code back to endia.big.

Set the bytes in ByteData

In the top part, you get bytes. You can also set the bytes while specifying endianness in much the same way you get the bytes. Add the following line to what you have written above.

byteData.setInt16(2, 256, Endian.big);
print(byteList);
Copy the code

Here you set two bytes (value: 256), starting with index two in the buffer. Print it out and you’ll see it.

[0, 1, 1, 0]
Copy the code

Does that make sense? The previous buffer contains:

[0, 1, 2, 4]
Copy the code

But you changed the last two bytes to 256, which in binary is:

00000001, 00000000,Copy the code

Or 1 and 0, when interpreted in terms of the greater Endian.

Ok, so much for endianness. Most of the time, you can accept the default British translation and not think about it.

Hexadecimal and binary

So far, I’ve written all binary numbers in this article as strings like 10101010, but it’s not very convenient to use this format when writing code because Dart’s default for integers is 10 base.

int x = 10000000;
Copy the code

That’s 10 million, not 128. : (

While it may be difficult to write binary directly, it is easy to write hexadecimal values directly in Dart. Just add 0x to the hexadecimal number, like that.

int x = 0x80;  // 10000000 binary
print(x);      // 128 decimal
Copy the code

Because the relationship between binary and hexadecimal is straightforward, this makes converting between the two a simple task. If you use binary a lot, the conversion table may be worth remembering.

hex  binary 
----------
0    0000
1    0001
2    0010
3    0011
4    0100
5    0101
6    0110
7    0111
8    1000
9    1001
A    1010
B    1011
C    1100
D    1101
E    1110
F    1111
Copy the code

Byte values will always be two blocks of hexadecimal values (or technically called 4-bit blocks of Nibbles, oh, the humor of those early programmers).

Let me give you a few more examples. If you override binary and just look at hexadecimal, can you find binary without looking at it?

hex       binary 
----------------
80        1000 0000
2A        0010 1010
F2F2F2    1111 0010 1111 0010 1111 0010
1F60E     0001 1111 0110 0000 1110
Copy the code

As a quick tidbit, the third value is the RGB hexadecimal value Medium uses for the gray shadow used as the background color of the code block. And the last one is the Unicode code point of a smiley face with sunglasses 😎.

Convert hexadecimal and binary strings

In Dart, an interesting trick that you may not know is that you can convert between different cardinals and get a string representation of a value.

Here is an example of converting a decimal number to a hexadecimal and binary string form.

String hex = 2020.toRadixString(16).padLeft(4.'0');
print(hex); // 07e4
String binary = 2020.toRadixString(2);
print(binary); / / 11111100100
Copy the code

Note: “Radix “means Radix.

  • Radix just means Radix.
  • The first example takes a decimal number2020, converts it to base -16 (that is, hexadecimal) and ensures length 4. This makes the7e4Turned out to be07e4.
  • The second example is decimal2020Converted toStringFormat binary.

You can use int.parse to convert in another way.

int myInt = int.parse('07e4', radix: 16);
print(myInt); / / 2020
myInt = int.parse('11111100100', radix: 2);
print(myInt); / / 2020
Copy the code

Convert Unicode strings to bytes and back again.

When I say bytes here, I’m just exponents. The String type in Dart is a list of Unicode numbers. Unicode numbers are called code points and can be as small as 0 or as large as 10FFFF. Here’s an example.

Character Unicode code point (hex) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- - H 48 65 6 c l e l 6 c o f f60e 😎 1Copy the code

The code point words Dart uses are runes. You can see that they look like this.

With Runes codePoints = 'Hello 😎. With Runes; print(codePoints); // (72, 101, 108, 108, 111, 128526)Copy the code

That’s the decimal version of their hexadecimal value.

A rune is an iterable set of int values. However, like the List

and Uint8List you saw at the beginning of this article, 64-bit or even 32-bit text storage isn’t efficient when most characters in the English-speaking world (except emoticons like 😎) only need 8 bits. Even most of the thousands of characters in existence can be encoded in fewer than 16 bits.

For this reason, most people use 8-bit or 16-bit encoding systems to encode Unicode values. When a Unicode value is too large to fit in 8 or 16 bits, the system uses a special trick to encode larger values. The 8-bit encoding system is called UTF-8. The 16-bit encoding system is called UTF-16.

The encoding of UTF-8 and UTF-16 is super interesting, especially the tricks they use to encode large Unicode code points. If you’re interested in this sort of thing (and you probably are if you’re still reading), you should definitely watch the video below. This is the best video I’ve seen on the subject.

www.youtube.com/watch?v=HhU…

Utf-16 conversion in Dart

While Unicode code points can be used as an Iterable when you query runes, Dart internally uses UTF-16 as the actual encoding for strings. These 16-bit values are called code units rather than code points.

The conversion to get utF-16 code units from a string is simple.

List<int> codeUnits = 'Hello 😎'.codeUnits;
print(codeUnits); // [72, 101, 108, 108, 111, 55357, 56846]
Copy the code

Notes.

  • You might think, “Hey, oneintIt’s 64 bytes, not 16!” This is just how it represents you externally after you’ve done the transformation.intThe type should be generic, and you shouldn’t care how many bytes it uses. Internally, a string is a list of 16-bit integers.
  • As you can see, it requires two UTF-16 values to represent 😎.55357and56846, that is,D83DandDE0EIn hexadecimal. These two numbers are called substitution pairs, and if you watched the video above, you’ll know all about it.
Decimal Hex Binary -------------------------------- 55357 D83D 1101100000111101 (high surrogate) 56846 DE0E 1101111000001110 (low surrogate) String Hex Binary ---------------------------------------- F60E 0000 1111 0110 0000 1110 + 10000 0001 0000 0000 0000 0000 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 😎 f60e 1 0001 1111 0110 0000 1110Copy the code

Since each UTF-16 code unit is the same length, it also makes it super easy to access any code unit through its index.

int index = 0;
int codeUnit = 'Hello 😎'.codeUnitAt(index);
print(codeUnit); / / 72
Copy the code

Converting from code units back to strings is also easy.

List<int> codeUnits = 'Hello 😎'.codeUnits;
final myString = String.fromCharCodes(codeUnits);
print(myString); / / Hello 😎
print(String.fromCharCode(72)); // H
Copy the code

While this is all well and good, there are some problems with UTF-16-based string manipulation. For example, if you delete only one code unit at a time, you are likely to forget substitution pairs (and morpheme clusters). When this happens, the smile is not so funny. Read the following articles to learn more about this topic.

  • Use Unicode and Grapheme clusters in Dart.
  • The Dart string operates correctly 👉

Convert utF-8 strings

When Dart uses UTF-16 encoding, the Internet uses UTF-8 encoding, which means that when you transfer text over the network, you need to convert your Dart string to a UTF-8 encoded list of integers. This means that when you transfer text over the network, you need to convert your Dart string to a UTF-8 encoded list of integers. Again, keep in mind that this is just a list of 8-bit binary numbers, specifically a Uint8List. The nice thing about 8-bit values is that you don’t need to worry about endianness.

If you watched the video I linked above and now understand how UTF-8 encoding works, you can write your own encoders and decoders. However, you don’t need to do this because they are already available in Dart’s Dart: Convert library.

Import the library like this.

import 'dart:convert';
Copy the code

Then you can simply convert the string to UTF-8 like this.

Uint8List encoded = utf8.encode('Hello 😎');
print(encoded); // [72, 101, 108, 108, 111, 240, 159, 152, 142]
Copy the code

Smiley is encoded as four 8-bit values this time.

Decimal Hex Binary ------------------------ 240 F0 11110000 159 9F 10011111 152 98 10011000 142 8E 10001110 String Hex Binary -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 😎 f60e 1 00001 1111 0110 0000 1110Copy the code

I’m telling you, you need to see that video. 😎

Converting from UTF-8 to String is just as simple. This time using the decoding method.

List<int> list = [72.101.108.108.111.240.159.152.142];
String decoded = utf8.decode(list);
print(decoded); / / Hello 😎
Copy the code

It’s ok to pass in a list of int values, but if any of them are larger than 8 bits, you’ll get an exception, so be aware of that.

Boolean logic

I assume that you know, &&, | |, and! Boolean logical operator.

// AND
print(true && true);   // true
print(true && false);  // false
print(false && false); // false
// OR
print(true || true);   // true
print(true || false);  // true
print(false || false); // false
// NOT
print(!true);          // false
print(!false);         // true
Copy the code

Well, these operators also have bitwise equivalents that work on binary numbers. If you see the false as 0, the true as 1, then you will get with an operator &, | and ~ similar results (plus an additional ^ XOR operator).

// AND
print(1 & 1);  / / 1
print(1 & 0);  / / 0
print(0 & 0);  / / 0
// OR
print(1 | 1);  / / 1
print(1 | 0);  / / 1
print(0 | 0);  / / 0
// XOR
print(1 ^ 1);  / / 0
print(1 ^ 0);  / / 1
print(0 ^ 0);  / / 0
// NOT
print(~1);     / / - 2
print(~0);     // -1
Copy the code

Well, the results are almost the same. The ~ bitwise NOT operator gives some strange numbers. We’ll come back to that later, although you might have guessed why.

Check out this video series if you want to get a bit more background on smarts before we see some examples of Dart.

  • The bitwise operator 1: AND operates
  • Bitwise operator 2: OR operation
  • Bit operator 3: the XOR algorithm

The Bitwise AND operator&

The bitwise & operator compares each pair of bits and gives a 1 in the resulting bit only if both of the input bits are 1.

Here’s an example.

final y = 0x4a;   / / 01001010
final x = 0x0f;   / / 00001111
final z = x & y;  / / 00001010
Copy the code

Why is this useful? Well, one thing that’s useful is to do an AND bit mask. Bitmasks are a way to filter out everything but the information you’re looking for.

You can see it in action in the Flutter TextPainter source code.

static bool _isUtf16Surrogate(int value) {
  return value & 0xF800= =0xD800;
}
Copy the code

This static method checks whether a UTF-16 value is a proxy pair. The pronoun ranges from 0xD800 to 0xDFFF. The value 0xF800 is a bit mask.

It’s a little bit easier to understand in Boolean form.

D800  1101 1000 0000 0000  (min)
DFFF  1101 1111 1111 1111  (max)
F800  1111 1000 0000 0000  (bitmask)
Copy the code

You’ll notice that the top five of any term is 11011. The next 11 digits could be anything. So that’s what bitmasks are for. It uses five 1 bits to match the pattern it is looking for, and 11 0 bits to ignore the rest.

Remember that our friend Smiley 😎 is made up of substitute pairs D83D and DE0E, so either value should return true. Let’s try the second one, DE0E.

DE0E  1101 1110 0000 1110  (test value)
F800  1111 1000 0000 0000  (bitmask)
-------------------------
D800  1101 1000 0000 0000  (result of & operation)
Copy the code

The result is D800, so it should be realistic, that is, yes, DE0E is a proxy. You can test it out for yourself.

bool _isUtf16Surrogate(int value) {
  return value & 0xF800= =0xD800;
}
print(_isUtf16Surrogate(0xDE0E)); // true
Copy the code

The two-bit OR operator|

In each pair a bitwise | operator, if any of input bits (or two) is 1, the result of 1.

Here’s an example.

final y = 0x4a;   / / 01001010
final x = 0x0f;   / / 00001111
final z = x | y;  / / 01001111
Copy the code

Why is this useful? Well, you can also use it as a bitmask to “turn on” certain bits and leave others unaffected. For example, colors are usually packed into a 32-bit integer, with the first 8 bits representing alpha (transparency), the next 8 bits being red, the next 8 bits being green, and the last 8 bits being blue.

So let’s say you have this translucent purple here.

If you want to keep this color, but want it to be completely opaque, you can do this using an OR bitmask. The hexadecimal value for this color is 802E028A.

alpha    red      green    blue
-----------------------------------
80       2E       02       8A
10000000 00101110 00000010 10001010
Copy the code

Completely transparent is 0x00 and completely opaque is 0xFF. So you make a bitmask that keeps the red, green, and blue values the same, but makes alpha 0xFF. In this case, we should expect FF2E028A after applying the bitmask.

10000000 00101110 00000010 10001010  (original)
11111111 00000000 00000000 00000000  (bitmask)
-----------------------------------
11111111 00101110 00000010 10001010  (result of | operation)
Copy the code

Since ORing anything with a 1 becomes a 1, this changes all alpha values to 1. ORing Any value with 0 does not change, so other values remain the same.

Try it in Dart.

final original = 0x802E028A;
final bitmask = 0xFF000000;
final opaque = original | bitmask;
print(opaque.toRadixString(16)); // ff2e028a
Copy the code

Know!

Side note: If you’re uncomfortable with a hexadecimal string being lowercase, you can always overwrite it to uppercase.

print('ff2e028a'.toUpperCase()); // FF2E028A
Copy the code

Bit XOR operator^

The bitwise ^ operator compares each pair of bits, with 1 in the resulting bit if the input bit is different.

Here’s an example.

final y = 0x4a;   / / 01001010
final x = 0x0f;   / / 00001111
final z = x ^ y;  / / 01000101
Copy the code

Why is this useful? In Dart, you’ll often see the ^ operator used to create hash codes. For example, here’s a Person class.

class Person {
  final String name;
  final int age;
  Person(this.name, this.age);
  @override
  bool operator= = (Object other) {
    return identical(this, other) ||
        other is Person &&
            runtimeType == other.runtimeType &&
            name == other.name &&
            age == other.age;
  }
  @override
  int get hashCode => name.hashCode ^ age.hashCode;
}
Copy the code

The hashCode in the last line is the interesting part. Let’s recreate it here.

final name = 'Bob';
final age = 97;
final hashCode = name.hashCode ^ age.hashCode;
print(name.hashCode); / / 124362681
print(age.hashCode);  / / 97
print(hashCode);      / / 124362712
print(name.hashCode.toRadixString(2).padLeft(32.'0'));
print(age.hashCode.toRadixString(2).padLeft(32.'0'));
print(hashCode.toRadixString(2).padLeft(32.'0'));
Copy the code

Here are the binary results.

00000111011010011001111110111001  (name hash code)
00000000000000000000000001100001  (age hash code)
--------------------------------
00000111011010011001111111011000  (result of ^ operation)
Copy the code

For hash codes, you want them to be as distributed as possible, so you don’t have hash collisions. If you do & operation, you will get more 0, if you by using the | operator, you’ll get more of 1. Because the ^ operator preserves the distribution of 0 and 1, it makes it a good candidate for creating new hash codes from other hash codes.

The Bitwise NOT operator~

The ~ operator reverses the value of a bit. You saw these confusing results earlier.

// NOT
print(~1);     / / - 2
print(~0);     // -1
Copy the code

It makes even more sense if you look at binary values. Here is the representation of the 64-bit 1.

0000000000000000000000000000000000000000000000000000000000000001
Copy the code

When you flip all these bits (i.e., the result of ~1), you get.

1111111111111111111111111111111111111111111111111111111111111110
Copy the code

If you’ve seen the video on the complement of two, you’ll remember that this is how you represent the number minus 2 in binary. For a similar story with ~0, problem solved.

By the way, here’s a joke for people who love Shakespeare.

2b|~2b
Copy the code

That’s the problem.

The displacement<< >>

There is one last topic to cover: bit shifts. You can shift to the right using the bit-left shift operator << and the bit-right shift operator >>.

You can learn more about bit shifts in this video.

www.youtube.com/watch?v=mjq…

Here is an example of a left shift in Dart.

final value = 0x01;  / / 00000001
print(value << 5);  / / 00100000
Copy the code

All the bits have been shifted to the left. 5,000,000, the binary value is 32. An interesting fact is that the easy way to multiply by two is to move one to the left.

print(7 << 1);  // 14 (decimal)
Copy the code

This is a shift to the right.

final value = 0x80;  / / 10000000
print(value >> 3);  / / 00010000
Copy the code

That’s the same thing as saying 128 moved 3 to the right is 16.

Why is this important? Well, one place you can see it in Dart is to extract values from a packaged integer. For example, the source code for the Color class has the following getter to extract a red value from a Color value.

/// The red channel of this color in an 8 bit value.
int get red => (0x00ff0000 & value) >> 16;
Copy the code

This first finds only the red part of the ARGB(alpha red green blue) value using the AND bitmask, AND then moves the result to the beginning. Here is our previous 802E028A purple effect.

alpha    red      green    blue
-----------------------------------
80       2E       02       8A
10000000 00101110 00000010 10001010  (original)
00000000 11111111 00000000 00000000  (bitmask)
-----------------------------------
00000000 00101110 00000000 00000000  (result of &)
-----------------------------------
00000000 00000000 00000000 00101110  (result of >> 16)
Copy the code

And this is in Dart.

final purple =  0x802E028A;
final redBitmask = 0x00FF0000;
final masked = purple & redBitmask;
final redPart = masked >> 16;
print(redPart.toRadixString(16)); // 2e
Copy the code

The key point

You did it! Here are the main points of the article.

  • Binary data can be represented by a list of integers.

  • Uint8List is an unsigned List of 8-bit integers that is more efficient than List

    for handling large amounts of binary data.

  • You can use BytesBuilder to add binary data to the list.

  • Byte data is supported by BytesBuffer, and you can get different views of the underlying bytes. For example, you can see byte blocks of 8, 16, 32, or 64 bits, which can be signed or unsigned integers.

  • For byte blocks larger than 8 bits, you need to pay attention to the number of bytes, which is affected by the underlying machine or storage format. You can use ByteData to specify large or small Endyne views.

  • Prefix the number with 0x, in hexadecimal notation.

  • The Dart string is a list of UTF-16 values, known as code units.

  • Convert the string to UTF-8 by using the DART: Convert library.

  • Bits of logical operators is &, |, ^ and ~, shift operator is < < and > >.

  • Bit operators and bit masks allow you to access and manipulate individual bits.

Continue to

Several areas worth further investigation are byte streams and how to transform data on these streams. For example, when reading or downloading a file, you can get a byte stream. Instead of waiting for the entire file to arrive and processing it, start converting it to utF-8 values (or other values) as soon as you get the byte stream. If anything is unclear, leave a comment or ask a question.

www.twitter.com/FlutterComm

Translation via www.DeepL.com/Translator (free version)