Encoding, digest, and encryption (I) – byte encoding

The foreword 0.

The reason for choosing this topic is that in the interview process, many job seekers give a common answer to the question “Please list the commonly used encryption algorithms” : “USED MD5 and Base64”, and some even say that “Base64 is symmetric encryption, MD5 is asymmetric encryption”. So, through the next three articles, we will introduce three common terms in programming: byte encoding, information summarization, and data encryption.

1. Introduction to coding

In computers, data is stored in a unit called a byte. The smallest storage unit is between 1 and 1bit. A bit has two states 0 and 1. 1 byte = 8 bit. Usually, an English letter takes up 1 byte, and Chinese characters take up 2 bytes when they are encoded by GBK. Utf-8 is a variable-length encoding, typically expressed in 0-4 bytes.

The above is limited to the characters that a computer can display on the screen. But 1byte typically represents 256 different data. Binary representation: 00000000-11111111, that is, 2^8. The number of visible characters in ASCII is less than 100. To display the entire contents of the 1byte representation, you need to encode it. It is usually used in hexadecimal format, 0x00-0xFF. 0x31 represents the character ‘1’, 0x01 represents the letter 1,0, x41 represents ‘A’ and 0x61 represents ‘A’. No longer list one by one, interested partners can refer to the ASCII code table.

2. Hexadecimal code

2.1 concept

Hexadecimal coding is based on the process of binary conversion. The following table lists some common numeric encodings and their meanings.

Here, we need to introduce the concept of cardinality. Base 2:0, 1. Base 10:0-9. Hexadecimal base 0-9, A-F. By looking at the cardinality representing a string of content, you can quickly determine the encoding method used.

The following table shows the relationship between the hexadecimal base and base 10 and base 2. Both are expressed in 1byte.

It is not hard to see that the hexadecimal system uses 4 bits to represent a radix (16 = 2^4).

2.2 conversion

To convert the number 100 to hexadecimal:

The calculation method is relatively simple, take the mod of 100 with 16. So 100 is equal to 6 times 16 plus 4. That is, 100 = 0x64. Convert to base 2, and convert to 4bit 0 and 1 for 6 and 4 respectively. 0110, 0100.

So 100 = 0x64 = 01100100

2.3 Code Implementation

According to the conversion rules in the previous section:

1. Converting a byte array to a hexadecimal string requires an independent operation on each byte. Take the higher and fourth digits, respectively, and convert them to two decimal numbers for the base index, and finally combine them into a hexadecimal representation.
2. Converting a hexadecimal string into a byte array requires groups of two hexadecimal bases. Find the decimal numbers and add the next four digits.

/** * hex base */
static char[] hex = {'0'.'1'.'2'.'3'.'4'.'5'.'6'.'7'.'8'.'9'.'A'.'B'.'C'.'D'.'E'.'F'};

/** * encoding: byte array to hexadecimal string **@param data
 * @return* /
public static final String encode(byte[] data) {
    if (data == null)
        return null;

    StringBuffer hexSrtBuff = new StringBuffer();
    for (byte b : data) {
        int height = b >> 4 & 0x0f; // Take the higher four digits
        int low = b & 0x0f;// take the lower four digits
        hexSrtBuff.append(hex[height]);
        hexSrtBuff.append(hex[low]);
    }
    return hexSrtBuff.toString();
}

/** * decode: hexadecimal string to byte array **@param hexStr
 * @return* /
public static final byte[] decode(String hexStr) {
    if (hexStr == null)
        return null;
    if (hexStr.length() % 2! =0) { // Invalid hexadecimal string argument
        throw new IllegalArgumentException("The hex string was illegal");
    }
    byte[] data = new byte[hexStr.length() / 2];
    for (int i = 0; i < hexStr.length(); i += 2) {
        char h = hexStr.charAt(i);
        char l = hexStr.charAt(i + 1);
        int height = (h >= hex[10] && h <= hex[15])? (h - hex[10] + 10) : (h - hex[0]);
        int low = (l >= hex[10] && l <= hex[15])? (l - hex[10] + 10) : (l - hex[0]);
        data[i / 2] = (byte) ((height << 4) + (low & 0x0f));
    }
    return data;
}
Copy the code

3. The Base64 encoding

The previous section introduced the hexadecimal coding rules and code implementation. It is not hard to see that the amount of storage required for a single hexadecimal encoding doubles. Although this is convenient for computer display, can be used for network communication, but the cost of storage space and transmission efficiency will be halved. Thus base64 encoding was born. Base64 encoding was born because of this.

3.1 concept

Base64 has a total of 64 cardinals. Each cardinality takes up 6 bits (64 = 2^6).

3.2 conversion

According to the index relation of the above table, try to convert several cases.

1. Character 1

   Base 10 represents 49. The hexadecimal value is 0x31. Base 2 indicates 00110001.

   There are 8 bits in total, not divisible by 6 bits, so not enough bits to complement 0. Add the last 12 digits 001100 010000. Corresponding to base64 index 12 16. base64 MQ==

   For the complement of 00, we need to mark it with =.

2. 1 a string

   Base 10 is 49, 65. The hexadecimal value is 0x31 0x41. Base 2 indicates 00110001 01000001.

   There are 16 bits, not divisible by 6 bits, so there are not enough bits to complement 0. Add the next 18 digits 001100 010100 000100. The base64 index is 12, 20, and 4. base64 MUE=

3. The string 1 aa

   Base 10 is 49, 65, 97. The hexadecimal value is 0x31 0x41 0x61. Base 2 indicates 00110001 01000001 01100001.

   There are 24 bits in total, and they are divisible by 6 bits, so there is no need to complement them. The last 24 bits are 001100 010100 000101 100001. The base64 index is 12, 20, 5, 33. base64 MUFh

According to the above conversion, the base64 encoded string length must be an integer multiple of 4. Perhaps 1, 2 bytes of data encoded in Base64 is not the best thing to do. But for 100 bytes of data encoding:

1. Hexadecimal code

   Length after encoding: 100 * 2 = 200.

2. Base64 encoding

   Length ⌈ 100/3 ⌉ *4 = 34 *4 = 136.

3.3 Code Implementation

Considering the scenario of complement, the implementation is complicated. For programming language entry, practice use.

/** * base 64 Base: 26 uppercase letters, 26 lowercase letters, 10 Arabic digits, '+, '/' */
static char[] base64 = {
        'A'.'B'.'C'.'D'.'E'.'F'.'G'.'H'.'I'.'J'.'K'.'L'.'M'.'N'.'O'.'P'.'Q'.'R'.'S'.'T'.'U'.'V'.'W'.'X'.'Y'.'Z'.'a'.'b'.'c'.'d'.'e'.'f'.'g'.'h'.'i'.'j'.'k'.'l'.'m'.'n'.'o'.'p'.'q'.'r'.'s'.'t'.'u'.'v'.'w'.'x'.'y'.'z'.'0'.'1'.'2'.'3'.'4'.'5'.'6'.'7'.'8'.'9'
        , '+'.'/'
};

/** * encoding: byte array to base64 * <p> * character '1' binary: 00110001 to base64 after 001100 010000 add two equal signs **@param data
 * @return* /
public static final String encode(byte[] data) {
    if (data == null)
        return null;
    StringBuffer base64StrBuff = new StringBuffer();
    int leftBit = 0; // Save the remaining bytes
    int index = 0;
    int num;
    for (index = 0; index < data.length; index++) {
        switch (index % 3) {
            case 0:
                num = data[index] >> 2 & 0x3f;  // take the first six digits
                leftBit = data[index] & 0x03; // Keep the last two
                base64StrBuff.append(base64[num]);
                break;
            case 1:
                num = (leftBit << 4 & 0x30) + (data[index] >> 4 & 0x0f); // Add the first four digits to the last two digits
                leftBit = data[index] & 0x0f; // Keep the last four digits
                base64StrBuff.append(base64[num]);
                break;
            case 2:
                num = (leftBit << 2 & 0x3c) + (data[index] >> 6 & 0x03); // Add the first two digits to the last four digits
                leftBit = data[index] & 0x3f;// Keep the last six
                base64StrBuff.append(base64[num]);
                base64StrBuff.append(base64[leftBit]);
                leftBit = 0;
                break; }}/** * completes the remaining bits and marks */ with an equal sign
    switch (index % 3) {
        case 0:
            break;
        case 1:
            base64StrBuff.append(base64[leftBit << 4 & 0x30]);
            base64StrBuff.append('=');
            base64StrBuff.append('=');
            break;
        case 2:
            base64StrBuff.append(base64[leftBit << 2 & 0x3c]);
            base64StrBuff.append('=');
            break;
    }

    return base64StrBuff.toString();
}

/** * decode: base64 transposition array **@param base64Str
 * @return* /
public static final byte[] decode(String base64Str) {
    if (base64Str == null)
        return null;
    if (base64Str.length() % 4! =0)
        throw new IllegalArgumentException("thr base64 string was illegal");
    // Check the number of equal signs at the end,
    int equalSignCount = 0;
    for (int i = base64Str.length() - 1; i > 0; i--) {
        if(base64Str.charAt(i) ! ='=') {
            break;
        }
        equalSignCount++;
    }
    // Convert to the length of the byte array
    int bytesLen = base64Str.length() / 4 * 3 - equalSignCount;
    byte[] data = new byte[bytesLen];
    int index = 0;
    for (int i = 0; i < base64Str.length(); i += 4) {
        // A group of four bytes is processed into three bytes
        int one = getCharIndex(base64Str.charAt(i));
        int two = getCharIndex(base64Str.charAt(i + 1));
        int three = getCharIndex(base64Str.charAt(i + 2));
        int four = getCharIndex(base64Str.charAt(i + 3));
        if (one < 0)
            break;
        int first = one << 2 & 0xfc;
        if (two < 0)
            break;
        first += (two >> 4 & 0x03);
        data[index++] = (byte) first;
        if (three < 0)
            break;
        int second = (two << 4 & 0xf0);
        second += (three >> 2 & 0x0f);
        data[index++] = (byte) second;
        if (four < 0)
            break;
        int third = (three << 6 & 0xc0);
        third += four;
        data[index++] = (byte) third;
    }
    return data;
}

/** * finds the index of the character in the string **@param c
 * @return* /
private static int getCharIndex(char c) {
    if (c >= 'A' && c <= 'Z')
        return c - 'A';
    else if (c >= 'a' && c <= 'z')
        return c - 'a' + 26;
    else if (c >= '0' && c <= '9')
        return c - '0' + 52;
    else if (c == '+')
        return 62;
    else if (c == '/')
        return 63;
    else
        return -1;

}
Copy the code

4. To summarize

Although hexadecimal and Base64 encodings are reversible and have corresponding decoding operations, they are not encryption algorithms. Encryption algorithms generally need to use a key. Only a correct key can decrypt the correct plaintext.

Think it works? I’ll give you a bonus.I’d like to make a reward

Here’s the AD: Flueky’s tech site