The Dart native SDK supports UTF-8 encoding by default. Both File and HttpClient interfaces provide UTF-8 encoding by default.

For example, in file.dart, the openWrite method defaults to ENCODING utF8, so the generated file defaults to encoding UTF-8.

  //file.dart
  IOSink openWrite({FileMode mode: FileMode.write, Encoding encoding: utf8});
Copy the code

This article provides a GBK COdec, compatible with native API stream interface, read and write files and Http stream API is more convenient.

For example, using the Stream API to generate a GBK file flows as follows:

import 'dart:io';
import 'package:fast_gbk/fast_gbk.dart';
void main() async {
  File output = File("gbk.txt");
  var stream = output.openWrite(encoding: gbk);
  stream.write("123");
  stream.writeln("456");
  stream.writeCharCode(0x41);
  await stream.close();
}
Copy the code

Method of use

  • Add the dependent

Add dependencies to pubspec.yaml

Dependencies: fast_gbk: ^ while

  • Update pubcache

flutter pub get

  • Add references to the DART file

import ‘package:fast_gbk/fast_gbk.dart’;

Demo

  • The simplest use scenario, direct encoding decode String:
import 'package:fast_gbk/fast_gbk.dart';
void main() async {
  var encoded = gbk.encode("The Yellow River flows into the sea at the end of the mountain in daylight.");
  var decoded = gbk.decode([176.215.200.213.210.192.201.189.190.161.163.172.187.198.186.211.200.235.186.163.193.247]);
}
Copy the code
  • Read the GBK encoded file
import 'dart:convert';
import 'dart:io';
import 'package:fast_gbk/fast_gbk.dart';
void main() {
  File gbkFile = File("gbkFile.txt");
  var stream = gbkFile.openRead();
  stream.transform(gbk.decoder)
      .transform(const LineSplitter())
      .listen((line) {
    stdout.writeln(line);
  });
}

Copy the code
  • Write GBK coded files
import 'dart:io';
import 'package:fast_gbk/fast_gbk.dart';
void main() async {
  File output = File("gbk.txt");
  var stream = output.openWrite(encoding: gbk);
  stream.write("123");
  stream.writeln("456");
  stream.writeCharCode(0x41);
  await stream.close();
}
Copy the code
  • Decode GBK HttpClient response
import 'dart:io';
import 'package:fast_gbk/fast_gbk.dart';
void main() async {
  var gbkWebUrl = "http://www.newsmth.net/nForum/#! mainpage";
  var httpClient = HttpClient();
  HttpClientRequest request = await httpClient.getUrl(Uri.parse(gbkWebUrl));
  HttpClientResponse response = await request.close();
  var responseBody = await response.transform(gbk.decoder).join();
  print(responseBody);
  httpClient.close();
}
Copy the code
  • Decoding response using Gbk Codec in DIO:

Set the options responseDecoder

      BaseOptions options = BaseOptions();
      options.responseDecoder = gbkDecoder;
      _client = Dio(options);
Copy the code

Define the GBK Decoder

  String gbkDecoder (List<int> responseBytes, RequestOptions options,
      ResponseBody responseBody) {
    String result =  gbk.decode(responseBytes);
    return result;
  }
Copy the code

Q&A

  • Q: Another wheel? How does it differ from the existing GBK CoDEC?

  • A: In terms of function, the existing GBK_COdec and GBK2UTF8 can support encode and decode interface, but do not support Stream interface.

Response.transform (GBK.decoder), GBk_codec and GBk2UTF8 cannot be used this way.

In terms of efficiency, the author of this article rewrites this module because of the lag generated when parsing HttpResponse using both GBK_codec and GBK2UTF8:

Gbk_codec uses a large number of String+= splice characters in the decoding, which causes the interface refresh to produce a lag. As shown in the flame chart below, it takes 3.8s to decode the interface home page refresh:

Gbk2utf8 is slightly more efficient, and the interface costs 240ms. It is found that GBK is converted to Unicode first, then unicode to UTF8, then UTF8 to String.

In fact, the Dart String is implemented in UTF-16 encoding and supports Unicode. Gbk2utf8 actually wastes half of the decoding time and is therefore inefficient.

The fast_GBK test results are as follows. The interface also takes 150ms, which is the fastest decoding among the three CODec, and the interface is obviously no longer stuck.

Because fast_GBK supports Stream interface, when encoding and decoding large amounts of data, decode will be more efficient than direct encode, decode, and more convenient to use.

  • Q: Is it stable? Any bugs?

  • A: Currently the test cases have covered all UTF-8 characters and GBK characters. If any bugs are found, the code has been open source on Github. Welcome to submit issue and pull Request. Pub link: pub.dev