Welcome to my GitHub

Github.com/zq2599/blog…

Content: all original article classification summary and supporting source code, involving Java, Docker, Kubernetes, DevOPS, etc.;

About the sink

In the red box, it is Sink. It can be seen that real-time data starts at Source and ends at Sink after the business logic is completed in Transformation. Therefore, Sink can be used to process calculation results, such as console output or save database:

About “Flink’s Sink Combat” series of articles

This paper is the first chapter of Flink’s Sink Actual Combat. It aims to have a preliminary understanding of Sink. Through the analysis and research of basic API and addSink method, it lays a solid foundation for the subsequent coding actual combat.

Full range of links

  1. “Flink sink Actual Combat: A Preliminary Study”
  2. Flink Sink Combat II: Kafka
  3. Flink’s Sink: Cassandra3
  4. “Flink sink Combat iv: Custom”

Start with an example code

  1. Here is a simple flink application code. The print method in the red box is the sink operation:

2. The following figure shows the official sink mode, which is the API of DataStream class and can be realized by calling directly. Print in the code just now is one of them:Java: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction: PrintSinkFunction4. Another common API is writeAsText, which uses the writeUsingOutputFormat method:WriteUsingOutputFormat addSink is called with OutputFormatSinkFunction: writeUsingOutputFormat addSink is called with OutputFormatSinkFunction6. AddSink is called behind both print and writeAsText. What about another common writeAsCsv method? Could it also be addSink? AddSink: writeUsingOutputFormat (); writeUsingOutputFormat ();7. To sum up, the key of data sink is the addSink input, namely the realization of SinkFunction interface. Through the class diagram, we can intuitively see how common sink capability is realized:

8. It can be seen from the above figure that abstract class RichSinkFunction is closely related to various sink capabilities, so we should focus on it and display method signatures in the class diagram, as shown below:9. As shown in the figure above, RichSinkFunction itself has no content, but it realizes SinkFunction and inherits AbstractRichFunction. It is the combination of RichFunction and SinkFunction. 10. The features of RichFunction are aboveFlink’s DataSource TrilogyIs the resource’s open and close; 11. What are the features of SinkFunction? The class diagram shows two invoke methods. Take a look at the official printsinkFunction.java:12. The writer. Write (record) source in PrintSinkOutputWriter. Java, as shown below:

summary

So far, we have a basic understanding of Flink sink:

  1. Responsible for the processing of real-time results (such as output or persistence);
  2. The main implementation is to call datastream.addsink;
  3. The main way to realize various sink capabilities is to realize the interface defined by addSink method.

In the following chapters, let’s practice the coding of sink. The direction of the practice is to experience the officially provided Sink capability and realize the user-defined sink capability.

Welcome to pay attention to the public number: programmer Xin Chen

Wechat search “programmer Xin Chen”, I am Xin Chen, looking forward to enjoying the Java world with you…

Github.com/zq2599/blog…