For beginners, learning Flink may not be an easy task. Reading documents is learning and, more importantly, practicing. But for a beginner to run a Flink SQL is really not easy, to build a variety of environments, really tired. Luckily, there’s a tool in the Flink ecosystem that can help you learn Flink more effectively: Zeppelin. This article is not about Flink on Zeppelin, but more about how to learn Flink using Zeppelin.

The Flink SQL Cookbook on Zeppelin is probably the best user experience yet. You can run various types of Flink SQL statements in a few easy steps from this article without writing any code. Without further ado, let’s get started.

This tutorial is ververica’s Flink-SQL-Cookbook (github.com/ververica/f…) An improved version of. All of these examples you can run in Zeppelin without writing any code. I’ve ported all the examples to Zeppelin.

Prepare the environment

Step 1

git clone https://github.com/zjffdu/flink-sql-cookbook-on-zeppelin.git
Copy the code

Inside this repo are some Zeppelin Notebooks filled with examples from Flink-SQL-Cookbook.

Step 2

Download Flink 1.12.1 (I haven’t tried any other versions of Flink, so if you’re interested, try it) and unzip it.

Step 3

Flink Faker at github.com/knaufk/flin…

Copy the compiled flink-faker-0.2.0.jar to the Lib directory of Flink. The Flink Faker is a special table source used to generate test data. We’ll use this Flink Faker in many of our examples.

Step 4

Run the following command to start the latest version of Zeppelin.

docker run -p 8081:8081 -p 8080:8080 --rm -v $PWD/logs:/logs -v / Users/jzhang/lot/flink - SQL - cookbook - on - zeppelin: / notebook - v/Users/jzhang/Java/lib/flink - 1.12.1: / flink - e ZEPPELIN_LOG_DIR='/logs' -e ZEPPELIN_NOTEBOOK_DIR='/notebook' --name zeppelin apache/ Zeppelin :0.9.0Copy the code

Note the two directories here:

  1. / Users/jzhang/lot/flink – SQL – cookbook – on – zeppelin (this is Step 1 in the clone repo directory)
  2. / Users/jzhang/Java/lib/flink – 1.12.1 (this is Step 2 after download and unpack the flink directory)

These two directories are my own local directories, please replace them with your own directories.

Experience the Flink Sql Cookbook tutorial

Now that the tutorial environment is ready, open your browser to http://localhost:8080 and begin your Flink Sql learning journey.

This is Zeppelin’s UI, and there is already a folder called Flink Sql Cookbook that contains all the Flink Sql tutorials. First we need to configure the Flink interpreter, click on the menu in the upper right corner, select Interpreter, go to Flink Interpreter, Change FLINK_HOME to /flink (the same flink we mounted in the docker command above) and click restart Interpreter.

If you encounter one of the following errors, please scroll down to see if there is a username in Depenendies, if so, delete it and save it (this is a front-end bug from Zeppelin, the community is fixing it)

Example 1: Filtering Data

Let’s take a look at Foundations/04 Filtering Data.

The first Paragraph creates a server_logs table. The second Paragraph uses the SELECT WHERE statement to filter the data in the table and takes the latest 10 data in chronological order. As you can see, the data is updated every 3 seconds, and status_code is always 401 or 403, which verifies our SQL logic. There is also a link to the FLINK JOB in the upper right corner, where you can click to see the JOB details.

Example 2: Lateral Table Join

Let’s look at an example of a Lateral Table Join, which is one of the Join types in Flink SQL. Beginners will feel a little confused when they see this noun at first, and they will also feel vaguely confused after checking the information online. If there is a more intuitive example for you at this time, it should be very helpful for your understanding. Open your Joins/06 Lateral Table Join and run. This should look like this.

Here I will use these two examples, there are many more useful examples (as shown below), you can learn by yourself, you can try to modify the SQL and then run to see what the difference is.

The above are the learning materials I spent two days over the weekend sorting out, hoping to help you learn Flink and make progress together. However, there is still room for improvement in this tutorial, if you are interested in it, there are still 3 things that can be improved:

The instructions in each Note are in English and can be translated into Chinese to make it easier for more people to learn. Now each tutorial is in text form, if someone can make a short video for each tutorial, I think it will be better. Add more case tutorials, now there’s a lot of content, but there’s room for more.