In daily scenarios of big data, Flink can be applied to every link of the whole stack of big data +AI, from data producers to data collection, data processing and data application (BI+AI). As a new generation of open source big data computing engine, Flink not only meets the real-time demand of the industry, but also can get through the whole link of end-to-end data value mining.
Now, how does Flink apply in each link of big data? What significant value does it bring? What other ways will Flink evolve in the future?
From April 25th to 26th, Flink Forward, the first online conference of Apache top projects in the world, will be officially launched in Chinese. Focus on the classic scenes and business stories of real-time computing of Major Internet companies such as Alibaba, Google, AWS, Uber, Netflix, DellEMC, Weibo, Didi, etc., and benefit global developers with super lineup and quality issues! The highlights are in Chinese, translated and interpreted by Flink’s core contributors, and you can watch them online for free.
Highlights:
- Detailed explanation of the whole process in Chinese, several heavyweight guests such as Flink PMC, Committer, and front-line technical experts will explain 18 high-quality talks in Chinese, allowing you to understand more clearly and thoroughly.
- Learn the latest trends of Flink from a variety of perspectives, including technical depth, community ecology and future development. Share with you the Roadmap of Flink’s long-term plan.
- Rich classic scenes and business stories of domestic and foreign first-line enterprises, and precious practical experience in production environment.
- A variety of Flink peripheral gifts, participate in the live interaction can win the community peripheral gift package.
Previously, Little Squirrel has introduced in detail the live agenda of the April 25 show in Beijing in the morning and Hangzhou in the afternoon. The following will share what to expect in Shanghai on April 26.
4/26, Shanghai live what to watch
Round table | Keynote: Stream analytics made real with Pravega and Apache Flink
As streaming applications evolve, so do their supporting systems. Today’s applications are no longer islands, but part of an ecosystem. As a result, the market is more demanding than ever about the capabilities that streaming systems and platforms provide. With these requirements in mind, we designed and built Pravega.
Pravega is a native streaming storage system with streams at its core. Pravega enables applications to inject and store unlimited data streams in a flexible and consistent manner. By combining Pravega with Apache Flink, we can build data pipelining to track and process both historical and real-time data, while satisfying end-to-end exactly once semantics. Pravega and Apache Flink have become an important part of Dell Technologies’ Streaming Data Platform products. In this talk, we will share our practical experience, including real-time video stream analysis and real life use cases for autonomous driving.
**Srikanth Satya, Vice President, Dell Technologies, Engineering
** Speaker: ** Teng Yu, Software Development Director of Dell Technology Group.
Round table | Google: machine learning workflow distributed processing
Machine learning processes in production environments typically require very large computing and system resources and need to be distributed across clusters. The cost of on-premise or cloud-based infrastructure requires the most efficient use of resources, making distributed processing pipeline frameworks such as Apache Flink ideal for machine learning workflows.
At the same time, machine learning in production environment must solve the problems of modern software methodology and some special problems. Different types of machine learning have different requirements, often driven by different data lifecycles and fact sources; Implementations often suffer from modularity, scalability, and extensibility limitations. In this talk, we will discuss machine learning applications in production environments and review TensorFlow Extended (TFX), Flink, Apache Beam, and Google’s experience using machine learning in production.
Sharing Guests:
- Ahmet Altay, Senior Software Engineer at Google, Apache Beam PMC.
- Reza Rokni is a development advocate for Google Cloud Dataflow and Apache Beam.
- Robert Crowe, Google data scientist and TensorFlow enthusiast.
** Commentary guests: ** Qin Jiangjie, Apache Flink PMC, Senior technical expert of Alibaba.
Round table | AWS: how to provide application in full custody Apache Flink service high availability
AWS provides customers with a fully hosted service to run the Apache Flink application Amazon Kinisis Data Analytics. We also face unique challenges in serving our customers. One challenge is problem attribution. How do you determine whether a runtime error is caused by an error in the customer code or by a problem with the underlying infrastructure?
In this talk, we will describe how to automatically classify errors and how to restore application availability through programmatic or manual intervention. This is critical to ensuring the availability of applications we expect from AWS services and maintaining sustainable operations. We also review how to design improvements to the Apache Flink engine to disambiguate, and how to ensure the high availability of Apache Flink applications.
Sharing Guests:
- Ryan Nienhuis is an AWS development engineer.
- Tirtha Chatterjee is an AWS development engineer.
** Commentary guest: ** Zhang Jianfeng (Jian Feng), Apache Member, Senior technical expert of Alibaba.
Speech | Production – Ready Flink and Hive Integration – what story you can tell now?
This talk will share the full Flink-Hive integration released in Flink 1.10. This section describes some common use cases where Flink users need to read and write Hive data and reference Hive UDF functions. To illustrate how users can easily solve these problems, I’ll detail the Flink-Hive integration with examples. Attendees will be able to learn:
- How to read and write Hive data locally in Flink (tables, views, partitions)
- Syntax and SQL commands supported by Flink
- Optimization of Flink integration into Hive
- How to reuse Hive UDFs in Flink
** Sharing guests: ** Li Bowen, Alibaba technical expert.
** Commentary guest: ** Rui Li (Tianyi), Apache HBase PMC, Alibaba technical expert
Flink Forward Global Online Conference
Best way to watch
The live broadcast will be held on Flink Forward’s official website. Click “Read the original article” or copy the link below for more details. You can book the live broadcast after registering and logging in. At that time, the community will remind everyone to participate in the form of SMS notification in advance.
The conference website live booking: developer.aliyun.com/live/2594?s…
The following information is displayed after the reservation is successful:
**Tips: The ** community has prepared a wealth of peripheral gifts, participate in the live interaction, ask questions to receive, there is a surprise lucky draw
Full Edition Agenda
Flink Forward global live broadcast highlights are divided into four parts: Key Keynote topics, Flink best Practices, in-depth technology applications, and community ecology. In the form of live broadcast in Beijing, Shanghai, and Hangzhou in turn, you will understand Flink’s core advantages and future development through practice cases of diverse scenarios.
■ Live: April 25-26
■ Sharing guests:
- Apache Member, Flink PMC
- Apache Flink core contributor
- Dachang first-line technical experts
■ Detailed agenda:
April 25-26, Flink Forward global live Chinese highlights! To learn more about the conference, you can scan the qr code below the nail into the group consultation ~
If you are interested in the live broadcast of Flink Forward Virtual Conference 2020, click the link below to learn more about the full Flink Forward Virtual Conference 2020 agenda and register for booking!
www.flink-forward.org/sf-2020/con…