Spark – How does the RPC of each component communicate
RpEndpointRef: A reference to the rpEndpoint. That is, it points to the server's rpEndpoint, so rpEndpointRef has the server's RPC address.
Read More
Category: spark
Spark – RDD is used
December 14, 2023
by Barry Harris
No Comments
Resilient Distributed Dataset (RDD) is a core concept in Spark. Simply put, you create an RDD in Spark and use it to manipulate the data...
Read More
Super detailed walkthrough! Databricks Data Insights – Enterprise-class, fully hosted Spark big data analytics platform and case studies
December 14, 2023
by Russell Bibi-Burke
No Comments
Lecturer: Zong Ze, Alibaba Cloud Technical Expert, Head of Open Platform - Ecological Enterprise Team in the Computing Platform Business Unit
Read More
Application of Databricks Data Insights in HVAC and Buildings of Midea
December 14, 2023
by Reece Iqbal
No Comments
Midea HVAC and Building Business Unit (hereinafter referred to as Midea HVAC) is one of the five divisions of Midea Group. Its products cover multi-unit...
Read More
Kylin on Parquet introduction and quick start
December 14, 2023
by 龍美玲
No Comments
Reprinted from the Apache Kylin official account (also by me) : link to the original article. This article is a summary of the introduction by...
Read More
Kylin 4.0 TOPN implementation principle introduction
December 14, 2023
by Leanne Cooksley
No Comments
Apache Kylin is an open source distributed analysis engine that provides SQL query interfaces on top of Hadoop and multi-dimensional analysis (OLAP) capabilities to support...
Read More
Apache Kylin 4.0 precise de-duplication of the global dictionary principle
December 14, 2023
by Tiya Wali
No Comments
With large data sets, accurate de-duplication and fast query response can be challenging. We know that the most commonly used processing method for accurate reduplication...
Read More
Spark – Distributed high availability cluster installation
December 14, 2023
by Shlok Chandra
No Comments
Spark Release Archives is available on the download page of the official website. I chose Spark-2.4.5, but I chose version 2.7.7 for Hadoop. So the...
Read More
Spark – Use of the Spark Shell
December 13, 2023
by Anya Ratti
No Comments
$SPARK_HOME/bin/spark-shell = $SPARK_HOME/bin/spark-shell = $SPARK_HOME/bin/spark-shell = $SPARK_HOME/bin/spark-shell = $SPARK_HOME/bin/spark-shell = $SPARK_HOME/bin/spark-shell Once in, you can see that SC and Spark have been initialized.
Read More
Spark – Spark Submit used
December 13, 2023
by 朱惠婷
No Comments
Spark Submit is used to start the applications in the cluster, and it runs the same commands as the Spark Shell. {code... } --class: the...
Read More