How does openLooKeng implement Hash Join from build to use?
Hash Join is one of the common ways to Join multiple tables. So how do you build and implement Hash Joins on openLooKeng? What Join...
Read More
Tag: Big data
ReplacingMergeTree: Implement Clickhouse data updates
January 21, 2024
by Brooke Miller
No Comments
Abstract: Clickhouse, as an OLAP database, has very limited transaction support. This article mainly introduces how to update and delete Clickhouse data by ReplacingMergeTree.
Read More
How to test a data lake in the lab
January 21, 2024
by David Gray
No Comments
Abstract: How to test the function, performance, reliability and other aspects of tens of thousands of data lakes in the laboratory has also become a...
Read More
This is an alternative to Excel, but 90% of people don’t use it!
January 21, 2024
by Prisha Dugar
No Comments
In daily work, when it comes to tables, calculations, charts and analysis of data, Excel must be the first choice. However, I also see many...
Read More
Big data development -Spark-RDD Operation Case – HTTP log Analysis
January 21, 2024
by 劉雅芳
No Comments
1. In the production environment, how to handle the configuration file && table data processing configuration file, or configuration table, usually stored in the online...
Read More
Spark Core
January 21, 2024
by Tania Wilton
No Comments
Before Spark, mature computing systems such as MapReduce exist and provide high-level apis (Map/Reduce) to implement distributed computing by running computing in clusters and providing...
Read More
Big data learning routes share Lambda expressions
January 21, 2024
by 劉中山
No Comments
//contains Compares objects to equals. TreeSet also implements the Comparable interface and overwrites the compareTo method. 1. Use generics no longer need to be fault-tolerant,...
Read More
[Spark Refinement] Map operator of the four RDD operators that must be mastered
January 21, 2024
by Olivia Black
No Comments
Small knowledge, big challenge! This article is participating in the creation activity of "Essential Tips for Programmers". Given the mapping function f, map(f) counts the...
Read More
Big data development -Spark- Persistence and caching of RDD
January 21, 2024
by 劉雅芳
No Comments
1. The RDD cache mechanism that persistSpark is very fast is that RDD supports caching. After a successful cache, the data set is fetched directly...
Read More
【 K8S series 1】 Comparison between Spark on K8S and Spark on K8S Operator
January 20, 2024
by Melissa Hunter
No Comments
| | implementation way for spark submit way, whether the client to submit or cluster to submit, are SparkApplication inheritance. Client submission, subclass is JavaMainApplication,...
Read More
V\P\N Automatically disconnects the reconnection script
January 20, 2024
by 徐冠霖
No Comments
After deploying the scheduled pumping plan, I found that sometimes V\P\N would be disconnected, leading to the failure of pumping, so I modified it combining...
Read More
Real-time analysis using Pulsar Functions
January 20, 2024
by Madeleine Burton
No Comments
How quickly to process, understand, and respond to data is critical for many event-driven applications. In analysis and data processing for these scenarios, calculating precise...
Read More
Finally, someone has made big data, cloud computing technology architecture and practical technology clear
January 20, 2024
by 熊雅涵
No Comments
With the rapid development of big data cloud computing and the blooming of relevant technology hotspots, major manufacturers in the industry have formulated corresponding strategies...
Read More
Ali Cloud on the SaaS industry thinking, see this article is enough
January 20, 2024
by Raghav Kaur
No Comments
In the past two decades, with the deepening of reform and opening up, as well as China's demographic dividend and other factors, a large number...
Read More
So the reader is pleased to offer the best | a readers after sharing
January 20, 2024
by Craig Jensen-Middleton
No Comments
This is what a reader of xiaobian did in the group after he liked the offer. The article hides the reader's personal information. Xiaobian shares...
Read More
Windows10 system Hadoop and Hive development environment construction pit filling guide
January 20, 2024
by Kevin Morgan
No Comments
At present, the author needs to build a data platform, and has found that there are a lot of pits in the installation and operation...
Read More
Post recommendation | what the next generation of Pulsar message platform
January 20, 2024
by Arnav Dube
No Comments
Pulsar is a Yahoo! Open-source messaging middleware in 2016 and Apache's top level project in 2018. In my previous articles I have written about many...
Read More
Struct complex data type UDF preparation, GenericUDF preparation
January 20, 2024
by Scott Faulkner
No Comments
Java UDF supports multiple data types: BIGINT, STRING, DOUBLE, BOOLEAN, ARRAY, MAP, STRUCT, etc. And the Writable parameter. Java UDFs use complex data types...
Read More
Data center network technology upstarts: VXLAN and campus network virtualization
January 20, 2024
by Anika Gill
No Comments
1 Overview Problems on a traditional DC network The number of VMS is limited by device entry specifications On a traditional Layer 2 network, the...
Read More
Dynamic Proxy mode (Design Mode 10)
January 20, 2024
by John Armstrong
No Comments
Handler interface class: handles proxy access to the real role of the client execution result: ··· · From the result, we can see that whenever...
Read More
Hadoop2.5.2 learning 13-mr sina weibo tf-idf algorithm introduction
January 20, 2024
by Steven Smith
No Comments
Term Frequency -- Inverse Document Frequency (TF-IDF) is a commonly used weighting technique for information retrieval and data mining. Tf-idf is a statistical method used...
Read More
User Portrait Productization — Building real-time User Portrait from scratch (6)
January 20, 2024
by Kent Cooke MD
No Comments
After developing the user tag, how to apply the tag to the reality is a very important problem. Only the product design can make the...
Read More
“The Power of the Clickhouse Array” 2-3
January 20, 2024
by Tejas Chaudhry
No Comments
To sum up, groupArray() builds arrays from grouped values, and sortArray() sorts these values, using one array as the sorting key for the others. All...
Read More
The real-time data warehouse of the whole scene was constructed based on Flink
January 20, 2024
by Nayantara Deshpande
No Comments
Real time warehouse construction practice. Although real-time computing has only become popular in recent years, in the early stage, some companies have the demand for...
Read More
Hadoop 3.0 cluster deployment, super detailed -Ali0th
January 20, 2024
by Dhanush Kota
No Comments
After deploying Standalone Hadoop in the previous article, I tried to deploy a Hadoop cluster. A Hadoop cluster requires at least three machines because the...
Read More
How to deal with stolen data in a big data cluster? Transparent encryption can be tried
January 19, 2024
by Callum Hussain
No Comments
Abstract: In traditional big data clusters, user data is stored in HDFS in plain text. Cluster maintenance personnel or malicious attackers can bypass HDFS permission...
Read More
ClickHouse Cluster Construction (1)
January 19, 2024
by Todd Wright
No Comments
ClickHouse is a columnary-storage database (DBMS) open-source by Yandex of Russia in 2016 for online analytical processing queries (OLAP) that generate reports of analytical data...
Read More
Memoirs of Ali HBase HIGH availability in the Eight-year War of Resistance
January 19, 2024
by 李淑娟
No Comments
Since 2017, Ali HBase has moved to public cloud. We have planned to gradually provide ali's internal HIGH availability technology to external customers. At present,...
Read More
Development of intelligent police control and early warning platform development of big data police research and judgment system
January 19, 2024
by Wendy Thomas-Wade
No Comments
One, one key to control the capture of "fugitives" two, intelligent empowerment, complete the "impossible" to complete the task three, science and technology to assist...
Read More
Talk about Lambda architecture
January 19, 2024
by Maria Harvey
No Comments
In a data analysis scenario, we might encounter such problems. For example, if we want to make a recommendation system, if we do it with...
Read More