Abstract:

Aliyun e-MapReduce practice

  • How do I install Kafka components on E-MapReduce using boot operations

The Kafka component does not exist in the current e-MapReduce and needs to be installed. This article describes how to install kafka_2.10-0.10.0.0 with an E-MapReduce boot operation.

information

  • Big data industry has become a new favorite data source in the capital market or the core competitiveness of big data companies recently, the development tide of big data is sweeping the world, and capital is also keen to pursue high-growth markets. Big data is a strategic emerging industry in China, and the investment community continues to be optimistic about the development of the big data industry in the future, this trend is significantly enhanced.
  • As A big data operation and maintenance company, LinkedSee focuses on helping enterprises solve the needs of hardware operation and maintenance. LinkedSee’s judgment on the market is that small and micro enterprises tend to use public cloud more to save costs. Although they do not need to maintain the equipment room themselves, they still need to monitor the maintenance status of others. Therefore, it is appropriate for these enterprises to provide alarm services. And large enterprises will continue with privatisation plan, even if there is business migration to the total of the cloud, the demand of the management of computer room is not disappear, but is passed on to the underlying such as ali cloud IaaS vendors, that is to say, the vendor’s hardware maintenance cost is higher and higher, for this kind of situation, to provide a set of monitoring maintenance plan is more appropriate.
  • Cloud service providers are beginning to offer customers more options, including hosting around the world, more virtualization instance configurations, and workload optimization mechanisms, as well as more options for managing and analyzing data within the cloud. This article analyzes what changes the IaaS public cloud market will see in 2017

technology

  • Catalyst is a functional relational query optimization framework in Spark SQL. In this talk, we extract the key TreeNode and Rule structures in Catalyst. A complete compiler optimized Brainfuck interpreter with less than 300 lines of code was implemented. Through this mini-interpreter, viewers will fully understand the basic workings of Catalyst and appreciate the power of functional, declarative programming
  • This article introduces the new Apache Flink 1.2.0 features. On Apache Flink version 1.1+, The main focus of the community is on Operations, Ecosystem, Broader Audience and Application Features
  • HBase RegionServer Breakdown Data recovery To prevent data loss due to RegionServer process exceptions after data is written to the cache, data is written to the HLog in sequence before being written to the cache. If RegionServer breaks down or other faults occur, the HLog can be played back to recover data to prevent data loss. The most important aspect of HBase fault recovery is how to recover lost data through HLog playback
  • This paper introduces some practices of Meituan in real-time log monitoring/query using Spark and ES