preface
A view is a virtual table that is exported from one or more tables. For example, in a class, the head teacher needs to know the scores of all the students in the class so as to know the overall situation for overall guidance, while the math teacher only needs to know the scores of the students in math.
We hear a lot about views in databases. In fact, openLooKeng, a big data multi-source query analysis engine, also has a view concept. The following will use Hive Connector to explain the existence of several views.
VDM connector
This connector is a new connector developed by openLooKeng. It can be used in scenarios where multiple data sources need to be managed or accessed in the same session or view, and users do not need to care about the distribution and source of data.
The Virtual Data Mart (VDM) connector aims to bring this feature to openLooKeng.
VDM connector support:
· Create, update, and delete views that combine multiple directories
· Access real data through views
· Manage user rights through views
· Record each user’s use of the VDM view
For detailed configuration, please refer to the official website:
Openlookeng. IO/useful – cn/docs /…
According to the guidance on the official website, we know that the back-end storage of the VDM connector can support HDFS and relational databases, and mysql is used for storage here.
The above is a view stored in the VDM based on a Student table in the Hive Connector.
View in Hive
In addition to the VDM connector mentioned above, openLooKeng views can also be stored in the Hive Connector.
Openlookeng. IO/useful – cn/docs /…
Configure the Hive Connector so that we can follow the example above to create and query views.
The above openLooKeng view is created based on the Hive Connector and stored in the Hive Connector.
Hive View
A view created in Hive is a view created in Hive. OpenLooKeng also supports access to views created in Hive.
Create a Hive View in Beeline
We then query the view in openLooKeng and see that the query result is the same as that in Hive.
The principle of analytic
This section describes the process of creating and querying a view. The following figure shows the process of creating a view:
DataDefinitionExecution is called when openLooKeng receives a view creation request, and then the corresponding MetaDataManager or HiveMetadata is called if it is a Hive connector. The VDM connector invokes the VdmMetadata processing to encode the relevant SQL statements in the view into viewData, which is stored in the Connector.
Here is the view query flow:
This is different from the view creation process. When a request arrives at the server, SqlQueryExectuion will handle it and parse the syntax. When doing visitTable, MetadataManager will obtain the view through the metadata of the corresponding connector. The hive connector is HiveMetadata, VDM is VdmMetadata, and the actual table is obtained. If the table is a view, then the view is analyzed by StatementAnalyzer#paserView and #analyzeView. Until the final physical table is parsed. In HiveMetadata#getView, if it is a Hive View, it is converted by HiveMetadata into openLooKeng View, which is then given to StatementAnalyzer for parsing. The process is handled just like the View of openLooKeng.
That’s how you create and query views in openLooKeng.
# openLooKeng, Make Big Data Simplified
OpenLooKeng is an open source, high-performance, multi-source analysis engine for big data. It provides a unified SQL interface and cross-data source/data center analysis capability, providing a simple data analysis experience for big data users. If you have anything you want to communicate, feel free to post an Issue in the community code storehouse. Also welcome to add small assistant wechat, into the exclusive technical exchange group.
Community official openlookeng. IO
Code warehouse address gitee.com/openlookeng
Community public account openLooKeng
Community helper wechat openLooKengoss
