Query component
The query is the seventh category in the transformation. Queries are used to query data in the data source and merge it into the master data stream.
HTTP client
Submit the request using GET to GET the returned page content.
Operation example:
Custom constant data
It’s used to generate some unchanging data.
Operation example:
Specifying constant data
Operation example:
Database query
The left connection in the database. Left join is two tables to perform the left associated query, the left table data all query out.
Operation example:
Database connection
You can perform two database queries, and a single parameter table entry.
Operation example:
Stream query
The data is loaded into memory before the query, and only equivalent queries can be performed.
Operation example:
Process components
Processes are the sixth category within transformations. Flow is mainly used to control data flow and data flow.
Switch/case allows data flow from one path to multiple paths.
Operation example:
Filtering records allows data to flow from one path to two.
Operation example:
An empty operation is typically the end point of a data stream.
It is often used in the Kettle sample example, but rarely used in real development.
Operation example:
Termination is the end point of the data flow
An error will be reported if any data arrives here. It is used to verify data.
Operation example:
Connected components
Connections are the eighth category in a transformation. Joins are result sets joined by keywords.
The merge record
Is used to merge data from two different sources, old data and new data. This step matches, compares, and merges the old data and new data according to the specified keywords. Mainly used for the comparison of old and new data, very good oh!
1) Parameters to be set:
1.1), old data sources: steps of old data sources.
1.2) New data sources: steps for new data sources.
2) Flag field:
Sets the name of the flag field used to hold the result of the comparison, which can be any of the following.
2.1） “identical”
Old data is the same as new data.
2.2.) “changed”
The numbers have changed.
2.3.) the “new”
There are records in the new data that are not in the old data.
2.4) “does”
Records that are present in the old data but not in the new data.
3) Key fields:
Used to locate the same record in two data sources.
3.1) Compare fields
For the same record in both data sources, specify the fields to be compared.
3.2) The merged data will include all the data from the old data source and the new data source. For the changed data, the new data will be used to replace the old data, and an identifier field will be used in the result to specify the comparison result between the old and new data.
4) Attention:
4.1), old data and new data need to be sorted according to key fields in advance.
4.2), old data and new data should have the same field name.
