Clickhouse (OLAP- Online Analytical Processing)

Unlike traditional OLTP (Online Transaction Processing), column databases are better suited for OLAP scenarios.

There are many column databases, but clickHouse is the focus here.

The following figure shows the difference between row and column query data:

Line:

\

The column type:

\

Summary: For analysis queries, you typically only need to read a small number of columns from a table. In a column database you can read only as much data as you need. For example, if you only need to read 5 out of 100 columns, this will help you reduce I/O consumption by at least 20 times.

  • Docker installation run ClickHouse steps:
  1. Pull the mirror
docker pull yandex/clickhouse-server
Copy the code

2. Mount the configuration file to all IP addresses and set user names and passwords

(Attach the configuration file users. XML, whose path is in the container /etc/clickhouse-server/users.xml)

3. Run

Docker run - name = clickhouse server - test - p - 8123-8123 - v users. XML: / etc/clickhouse - server/users. XML - d yandex/clickhouse-server:latestCopy the code
  • Basic database operations

1, connect to database, use click-house-client

Mysql > select * from database where databases are displayed

3, create a database and mysql is different, there are a lot of different configuration can be set, (attach clickhouse official address: [Chinese] clickhouse.com/docs/zh/dev… [US] clickhouse.com/docs/en/

)

  • The Python client connects to clickhouse-server
from clickhouse_driver importThe Client host = 'XXX. XXX. XXX. XXX'# server address ='8123'# port user ='default'# database =' voc_data'  # database
send_receive_timeout = 5  # timeout
client = Client(host=host)

Create database
db_sql = 'CREATE DATABASE IF NOT EXISTS voc_data; '
ans = client.execute(db_sql)
print(ans)
Copy the code