Performance analysis method

methods To solve the problem conclusion
Slow Query Analyze the SQL in question By performing process status analysis
Explain Shows how mysql uses indexes to process SELECT statements and join tables. It helps to choose better indexes and write better queries Through plan analysis
Profile Query how long the SQL will take to execute, and see how much CPU/Memory usage, Systemlock, Table lock, etc Through process analysis

Slow Query

The query command

By executing commandsSHOW PROCESSLISTSHOW FULL PROCESSLISTYou can view all SQL execution processes as follows:

Field meaning

field meaning
Id The PROCESS Id.
ProcessId Unique identifier of a task. ProcessId is used when executing KILL PROCESS
User The current user
Host Displays the host name of the client issuing this statement, including the IP and port number
DB Displays which database the PROCESS is currently connected to
Command Displays the commands executed by the current connection, namely, sleep, query, and connect commands
Time Displays the Command execution time, in seconds
State Displays the execution status of the SQL statement under the current connection
Info Display SQL statements.

Note If the FULL keyword is not used, you can view only the first 100 characters of the Info field in each record.

Thread Command value

value meaning
Binlog Dump This is the thread on the primary server that sends binary log content to the slave server
Table Dump The thread sends the table contents to the slave server
Change user The thread is performing a change user operation
Close stmt The thread is closing the prepared statement
Connect In replication, the slave server connects to its master server
Connect Out The slave server is connecting to its master server in replication
Create DB The thread is performing a create-database operation
Daemon This thread is inside the server, not the thread to which the service client is connected
Debug The thread is generating debug information
Delayed insert A thread is a deferred insert handler
Drop DB The thread is performing a drop-database operation
Execute The thread is executing a prepared statement.
Fetch The thread is executing the result of a prepared statement
Field List The thread is retrieving table column information
Init DB The thread is selecting the default database
Kill A thread is killing another thread
Long Data The thread retrieves long data as a result of executing a prepared statement
Ping The thread is processing a server ping request
Prepare The thread is generating an execution plan for the statement
Processlist The thread is generating information about the server thread
Query The thread is executing a statement
Quit Thread terminating
Refresh Threads are refreshing tables, logs, or caches, or resetting state variables or copying server information
Register Slave The thread is registering the slave server
Reset stmt The thread is resetting a prepared statement
Set option The thread is setting or resetting client statement execution options
Shutdown The thread is shutting down the server
Sleep The thread is waiting for a client to send it a new statement
Statistics The thread is generating server status information

Thread State value

General thread state value

value meaning
After create When a thread creates a table (including an internal temporary table), it does so at the end of the function that creates the table. This state is used even if the table cannot be created due to some error
Analyzing The thread is calculating MyISAM TABLE key distribution (e.g., for ANALYZE TABLE)
checking permissions The thread is checking whether the server has the necessary permissions to execute the statement
Checking table The thread is performing a table check operation
cleaning up The thread has processed a command and is preparing to free memory and reset some state variables
closing tables The thread flushs the changed table data to disk and closes the used table. This should be a quick operation. If not, verify that you do not have a complete disk and that the disk is not heavily used
copy to tmp table The thread is processing an ALTER TABLE statement. This state occurs after the table with the new structure has been created, but before the rows are copied to the table. For threads in this state, you can use performance mode to get the progress of the replication operation
Copying to group table If the statement has different ORDER BY and GROUP BY criteria, the rows are grouped and copied to a temporary table
Creating index ALTER TABLE; ENABLE KEYS A MyISAM table
Creating sort index The thread is processing a thread whose SELECT uses internal temporary table resolution
creating table The thread is creating a table, which includes creating temporary tables
committing alter table to storage engine The server has completed emplacing the ALTER TABLE and submitting the results
deleting from main table The server is performing the first part of a multi-table drop, which deletes only from the first table and saves the columns and offsets to be deleted from the other (reference) tables
deleting from reference tables The server is performing the second part of the multi-table deletion and removing the matching rows from the other tables
discard_or_import_tablespace ALTER TABLE; DISCARD TABLESPACE or ALTER TABLE… The IMPORT in TABLESPACE statement
end This occurs before the ALTER TABLE, CREATE VIEW, DELETE, INSERT, SELECT, or UPDATE statements are completed, but the cleanup is complete
executing The thread has started executing a statement
Execution of init_command The thread is executing a statement in the value of the init_command system variable
freeing items The thread has executed a command, and some of the release of the project done in this state involves the query cache, which is usually cleaned up later
FULLTEXT initialization The server is preparing to perform a natural language full-text search
init This operation occurs before initializing ALTER TABLE, DELETE, INSERT, SELECT, or UPDATE. The actions taken by the server in this state include flushing binary logs, Innodb logs, and some query cache cleanup operations. For the final state, it is possible to change the data in the table and then delete the query cache entry, write the event to the binary log, and free the memory buffer, including the BLOB
Killed Executing the KILL statement sends a statement to the thread that should be interrupted the next time the KILL flag is checked. This flag is checked in every main loop of MySQL, but in some cases it can take a very short time for threads to die. If a thread is locked by another thread, the kill takes effect as soon as the other thread releases the lock
Locking system tables The thread is trying to lock a system table (for example, a time zone or log table)
login The initial state of the connection thread until the client is successfully authenticated
manage keys The server enables or disables table indexing
NULL This state is used for the SHOW PROCESSLIST state
Opening system tables The thread tries to open a system table (for example, a time zone or log table)
Opening tables The thread is trying to open a table, which should be a very fast program unless something prevents it from opening. For example, an ALTER TABLE or a LOCK TABLE statement can prevent opening a TABLE until the statement is complete. You may also want to pay attention to whether the value of the table_open_cache parameter is large enough. For system tables, use the Opening System tables state
optimizing The server is performing the initial optimization of the query
preparing This state occurs during query optimization
Purging old relay logs The thread is removing unwanted relay log files
query end This state occurs before the freeing Items state after query processing
Removing duplicates The use of SELECT DISTINCT in this query prevents MySQL from optimizing different operations at an early stage. Therefore, MySQL needs an extra stage to remove all duplicate rows and then send the results to the client
removing tmp table After processing the statement, the thread is deleting an internal temporary table SELECT. If no temporary tables are created, this state is not used
rename The thread is renaming a table
rename result table The thread is processing an ALTER TABLE statement, has created a new TABLE, and renamed it to replace the original TABLE
Reopen tables The thread acquired the lock on the table, but noticed it after acquiring the lock on the underlying table structure change. It releases the lock, closes the table, and tries to reopen it
Repair by sorting Fix code is using sort to create indexes
preparing for alter table The server is preparing to execute ALTER TABLE in place
Repair done The thread has completed a multithreaded repair of a MyISAM table
Repair with keycache Repair code creates keys one by one using the key cache, which is much slower than Repair by sorting
Rolling back The thread is rolling back the transaction
Saving state For MyISAM table operations (such as repair or analysis), the thread saves the new table state to. MYI file header. State includes information such as row count, AUTO_INCREMENT counter, and key distribution
Searching rows for update The thread is in the first phase to find all matching rows before updating. You must do this if UPDATE wants to change the index used to find the row involved
setup The thread is starting an ALTER TABLE operation
Sorting for group The thread is doing a meeting for a GROUP BY
Sorting for order The thread is doing an ORDER BY satisfying an ORDER BY
Sorting index The thread is to sort the index pages for more efficient access during MyISAM table optimization operations
Sorting result For a SELECT statement, this is similar to Creating sort Index, but for non-temporary tables
statistics The server is calculating statistics to develop a query execution plan. If a thread is in this state for an extended period of time, the server may be disk-bound and perform other work
update The thread is preparing to start updating the table
Updating The thread is searching for rows to update and updating them
updating main table The server is performing the first part of a multi-table update, which updates only the first table and saves the columns and offsets used to update the other (reference) tables
updating reference tables The server is performing the second part of the multi-table update and updating the matching rows from other tables
User lock The thread will require or is waiting for an advisory lock requested by GET_LOCK() call. Because of SHOW PROFILE, this state means that the thread is requesting a lock (without waiting for it)
User sleep The thread calls a SLEEP() call

Faulty thread status value

value meaning
logging slow query The thread is writing a statement to the slow query log
altering table The server is performing an in-place ALTER TABLE
Receiving from client The server is reading packets from the client
Copying to tmp table The server is copying disk to in-memory temporary tables, which are temporary tables created directly on disk rather than from memory to disk.
Copying to tmp table on disk The state of when a thread copies a temporary table from disk to memory after changing it from memory to a disk-based format to save memory
Creating tmp table A thread is creating temporary tables in memory or on disk. If tables are created in memory but later converted to disk tables, the state in this operation will be one of Copying to TMP table on disk
Sending data The thread is reading and processing the rows of the SELECT statement and sending the data to the client. Because operations that occur during this state tend to perform a large number of disk accesses (reads), it is typically the longest running state over the lifetime of a given query
Sending to client The server is writing packets to the client
Waiting for commit lock FLUSH TABLES WITH READ LOCK waiting for LOCK commit
Waiting for global read lock FLUSH TABLES WITH READ LOCK Waiting for a global READ LOCK or read_only setting a global system variable
Waiting for tables The thread gets a notification that the underlying structure of the table has changed and needs to reopen the table to get the new structure. However, to reopen the table, you must wait until all other threads close the table. If another thread has used FLUSH TABLES or one of the following statements: FLUSH TABLES TBL_NAME, ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE, or OPTIMIZE TABLE will be notified
Waiting for table flush The thread is executing FLUSH TABLES and is waiting for all threads to close their TABLES, or to get a notification that the underlying structure in the table has changed and needs to reopen the table for the new structure. However, to reopen the table, you must wait until all other threads have closed the table. If another thread has used FLUSH TABLES or one of the following statements: FLUSH TABLES TBL_NAME, ALTER TABLE, RENAME TABLE, REPAIR TABLE, ANALYZE TABLE, or OPTIMIZE TABLE all issue this notification
Waiting for lock_type lock The server is waiting for THR_LOCK to obtain a lock or lock from the metadata locking subsystem, where LOCK_type indicates the type of lock. Waiting for table level lock; These states indicate waiting for a metadata lock: Waiting for event metadata lock, Waiting for global read lock, Waiting for Schema metadata lock, Waiting for stored Function metadata lock, Waiting for stored procedure metadata lock, Waiting for table metadata lock, Waiting for trigger metadata lock
Writing to net The server is Writing packets to the network. If a thread has been executing for a long time and is in the Writing to NET state, then it is sending packets to the network, try to adjust the max_allowed_packet size. In addition, this can cause a lot of blocking in other threads
Waiting on cond The thread waits for the condition to become the general state of true, with no specific state information available
System lock The thread has called mysql_lock_tables() and the thread state has never been updated. This is a very common state and can occur for many reasons. For example, a thread will request or be waiting for an internal or external system lock on a table. This can happen when InnoDB waits for table-level locks while performing table locks. If this state is caused by requesting an external lock and you are not using multiple mysqLD server MyISAM that are accessing the same table, you can disable the external system lock using the -skip-external-locking option. However, external locking is disabled by default, so this option will most likely not work. Because of SHOW PROFILE, this state means that the thread is requesting a lock (without waiting for it). For system tables, use the Locking System Tables state

Query the cache status value

value meaning
checking privileges on cached query The server is checking whether the user has access to the cached query results
checking query cache for query The server is checking to see if the current query exists in the query cache
invalidating query cache entries The query cache entry is marked as invalid because the underlying table has changed
sending cached result to client The server is fetching the results of the query from the query cache and sending them to the client
storing result in query cache The server stores the query results in the query cache
Waiting for query cache lock This state occurs when the session is waiting to take a query cache lock. This may require some QUERY caching operations, such as INSERT or DELETE statements that invalidate the QUERY CACHE, RESET QUERY CACHE, and so on

Event scheduler thread state value

These states apply to the event scheduler thread, creating the thread that executes the scheduled event or terminates the scheduler.

value meaning
Clearing The scheduler thread, or thread executing an event, is terminating and is about to end
Initialized The scheduler thread or thread that will execute the event is initialized.
Waiting for next activation The scheduler has a non-empty event queue, but the next activation is in the future
Waiting for scheduler to stop Thread aSET GLOBAL event_scheduler=OFFAnd is waiting for the scheduler to stop
Waiting on empty queue The scheduler’s event queue is empty; it is sleeping

Primary library thread state value

The following list shows the most common states you might see in the State column of the Binlog Dump thread on the primary server in a master-slave replication. If the Binlog Dump thread is not visible on the master server, this means that replication is not running, that is, no Slave hosts are currently connected.

value meaning
Sending binlog event to slave Binary logs are made up of events, and an event usually adds some additional information to an update. The thread has read an event from the binary log and is sending it to the slave server.
Finished reading one binlog; switching to next binlog The thread has finished reading the binary log file and is opening the next log file to send to the slave server
Has sent all binlog to slave; waiting for binlog to be updated The thread has read all major updates from the binary log and has sent them to the slave server. The thread is now idle, waiting for new events to appear in the binary log caused by new updates on the primary server
Waiting to finalize termination A very simple state that occurs when a thread stops

Slave LIBRARY I/O thread status value

value meaning
Connecting to master The thread is trying to connect to the primary server
Checking master version A temporary state that occurs immediately after a connection is established to the master server
Registering slave on master A temporary state that occurs immediately after a connection is established to the master server
Requesting binlog dump A temporary state that occurs immediately after a connection is established to the master server. The thread sends a request to the main server for the contents of the binary log starting with the requested binary log file name and location
Waiting to reconnect after a failed binlog dump request If the binary log dump request fails (because there is no connection), the thread goes to sleep and then periodically tries to reconnect. You can specify the interval between retries using the – master-connect-retry option
Reconnecting after a failed binlog dump request The thread is trying to reconnect to the primary server
Waiting for master to send event The thread has connected to the primary server and is waiting for the binary log event to arrive. If the primary server is idle, it lasts a long time. If the wait persists slave_read_timeout seconds, a timeout occurs. At this point, the thread thinks the connection has been broken and attempts to reconnect
Queueing master event to the relay log The thread has read an event and is copying it to the relay log for processing by the SQL thread
Waiting to reconnect after a failed master event read An error occurred while reading (due to no connection) and the thread will sleep master-connect-retry seconds before attempting to reconnect
Reconnecting after a failed master event read Waiting for Master to Send Event Waiting for Master to send Event Waiting for Master to send Event Waiting for Master to send Event
Waiting for the slave SQL thread to free enough relay log space A non-zero relay_LOG_space_limit value is being used, and the relay log has grown to its combined size beyond this value. The I/O thread is waiting until the SQL thread processes the relay log content and removes part of the relay log file to free up enough space
Waiting for slave mutex on exit A very simple state that occurs when a thread stops

SQL thread state value from the library

value meaning
Reading event from the relay log The thread has read an event from the descendant log and is ready to process the event
Has read all relay log; waiting for the slave I/O thread to update it The thread has processed all events in the relay log file and is now waiting for the I/O thread to write new events to the relay log
Waiting for slave mutex on exit A very simple state that occurs when a thread stops

Connect thread state values from the library

These thread states occur on replicated slave libraries, but are associated with connected threads, not WITH I/O or SQL threads.

value meaning
Changing master The thread is processing the CHANGE MASTER TO statement
Killing slave The thread is processing a STOP SLAVE statement
Opening master dump table This state occurs after the Creating Table from Master dump
Reading master dump table data This state occurs after the Opening master dumps the table
Rebuilding the index on master dump table This state occurs after Reading master and dumping table data

Explain

Use the EXPLAIN keyword to simulate the optimizer’s execution of SQL statements, analyzing performance bottlenecks in your query or structure. Add the EXPLAIN keyword before the SELECT statement, MySQL will set a tag on the query, and executing the query will return information about the execution plan, rather than executing the SQL statement

Note: If a subquery is included in from, the subquery is still executed, putting the results into a temporary table

explain

The sample table

DROP TABLE IF EXISTS `actor`;
CREATE TABLE `actor` (`id` INT(11) NOT NULL.5`name` VARCHAR(4) DEFAULT NULL,`update_time` DATETIME DEFAULT NULL.PRIMARY KEY (`id`)) ENGINE=INNODB DEFAULT CHARSET=utf8;
INSERT INTO `actor` (`id`, `name`, `update_time`) VALUES (1.'a'.'2017 ‐ ‐ 22 15:27:18 12'), (2.'b'.'2017 ‐ ‐ 22 15:27:18 12'), (3.'c'.'2017 ‐ ‐ 22 15:27:18 12');

DROP TABLE IF EXISTS `film`;
CREATE TABLE `film` (`id` INT(11) NOT NULL AUTO_INCREMENT,`name` VARCHAR(10) DEFAULT NULL.PRIMARY KEY (`id`),KEY `idx_name` (`name`)) ENGINE=INNODB DEFAULT CHARSET=utf8;
INSERT INTO `film` (`id`, `name`) VALUES (3.'film0'), (1.'film1'), (2.'film2');

DROP TABLE IF EXISTS `film_actor`;
CREATE TABLE `film_actor` (`id` INT(11) NOT NULL,`film_id` INT(11) NOT NULL,`actor_id` INT(11) NOT NULL,`remark` VARCHAR(255) DEFAULT NULL.PRIMARY KEY (`id`),KEY `idx_film_actor_id` (`film_id`,`actor_id`)
Copy the code

Query the SQL execution plan.

  • Id column The ID column is the serial number of the select. As many as there are select, there are several ids, and the order of ids increases in the order in which the select appears.

    The larger the ID column is, the higher the execution priority is. If the IDS are the same, the execution is performed from the top down. If the ID is NULL, the execution is performed last.

  • The select_Type column indicates whether the corresponding row is a simple or complex query.

value meaning
simple A simple query that does not contain subqueries or unions
primary Complex query, the outermost select in a combination
subquery Subqueries included in select (not in the FROM clause)
derived Subqueries contained in the FROM clause

MySQL stores subquery results in a temporary table, also known as a derived table

  • The table column indicates which table the explain row is accessing

    • whenfromWhen there are subqueries in the clause, the table column isderivenNFormat that represents the current query dependencyid=NThe query is executed firstid=NThe query
    • When you haveunion, the value of the table column of the UNION RESULT is <union1,2>, 1 and 2 represent the IDS of select rows participating in the UNION.
  • The type column represents the association type or access type, that is, how does MySQL decide to look for rows in the table, the approximate range of data row records

    System > const > eq_ref > ref > range > index > all Generally speaking, it is necessary to ensure that the query reaches the range level, and ref is the best.

value meaning
NULL Mysql can decompose queries during the optimization phase without having to access tables or indexes during the execution phase. For example, if you select a minimum value in the index column, you can do this by looking up the index separately without accessing the table during execution
const, system Mysql can optimize part of a query and convert it to a constant (see here)show warningsResults). Used forprimary keyorunique key, so the table has at most one matching row, read 1 times, the speed is relatively fast. Mysql > explain extended select * from (select * from film where id = 1) TMP; mysql> show warnings;
eq_ref primary keyorunique keyAll parts of the index are joined, and at most one qualifying record is returned. This is probably the best type of join besides const, and it does not appear in simple SELECT queries. mysql> explain select * from film_actor left join film on film_actor.film_id = film.id;
ref Instead of using a unique index, eq_ref uses a normal index or a partial prefix of a unique index, which is compared to a value and may find multiple rows that match the condition. 1.Simple SELECT query. Mysql > explain select * from film where name = ‘film1’; 2.Associated table query. Idx_film_actor_id is the combined index of film_id and actor_id, using the left prefix film_id of film_actor. mysql> explain select film_id from film left join film_actor on film.id =film_actor.film_id; The range scan is usually performed in(), between,>,<, >=, etc. Use an index to retrieve rows in a given range. mysql> explain select * from actor where id > 1;
index Scan a full table index, which is usually faster than ALL. mysql> explain select * from film
all Full table scan, which means mysql has to look for rows from beginning to end. Mysql > explain select * from actor mysql> explain select * from actor
  • The possible_keys column shows which indexes the query might use to find it. Possible_keys may have columns while key is NULL when explaining possible_keys. In this case, it is because there is not much data in the table. Mysql thinks that the index is not helpful for the query and selects the full table query. If the column is NULL, there is no associated index. In this case, you can improve query performance by examining the WHERE clause to see if you can create an appropriate index, and then using Explain to see the effect.

  • The key column shows which index mysql actually uses to optimize access to the table. If no index is used, the column is NULL. If you want to force mysql to use or ignore indexes in the possible_keys column, use force index and ignore index in the query

    Mandatory INDEX: EXPLAIN SELECT *FROM account_info FORCE INDEX(I_SEQ); EXPLAIN SELECT *FROM account_info IGNORE INDEX(I_SEQ);

  • The key_len column shows the number of bytes used by mysql in the index. This value can be used to figure out which columns are used in the index.

    For example, the joint index idx_FILM_actor_id for film_actor consists of two int columns film_id and actor_id, and each int is 4 bytes. From key_len=4 in the result, it can be inferred that the query uses the first column, the film_ID column, to perform the index lookup. mysql> explain select * from film_actor where film_id = 2

Key_len is computed as follows

type Len Length (bytes) note
char(n) n Character types
varchar(n) 2 bytes The value is a string of 3n+ 2 bytes if it is UTF-8 Character types
tinyint 1 Numeric types
smallint 2 Numeric types
int 4 Numeric types
bigint 8 Numeric types
date 3 Time to type
timestamp 4 Time to type
datetime 8 Time to type
  • If the field is allowed NULL, 1 byte is required to record whether the index is NULL
  • The maximum length of the string is 768 bytes. If the string is too long, mysql will perform a process similar to the left prefix index to extract the first half of the string for indexing
  • The ref column shows the columns or constants used by the table to find values in the index of the key column. Common examples are const (constant), field names (e.g. Film.id).
  • The rows column is the number of rows that mysql estimates to be read and tested. Note that this is not the number of rows in the result set
  • The Extra column displays additional information.
value meaning
Using index Mysql > explain select film_id from film_actor where film_id = 1;
Using where Mysql > explain select * from actor where name = ‘a’; mysql> explain select * from actor where name = ‘a’;
Using index condition The columns of the query are not completely covered by the index. The where condition is a range of leading columns. mysql> explain select * from film_actor where film_id > 1;
Using temporary Mysql needs to create a temporary table to process the query. This situation is generally optimized
Using filesort Sort externally rather than by index, sort from memory when the data is small, otherwise it needs to be done on disk. In this case, index optimization is also generally considered
Select tables optimized away Mysql > explain select min(id) from film;

explain extended

Additional query optimization information will be provided in addition to explain.

EXPLAIN EXTENDED SELECT * FROM account_info LIMIT 1 OFFSET 5;



Follow throughshow warningsCommand to get an optimized query to see what the optimizer optimized.

explain partitions

Added a PARTITIONS field that shows the partitions the query will access if the query is based on a partitioned table

Query Profiler

Query Profiler is a Query diagnostic analysis tool that comes with MYSQL to analyze performance bottlenecks of a SQL statement. Normally, explain and slow Query logs cannot be used for precise analysis, but Query Profiler can locate the resource consumption of an SQL statement, such as CPU, IO, and time spent executing the SQL statement. However, this tool is only available in MYSQL 5.0.37 and above.

Viewing Function Status

This function is disabled by default. You need to manually enable it. You can perform the following operations to check whether the function is enabled on the current mysql server.

  • profilingParameter values forOFF“Indicates that the function is not enabled.
  • profiling_history_sizeParameter values for15Indicates that the query history of the last 15 times is recorded
mysql> show variables like '%profiling%';
Copy the code

Enabling Performance Collection

Run the following command to open it:

mysql> SET profiling=1;
Copy the code

Viewing the Performance List

mysql> show profiles;
Copy the code
  • Query_IDIs the ID of the query statement executed

Querying Performance Details

mysql> show profile all for query 145; 
Copy the code



AllYes View all fields. You can specify fields to view. The meanings are as follows:

field meaning
ALL Displays all cost information
BLOCK IO Displays block IO related costs
CONTEXT SWITCHES Context switch related costs
CPU Display CPU overhead information
IPC Displays sending and receiving cost information
MEMORY Display memory costs
PAGE FAULTS Displays cost information about page errors
SOURCE Displays cost information related to Source_function, Source_file, and Source_line
SWAPS Displays information about the cost of switching times

To summarize the Query Profiler usage process:

  • set profiling=1; // Open profile analysis
  • Execute SQL;
  • show profiles; Sql1,sql2
  • SHOW profile CPU,BLOCK IO io FOR query 1; // Check the CPU and I/O consumption
  • set profiling=0; // Disable profile analysis

reference

Mysql SHOW PROCESSLIST helps you understand the state changes of the primary and secondary replication threads in Mysql. Explain best practices and indexing. PDF