Feed: Planet MySQL
;
Author: Lalit
;
This Blog Post is an overview of MySQL Partitioning and its operations, Will explore following points for MySQL Partitioning,
- About Partitioning
- Partition Types
- Partition table info and SELECT Operations
- NULL Column value and MySQL Partition
- Partition table management
About Partitioning:
- It’s a plugin named as “Partition”
- MySQL Partitioning has inbuilt support for binary install setup. If you are using MySQL source then build must be configured with the -DWITH_PARTITION_STORAGE_ENGINE
- To disable partitioning start server with –skip-partition [ This will not allow any access/manipulation to partition table data.]
- For MySQL commercial edition Partitioning supported by MySQL Enterprise and MySQL Cluster edition, Partitioning not supported by MySQL Standard edition.
- MySQL Community binaries provided by Oracle include partitioning support.
- No Special configuration needed in my.cnf file.
- MySQL partitioning not supported for MERGE, CSV, or FEDERATEDstorage engines.
- Metadata information about partition tables in INFORMATION_SCHEMA.PARTITIONS
- Data and Index together will be part of Partitioning for InnoDB Storage engine.
- For MyISAM tables, data and indexes can be stored across different disk/location.
SHOW PLUGINS; SELECT PLUGIN_NAME, PLUGIN_VERSION, PLUGIN_STATUS FROM INFORMATION_SCHEMA.PLUGINS WHERE PLUGIN_NAME='partition';
Partition Types:
RANGE Partitioning: Partitioning based on Range values in column . For example, partitioning data for each year.
Example:
CREATE TABLE pt_range ( inc_id BIGINT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP )ENGINE=INNODB PARTITION BY RANGE( YEAR(created_date) ) ( PARTITION p0 VALUES LESS THAN (2000), PARTITION p1 VALUES LESS THAN (2003), PARTITION p2 VALUES LESS THAN (2006), PARTITION p3 VALUES LESS THAN (2009), PARTITION p4 VALUES LESS THAN (2012), PARTITION p5 VALUES LESS THAN (2015), PARTITION p6 VALUES LESS THAN (2018), PARTITION p7 VALUES LESS THAN MAXVALUE );
NOTE:If you are using primary key in partition table, then partitioning column must be a part of primary key otherwise table creation will fail with the following error,
ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table’s partitioning functionFor Date and Time type columns only DATE and DATETIME datatypes are supported for RANGE Partitioning. For example use of TIMESTAMP data type in partitioning column will throw the following error,
ERROR 1486 (HY000): Constant, random or timezone-dependent expressions in (sub)partitioning function are not allowed.
With Primary key:
CREATE TABLE pt_range ( inc_id bigint NOT NULL AUTO_INCREMENT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (inc_id,created_date) )ENGINE=INNODB PARTITION BY RANGE( YEAR(created_date) ) ( PARTITION p0 VALUES LESS THAN (2000), PARTITION p1 VALUES LESS THAN (2003), PARTITION p2 VALUES LESS THAN (2006), PARTITION p3 VALUES LESS THAN (2009), PARTITION p4 VALUES LESS THAN (2012), PARTITION p5 VALUES LESS THAN (2015), PARTITION p6 VALUES LESS THAN (2018), PARTITION p7 VALUES LESS THAN MAXVALUE );
LIST Partitioning: Partitioning of data the basis of list of values in partition column. For example column with status_id
Example:
CREATE TABLE pt_list ( inc_id bigint NOT NULL AUTO_INCREMENT, Inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, status_id tinyint NOT NULL, PRIMARY KEY (inc_id,created_date,status_id) ) PARTITION BY LIST (status_id) ( PARTITION p1 VALUES IN (1,2), PARTITION p2 VALUES IN (3), PARTITION p3 VALUES IN (4,5) );
Key point:
- Partitioning column must be an integer type.
List Column Portioning: This is a variant of LIST partitioning allows the use columns of types other than integer types for partitioning columns, as well as to use multiple columns as partitioning keys.
Example:
CREATE TABLE pt_list_column (
inc_id bigint NOT NULL,
inc_tital VARCHAR(250) NOT NULL,
inc_desc TEXT,
owner VARCHAR(250),
created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
dept_name varchar(100) NOT NULL DEFAULT 'other',
KEY (dept_name)
)
PARTITION BY LIST COLUMNS (dept_name) (
PARTITION p1 VALUES IN ('support','development'),
PARTITION p2 VALUES IN ('sales'),
PARTITION p3 VALUES IN ('other','admin')
);
On Disk:
pt_key.frm pt_key#P#p0.ibd pt_key#P#p1.ibd pt_key#P#p2.ibd pt_key#P#p3.ibd pt_key#P#p4.ibd
Key point:
- Supported in MySQL 5.6 version.
- Enum datatype for partition column not supported.
HASH Partitioning: MySQL Hashing will do an even distribution of data for specified partitioning column with given number of partitions.
Example:
CREATE TABLE pt_hash ( inc_id bigint NOT NULL AUTO_INCREMENT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, status_id tinyint NOT NULL, PRIMARY KEY (inc_id,created_date,status_id) ) PARTITION BY HASH (status_id) PARTITIONS 5;
On Disk:
pt_hash#P#p0.ibd pt_hash#P#p1.ibd pt_hash#P#p2.ibd pt_hash#P#p3.ibd pt_hash#P#p4.ibd
Key Points:
- If you do not include a partition clause, the number of partitions defaults to 1.
LINEAR HASH Partitioning: Linear hashing utilizes a linear powers-of-two algorithm whereas regular hashing employs the modulus of the hashing function’s value.
Example:
CREATE TABLE pt_linear_hash ( inc_id bigint NOT NULL AUTO_INCREMENT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, status_id tinyint NOT NULL, PRIMARY KEY (inc_id,created_date,status_id) ) PARTITION BY LINEAR HASH (YEAR(created_date)) PARTITIONS 12;
KEY Partitioning: Partitioning by key is similar to partitioning by hash, except that where hash partitioning employs a user-defined expression, the hashing function for key partitioning is supplied by the MySQL server.
Example:
CREATE TABLE pt_key ( inc_id bigint NOT NULL AUTO_INCREMENT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, status_id tinyint(5) NOT NULL, PRIMARY KEY (inc_id,created_date,status_id) ) PARTITION BY KEY (status_id) PARTITIONS 5;
Key point:
- Default partitioning on the basis of a primary key. If there no primary key then it will use available unique key.
Subpartitioning: [Composite Partitioning ]
Example:
CREATE TABLE pt_sub ( inc_id bigint NOT NULL AUTO_INCREMENT, inc_tital VARCHAR(250) NOT NULL, inc_desc TEXT, owner VARCHAR(250), created_date DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, status_id tinyint NOT NULL, PRIMARY KEY (inc_id,created_date,status_id) ) PARTITION BY LIST (status_id) SUBPARTITION BY HASH ( YEAR(created_date)) SUBPARTITIONS 12( PARTITION p1 VALUES IN (1,2) ENGINE = InnoDB, PARTITION p2 VALUES IN (3) ENGINE = InnoDB, PARTITION p3 VALUES IN (4,5) ENGINE = InnoDB );
On Disk:
pt_sub.frm pt_sub#P#p1#SP#p1sp4.ibd pt_sub#P#p2#SP#p2sp10.ibd pt_sub#P#p2#SP#p2sp6.ibd pt_sub#P#p3#SP#p3sp1.ibd pt_sub#P#p3#SP#p3sp8.ibd pt_sub#P#p1#SP#p1sp0.ibd pt_sub#P#p1#SP#p1sp5.ibd pt_sub#P#p2#SP#p2sp11.ibd pt_sub#P#p2#SP#p2sp7.ibd pt_sub#P#p3#SP#p3sp2.ibd pt_sub#P#p3#SP#p3sp9.ibd pt_sub#P#p1#SP#p1sp10.ibd pt_sub#P#p1#SP#p1sp6.ibd pt_sub#P#p2#SP#p2sp1.ibd pt_sub#P#p2#SP#p2sp8.ibd pt_sub#P#p3#SP#p3sp3.ibd pt_sub#P#p1#SP#p1sp11.ibd pt_sub#P#p1#SP#p1sp7.ibd pt_sub#P#p2#SP#p2sp2.ibd pt_sub#P#p2#SP#p2sp9.ibd pt_sub#P#p3#SP#p3sp4.ibd pt_sub#P#p1#SP#p1sp1.ibd pt_sub#P#p1#SP#p1sp8.ibd pt_sub#P#p2#SP#p2sp3.ibd pt_sub#P#p3#SP#p3sp0.ibd pt_sub#P#p3#SP#p3sp5.ibd pt_sub#P#p1#SP#p1sp2.ibd pt_sub#P#p1#SP#p1sp9.ibd pt_sub#P#p2#SP#p2sp4.ibd pt_sub#P#p3#SP#p3sp10.ibd pt_sub#P#p3#SP#p3sp6.ibd pt_sub#P#p1#SP#p1sp3.ibd pt_sub#P#p2#SP#p2sp0.ibd pt_sub#P#p2#SP#p2sp5.ibd pt_sub#P#p3#SP#p3sp11.ibd pt_sub#P#p3#SP#p3sp7.ibd
Partition table info and SELECT Operations:
SELECT TABLE_NAME,PARTITION_NAME,SUBPARTITION_NAME,PARTITION_METHOD,SUBPARTITION_METHOD FROM INFORMATION_SCHEMA.PARTITIONS WHERE TABLE_SCHEMA='db_name';
Sample data:
Partition details: PARTITION BY RANGE( YEAR(created_date) ) ( PARTITION p0 VALUES LESS THAN (2000), PARTITION p1 VALUES LESS THAN (2003), PARTITION p2 VALUES LESS THAN (2006), PARTITION p3 VALUES LESS THAN (2009), PARTITION p4 VALUES LESS THAN (2012), PARTITION p5 VALUES LESS THAN (2015), PARTITION p6 VALUES LESS THAN (2018), PARTITION p7 VALUES LESS THAN MAXVALUE ); mysql> select * from pt_range; +--------+-------------------+-----------------+-------+---------------------+ | inc_id | Inc_tital | inc_desc | owner | created_date | +--------+-------------------+-----------------+-------+---------------------+ | 1 | login issue | unable to login | lalit | 1999-06-26 12:33:21 | | 2 | print issue | unable to print | XYZ | 2000-09-28 12:33:21 | | 3 | print issue again | | maren | 2001-10-26 02:23:21 | | 4 | scan issue | NULL | maren | 2005-06-26 02:23:21 | | 5 | scan issue | NULL | abc | 2010-07-26 02:23:21 | | 6 | scan issue | NULL | abc | 2013-07-26 02:23:21 | | 7 | scan issue | NULL | abc | 2015-07-26 02:23:21 | | 8 | urgent | NULL | NULL | 2016-07-26 02:23:21 | | 9 | test | NULL | NULL | 2017-09-28 12:38:35 | +--------+-------------------+-----------------+-------+---------------------+ 9 rows in set (0.00 sec) mysql> SELECT * FROM pt_range PARTITION (p5); +--------+------------+----------+-------+---------------------+ | inc_id | Inc_tital | inc_desc | owner | created_date | +--------+------------+----------+-------+---------------------+ | 6 | scan issue | NULL | abc | 2013-07-26 02:23:21 | +--------+------------+----------+-------+---------------------+ 1 row in set (0.00 sec) mysql> SELECT count(*) FROM pt_range WHERE YEAR(created_date)='2000'; +----------+ | count(*) | +----------+ | 1 | +----------+ 1 row in set (0.00 sec) mysql> SELECT * FROM pt_range PARTITION (p1); +--------+-------------------+-----------------+-------+---------------------+ | inc_id | Inc_tital | inc_desc | owner | created_date | +--------+-------------------+-----------------+-------+---------------------+ | 2 | print issue | unable to print | XYZ | 2000-09-28 12:33:21 | | 3 | print issue again | | maren | 2001-10-26 02:23:21 | +--------+-------------------+-----------------+-------+---------------------+ 2 rows in set (0.00 sec)
SELECT Analysis using EXPLAIN:
mysql> explain SELECT * FROM pt_range WHERE YEAR(created_date)='2000'; +----+-------------+----------+----------------------+------+---------------+------+---------+------+------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+----------+----------------------+------+---------------+------+---------+------+------+----------+-------------+ | 1 | SIMPLE | pt_range | p0,p1,p3,p4,p5,p6,p7 | ALL | NULL | NULL | NULL | NULL | 6 | 100.00 | Using where | +----+-------------+----------+----------------------+------+---------------+------+---------+------+------+----------+-------------+ 1 row in set, 1 warning (0.00 sec) mysql> explain SELECT * FROM pt_range PARTITION (p1) WHERE YEAR(created_date)='2000'; +----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------------+ | 1 | SIMPLE | pt_range | p1 | ALL | NULL | NULL | NULL | NULL | 2 | 100.00 | Using where | +----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------------+ 1 row in set, 1 warning (0.00 sec)
NULL Column value and MySQL Partition:
Ideally, it’s better to avoid NULL value in partition columns by enforcing the NOT NULL constraint, where ever it possible.
If NULL is allowed for partition column then, partition will handle null values as follows,
| Type of Partition | Handling NULL | Example |
| RANGE Partitioning | For partition column NULL value will be inserted into lower value partition. | PARTITION p0 VALUES LESS THAN (2000),
PARTITION p1 VALUES LESS THAN (2003), PARTITION p2 VALUES LESS THAN (2006) NULL Value will go into partition p0 |
| LIST Partitioning | NULL should be part of one of the partition list value.
If NULL is not included in any partition list, Insert operation will fail with the error. |
PARTITION BY LIST (status_id) (
PARTITION p1 VALUES IN (1,2) INSERT INTO pt_list(status_id) values(NULL); ERROR 1504 (HY000): Table has no partition for value NULL ALTER TABLE pt_list REORGANIZE PARTITION p1 INTO (PARTITION p1 VALUES IN (1,2,NULL)); Now NULL values will go into p1 partition. |
| HASH and KEY Partitioning | For both types of portioning NULL treated as 0 (zero) | Hence NULL values will go to very first partition (eg . p0) |
Partition table management:
Adding new partition:
CREATE TABLE `pt_list` ( `inc_id` bigint(20) NOT NULL, `Inc_tital` varchar(250) NOT NULL, `inc_desc` text, `owner` varchar(250) DEFAULT NULL, `created_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, `status_id` tinyint(4) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 PARTITION BY LIST (status_id) (PARTITION p1 VALUES IN (1,2) ENGINE = InnoDB, PARTITION p2 VALUES IN (3) ENGINE = InnoDB, PARTITION p3 VALUES IN (4,5) ENGINE = InnoDB); mysql> ALTER TABLE pt_list ADD PARTITION (PARTITION p4 VALUES IN (6,7,8)); Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.02 sec) Records: 0 Duplicates: 0 Warnings: mysql> show create table pt_listG *************************** 1. row *************************** Table: pt_list Create Table: CREATE TABLE `pt_list` ( `inc_id` bigint(20) NOT NULL, `Inc_tital` varchar(250) NOT NULL, `inc_desc` text, `owner` varchar(250) DEFAULT NULL, `created_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, `status_id` tinyint(4) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 /*!50100 PARTITION BY LIST (status_id) (PARTITION p1 VALUES IN (1,2) ENGINE = InnoDB, PARTITION p2 VALUES IN (3) ENGINE = InnoDB, PARTITION p3 VALUES IN (4,5) ENGINE = InnoDB, PARTITION p4 VALUES IN (6,7,8) ENGINE = InnoDB) */ 1 row in set (0.00 sec)
Alter/Change Partition:
mysql> ALTER TABLE pt_list REORGANIZE PARTITION p1 INTO (PARTITION p1 VALUES IN (1,2,NULL)); Query OK, 0 rows affected (0.11 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> show create table pt_listG *************************** 1. row *************************** Table: pt_list Create Table: CREATE TABLE `pt_list` ( `inc_id` bigint(20) NOT NULL, `Inc_tital` varchar(250) NOT NULL, `inc_desc` text, `owner` varchar(250) DEFAULT NULL, `created_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, `status_id` tinyint(4) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8 /*!50100 PARTITION BY LIST (status_id) (PARTITION p1 VALUES IN (NULL,1,2) ENGINE = InnoDB, PARTITION p2 VALUES IN (3) ENGINE = InnoDB, PARTITION p3 VALUES IN (4,5) ENGINE = InnoDB, PARTITION p4 VALUES IN (6,7,8) ENGINE = InnoDB) */ 1 row in set (0.00 sec)
DROP Partition:
mysql> show create table pt_rangeG *************************** 1. row *************************** Table: pt_range Create Table: CREATE TABLE `pt_range` ( `inc_id` bigint(20) NOT NULL AUTO_INCREMENT, `Inc_tital` varchar(250) NOT NULL, `inc_desc` text, `owner` varchar(250) DEFAULT NULL, `created_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`inc_id`,`created_date`) ) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 /*!50100 PARTITION BY RANGE ( YEAR(created_date)) (PARTITION p0 VALUES LESS THAN (2000) ENGINE = InnoDB, PARTITION p1 VALUES LESS THAN (2003) ENGINE = InnoDB, PARTITION p2 VALUES LESS THAN (2006) ENGINE = InnoDB, PARTITION p3 VALUES LESS THAN (2009) ENGINE = InnoDB, PARTITION p4 VALUES LESS THAN (2012) ENGINE = InnoDB, PARTITION p5 VALUES LESS THAN (2015) ENGINE = InnoDB, PARTITION p6 VALUES LESS THAN (2018) ENGINE = InnoDB, PARTITION p7 VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */ 1 row in set (0.00 sec) mysql> select * from pt_range; +--------+-------------------+-----------------+-------+---------------------+ | inc_id | Inc_tital | inc_desc | owner | created_date | +--------+-------------------+-----------------+-------+---------------------+ | 1 | login issue | unable to login | lalit | 1999-06-26 12:33:21 | | 2 | print issue | unable to print | XYZ | 2000-09-28 12:33:21 | | 3 | print issue again | | maren | 2001-10-26 02:23:21 | | 4 | scan issue | NULL | maren | 2005-06-26 02:23:21 | | 5 | scan issue | NULL | abc | 2010-07-26 02:23:21 | | 6 | scan issue | NULL | abc | 2013-07-26 02:23:21 | | 7 | scan issue | NULL | abc | 2015-07-26 02:23:21 | | 8 | urgent | NULL | NULL | 2016-07-26 02:23:21 | | 9 | test | NULL | NULL | 2017-09-28 12:38:35 | +--------+-------------------+-----------------+-------+---------------------+ 9 rows in set (0.00 sec) mysql> ALTER TABLE pt_range DROP PARTITION p2; Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> select * from pt_range; +--------+-------------------+-----------------+-------+---------------------+ | inc_id | Inc_tital | inc_desc | owner | created_date | +--------+-------------------+-----------------+-------+---------------------+ | 1 | login issue | unable to login | lalit | 1999-06-26 12:33:21 | | 2 | print issue | unable to print | XYZ | 2000-09-28 12:33:21 | | 3 | print issue again | | maren | 2001-10-26 02:23:21 | | 5 | scan issue | NULL | abc | 2010-07-26 02:23:21 | | 6 | scan issue | NULL | abc | 2013-07-26 02:23:21 | | 7 | scan issue | NULL | abc | 2015-07-26 02:23:21 | | 8 | urgent | NULL | NULL | 2016-07-26 02:23:21 | | 9 | test | NULL | NULL | 2017-09-28 12:38:35 | +--------+-------------------+-----------------+-------+---------------------+ 8 rows in set (0.00 sec) mysql> show create table pt_rangeG *************************** 1. row *************************** Table: pt_range Create Table: CREATE TABLE `pt_range` ( `inc_id` bigint(20) NOT NULL AUTO_INCREMENT, `Inc_tital` varchar(250) NOT NULL, `inc_desc` text, `owner` varchar(250) DEFAULT NULL, `created_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`inc_id`,`created_date`) ) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 /*!50100 PARTITION BY RANGE ( YEAR(created_date)) (PARTITION p0 VALUES LESS THAN (2000) ENGINE = InnoDB, PARTITION p1 VALUES LESS THAN (2003) ENGINE = InnoDB, PARTITION p3 VALUES LESS THAN (2009) ENGINE = InnoDB, PARTITION p4 VALUES LESS THAN (2012) ENGINE = InnoDB, PARTITION p5 VALUES LESS THAN (2015) ENGINE = InnoDB, PARTITION p6 VALUES LESS THAN (2018) ENGINE = InnoDB, PARTITION p7 VALUES LESS THAN MAXVALUE ENGINE = InnoDB) */ 1 row in set (0.00 sec)
Advantages of partitioning:
- Allow to store more data efficiently.
- An unused portion of data can be easily removed from a table.
- Due to the partitioning of data on physical level Read/Write operations are faster as compare non-partition table.
Refer MySQL official PARTITIONING_DOC for more…
Advertisements