Tuesday, August 28, 2012

Import Big InnoDB Tables

Import Big InnoDB Tables

  1. turn off the logs;
  2. turn off unique key check if the table has;
  3. turn off foreign key check;
  • When importing data into InnoDB, make sure that MySQL does not have autocommit mode enabled because that requires a log flush to disk for every insert. To disable autocommit during your import operation, surround it with SET autocommit and COMMIT statements:  
  • SET autocommit=0; 
  • ... SQL import statements ... 
  • COMMIT; 
  • If you use the mysqldump option --opt, you get dump files that are fast to import into an InnoDB table, even without wrapping them with the SET autocommit and COMMIT statements. 
  • If you have UNIQUE constraints on secondary keys, starting from MySQL 3.23.52 and 4.0.3, you can speed up table imports by temporarily turning off the uniqueness checks during the import session: 
  • SET unique_checks=0; 
  • ... SQL import statements ... 
  • SET unique_checks=1; 
  • For big tables, this saves a lot of disk I/O because InnoDB can use its insert buffer to write secondary index records in a batch. Be certain that the data contains no duplicate keys.
  • # If you have FOREIGN KEY constraints in your tables, starting from MySQL 3.23.52 and 4.0.3, you can speed up table imports by turning the foreign key checks off for a while in the import session:  
  • SET foreign_key_checks=0; 
  • ... SQL import statements ... 
  • SET foreign_key_checks=1; 
  • For big tables, this can save a lot of disk I/O. 
  •  
  • If the above solution still can not quick your import process. Try mysqlimport. You can use mysqldump to dump a sql file and a text file. There is an example, look like this:

How to shrink an ibdata1 file with minimal MySQL downtime

The default mySQL configuration for InnoDB database tables creates a massive storage file called ‘ibdata1′. Basically, the ibdata1 file contains the table data of your InnoDB tables. In large production environments, this file can grow to be extremely large. On some of the servers I administer, I’ve seen this file exceed sizes of 30GB. Fixing the file size obviously has the effect of limiting the total amount of data which can be stored in InnoDB tables, so that’s not a viable option.

The ibdata1 file is by default ‘auto-growing’, so it will inflate as more data is put into InnoDB tables. After records are deleted from InnoDB tables, the file will contain pages marked as “free” which could be used for future data, but the file itself is unshrinkable.

The inability to shrink this file is a particularly annoying feature of MySQL. The ibdata1 file can’t actually be shrunk unless you delete all databases, remove the files and reload a dump. I’ve come up with a solution to do this with minimal downtime for the mySQL service (the length of time it takes for a normal service restart).

To do this, it’s necessary that you have enough disk space available to double your mySQL storage footprint. We’re in essence going to be spawning up a new service in a temporary ‘staging’ area, and then importing the data back in using the file-per-table option.
CAUTION: Be sure you have backups of your data before performing these operations. It is possible that something could go horribly wrong and ruin your day/week/month/year.


IMPORTANT: Before shrinking the ibdata file, be sure my.cnf is configured to use separate files per innoDB table:
sed ‘/innodb_file_per_table/d’ -i /etc/my.cnf
echo ‘innodb_file_per_table’ >> /etc/my.cnf
1. Get list of all InnoDB databases
mysql> SELECT `TABLE_SCHEMA`,`TABLE_NAME` FROM information_schema.tables WHERE ENGINE = ‘InnoDB’
2. Dump all of the databases to /root/all-databases.sql
/usr/bin/mysqldump –extended-insert –all-databases –add-drop-database –disable-keys –flush-privileges –quick –routines –triggers > /root/all-databases.sql
3. Prepare a secondary venue for which we can perform our magic
mkdir /var/lib/mysql2 # create a new staging area
rsync -avz /var/lib/mysql/mysql /var/lib/mysql2 # copy the mysql database w/grants
chown -R mysql.mysql /var/lib/mysql2 # set permissions correctly
4. Spin up the new mysqld instance, allowing it to create a fresh ibdata1 environment
Notes: to avoid conflicts we’re using temporary pid/socket/error files here, and skipping networking binding of 3306
/usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql2 –plugin-dir=/usr/lib64/mysql/plugin –user=mysql –log-error=/var/lib/mysql2/error.log –pid-file=/var/lib/mysql2/temp.pid –skip-networking –socket=/var/lib/mysql2/dirty.sock
5. Check out our new environment, ensure there’s no databases other than information_schema and mysql
# ls -lah /var/lib/mysql2
total 29M
drwxr-xr-x 3 mysql mysql 4.0K Aug 2 08:46 ./
drwxr-xr-x 28 root root 4.0K Aug 2 08:39 ../
srwxrwxrwx 1 mysql mysql 0 Aug 2 08:46 dirty.sock=
-rw-rw—- 1 mysql mysql 7.2K Aug 2 08:46 error.log
-rw-rw—- 1 mysql mysql 18M Aug 2 08:41 ibdata1
-rw-rw—- 1 mysql mysql 5.0M Aug 2 08:46 ib_logfile0
-rw-rw—- 1 mysql mysql 5.0M Aug 2 08:41 ib_logfile1

drwx–x–x 2 mysql mysql 4.0K Jun 27 07:27 mysql/
-rw-rw—- 1 mysql mysql 5 Aug 2 08:46 temp.pid # mysql -S /var/lib/mysql2/dirty.sock
mysql> show databases;
+——————–+
| Database |
+——————–+
| information_schema |
| mysql |
+——————–+
2 rows in set (0.00 sec)
6. In order for a successful import, we need to be able to drop the log tables. To do this we need to temporarily disable them:
mysql> SET @old_log_state = @@global.general_log;
mysql> SET GLOBAL general_log = ‘OFF’;
mysql> ALTER TABLE mysql.general_log ENGINE = MyISAM;
mysql> SET @old_log_state = @@global.slow_query_log;
mysql> SET GLOBAL slow_query_log = ‘OFF’;
mysql> SET GLOBAL log_slow_queries = ‘OFF’;
mysql> ALTER TABLE mysql.slow_query_log ENGINE = MyISAM;
7. While still in the console for the secondary server, disable foreign key settings and import, then enable foreign key checks again
# mysql -S /var/lib/mysql2/dirty.sock
mysql> SET FOREIGN_KEY_CHECKS=0;
mysql> SOURCE /root/all-databases.sql;
mysql> SET FOREIGN_KEY_CHECKS=1;
8. Finally, shut down both servers and move the new mysql directory into place. Upon restart, your conversion and shrinkage should be successful.
# service mysql stop
# killall mysqld
# mv /var/lib/mysql /var/lib/mysql.old
# mv /var/lib/mysql2 /var/lib/mysql
# service mysql start

Note: logging will automatically be turned back on provided you have the appropriate settings defined in your my.cnf

Monday, August 27, 2012

Common utilities for MySQL: openark tool kit

oak-online-alter-table: Perform a non-blocking ALTER TABLE operation

oak-online-alter-table

NAME

oak-online-alter-table: Perform a non-blocking ALTER TABLE operation

SYNOPSIS

Add an index on the Population column, City table, world database:
oak-online-alter-table --database=world --table=City --alter="ADD KEY(Population)"
Same as above, specify database with table name:
oak-online-alter-table --table=world.City --alter="ADD KEY(Population)"
Perform multiple alters:
oak-online-alter-table --table=world.City --alter="ADD KEY(Population), MODIFY Name VARCHAR(64) CHARSET utf8 NOT NULL"
Add column, drop column, modify AUTO_INCREMENT:
oak-online-alter-table --table=world.City --alter="ADD COLUMN Mayor VARCHAR(64) CHARSET utf8 DEFAULT '', DROP COLUMN district, MODIFY id SMALLINT AUTO_INCREMENT"
Do not ALTER, just rebuild City:
oak-online-alter-table --database=world --table=City
Create a ghost table called City_mirror, which is updated online from City table:
oak-online-alter-table --database=world --table=City --alter="ADD KEY(Population)" --ghost=City_mirror
Perform ALTER in chunks of 5000 rows:
oak-online-alter-table --table=world.City --alter="ADD KEY(Population)" --chunk-size=5000
Perform ALTER in chunks of 5000 rows, sleep for 20 milliseconds between chunks:
oak-online-alter-table --table=world.City --alter="ADD KEY(Population)" --chunk-size=5000 --sleep=20
Perform a cleanup for an aborted run:
oak-online-alter-table --database=world --table=City --cleanup
Provide connection parameters. Prompt for password:
oak-online-alter-table --database=world --table=City --alter="ADD KEY(Population)" --user=root --ask-pass --socket=/tmp/mysql.sock
Use a defaults file for connection parameters:
oak-online-alter-table --database=world --table=City --alter="ADD KEY(Population)" --defaults-file=/home/myuser/.my-oak.cnf

DESCRIPTION

oak-online-alter-table allows for non blocking ALTER TABLE operations, table rebuilds and creating a table's ghost. To run this utility, the following requirements must be met:
  • The table has at least one single-column UNIQUE KEY
  • Altered table shares a single-column UNIQUE KEY with the original table
  • No ‘AFTER’ triggers are defined on the table (the utility creates its own triggers for the duration of the operation)
  • The table has no FOREIGN KEYs
  • Table name is no longer than 57 characters
The utility provides with three basic functionalities:
  • A non blocking ALTER TABLE operation: adding columns or keys, dropping columns or keys, modifying columns, converting table character set etc., are all supported, under the following restrictions:
    • ADD COLUMN (new column must have a default value)
    • DROP COLUMN (as long as there’s still a shared single-column UNIQUE KEY between the old table and the altered one)
    • MODIFY COLUMN (change type, including UNIQUE KEY columns)
    • ADD KEY (normal, unique, fulltext etc.)
    • DROP KEY (as long as there’s still a shared single-column UNIQUE KEY between the old table and the altered one)
    • Change ENGINE: works, but great care should be taken when dealing with non-transactional engines
    • Adding FOREIGN KEY constraints is possible
    • More… Not all ALTERS have been tested, e.g. CONVERT, partition handling…
  • An empty ALTER, which rebuilds a table: potentially releasing disk space and organizing rows in a more optimized way on disk.
  • (May not be supported in future versions): Creating a live 'ghost' table, which is synchronized with the original table, as long as the following do not occur:
    • An ALTER TABLE on the original table
    • A TRUNCATE on the original table
    • LOAD DATA INFILE into the original table
    • OPTIMIZE on the original table
While the utility runs, it is allowed to invoke INSERT, UPDATE, DELETE, REPLACE on the original table. It is not allowed to TRUNCATE, ALTER, REPAIR, OPTIMIZE or otherwise perform a table-wide operation on the original table. The utility works with InnoDB tables, MyISAM tables, or otherwise any table-level locking storage engine (MEMORY, ARCHIVE). There is currently no support for PBXT or Falcon. The utility works by creating a ghost table for the duration of the runtime, which is slowly synchronized with the original table. When the synchronization is complete, it takes place of the original table. To do this, the utility must create AFTER INSERT, AFTER UPDATE, AFTER DELETE triggers on the original table. Synchronizing the ghost table with the original tables takes place in several steps. In one of those steps, data is copied from the original table to the ghost table. This is done in chunks of rows, the number of which is configurable using the chunk-size option. While a chunk is being copied, there is a read lock on the table (MyISAM, ARCHIVE, MEMORY) or on the rows contained in the chunk (InnoDB). The smaller the chunk - the faster the locks are removed, and the more concurrency is allowed; but also the longer it will take for the entire operation to complete. For write intensive application, it may be advisable to allow for pauses between chunks, so as to make as little impact as possible. This can be configured with the sleep option. While sleeping between chunks, no locks are placed. Even so, a performance impact is noticed while the application runs, and this is due to the triggers added on the table, and the propagation of DML statements to the ghost table. It is required to have enough disk space to accomodate the altered table (as in a normal ALTER TABLE). Only when the operation completes can there be a disk space regaining (depending on your storage engine and configuration).

OPTIONS

-a ALTER_STATEMENT, --alter=ALTER_STATEMENT
Comma delimited ALTER statement details, excluding the 'ALTER TABLE t' itself. When this options is not used, an 'empty' ALTER is performed, essentially rebuilding the table.
--ask-pass
Prompt for password.
-c CHUNK_SIZE, --chunk-size=CHUNK_SIZE
Number of rows to act on in chunks. Default: 1000. The lower the number, the shorter any locks are held, but the more operations required and the more total running time. Do not use very low values when the PRIMARY KEY, or otherwise the only UNIQUE KEY are on textual columns, as values from such keys are reused when working the chunks. If you're not sure - stick with the defaults.
--cleanup
Remove custom triggers, ghost table from possible previous runs. In case a previous run was abruptly terminated, this option removes all custom data this utility may have created. It is not necessary to run with this option after a normal completion.
-d DATABASE, --database=DATABASE
Database name (required unless table is fully qualified)
--defaults-file=DEFAULTS_FILE
Read from MySQL configuration file. Overrides --user, --password, --socket, --port.
Configuration needs to be in the following format:
[client]
user=my_user
password=my_pass
socket=/tmp/mysql.sock
port=3306
-g GHOST, --ghost=GHOST
Table name to serve as ghost. When this option is used, a table by this name is created and synchronized with the original table. The original table is thereafter unaltered, and the three AFTER INSERT, AFTER UPDATE and AFTER DELETE triggers are maintained. To perform an ALTER TABLE, do not use this option. [May be removed in future versions]
-H HOST, --host=HOST
MySQL host (default: localhost)
-l, --lock-chunks
Use LOCK TABLES for each chunk. This option enforces a higher locking mechanism, and is at current available as preparation to be able to work with unsupported engines. It is not required nor advisable to use this option with MyISAM or InnoDB engines. [May be removed in future versions]. -p PASSWORD, --password=PASSWORD
MySQL password
-P PORT, --port=PORT
TCP/IP port (default: 3306)
-q, --quiet
Quiet mode, do not verbose. Verbose is on by default, use this option to turn it off.
--sleep=SLEEP_MILLIS
Number of milliseconds to sleep between chunks. Default: 0. Use a higher value if your system suffers from high load average. The higher the value, the more time the system is allowed to 'rest', but also the longer the runtime, and the more overhead (over time) from the triggers effect. You may also wish to set a higher number if slaves are finding it hard otherwise to catch up.
-S SOCKET, --socket=SOCKET
MySQL socket file. Only applies when host is localhost
-t TABLE, --table=TABLE
Table with AUTO_INCREMENT column to alter (optionally fully qualified as database_name.table_name, in which case --database is not required)
-u USER, --user=USER
MySQL user
-v, --verbose
Print user friendly messages. Enabled by default.

Steps to take to reset slave and delete old connect info alt. break master-master seup.


On the slave server(s): 
alt.
on the Master when you want to break a master-master replication setup:

1.      Run 'stop slave' to stop replication.
2.      Run 'reset slave' to tell the slave server to forget it's position in the binary log retrieved from the master server.
3.      Add 'skip-slave-start' to my.cnf to prevent replication from starting when you restart MySQL.

Done.

Tuesday, August 21, 2012

Queries to use and diagnose blocking and locking threads 

... only works in MySQL version 5.5.x

This is the Query that is blocking :

--------------
SELECT r.trx_id waiting_trx_id,  r.trx_mysql_thread_id waiting_thread,
       r.trx_query waiting_query,
       b.trx_id blocking_trx_id, b.trx_mysql_thread_id blocking_thread,
       b.trx_query blocking_query
FROM       information_schema.innodb_lock_waits w
INNER JOIN information_schema.innodb_trx b  ON  b.trx_id = w.blocking_trx_id
INNER JOIN information_schema.innodb_trx r  ON  r.trx_id = w.requesting_trx_id
--------------

Empty set (0.00 sec)

=======================================================================

Tell you how long it has waited and from which host and port

... only works in MySQL version 5.5.x

--------------
select r.trx_id as waiting_trx_id, r.trx_mysql_thread_id as waiting_thread,
 timestampdiff(second, r.trx_wait_started, current_timestamp) as wait_time,
 r.trx_query as waiting_query,
 l.lock_table as waiting_table_lock,
 b.trx_id as blocking_trx_id, b.trx_mysql_thread_id as blocking_thread,
 substring(p.host, 1, instr(p.host, ':') - 1) as blocking_host,
 substring(p.host,    instr(p.host, ':') + 1) as blocking_port,
 if(p.command = "Sleep" , p.time, 0) as idle_in_trx,
 b.trx_query as blocking_query
FROM INFORMATION_SCHEMA.INNODB_LOCK_WAITS AS w
INNER JOIN INFORMATION_SCHEMA.INNODB_TRX  AS b ON b.trx_id = w.blocking_trx_id
INNER JOIN INFORMATION_SCHEMA.INNODB_TRX  AS r ON b.trx_id = w.requesting_trx_id
INNER JOIN INFORMATION_SCHEMA.INNODB_LOCKS AS l ON w.requested_lock_id = l.lock_id
LEFT  JOIN INFORMATION_SCHEMA.PROCESSLIST  AS p ON p.id = b.trx_mysql_thread_id
ORDER BY wait_time DESC
--------------

Empty set (0.00 sec)

========================================================================

... show how many queries are blocked on which threads...

--------------
SELECT CONCAT('thread ' , b.trx_mysql_thread_id , ' from ' , p.host) as who_blocks,
       IF(p.command = "Sleep", p.time, 0 ) as idle_in_trx,
       MAX(TIMESTAMPDIFF(SECOND, r.trx_wait_started, NOW() )) as max_wait_time,
       COUNT(*) as num_waiters
FROM INFORMATION_SCHEMA.INNODB_LOCK_WAITS AS w
INNER JOIN INFORMATION_SCHEMA.INNODB_TRX  AS b ON b.trx_id = w.blocking_trx_id
INNER JOIN INFORMATION_SCHEMA.INNODB_TRX  AS r ON b.trx_id = w.requesting_trx_id
LEFT  JOIN INFORMATION_SCHEMA.PROCESSLIST AS p ON p.id = b.trx_mysql_thread_id
GROUP BY who_blocks ORDER BY num_waiters DESC
--------------

Empty set (0.00 sec)

MySQL String Functions :


Name Description
ASCII() Return numeric value of left-most character
BIN() Return a string representation of the argument
BIT_LENGTH() Return length of argument in bits
CHAR_LENGTH() Return number of characters in argument
CHAR() Return the character for each integer passed
CHARACTER_LENGTH() A synonym for CHAR_LENGTH()
CONCAT_WS() Return concatenate with separator
CONCAT() Return concatenated string
CONV() Convert numbers between different number bases
ELT() Return string at index number
EXPORT_SET() Return a string such that for every bit set in the value bits, you get an on string and for every unset bit, you get an off string
FIELD() Return the index (position) of the first argument in the subsequent arguments
FIND_IN_SET() Return the index position of the first argument within the second argument
FORMAT() Return a number formatted to specified number of decimal places
HEX() Return a string representation of a hex value
INSERT() Insert a substring at the specified position up to the specified number of characters
INSTR() Return the index of the first occurrence of substring
LCASE() Synonym for LOWER()
LEFT() Return the leftmost number of characters as specified
LENGTH() Return the length of a string in bytes
LOAD_FILE() Load the named file
LOCATE() Return the position of the first occurrence of substring
LOWER() Return the argument in lowercase
LPAD() Return the string argument, left-padded with the specified string
LTRIM() Remove leading spaces
MAKE_SET() Return a set of comma-separated strings that have the corresponding bit in bits set
MID() Return a substring starting from the specified position
OCT() Return a string representation of the octal argument
OCTET_LENGTH() A synonym for LENGTH()
ORD() If the leftmost character of the argument is a multi-byte character, returns the code for that character
POSITION() A synonym for LOCATE()
QUOTE() Escape the argument for use in an SQL statement
REGEXP Pattern matching using regular expressions
REPEAT() Repeat a string the specified number of times
REPLACE() Replace occurrences of a specified string
REVERSE() Reverse the characters in a string
RIGHT() Return the specified rightmost number of characters
RPAD() Append string the specified number of times
RTRIM() Remove trailing spaces
SOUNDEX() Return a soundex string
SOUNDS LIKE Compare sounds
SPACE() Return a string of the specified number of spaces
STRCMP() Compare two strings
SUBSTRING_INDEX() Return a substring from a string before the specified number of occurrences of the delimiter
SUBSTRING(), SUBSTR() Return the substring as specified
TRIM() Remove leading and trailing spaces
UCASE() Synonym for UPPER()
UNHEX() Convert each pair of hexadecimal digits to a character
UPPER() Convert to uppercase

ASCII(str)

Returns the numeric value of the leftmost character of the string str. Returns 0 if str is the empty string. Returns NULL if str is NULL. ASCII() works for characters with numeric values from 0 to 255.
mysql> SELECT ASCII('2');
+---------------------------------------------------------+
| ASCII('2')                                              |
+---------------------------------------------------------+
| 50                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT ASCII('dx');
+---------------------------------------------------------+
| ASCII('dx')                                             |
+---------------------------------------------------------+
| 100                                                     |
+---------------------------------------------------------+
1 row in set (0.00 sec)

BIN(N)

Returns a string representation of the binary value of N, where N is a longlong (BIGINT) number. This is equivalent to CONV(N,10,2). Returns NULL if N is NULL.
mysql> SELECT BIN(12);
+---------------------------------------------------------+
| BIN(12)                                                 |
+---------------------------------------------------------+
| 1100                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

BIT_LENGTH(str)

Returns the length of the string str in bits.
mysql> SELECT BIT_LENGTH('text');
+---------------------------------------------------------+
| BIT_LENGTH('text')                                      |
+---------------------------------------------------------+
| 32                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

CHAR(N,... [USING charset_name])

CHAR() interprets each argument N as an integer and returns a string consisting of the characters given by the code values of those integers. NULL values are skipped.
mysql> SELECT CHAR(77,121,83,81,'76');
+---------------------------------------------------------+
| CHAR(77,121,83,81,'76')                                 |
+---------------------------------------------------------+
| MySQL                                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)

CHAR_LENGTH(str)

Returns the length of the string str, measured in characters. A multi-byte character counts as a single character. This means that for a string containing five two-byte characters, LENGTH() returns 10, whereas CHAR_LENGTH() returns 5.
mysql> SELECT CHAR_LENGTH("text");
+---------------------------------------------------------+
| CHAR_LENGTH("text")                                     |
+---------------------------------------------------------+
| 4                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

CHARACTER_LENGTH(str)

CHARACTER_LENGTH() is a synonym for CHAR_LENGTH().

CONCAT(str1,str2,...)

Returns the string that results from concatenating the arguments. May have one or more arguments. If all arguments are non-binary strings, the result is a non-binary string. If the arguments include any binary strings, the result is a binary string. A numeric argument is converted to its equivalent binary string form; if you want to avoid that, you can use an explicit type cast, as in this example:
mysql> SELECT CONCAT('My', 'S', 'QL');
+---------------------------------------------------------+
| CONCAT('My', 'S', 'QL')                                 |
+---------------------------------------------------------+
| MySQL                                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)

CONCAT_WS(separator,str1,str2,...)

CONCAT_WS() stands for Concatenate With Separator and is a special form of CONCAT(). The first argument is the separator for the rest of the arguments. The separator is added between the strings to be concatenated. The separator can be a string, as can the rest of the arguments. If the separator is NULL, the result is NULL.
mysql> SELECT CONCAT_WS(',','First name','Last Name' );
+---------------------------------------------------------+
| CONCAT_WS(',','First name','Last Name' )                |
+---------------------------------------------------------+
| First name, Last Name                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)

CONV(N,from_base,to_base)

Converts numbers between different number bases. Returns a string representation of the number N, converted from base from_base to base to_base. Returns NULL if any argument is NULL. The argument N is interpreted as an integer, but may be specified as an integer or a string. The minimum base is 2 and the maximum base is 36. If to_base is a negative number, N is regarded as a signed number. Otherwise, N is treated as unsigned. CONV() works with 64-bit precision.
mysql> SELECT CONV('a',16,2);
+---------------------------------------------------------+
| CONV('a',16,2)                                          |
+---------------------------------------------------------+
| 1010                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

ELT(N,str1,str2,str3,...)

Returns str1 if N = 1, str2 if N = 2, and so on. Returns NULL if N is less than 1 or greater than the number of arguments. ELT() is the complement of FIELD().
mysql> SELECT ELT(1, 'ej', 'Heja', 'hej', 'foo');
+---------------------------------------------------------+
| ELT(1, 'ej', 'Heja', 'hej', 'foo')                      |
+---------------------------------------------------------+
| ej                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

EXPORT_SET(bits,on,off[,separator[,number_of_bits]])

Returns a string such that for every bit set in the value bits, you get an on string and for every bit not set in the value, you get an off string. Bits in bits are examined from right to left (from low-order to high-order bits). Strings are added to the result from left to right, separated by the separator string (the default being the comma character .,.). The number of bits examined is given by number_of_bits (defaults to 64).
mysql> SELECT EXPORT_SET(5,'Y','N',',',4);
+---------------------------------------------------------+
| EXPORT_SET(5,'Y','N',',',4)                             |
+---------------------------------------------------------+
| Y,N,Y,N                                                 |
+---------------------------------------------------------+
1 row in set (0.00 sec)

FIELD(str,str1,str2,str3,...)

Returns the index (position starting with 1) of str in the str1, str2, str3, ... list. Returns 0 if str is not found.
mysql> SELECT FIELD('ej', 'Hej', 'ej', 'Heja', 'hej', 'foo');
+---------------------------------------------------------+
| FIELD('ej', 'Hej', 'ej', 'Heja', 'hej', 'foo')          |
+---------------------------------------------------------+
| 2                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

FIND_IN_SET(str,strlist)

Returns a value in the range of 1 to N if the string str is in the string list strlist consisting of N substrings.
mysql> SELECT FIND_IN_SET('b','a,b,c,d');
+---------------------------------------------------------+
| SELECT FIND_IN_SET('b','a,b,c,d')                       |
+---------------------------------------------------------+
| 2                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

FORMAT(X,D)

Formats the number X to a format like '#,###,###.##', rounded to D decimal places, and returns the result as a string. If D is 0, the result has no decimal point or fractional part.
mysql> SELECT FORMAT(12332.123456, 4);
+---------------------------------------------------------+
| FORMAT(12332.123456, 4)                                 |
+---------------------------------------------------------+
| 12,332.1235                                             |
+---------------------------------------------------------+
1 row in set (0.00 sec)

HEX(N_or_S)

If N_or_S is a number, returns a string representation of the hexadecimal value of N, where N is a longlong (BIGINT) number. This is equivalent to CONV(N,10,16).
If N_or_S is a string, returns a hexadecimal string representation of N_or_S where each character in N_or_S is converted to two hexadecimal digits.
mysql> SELECT HEX(255);
+---------------------------------------------------------+
| HEX(255)                                                |
+---------------------------------------------------------+
| FF                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT 0x616263;
+---------------------------------------------------------+
| 0x616263                                                |
+---------------------------------------------------------+
| abc                                                     |
+---------------------------------------------------------+
1 row in set (0.00 sec)

INSERT(str,pos,len,newstr)

Returns the string str, with the substring beginning at position pos and len characters long replaced by the string newstr. Returns the original string if pos is not within the length of the string. Replaces the rest of the string from position pos if len is not within the length of the rest of the string. Returns NULL if any argument is NULL.
mysql> SELECT INSERT('Quadratic', 3, 4, 'What');
+---------------------------------------------------------+
| INSERT('Quadratic', 3, 4, 'What')                       |
+---------------------------------------------------------+
| QuWhattic                                               |
+---------------------------------------------------------+
1 row in set (0.00 sec)

INSTR(str,substr)

Returns the position of the first occurrence of substring substr in string str. This is the same as the two-argument form of LOCATE(), except that the order of the arguments is reversed.
mysql> SELECT INSTR('foobarbar', 'bar');
+---------------------------------------------------------+
| INSTR('foobarbar', 'bar')                               |
+---------------------------------------------------------+
| 4                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LCASE(str)

LCASE() is a synonym for LOWER().
LEFT(str,len)
Returns the leftmost len characters from the string str, or NULL if any argument is NULL.
mysql> SELECT LEFT('foobarbar', 5);
+---------------------------------------------------------+
| LEFT('foobarbar', 5)                                    |
+---------------------------------------------------------+
| fooba                                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LENGTH(str)

Returns the length of the string str, measured in bytes. A multi-byte character counts as multiple bytes. This means that for a string containing five two-byte characters, LENGTH() returns 10, whereas CHAR_LENGTH() returns 5.
mysql> SELECT LENGTH('text');
+---------------------------------------------------------+
| LENGTH('text')                                          |
+---------------------------------------------------------+
| 4                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LOAD_FILE(file_name)

Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full pathname to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes.
If the file does not exist or cannot be read because one of the preceding conditions is not satisfied, the function returns NULL.
As of MySQL 5.0.19, the character_set_filesystem system variable controls interpretation of filenames that are given as literal strings.
mysql> UPDATE table_test
    -> SET blob_col=LOAD_FILE('/tmp/picture')
 -> WHERE id=1;
...........................................................

LOCATE(substr,str), LOCATE(substr,str,pos)

The first syntax returns the position of the first occurrence of substring substr in string str. The second syntax returns the position of the first occurrence of substring substr in string str, starting at position pos. Returns 0 if substr is not in str.
mysql> SELECT LOCATE('bar', 'foobarbar');
+---------------------------------------------------------+
| LOCATE('bar', 'foobarbar')                              |
+---------------------------------------------------------+
| 4                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LOWER(str)

Returns the string str with all characters changed to lowercase according to the current character set mapping.
mysql> SELECT LOWER('QUADRATICALLY');
+---------------------------------------------------------+
| LOWER('QUADRATICALLY')                                  |
+---------------------------------------------------------+
| quadratically                                           |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LPAD(str,len,padstr)

Returns the string str, left-padded with the string padstr to a length of len characters. If str is longer than len, the return value is shortened to len characters.
mysql> SELECT LPAD('hi',4,'??');
+---------------------------------------------------------+
| LPAD('hi',4,'??')                                       |
+---------------------------------------------------------+
| ??hi                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

LTRIM(str)

Returns the string str with leading space characters removed.
mysql> SELECT LTRIM('  barbar');
+---------------------------------------------------------+
| LTRIM('  barbar')                                       |
+---------------------------------------------------------+
| barbar                                                  |
+---------------------------------------------------------+
1 row in set (0.00 sec)

MAKE_SET(bits,str1,str2,...)

Returns a set value (a string containing substrings separated by .,. characters) consisting of the strings that have the corresponding bit in bits set. str1 corresponds to bit 0, str2 to bit 1, and so on. NULL values in str1, str2, ... are not appended to the result.
mysql> SELECT MAKE_SET(1,'a','b','c');
+---------------------------------------------------------+
| MAKE_SET(1,'a','b','c')                                 |
+---------------------------------------------------------+
| a                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

MID(str,pos,len)

MID(str,pos,len) is a synonym for SUBSTRING(str,pos,len).

OCT(N)

Returns a string representation of the octal value of N, where N is a longlong (BIGINT) number. This is equivalent to CONV(N,10,8). Returns NULL if N is NULL.
mysql> SELECT OCT(12);
+---------------------------------------------------------+
| OCT(12)                                                 |
+---------------------------------------------------------+
| 14                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

OCTET_LENGTH(str)

OCTET_LENGTH() is a synonym for LENGTH().

ORD(str)

If the leftmost character of the string str is a multi-byte character, returns the code for that character, calculated from the numeric values of its constituent bytes using this formula:
  (1st byte code)
+ (2nd byte code � 256)
+ (3rd byte code � 2562) ...
If the leftmost character is not a multi-byte character, ORD() returns the same value as the ASCII() function.
mysql> SELECT ORD('2');
+---------------------------------------------------------+
| ORD('2')                                                |
+---------------------------------------------------------+
| 50                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)

POSITION(substr IN str)

POSITION(substr IN str) is a synonym for LOCATE(substr,str).

QUOTE(str)

Quotes a string to produce a result that can be used as a properly escaped data value in an SQL statement. The string is returned enclosed by single quotes and with each instance of single quote (.'.), backslash (.\.), ASCII NUL, and Control-Z preceded by a backslash. If the argument is NULL, the return value is the word .NULL. without enclosing single quotes.
mysql> SELECT QUOTE('Don\'t!');
+---------------------------------------------------------+
| QUOTE('Don\'t!')                                        |
+---------------------------------------------------------+
| 'Don\'t!'                                               |
+---------------------------------------------------------+
1 row in set (0.00 sec)
NOTE: Please check if your installation has any bug with this function then don't use this function.

expr REGEXP pattern

This function performs a pattern match of expr against pattern. Returns 1 if expr matches pat; otherwise it returns 0. If either expr or pat is NULL, the result is NULL. REGEXP is not case sensitive, except when used with binary strings.
mysql> SELECT 'ABCDEF' REGEXP 'A%C%%';
+---------------------------------------------------------+
| 'ABCDEF' REGEXP 'A%C%%'                                 |
+---------------------------------------------------------+
| 0                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)
Another example is:
mysql> SELECT 'ABCDE' REGEXP '.*';
+---------------------------------------------------------+
|  'ABCDE' REGEXP '.*'                                    |
+---------------------------------------------------------+
| 1                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)
Let's see one more example:
mysql> SELECT 'new*\n*line' REGEXP 'new\\*.\\*line';
+---------------------------------------------------------+
| 'new*\n*line' REGEXP 'new\\*.\\*line'                   |
+---------------------------------------------------------+
| 1                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

REPEAT(str,count)

Returns a string consisting of the string str repeated count times. If count is less than 1, returns an empty string. Returns NULL if str or count are NULL.
mysql> SELECT REPEAT('MySQL', 3);
+---------------------------------------------------------+
| REPEAT('MySQL', 3)                                      |
+---------------------------------------------------------+
| MySQLMySQLMySQL                                         |
+---------------------------------------------------------+
1 row in set (0.00 sec)

REPLACE(str,from_str,to_str)

Returns the string str with all occurrences of the string from_str replaced by the string to_str. REPLACE() performs a case-sensitive match when searching for from_str.
mysql> SELECT REPLACE('www.mysql.com', 'w', 'Ww');
+---------------------------------------------------------+
| REPLACE('www.mysql.com', 'w', 'Ww')                     |
+---------------------------------------------------------+
| WwWwWw.mysql.com                                        |
+---------------------------------------------------------+
1 row in set (0.00 sec)

REVERSE(str)

Returns the string str with the order of the characters reversed.
mysql> SELECT REVERSE('abcd');
+---------------------------------------------------------+
| REVERSE('abcd')                                         |
+---------------------------------------------------------+
| dcba                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

RIGHT(str,len)

Returns the rightmost len characters from the string str, or NULL if any argument is NULL.
mysql> SELECT RIGHT('foobarbar', 4);
+---------------------------------------------------------+
| RIGHT('foobarbar', 4)                                   |
+---------------------------------------------------------+
| rbar                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

RPAD(str,len,padstr)

Returns the string str, right-padded with the string padstr to a length of len characters. If str is longer than len, the return value is shortened to len characters.
mysql> SELECT RPAD('hi',5,'?');
+---------------------------------------------------------+
| RPAD('hi',5,'?')                                        |
+---------------------------------------------------------+
| hi???                                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)

RTRIM(str)

Returns the string str with trailing space characters removed.
mysql> SELECT RTRIM('barbar   ');
+---------------------------------------------------------+
| RTRIM('barbar   ')                                      |
+---------------------------------------------------------+
| barbar                                                  |
+---------------------------------------------------------+
1 row in set (0.00 sec)

SOUNDEX(str)

Returns a soundex string from str. Two strings that sound almost the same should have identical soundex strings. A standard soundex string is four characters long, but the SOUNDEX() function returns an arbitrarily long string. You can use SUBSTRING() on the result to get a standard soundex string. All non-alphabetic characters in str are ignored. All international alphabetic characters outside the A-Z range are treated as vowels.
mysql> SELECT SOUNDEX('Hello');
+---------------------------------------------------------+
| SOUNDEX('Hello')                                        |
+---------------------------------------------------------+
| H400                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

expr1 SOUNDS LIKE expr2

This is the same as SOUNDEX(expr1) = SOUNDEX(expr2).

SPACE(N)

Returns a string consisting of N space characters.
mysql> SELECT SPACE(6);
+---------------------------------------------------------+
| SELECT SPACE(6)                                         |
+---------------------------------------------------------+
| '      '                                                |
+---------------------------------------------------------+
1 row in set (0.00 sec)

STRCMP(str1, str2)

Compares two strings and returns 0 if both strings are equal, it returns -1 if the first argument is smaller than the second according to the current sort order otherwise it returns 1.
mysql> SELECT STRCMP('MOHD', 'MOHD');
+---------------------------------------------------------+
| STRCMP('MOHD', 'MOHD')                                  |
+---------------------------------------------------------+
| 0                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)
Another example is:
mysql> SELECT STRCMP('AMOHD', 'MOHD');
+---------------------------------------------------------+
| STRCMP('AMOHD', 'MOHD')                                 |
+---------------------------------------------------------+
| -1                                                      |
+---------------------------------------------------------+
1 row in set (0.00 sec)
Let's see one more example:
mysql> SELECT STRCMP('MOHD', 'AMOHD');
+---------------------------------------------------------+
| STRCMP('MOHD', 'AMOHD')                                 |
+---------------------------------------------------------+
| 1                                                       |
+---------------------------------------------------------+
1 row in set (0.00 sec)

SUBSTRING(str,pos)

SUBSTRING(str FROM pos)

SUBSTRING(str,pos,len)

SUBSTRING(str FROM pos FOR len)

The forms without a len argument return a substring from string str starting at position pos. The forms with a len argument return a substring len characters long from string str, starting at position pos. The forms that use FROM are standard SQL syntax. It is also possible to use a negative value for pos. In this case, the beginning of the substring is pos characters from the end of the string, rather than the beginning. A negative value may be used for pos in any of the forms of this function.
mysql> SELECT SUBSTRING('Quadratically',5);
+---------------------------------------------------------+
| SSUBSTRING('Quadratically',5)                           |
+---------------------------------------------------------+
| ratically                                               |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT SUBSTRING('foobarbar' FROM 4);
+---------------------------------------------------------+
| SUBSTRING('foobarbar' FROM 4)                           |
+---------------------------------------------------------+
| barbar                                                  |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT SUBSTRING('Quadratically',5,6);
+---------------------------------------------------------+
| SUBSTRING('Quadratically',5,6)                          |
+---------------------------------------------------------+
| ratica                                                  |
+---------------------------------------------------------+
1 row in set (0.00 sec)

SUBSTRING_INDEX(str,delim,count)

Returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. SUBSTRING_INDEX() performs a case-sensitive match when searching for delim.
mysql> SELECT SUBSTRING_INDEX('www.mysql.com', '.', 2);
+---------------------------------------------------------+
| SUBSTRING_INDEX('www.mysql.com', '.', 2)                |
+---------------------------------------------------------+
| www.mysql                                               |
+---------------------------------------------------------+
1 row in set (0.00 sec)

TRIM([{BOTH | LEADING | TRAILING} [remstr] FROM] str)

TRIM([remstr FROM] str)

Returns the string str with all remstr prefixes or suffixes removed. If none of the specifiers BOTH, LEADING, or TRAILING is given, BOTH is assumed. remstr is optional and, if not specified, spaces are removed.
mysql> SELECT TRIM('  bar   ');
+---------------------------------------------------------+
| TRIM('  bar   ')                                        |
+---------------------------------------------------------+
| bar                                                     |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT TRIM(LEADING 'x' FROM 'xxxbarxxx');
+---------------------------------------------------------+
| TRIM(LEADING 'x' FROM 'xxxbarxxx')                      |
+---------------------------------------------------------+
| barxxx                                                  |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT TRIM(BOTH 'x' FROM 'xxxbarxxx');
+---------------------------------------------------------+
| TRIM(BOTH 'x' FROM 'xxxbarxxx')                         |
+---------------------------------------------------------+
| bar                                                     |
+---------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT TRIM(TRAILING 'xyz' FROM 'barxxyz');
+---------------------------------------------------------+
| TRIM(TRAILING 'xyz' FROM 'barxxyz')                     |
+---------------------------------------------------------+
| barx                                                    |
+---------------------------------------------------------+
1 row in set (0.00 sec)

UCASE(str)

UCASE() is a synonym for UPPER().

UNHEX(str)

Performs the inverse operation of HEX(str). That is, it interprets each pair of hexadecimal digits in the argument as a number and converts it to the character represented by the number. The resulting characters are returned as a binary string.
mysql> SELECT UNHEX('4D7953514C');
+---------------------------------------------------------+
| UNHEX('4D7953514C')                                     |
+---------------------------------------------------------+
| MySQL                                                   |
+---------------------------------------------------------+
1 row in set (0.00 sec)
The characters in the argument string must be legal hexadecimal digits: '0' .. '9', 'A' .. 'F', 'a' .. 'f'. If UNHEX() encounters any non-hexadecimal digits in the argument, it returns NULL.

UPPER(str)

Returns the string str with all characters changed to uppercase according to the current character set mapping.
mysql> SELECT UPPER('Allah-hus-samad');
+---------------------------------------------------------+
| UPPER('Allah-hus-samad')                                |
+---------------------------------------------------------+
| ALLAH-HUS-SAMAD                                         |
+---------------------------------------------------------+
1 row in set (0.00 sec)