Upgrade guide to MySQL 4.* or 5.0.*

1. Straight upgrade, suggested for 4.1 => 5.0 migration

The myisam storage engine in version 4.1 was already mature enough to allow a direct upgrade to the next major version of MySQL.

Note: This is not true for MERGE tables. You will likely run into trouble if you attempt a direct upgrade for this (rarely used) type of table. You should dump and recreate those tables and restore their contents in the process of upgrading. If unsure, you should begin with Upgrading from old versions of MySQL.

For this step two shells are required because locks belong to the mysql session.

Code Listing 1.1: Straight upgrade from 4.1 to 5.0.*

# quickpkg dev-db/mysql
# alias MYSQL="mysql --user=root --password='your_password'"
# DATADIR=$(MYSQL --batch --raw --silent --skip-column-names \
     --execute='SHOW variables LIKE "datadir";' \
     | sed -e 's|datadir[ \t]||')

(This next step should be done in the second shell)
# mysql --user=root --password='your_password'
mysql> FLUSH TABLES WITH READ LOCK;

(Return to the first shell to run this command)
# tar -cjpvf ~/mysql.$(date +%F"T"%H-%M).tar.bz2 \
     /etc/conf.d/mysql /etc/mysql/my.cnf "${DATADIR}"

(The following commands should be done in the second shell)
mysql> UNLOCK TABLES;
mysql> quit

(Return to the first shell for the rest of the upgrade)
# tar -tjvf ~/mysql.*.tar.bz2
# emerge -av ">dev-db/mysql-5.0"
# dispatch-conf
# revdep-rebuild
# /etc/init.d/mysql restart
# mysql_upgrade_shell --user=root --password='your_password' \
     --protocol=tcp --datadir="${DATADIR}"
# /etc/init.d/mysql restart
# unset DATADIR
# unalias MYSQL

2. Upgrading from old versions of MySQL

Users upgrading from an old version (<4.0.24) of MySQL will first have to install MySQL 4.0.25. If you are already running a more recent version, you can skip this section and continue with the next one.

Code Listing 2.1: Simple upgrade

# emerge -av --buildpkg "<mysql-4.1"

3. Creating a backup of your current data

Important: Values inside primary keys are handled differently in various MySQL versions, see bug #108502 for more details, it is higly recommended to scan your tables for values of "0" (zero) or less and update them to a value greater than or equal to "1".

One of the most important tasks that every database administrator has to perform is backing up data. Here we go:

Code Listing 3.1: Dump of all databases

# mysqldump \
  -uroot \
  --password='your_password' \
  -hlocalhost \
  --all-databases \
  --opt \
  --allow-keywords \
  --flush-logs \
  --hex-blob \
  --master-data \
  --max_allowed_packet=16M \
  --quote-names \
  --result-file=BACKUP_MYSQL_4.0.SQL

Now a file named BACKUP_MYSQL_4.0.SQL should exist, which can be used later to recreate your data. The data is described in the MySQL dialect of SQL, the Structured Query Language.

Now would also be a good time to see if the backup you have created is working.

4. Upgrading from recent versions of MySQL

If you have skipped step #1, you now have to create a backup package (of the database server, not the data) of the currently installed version:

Code Listing 4.1: Binary package backup

# quickpkg dev-db/mysql

Now it's time to clean out the current version and all of its data:

Code Listing 4.2: Uninstall MySQL

# /etc/init.d/mysql stop
# emerge -C mysql
# tar cjpvf ~/mysql.$(date +%F"T"%H-%M).tar.bz2 /etc/mysql/my.cnf /var/lib/mysql/
# ls -l ~/mysql.*
# rm -rf /var/lib/mysql/ /var/log/mysql

Note: Now two different backups should exist: the SQL one, which is portable between various versions of MySQL, and the other one that will allow you to quickly restore your database. This is covered later in this document in more detail.

After you get rid of your old MySQL installation, you can now install the new version. Note that revdep-rebuild is necessary for rebuilding packages linking against MySQL.

Code Listing 4.3: Upgrading the binaries

# emerge -av ">mysql-4.1"
(Update your config files, you may also use dispatch-conf)
# etc-update
# revdep-rebuild

Now configure the newly installed version of MySQL and restart the daemon:

Code Listing 4.4: Configure MySQL 4.1 base setup

# emerge --config =mysql-4.1.<micro_version>
# /etc/init.d/mysql start

Finally you can import the backup you have created during step #2.

Important: The default /etc/mysql/my.cnf file sets binary logging on (log-bin) by default. This will log every single transaction that modifies data. If run on a very large database (1GB or more), this could create extremely large files that take up disk space rather quickly. If you are low on space, disabling binary logging might be a good idea.

Important: The default character set in MySQL 4.1 and above is utf8. If the data contain non-ASCII characters, you may want to preserve the default character set of the database replacing all occurrences of utf8 with latin1 in the /etc/mysql/my.cnf config file. More information can be found Charset conversion chapter.

Important: The administrative mysql database that contains user names, passwords amongst other things is and must be encoded in utf8.

Older mysqldump utilities may export tables in the wrong order when foreign keys are involved. To work around this problem, surround the SQL with the following statements:

Code Listing 4.5: Fixing foreign key checks

SET FOREIGN_KEY_CHECKS=0
SET FOREIGN_KEY_CHECKS=1

Next, import the backup.

Code Listing 4.6: Importing the SQL backup

# cat BACKUP_MYSQL_4.0.SQL \
     | mysql \
     -uroot \
     --password='your_password' \
     -hlocalhost \
     --max_allowed_packet=16M

# mysql_fix_privilege_tables \
     --defaults-file=/etc/mysql/my.cnf \
     --user=root \
     --password='your_password'

If you restart your MySQL daemon now and everything goes as expected, you have a fully working version of 4.1.x.

Code Listing 4.7: Restart the MySQL instance

# /etc/init.d/mysql restart

If you encountered any problems during the upgrade process, please report them on Bugzilla.

5. Recover the old installation of MySQL 4.0

If you are not happy with MySQL 4.1, it's possible to go back to MySQL 4.0.

Code Listing 5.1: Reverting to the previous version

# /etc/init.d/mysql stop
# emerge -C mysql
# rm -rf /var/lib/mysql/ /var/log/mysql
# emerge --usepkgonly "<mysql-4.1"
(Replace <timestamp> with the one used when creating the backup.)
# tar -xjpvf mysql.<timestamp>.tar.bz2 -C /
# /etc/init.d/mysql start

Important: If packages other than dev-db/mysql have been emerged following this guide, you need to run revdep-rebuild to ensure that every client is using the correct mysqlclient shared object.

6. On charset conversion:

Introduction

This chapter is not intended to be an exhaustive guide on how to do such conversions, rather a short list of hints on which the reader can elaborate.

Converting a database may be a complex task and difficulty increases with data variancy. Things like serialized object and blobs are one example where it's difficult to keeps pieces together.

Indexes

Every utf-8 character is considered 3 bytes long within an index. Indexes in MySQL can be up to 1000 bytes long (767 bytes for InnoDB tables). Note that the limits are measured in bytes, whereas the length of a column is interpreted as number of characters.

MySQL can also create indexes on parts of a column, this can be of some help. Below are some examples:

Code Listing 6.1: Indexing using prefixes

$ mysql -uroot -p'your_password' test

mysql> SHOW variables LIKE "version" \G
*************************** 1. row ***************************
Variable_name: version
    Value: 5.0.24-log
1 row in set (0.00 sec)

mysql> CREATE TABLE t1 (
  ->   c1 varchar(255) NOT NULL default '',
  ->   c2 varchar(255) NOT NULL default ''
  ->   ) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

mysql> ALTER TABLE t1
  ->   ADD INDEX idx1 ( c1 , c2 );
ERROR 1071 (42000): Specified key was too long; max key length is 1000 bytes

mysql> ALTER TABLE t1
  ->   ADD INDEX idx1 ( c1(165) , c2(165) );
Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> CREATE TABLE t2 (
  ->   c1 varchar(255) NOT NULL default '',
  ->   c2 varchar(255) NOT NULL default ''
  ->   ) ENGINE=MyISAM DEFAULT CHARSET=sjis;
Query OK, 0 rows affected (0.00 sec)

mysql> ALTER TABLE t2
  ->   ADD INDEX idx1 ( c1(250) , c2(250) );
Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> CREATE TABLE t3 (
  ->   c1 varchar(255) NOT NULL default '',
  ->   c2 varchar(255) NOT NULL default ''
  ->   ) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Query OK, 0 rows affected (0.00 sec)

mysql> ALTER TABLE t3
  ->   ADD INDEX idx1 ( c1 , c2 );
Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

Environment

The system must be configured to support the UTF-8 locale. You will find more information in our Using UTF-8 with Gentoo and Localization Guide documents.

In this example, we set some shell environment variables to make use of the English UTF-8 locale in /etc/env.d/02locale:

Code Listing 6.2: /etc/env.d/02locale

LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8

Be sure to run env-update && source /etc/profile afterward.

iconv

iconv, provided by sys-libs/glibc, is used to convert text files from one charset to another. The app-text/recode package can be used as well.

Code Listing 6.3: Using iconv

(From latin1 to utf8)
$ iconv -f ISO-8859-15 -t UTF-8 file1.sql > file2.sql

(From Japanese to utf8)
$ iconv -f ISO2022JP -t UTF-8 file1.sql > file2.sql

iconv can be used to recode a sql dump even if the environment is not set to utf8.

SQL Mangling

It's possible to use the CONVERT() and CAST() MySQL functions to convert data in your SQL scripts.

Apache (webserver)

To use utf-8 with apache, you need to adjust the following variables in httpd.conf: AddDefaultCharset, CharsetDefault, CharsetSourceEnc. If your source html files aren't encoded in utf-8, they must be converted with iconv or recode.

The contents of this document, unless otherwise expressly stated, are licensed under the CC-BY-SA-2.5 license. The Gentoo Name and Logo Usage Guidelines apply.

Page updated June 4, 2007

Summary: The MySQL herd is proud to announce that MySQL 5.0 will soon be found in Gentoo's stable tree. This document describes how to upgrade to MySQL 4.* and to 5.0.*.

Michael Kohl
Author

Francesco Riosa
Author

Donate to support our development efforts.