Gentoo Logo

Disclaimer : The original version of this article was first published on IBM developerWorks, and is property of Westtech Information Services. This document is an updated version of the original article, and contains various improvements made by the Gentoo Linux Documentation team.
This document is not actively maintained.


Learning Linux LVM, Part 2

1.  The cvs.gentoo.org upgrade

Introduction

In my first LVM article, I explained the concepts behind LVM. Now it's time to put LVM into action. In this article, I'm going to set up LVM on the official Gentoo Linux cvs server, cvs.gentoo.org. Although cvs.gentoo.org has only one hard drive, LVM's flexibility still provides an incredible improvement over the standard static partitioning approach. I'll show you all the steps of the LVM conversion process, so that if you're interested you can perform a similar conversion on one of your machines.

Warning: Because implementing LVM is a major change to the system (involving the creation of new partitions and other potentially hazardous actions) it's a really good idea to perform a full system backup before beginning this process. If you're not going to perform a backup, I hope you're using a test box with no important data on it. I should mention that I didn't experience any problems while converting to LVM, but it's best to be prepared in case something goes wrong.

That said, let's continue. Before starting the conversion process, I upgraded cvs.gentoo.org so that it was using the following packages. At the time I performed the LVM transition, these were the latest versions available (see Resources later in this article):

  • Linux kernel 2.4.1-ac19
  • LVM 0.9.1_beta5
  • reiserfs-utils 3.6.25

Now, for the hard drive. cvs.gentoo.org had a nice new IBM 45 GB hard drive sitting in it; however, when I installed Gentoo Linux on cvs, I only partitioned about 10 gigabytes of the drive, keeping the remaining 35 GB for future partitions. Such are the little tricks you need to employ when not using LVM -- leaving part of the drive unpartitioned is a primitive but effective way to allow for future expansion. However, with LVM there is a better approach.

The space problem

In the past few weeks, I had been noticing that my root ReiserFS partition had been slowly filling up, as you can see from this df output:

Code Listing 1.1: Shrinking free space

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   6989312   2775888  72% /
tmpfs                   269052         0    269052   0% /dev/shm

Now, a 72% full root partition isn't exactly a crisis, but it isn't a wonderful situation either. ReiserFS, like many other filesystems, starts slowing down as it gets more and more full, and it was just a matter of time before my root filesystem would fill up completely and filesystem performance would take a hit.

I decided to fix this problem by using LVM to create a new logical volume out of the 35 GB of currently unpartitioned space at the end of my hard drive. Then, I'd create a filesystem on this volume and move a good chunk of the contents of /dev/hda3 to it.

If you're thinking of making a similar transition on one of your machines, the first thing you need to do is find a suitable piece of your root filesystem to move to a logical volume. For me, the choice was easy -- my /home tree was taking up around 5.7 GB. By moving /home to its own LVM logical volume, my root filesystem would then be at about 20% capacity. Since most new data is being added to /home, my root filesystem would likely stay at around 20% capacity as well -- a very healthy situation.

The beginnings of a solution

To begin the conversion, I first had to partition the unused space at the end of my hard drive. Using cfdisk, I created a 35 GB partition (/dev/hda5) and set the partition type of the partition to 8E (the official LVM partition type). After this change, I rebooted to force a reread of my partition table. After the reboot, my partition table looked like this:

Code Listing 1.2: The new partition table

# sfdisk -l
Disk /dev/hda: 89355 cylinders, 16 heads, 63 sectors/track
Units = cylinders of 516096 bytes, blocks of 1024 bytes, counting from 0
   Device Boot Start     End   #cyls   #blocks   Id  System
/dev/hda1   *      0+    247     248-   124960+  83  Linux
/dev/hda2        248     743     496    249984   82  Linux swap
/dev/hda3        744   20119   19376   9765504   83  Linux
/dev/hda4      20120   89354   69235  34894440    5  Extended
/dev/hda5      20120+  89354   69235- 34894408+  8e  Linux LVM

Now that I had an empty 35 GB partition, I was ready to initialize it for LVM. Here's the procedure -- first, I would initialize the 35 gigabytes as a physical volume; then, I would create a volume group using this physical volume, and finally, I would allocate some of the extents on the volume group, creating a logical volume that would contain my new filesystem and house all the files currently in /home.

To begin the process, I used the pvcreate command to initialize /dev/hda5 as a physical volume:

Code Listing 1.3: Creating the physical volume

# pvcreate /dev/hda5
pvcreate -- physical volume "/dev/hda5" successfully created

pvcreate set up a special "accounting" area on /dev/hda5, called the VGDA (volume group descriptor area). LVM uses this area to keep track of how the physical extents are allocated, among other things.

My next step was to create a volume group and add /dev/hda5 to this group. The volume group would act as a pool of extents (chunks of storage blocks). Once the volume group was created, I could create as many logical volumes as I wanted. I decided that my volume group would be called "main":

Code Listing 1.4: Creating the volume group

# vgcreate main /dev/hda5
vgcreate -- INFO: using default physical extent size 4 MB
vgcreate -- INFO: maximum logical volume size is 255.99 Gigabyte
vgcreate -- doing automatic backup of volume group "main"
vgcreate -- volume group "main" successfully created and activated

The vgcreate command did a couple of things. In addition to creating the "main" volume group, it also set up /dev/hda5 to use 4 MB extents, the default extent size. This means that any logical volumes I create from this volume group can be expanded and shrunk in 4 MB increments.

Due to kernel limitations, the extent size determines the maximum size that a logical volume can be. As you can see from the above output, a 4 MB extent size imposes a logical volume size limitation of 256 gigabytes, which is an easily attainable logical volume size if you're adding several high-capacity drives to your volume group. If your volumes could end up being greater than 256 GB apiece, I recommend specifying a larger extent size at vgcreate time. Extents can range anywhere from 8 KB to 512 MB, and must always be a multiple of two. By increasing the extent size above 4 MB, the maximum physical volume size will be scaled accordingly, up to a maximum of 1 petabyte (although the current real-world size limit is 2 terabytes on x86 systems). For example, if I wanted to create a volume group with 32 megabyte extents, I'd type:

Code Listing 1.5: A larger extent size

# vgcreate -s 32M main /dev/hda5

32 MB is a good extent size, since a 32 MB granularity is still manageable and pushes the maximum logical volume size to 2 terabytes to boot. Once your volume group is created, you can view its information by typing vgdisplay:

Code Listing 1.6: Examining volume information

# vgdisplay
--- Volume group ---
VG Name               main
VG Access             read/write
VG Status             available/resizable
VG #                  0
MAX LV                256
Cur LV                0
Open LV               0
MAX LV Size           255.99 GB
Max PV                256
Cur PV                1
Act PV                1
VG Size               33.28 GB
PE Size               4 MB
Total PE              8519
Alloc PE / Size       0 / 0
Free  PE / Size       8519 / 33.28 GB
VG UUID               2qC2H2-iA8s-qW6F-cwXx-JVIh-I6VC-VVCGmn

Now that I had my volume group, I was ready to create a logical volume. I decided to initially make it 8 gigabytes in size and call it "lv_home":

Code Listing 1.7: Creating the logical volume

# lvcreate -L8G -nlv_home main
lvcreate -- doing automatic backup of "main"
lvcreate -- logical volume "/dev/main/lv_home" successfully created

Then, I created a filesystem on the volume:

Code Listing 1.8: Filesystem creation

# mkreiserfs /dev/main/lv_home


  <----------- MKREISERFSv2 ----------->

   Block size 4096 bytes
   Block count 2097152
   Used blocks 8275
           Journal - 8192 blocks (18-8209), journal header is in block 8210
           Bitmaps: 17, 32768, 65536, 98304, 131072, 163840,
           196608, 229376, 262144, 294912, 327680, 360448,
           393216, 425984, 458752, 491520, 524288, 557056,
           589824, 622592, 655360, 688128, 720896, 753664,
           786432, 819200, 851968, 884736, 917504, 950272,
           983040, 1015808, 1048576, 1081344, 1114112,
           1146880, 1179648, 1212416, 1245184, 1277952,
           1310720, 1343488, 1376256, 1409024, 1441792,
           1474560, 1507328, 1540096, 1572864, 1605632,
           1638400, 1671168, 1703936, 1736704, 1769472,
           1802240, 1835008, 1867776, 1900544, 1933312,
           1966080, 1998848, 2031616, 2064384
   Root block 8211
Hash function "r5"
ATTENTION: ALL DATA WILL BE LOST ON '/dev/main/lv_home'! (y/n)y
journal size 8192 (from 18)
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..done.

Now that the filesystem was created, I could mount it at /mnt/newhome:

Code Listing 1.9: Mounting the new volume

# mkdir /mnt/newhome
# mount /dev/main/lv_home /mnt/newhome
# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   6989840   2775360  72% /
tmpfs                   291388         0    291388   0% /dev/shm
/dev/main/lv_home      8388348     32840   8355508   1% /mnt/newhome

As you can see above, I was almost ready to copy over all my data in /home. Before I began, I dropped to runlevel 1 to ensure that no users or processes would be accessing or modifying files in /home as they were being copied over:

Code Listing 1.10: Runlevel 1

# init 1

Then, I began copying the files:

Code Listing 1.11: Copying files to the new directory

# cp -avx /home/* /mnt/newhome

The copy completed in about ten minutes. Then, I backed up my original /home to /home.old, just in case something was wrong with my copy. I created a new mount point, and remounted the new home at /home:

Code Listing 1.12: The new mount point

# cd /
# mv home home.old
# mkdir home
# umount /mnt/newhome
# mount /dev/main/lv_home /home

Then, it was time to set up the server so that my new /home partition would be available every time the machine started up. First, I modified my /etc/fstab so that it included a new /home entry:

Code Listing 1.13: Editing fstab


#fs                 mountpoint       type         opts          dump/pass
/dev/hda3           /                reiserfs     defaults      1 1
/dev/main/lv_home   /home            reiserfs     defaults      2 2
/dev/hda2           none             swap         sw            0 0
/dev/hda1           /boot            reiserfs     noauto        0 0
/dev/cdrom          /mnt/cdrom       iso9660      noauto,ro     0 0
proc                /proc            proc         defaults      0 0
none                /dev/pts         devpts       mode=620      0 0
tmpfs               /dev/shm         tmpfs        defaults      0 0

Then, I made minor modifications to my initialization scripts. I modified my "checkroot" startup script so that the following commands would run immediately after my root partition was remounted read/write:

Code Listing 1.14: Modifying the startup script

/sbin/vgscan
/sbin/vgchange -a y

Then, I modified my filesystem unmounting script that gets run on shutdown, so that the following command would run immediately after all filesystems were unmounted:

Code Listing 1.15: Modifying the shutdown script

/sbin/vgchange -a n

Once I had completed these steps, I rebooted the machine, and to my delight everything worked perfectly. After a day or so of absolutely no problems, I deleted /home.old to free up some space on my root filesystem. Yay! The transition to LVM was a success.

The beauty of LVM

While the transition to LVM is a bit of an ordeal, once the transition is complete, managing filesystems becomes tremendously easier. As an example, I decided to resize my new /home logical volume, adding about 2 gigabytes worth of space to the end of the filesystem. First, I added additional capacity to my "lv_home" logical volume, and then I used the resize_reiserfs utility to expand the filesystem so that it would use this additional capacity. Here are the two commands that did all this:

Code Listing 1.16: Using additional space

# lvextend -L+2G /dev/main/lv_home
# resize_reiserfs -f /dev/main/lv_home

In about a second, I had enlarged my /home filesystem by 2 GB; amazingly, I didn't need to reboot, drop to runlevel 1, or even unmount /home to perform the resize. Everything continued to work as it had before. Isn't that great? Here's the current state of my filesystems:

Code Listing 1.17: Filesystem space

# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hda3              9765200   1413340   8351860  15% /
/dev/main/lv_home     10485436   5609836   4875600  54% /home

You can see how LVM really can make an administrator's work a whole lot easier. In the future, I hope to move additional parts of my root filesystem over to LVM, and eventually even convert my root filesystem over to an LVM logical volume. The resources below will help you learn even more about LVM.

Resources

  • For LVM concepts and advice on get the latest kernel patches and tools installed on your system, see my previous article, Learning Linux LVM, Part 1.
  • Download the LVM tarball from RedHat.
  • The impatient will want to check out Heinz Mauelshagen's LVM HOWTO, which contains more examples on how to set up volume groups and logical volumes.
  • There's also an interesting HOWTO that shows you how to set up your root filesystem on a logical volume. Once LVM-0.9.1_final is out, I may try doing this.
  • Andreas Dilger is involved with the Linux LVM project and has a nice-looking online ext2 filesystem resizer.
  • ReiserFS is an excellent filesystem (especially in combination with LVM). If you're using ReiserFS, you'll want to grab the reiserfs-utils tarball, which contains a program called reiserfs_resize -- allowing online resizing of ReiserFS filesystems.
  • For more information on setting up Linux software RAID volumes, see Part 1 and Part 2 of Daniel's series on software RAID.
  • For a refresher, see Daniel's tutorial on compiling the Linux kernel.


Print

Page updated January 22, 2006

Summary: In this article, Daniel shares his experiences converting cvs.gentoo.org's /home filesystem to an LVM logical volume. After the transition, we get to see the benefits of LVM when cvs.gentoo.org's /home partition is dynamically resized in real-time, without rebooting, unmounting /home, or even dropping to runlevel 1. All processes continue to work without any interruption. Daniel's step-by-step details of the conversion will help anyone interested in peforming a similar transition on their own machine.

Daniel Robbins
Author

Donate to support our development efforts.

Copyright 2001-2014 Gentoo Foundation, Inc. Questions, Comments? Contact us.