认识和使用LVM,过一遍!

疑惑

最近在反复搭建ceph集群过程中,总是遇到osd创建不成功的问题,疑似硬盘残留信息,排查中引出了很多陌生的命令,比如vgremove等,于是打算重新了解这部分。

LVM是什么?

逻辑卷管理器(LVM,Logical Volume Manager)是一种把硬盘空间分配成逻辑卷的方法。

看到定义可能还比较懵,不妨结合场景:

有一块系统盘空间随着时间推移容量需要扩容该怎么做?

这时候使用LVM就比较方便了,它可以弹性的调整文件系统的容量,可以整合多个物理分区在一起,让这些分区看起来像是一个磁盘一样。通俗理解就是它可以将很多硬盘/分区全部拿过来作为一个资源池,然后自己再随意划分成逻辑层面的分区,那么这个分区后期进行扩容缩容删除就比较方便了!

LVM相关概念

1、Physical Volume,PV,物理卷

[root@node3 ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x172f2548.

Command (m for help): p                                        ###输入 p 查看分区情况

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x172f2548

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n                                      ###输入 n 创建新分区
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p                                        ### 默认是 p ,创建主分区,视情况而定
Partition number (1-4, default 1): 1                         ###这里做第一块主分区
First sector (2048-20971519, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519):
Using default value 20971519
Partition 1 of type Linux and of size 10 GiB is set

Command (m for help): p                                      ###再次输入 p 查看分区情况

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x172f2548

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    20971519    10484736   83  Linux

Command (m for help): w                                 ###最后输入 w 保存从sdb分出来的sdb1    
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

fdisk -l查看是否分区成功

[root@node3 ~]# fdisk -l

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x172f2548

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    20971519    10484736   83  Linux

可以看到实际分区Id字段为83,使用时需要通过 fdisk 命令将Id字段调整为8e(LVM 的标识符),再经过 pvcreate 命令将它转为 LVM 最底层的物理卷(PV),之后这些PV才能够被利用

[root@node3 ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): t                                  ###输入 t 修改分区类型
Selected partition 1
Hex code (type L to list all codes): L                   ###输入 L 列出所有分区类型

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor
14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT
1e  Hidden W95 FAT1 80  Old Minix
Hex code (type L to list all codes): 8e                ###输入 8e 修改为指定分区类型
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p                                ######输入 p 查看分区情况

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x172f2548

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    20971519    10484736   8e  Linux LVM

Command (m for help): w                                ###最后输入 w 保存修改的分区类型
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

使用pvcreate创建pv

[root@node3 ~]# pvcreate /dev/sdb1
  Physical volume "/dev/sdb1" successfully created.

2、Volume Group,VG,卷组

VG可以理解为很多物理卷(PV)所组成的组

[root@node3 ~]# vgcreate storage /dev/sdb1
  Volume group "storage" successfully created

[root@node3 ~]# vgdisplay
  --- Volume group ---
  VG Name               storage
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  1
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               <10.00 GiB
  PE Size               4.00 MiB
  Total PE              2559
  Alloc PE / Size       0 / 0
  Free  PE / Size       2559 / <10.00 GiB
  VG UUID               fK1Wcs-B1BL-TY3G-IRiJ-7JnV-wd0T-CRFx95

使用vgextend添加 pv 到 vg

[root@node3 ~]# vgextend storage /dev/sdc
sdc   sdc1

[root@node3 ~]# vgextend storage /dev/sdc1
  Physical volume "/dev/sdc1" successfully created.
  Volume group "storage" successfully extended
[root@node3 ~]# vgdisplay
  --- Volume group ---
  VG Name               storage
  System ID
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  2
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                0
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               19.99 GiB
  PE Size               4.00 MiB
  Total PE              5118
  Alloc PE / Size       0 / 0
  Free  PE / Size       5118 / 19.99 GiB
  VG UUID               fK1Wcs-B1BL-TY3G-IRiJ-7JnV-wd0T-CRFx95

可以看到两个 pv 已经成功添加到 vg 中

3、Logical Volume,LV,逻辑卷

LV 是由 VG 切分而来的,此时 LV 就可以当成一个分区格式化来使用了,

lvcreate 参数
-L 是直接制定容量,基础单位是MB,如果要切割出1G空间,则 -L 1G,MB以此类推。

-l 是以基本单元的个数为单位,每个基本单元的大小是4MB,-l 20,则是切割出4*20=80M空间。

-n 是指定生成逻辑卷的名称。

在卷组中切割出一个:名称为test1,大小为5G的空间:

[root@node3 ~]# lvcreate -n test1 -L 5G storage
  Logical volume "test1" created.

[root@node3 ~]# lvdisplay                        ###查看lv
  --- Logical volume ---
  LV Path                /dev/storage/test1
  LV Name                test1
  VG Name                storage
  LV UUID                WkSHKt-2GPL-sZX0-ZZeu-KGZB-9Gao-vufFmC
  LV Write Access        read/write
  LV Creation host, time node3, 2023-02-01 15:22:25 +0800
  LV Status              available
  # open                 0
  LV Size                5.00 GiB
  Current LE             1280
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     8192
  Block device           253:0

格式化、挂载使用

[root@node3 ~]# mkfs.ext4 /dev/storage/test1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
327680 inodes, 1310720 blocks
65536 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
[root@node3 ~]# mkdir /mnt/dir1

[root@node3 ~]# mount /dev/storage/test1 /mnt/dir1/

[root@node3 ~]# df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   1.9G     0  1.9G   0% /dev
tmpfs                      1.9G     0  1.9G   0% /dev/shm
tmpfs                      1.9G   20M  1.9G   2% /run
tmpfs                      1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda2                   19G  2.2G   17G  12% /
/dev/sda1                 1014M  142M  873M  14% /boot
tmpfs                      378M     0  378M   0% /run/user/0
/dev/mapper/storage-test1  4.8G   20M  4.6G   1% /mnt/dir1

4、Physical Extend,PE,物理扩展块

PE 是整个 LVM 最小的存储块,LVM 的 VG 最多仅能含有 65534 个 PE,一个 PE 默认的大小时 4M。
所以可以通过调整 PE 来调整 VG最大容量。

# vgdisplay
  --- Volume group ---
  VG Name               ceph-77651246-631b-44d8-9ece-1cc0af8e0b68
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  17
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                1
  Open LV               1
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               9.00 GiB
  PE Size               1.00 GiB
  Total PE              9
  Alloc PE / Size       9 / 9.00 GiB
  Free  PE / Size       0 / 0
  VG UUID               GdTfRy-au4B-DgAN-Ej2X-XgdK-hFNz-wDrLm1

LV 扩容、缩容

扩容

[root@node3 ~]# umount /mnt/dir1/                                  ###解除挂载

[root@node3 ~]# lvextend -L 15G /dev/storage/test1                 ###vg容量不够
  Insufficient free space: 2560 extents needed, but only 2558 available

[root@node3 ~]# lvextend -L 10G /dev/storage/test1             ###扩展后的分区容量,必须大于原容量
  Size of logical volume storage/test1 changed from 5.00 GiB (1280 extents) to 10.00 GiB (2560 extents).
  Logical volume storage/test1 successfully resized.

[root@node3 ~]# e2fsck -f /dev/storage/test1                    ###检查硬盘的完整性
e2fsck 1.42.9 (28-Dec-2013)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/storage/test1: 11/327680 files (0.0% non-contiguous), 58462/1310720 blocks

[root@node3 ~]# resize2fs /dev/storage/test1                     ###重置硬盘容量
resize2fs 1.42.9 (28-Dec-2013)
Resizing the filesystem on /dev/storage/test1 to 2621440 (4k) blocks.
The filesystem on /dev/storage/test1 is now 2621440 blocks long.

[root@node3 ~]# mount /dev/storage/test1 /mnt/dir1/              ###重新挂载使用
[root@node3 ~]# df -h
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   1.9G     0  1.9G   0% /dev
tmpfs                      1.9G     0  1.9G   0% /dev/shm
tmpfs                      1.9G   20M  1.9G   2% /run
tmpfs                      1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda2                   19G  2.2G   17G  12% /
/dev/sda1                 1014M  142M  873M  14% /boot
tmpfs                      378M     0  378M   0% /run/user/0
/dev/mapper/storage-test1  9.8G   23M  9.2G   1% /mnt/dir1

缩容

lvextend -L 3G /dev/storage/test1              ###操作参考扩容

LV删除、卸载

[root@node3 ~]# umount /mnt/dir1/                            ###解除挂载

[root@node3 ~]# lvremove /dev/storage/test1                  ###删除lv
Do you really want to remove active logical volume storage/test1? [y/n]: y
  Logical volume "test1" successfully removed

[root@node3 ~]# vgremove storage                             ###删除vg
Do you really want to remove volume group "storage" containing 1 logical volumes? [y/n]: y
Do you really want to remove active logical volume storage/test2? [y/n]: y
  Logical volume "test2" successfully removed
  Volume group "storage" successfully removed

[root@node3 ~]# pvremove /dev/sdb1                           ###删除pv
  Labels on physical volume "/dev/sdb1" successfully wiped.

over!!!

热门相关:超武穿梭   刺客之王   仗剑高歌   寂静王冠   时间都知道