2017-12-15

resize2fs: Operation not permitted While trying to add group

Recently, I encountered this problem when resizing an EXT4 file system in our NFS server.

# df -hT /export/rear
Filesystem           Type  Size  Used Avail Use% Mounted on
/dev/mapper/vgadm-lvrear
                     ext4  1.2T  1.2T   16M 100% /export/rear
# resize2fs /dev/mapper/vgadm-lvrear                           
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/mapper/vgadm-lvrear is mounted on /export/rear; on-line resizing required
old desc_blocks = 74, new_desc_blocks = 108
Performing an on-line resize of /dev/mapper/vgadm-lvrear to 450920448 (4k) blocks.
resize2fs: Operation not permitted While trying to add group #9472

# dmesg
EXT4-fs warning (device dm-28): ext4_group_add: No reserved GDT blocks, can't resize
EXT4-fs warning (device dm-28): ext4_group_add: No reserved GDT blocks, can't resize

I followed this Redhat KB Article - https://access.redhat.com/solutions/88483

# tune2fs -O ^has_journal /dev/mapper/vgadm-lvrear
tune2fs 1.41.12 (17-May-2010)
The has_journal feature may only be cleared when the filesystem is
unmounted or mounted read-only.

# dumpe2fs /dev/mapper/vgadm-lvrear | grep -i features
dumpe2fs 1.41.12 (17-May-2010)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Journal features:         journal_incompat_revoke

So solution would be either:

  • Go and install XFS (need additional subscription as the system is RHEL 6.8)
  • Or recreate the file system but with additional parameters --resize=max-online-resize

Full explanation has been posted here as well.

Thus, it has been decided by our Platform Architect that it's okay to move to XFS instead.  So I needed to enable a subscription and went ahead installing XFS.

$ sudo subscription-manager repos --list | grep scalefs
$ sudo subscription-manager repos --enable=rhel-6.4-scalefs-server-for-rhs-2.1-rpms

Here's what I did.

$ sudo yum install xfsprogs xfsdump -y

I made a request of 2x1TB LUNs from our Storage Team and created a new volume group called vgrear.

$ sudo pvcreate /dev/mapper/3600601604bc03800c862f9bc0bdce711
$ sudo pvcreate /dev/mapper/3600601604bc03800c662f9bc0bdce711
$ sudo vgcreate vgrear /dev/mapper/3600601604bc03800c862f9bc0bdce711 /dev/mapper/3600601604bc03800c662f9bc0bdce711

Created the logical volume group and the new file system (/export/rear_new).

$ sudo mkfs.xfs /dev/vgrear/lvrear
meta-data=/dev/vgrear/lvrear     isize=256    agcount=4, agsize=134217216 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=536868864, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=262143, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

$ sudo mkdir /export/rear_new
$ sudo mount /dev/vgrear/lvrear /export/rear_new
$ df -hT /export/rear_new
Filesystem           Type  Size  Used Avail Use% Mounted on
/dev/mapper/vgrear-lvrear
                     xfs   2.0T   33M  2.0T   1% /export/rear_new

Afterwards, I did an rsync (copy/transfer of files) from /export/rear to /export/rear_new.  And did the un/export, un/re-mounting of the file systems (did an lsof to ensure nothing is using the files/folders).  I also modified the /etc/fstab to reflect the changes I made in the VG/LV.

$ sudo rsync -avP /export/rear/* /export/rear_new/
sending incremental file list
a.out
...
$ sudo lsof /export/rear_new/
$ sudo lsof /export/rear
$ sudo exportfs -u *:/export/rear
$ sudo umount /export/rear; sudo umount /export/rear_new
$ sudo vi /etc/fstab
$ cat /etc/fstab | grep rear
/dev/vgadm/lvrear               /export/rear_old        ext4    ro,data=ordered         0 0
/dev/vgrear/lvrear              /export/rear            xfs     defaults,nodev  0 0
$ sudo mv /export/rear /export/rear_old
$ sudo mv /export/rear_new /export/rear
$ ls -l /export/ | grep rear
drwxrwxrwx 220 root   root    8192 Dec 13 18:56 rear
drwxrwxrwx 221 root   root   12288 Dec 13 06:23 rear_old
$ sudo mount /dev/vgrear/lvrear /export/rear
$ sudo mount /dev/vgadm/lvrear /export/rear_old
$ df -hT | grep rear
/dev/mapper/vgrear-lvrear
                     xfs    2.0T 1021G  1.1T  50% /export/rear
/dev/mapper/vgadm-lvrear
                     ext4   1.2T  1.0T  143G  88% /export/rear_old

And tested the archived file system /export/rear_old, and now it is read-only mode.

$ sudo touch /export/rear_old/test.ro
touch: cannot touch `/export/rear_old/test.ro': Read-only file system

No comments:

Post a Comment