Friday, September 28, 2012

How to files in AIX from a rm -rf / command

What to do if someone accidentally remove some system critical files in rootvg?

# rm -rf ~

In this case, the stuffs in root's home directory will be removed. You will see /admin, /dev, /bin, etc being deleted. If you are quick to notice the mistake and halt the rm command,

IMPORTANT: Keep your existing SSH session alive at all cost. Otherwise, working on a terminal via HDMC or similar is going to be painful.

h2. So, its "oh shit" right?

Hopefully, /lib is not removed yet, else you are in bigger shit.

ssh, rsync, scp all will no longer work. Let's do a little self repair before recovering the rest of the files.

Is your tar command gone? What commands do i have left?

h2. Recover mkdir

Read from http://coding-journal.com/restoring-your-unix-system-after-rm-rf/ about this one. Try the following.

# echo "mkdir 'bin', 0777;" | perl

This is on the assumption that you lost your mkdir command but still have perl. Here, the /bin directory is created. The full permission is just for this emergency purpose, you can probably change it later.

h2. Recovering /dev

If /dev is lost, you may need to create some of the more critical ones to enable scp and ssh to bring in your backups (mksysb). The steps below are used on AIX 7.1 SP4

# cd /dev
# mknod random c 36 0
# mknod urandom c 36 1
# mknod null c 2 2
# chmod 644 random urandom
# chmod 666 null

Go ahead and try a ssh or sync. If cannot, you may need to restart sshd.

# stopsrc -s sshd
# startsrc -s sshd


if you have another server with a similar make, you can also try to recreate the disk structure but this is not critical if you have a backup which you can extract later. The convention should follow a standard since IBM name all the basic disk the same way on AIX 7.1


# mknod hd1 b 10 8
# mknod  hd2 b 10 5
# mknod  hd3 b 10 7
# mknod  hd4 b 10 4
# mknod  hd5 b 10 1
# mknod  hd6 b 10 2
# mknod  hd8 b 10 3
# mknod  hd9var b 10 6
# mknod  hd10opt b 10 9
# mknod  hd11admin b 10 10
# chmod 660 hd1 hd10opt hd11admin hd2 hd3 hd4 hd5 hd6hd7hd8 hd9var

# mknod hd1 c 10 8

# mknod  hd2 c 10 5
# mknod  hd3 c 10 7
# mknod  hd4 c 10 4
# mknod  hd5 c 10 1
# mknod  hd6 c 10 2
# mknod  hd8 c 10 3
# mknod  hd9var c 10 6
# mknod  hd10opt c 10 9
# mknod  hd11admin c 10 10
# chmod 660 hd1 hd10opt hd11admin hd2 hd3 hd4 hd5 hd6hd7hd8 hd9var


h2. Lets bring back the files. 

After you bring in your backup, you can the commence restoration. I will list example using mksysb file.

Say we need to recover /dev, /admin, bosinst.data, etc. We just double check if the file is usable and whether the original files are inside this archive.

# restore -alvTf mksysb_mysever_date > /tmp/mksysb.server.txt

 Then proceed to restore.

# restore -xvqf mksysb_mysever_date ./bosinst.data
# mv bosinst.data /

# restore -xvqf mksysb_mysever_date ./dev
# cd ./dev
# mv * /dev/

# restore -xvqf mksysb_mysever_date ./admin
# cd ./admin
# mv * /admin/

# restore -xvqf mksysb_mysever_date ./.ssh
# mv .ssh /

# restore -xvqf mksysb_mysever_date ./.profile
# mv .profile /

so on and forth.

If you have another server with the similar make or build, you may want to go the extra step to verify if there are anything else that is still missing. 

In addition, go for a reboot at the nearest opportunity to ensure all is working well. Nothing is confirmed until it is tested and proven working.


No comments: