I have 3 disks which are part of the mirror group.
root@myserver # vxdisk list | grep ^c
c1t0d0s2 sliced rootdisk rootdg online
c1t1d0s2 sliced rootmirror1 rootdg online
c2t0d0s2 sliced - - online
c2t1d0s2 sliced - - online
Naming conventions:
- rootdisk
- rootmirror1
- rootmirror2
- rootmirror3
Now i want to add the mirror back. We are going to attach back device c2t0d0 into the rootdg disk group.
Always run the command vxdisk -ht to check what is the actual volume name to attach the disk to.
Setup the disk
root@myserver # vxdisksetup -if c2t0d0 (for VxVM 3.5)
root@myserver # vxdisksetup -if c2t0d0 format=sliced (for VxVM 4.0)
root@myserver # /usr/lib/vxvm/bin/vxdisksetup -if c2t0d0 format=sliced (for VxVM 4.1)
Add the disk into rootdg.
root@myserver # vxdg -g rootdg adddisk rootmirror2=c2t0d0
Mirror each of the partitions. Here i throttle the synchronisation speed to have less performance impact on the server.
root@myserver # vxassist -o slow=1500 -g rootdg mirror rootvol rootmirror2 &
root@myserver # vxassist -o slow=1500 -g rootdg mirror opt rootmirror2 &
root@myserver # vxassist -o slow=1500 -g rootdg mirror swapvol rootmirror2 &
root@myserver # vxassist -o slow=1500 -g rootdg mirror usr rootmirror2 &
root@myserver # vxassist -o slow=1500 -g rootdg mirror var rootmirror2 &
To monitor the progress of synchronisation.
root@myserver # vxtask list
TASKID PTID TYPE/STATE PCT PROGRESS
166 ATCOPY/R 07.88% 0/25166079/1982464 PLXATT opt opt-01
167 ATCOPY/R 25.29% 0/4194828/1060864 PLXATT swapvol swapvol-01
168 ATCOPY/R 15.13% 0/4194828/634880 PLXATT usr usr-01
169 ATCOPY/R 02.82% 0/16779312/473088 PLXATT var var-01
Saturday, October 31, 2009
How to add a mirror disk back to VxVM
How to ensure VTOC is updated from VxVM in SunOS?
It is important to keep the VTOC in sync with the setup in VxVM else you risk not being able to boot up the disk after un-encapsulating.
SUN also advised that if the VxVM disk mirrors are not identical to the first disk, this would result in unusable mirrors.
Also after creation of VxVM rootdisk and rootmirrors, you may want to create the underlying partition on the rootdisk and rootmirror.
Here, i will show the steps for setting up new disk as RAID1 mirror
root@myserver # vxdisksetup -if c2t0d0
root@myserver # vxdg -g rootdg adddisk rootmirror2=c2t0d0
root@myserver # vxassist -g rootdg mirror rootvol rootmirror2 &
root@myserver # vxassist -g rootdg mirror usr rootmirror2 &
root@myserver # vxassist -g rootdg mirror var rootmirror2 &
root@myserver # vxassist -g rootdg mirror opt rootmirror2 &
Take a look at the VTOC
root@myserver # prtvtoc /dev/dsk/c1t0d0s2
* /dev/dsk/c1t0d0s2 partition map
*
* Dimensions:
* 512 bytes/sector
* 107 sectors/track
* 27 tracks/cylinder
* 2889 sectors/cylinder
* 24622 cylinders
* 24620 accessible cylinders
*
* Flags:
* 1: unmountable
* 10: read-only
*
* Unallocated space:
* First Sector Last
* Sector Count Sector
* 0 2889 2888
* 8395434 4286577640 5777
* 71127180 4232235550 8395433
* 58730481 12396699 71127179
*
* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 2 00 5778 8389656 8395433
1 3 01 8395434 4194828 12590261
2 5 00 0 71127180 71127179
3 15 01 2889 2889 5777
4 14 01 5778 71121402 71127179
5 0 00 33564402 25166079 58730480
6 4 00 12590262 4194828 16785089
7 7 00 16785090 16779312 33564401
Now, the VTOC is not updated. we need to use VxVM command to update VTOC. not really advisable to update using "format" command manually since its more error prone.
Tag Description (2nd column from right)
0x00 UNASSIGNED
0x01 BOOT
0x02 ROOT
0x03 SWAP
0x04 USR
0x05 BACKUP
0x06 STAND
0x07 VAR
0x08 HOME
Flag Description (right-most column)
0x00 MOUNTABLE
0x01 UNMOUNTABLE
0x10 READ-ONLY
Note that Swap is NOT mountable.
root@myserver # vxprint -htg rootdg
DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
V NAME RVG KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg rootdg all all 0 1076053596.1025.myserver
dm rootdisk c1t0d0s2 sliced 2632 71121402 -
dm rootmirror1 c1t1d0s2 sliced 2632 71121402 -
dm rootmirror2 c2t0d0s2 sliced 2632 71121402 -
v opt - ENABLED ACTIVE 25166079 ROUND - fsgen
pl opt-01 opt ENABLED ACTIVE 25166079 CONCAT - RW
sd rootdisk-05 opt-01 rootdisk 33558624 25166079 0 c1t0d0 ENA
pl opt-02 opt ENABLED ACTIVE 25166079 CONCAT - RW
sd rootmirror1-05 opt-02 rootmirror1 12584484 25166079 0 c1t1d0 ENA
pl opt-03 opt ENABLED ACTIVE 25166079 CONCAT - RW
sd rootmirror2-05 opt-03 rootmirror2 33558624 25166079 0 c2t0d0 ENA
v rootvol - ENABLED ACTIVE 8389656 ROUND - root
pl rootvol-01 rootvol ENABLED ACTIVE 8389656 CONCAT - RW
sd rootdisk-01 rootvol-01 rootdisk 0 8389656 0 c1t0d0 ENA
pl rootvol-02 rootvol ENABLED ACTIVE 8389656 CONCAT - RW
sd rootmirror1-01 rootvol-02 rootmirror1 0 8389656 0 c1t1d0 ENA
pl rootvol-03 rootvol ENABLED ACTIVE 8389656 CONCAT - RW
sd rootmirror2-01 rootvol-03 rootmirror2 0 8389656 0 c2t0d0 ENA
v swapvol - ENABLED ACTIVE 4194828 ROUND - swap
pl swapvol-01 swapvol ENABLED ACTIVE 4194828 CONCAT - RW
sd rootdisk-02 swapvol-01 rootdisk 8389656 4194828 0 c1t0d0 ENA
pl swapvol-02 swapvol ENABLED ACTIVE 4194828 CONCAT - RW
sd rootmirror1-02 swapvol-02 rootmirror1 8389656 4194828 0 c1t1d0 ENA
pl swapvol-03 swapvol ENABLED ACTIVE 4194828 CONCAT - RW
sd rootmirror2-02 swapvol-03 rootmirror2 8389656 4194828 0 c2t0d0 ENA
v usr - ENABLED ACTIVE 4194828 ROUND - fsgen
pl usr-01 usr ENABLED ACTIVE 4194828 CONCAT - RW
sd rootdisk-03 usr-01 rootdisk 12584484 4194828 0 c1t0d0 ENA
pl usr-02 usr ENABLED ACTIVE 4194828 CONCAT - RW
sd rootmirror1-03 usr-02 rootmirror1 37750563 4194828 0 c1t1d0 ENA
pl usr-03 usr ENABLED ACTIVE 4194828 CONCAT - RW
sd rootmirror2-03 usr-03 rootmirror2 12584484 4194828 0 c2t0d0 ENA
v var - ENABLED ACTIVE 16779312 ROUND - fsgen
pl var-01 var ENABLED ACTIVE 16779312 CONCAT - RW
sd rootdisk-04 var-01 rootdisk 16779312 16779312 0 c1t0d0 ENA
pl var-02 var ENABLED ACTIVE 16779312 CONCAT - RW
sd rootmirror1-04 var-02 rootmirror1 41945391 16779312 0 c1t1d0 ENA
pl var-03 var ENABLED ACTIVE 16779312 CONCAT - RW
sd rootmirror2-04 var-03 rootmirror2 16779312 16779312 0 c2t0d0 ENA
For automated step, try this
# vxbootsetup
For manual step, try these
Notice that you did NOT create the disk partitions manually when you initialized the mirror disk. This is because VxVM automatically does its partitioning for you. This is also true for the root disk after it has been re-initialized and brought into rootdg. That is, the original rootdisk partition scheme has been replaced by Veritas's partition scheme. This means that the underlying partitions for each volume(swap, /opt, /var, /home) are not physically laid out on the drives corresponding to the veritas volumes.
In some situations though, you might want to be able to access the partitions corresponding to each volume by manually mounting the slice. In a recovery situation for example, you may want to boot from cdrom and access the /opt partition by manually mounting the slice without the veritas drivers loaded. This is not possible unless you create the underlying partition mapping directly to the opt volume using the "vxmksdpart" command.
All underlying partitions except / should be created. The / partition is automatically created and is mapped correctly to the corresponding veritas root volume.
One Caveat, you cannot use partition 3 or 4 because this is typically where Veritas stores is private and public region. The following is snipped from the "format" utility listing the Veritas partitions. This shows that the root partition exists but the swap, /var, /opt, home partitions are not mapped on the drive(Veritas knows where they exist on the drive because of its private and public regions):
Part Tag Flag Cylinders Size Blocks
0 root wm 1 - 2282 3.91GB (2282/0/0) 8194662 <=== "/" exists
1 unassigned wm 0 0 (0/0/0) 0 <=== Unassigned
2 backup wm 0 - 4923 8.43GB (4924/0/0) 17682084
3 - wu 0 - 0 1.75MB (1/0/0) 3591 <=== Private Region
4 - wu 1 - 4923 8.43GB (4923/0/0) 17678493 <=== Public Region
5 unassigned wm 0 0 (0/0/0) 0 <=== Unassigned
6 unassigned wm 0 0 (0/0/0) 0 <=== Unassigned
7 unassigned wm 0 0 (0/0/0) 0 <=== Unassigned
You may use the unassigned partitions which are 1, 5, 6, and 7 to map to the veritas volumes. Be sure to document which partitions map to the Veritas volumes for future reference. The "vxmksdpart" arguments are the subdisk (which maps directly to the volume), slice, tag and flag. Be sure to use the correct tag and flag for your particular Solaris version. Reference the man page for "fmthard" for correct options. Use the "vxprint" command to get the correct subdisk names: Generic command # /usr/lib/vxvm/bin/vxmksdpart
In my case
Rootdisk:
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootdisk-02 1 0x03 0x01 <=== swap
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootdisk-05 5 0x00 0x00 <=== /unassigned
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootdisk-03 6 0x04 0x00 <=== /usr
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootdisk-04 7 0x07 0x00 <=== /var
RootMirror:
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootmirror1-02 1 0x03 0x01 <=== swap
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootmirror1-05 5 0x00 0x00 <=== /unassigned
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootmirror1-03 6 0x04 0x00 <=== /usr
# /usr/lib/vxvm/bin/vxmksdpart -g rootdg rootmirror1-04 7 0x07 0x00 <=== /var
References:
http://sysunconfig.net/unixtips/jmr_Mirroring_Boot_Disks_Veritas_Volume_Manager.htm
http://unixway.com/vm/veritasvm/rootdg.html
How to unmount a online / mounted VXFS on 5.0 MP3
You may have encountered this
# umount -f /opt/app/oracle
UX:vxfs umount: ERROR: V-3-26360: file system /opt/app/oracle has been mount locked
Taken from the symantec support site - http://seer.entsupport.symantec.com/docs/313851.htm
Details:
Storage Foundation 5.0MP3 introduced a new feature called VxFS filesystem lock which disallows accidental unmounts when the file system resource is online. New umount option mntunlock is used to clear the lock and then unmount the filesystem. The offline script for the Mount resource will use this new option.
How to check if the filesystem is locked by VCS:
# mount -v | grep mntlock
Sometimes, it may be necessary to unmount a mount locked filesystem. This is for cases where VCS service groups having DiskGroup resources configured with UnMountVolumes attribute set and the volumes are mounted outside of VCS control (this is not very common). Use the VxFS umount command for Solaris and vxumount command for Linux to manually unmount the file system
Solaris:
# /opt/VRTS/bin/umount -o mntunlock=VCS
Linux:
# /usr/lib/fs/vxfs/umount -o mntunlock=VCS
Example:
# mount -v | grep mntlock
/dev/vx/dsk/oracledg/oracle on /opt/app/oracle type vxfs read/write/setuid/devices/mincache=direct/convosync=direct/delaylog/largefiles/qio/ioerror=mwdisable/mntlock=VCS/dev=46c2ee0 on Mon Sep 15 18:28:18 2008
# /opt/VRTS/bin/umount -o mntunlock=VCS /opt/app/oracle
HowTo setup Link-based IPMP with VCS for Solaris 10.
Thanks Bret Sanders for the knowledge. Have used this to setup and like to share.
Original: http://www.runningunix.com/2009/01/link-based-ipmp-setup-with-vcs/
We know that Solaris 10 allows us to setup link-based IPMP. Though a nice feature which the OS determines NIC availability solely on the NIC driver reporting the physical link status - UP or DOWN, setting link-based IPMP in VCS is a little different.
Previous versions used “probe-based” IPMP, where connectivity is tested by pinging something on the network from each interface.
While probe-based is actually a more thorough test (tests network layer 3 as well as 2), it is much more cumbersome to configure, and you need an extra IP address for each interface for “test” addresses. usually Link-based IPMP is sufficient for most applications.
For some reason, configuring link-based IPMP in VCS is somewhat tricky, and the documentation doesn’t seem to help much. It seems all the default values for VCS are for probe-based IPMP only.
To achieve link-based IPMP, There are values you need to change from the defaults in the MultiNICB resource:
Tells VCS to use mpathd for network link status
UseMpathd: 1
The default, /usr/sbin/in.mpathd is just incorrect - it doesn’t live there.
MpathCommand: /usr/lib/inet/in.mpathd -a
If you leave this at 1, it will overwrite your /etc/hostname.xxx files with probe-based IPMP configuration
ConfigCheck: 0
The “interface alias” for each device is not needed, leave them blank.
Device: (your IPMP interfaces here)
You want VCS to NOT ignore link status, since this is how link-based IPMP works.
IgnoreStatus: 0
Do not use your IPMP group name here, it’s not needed. VCS is not monitoring the group, mpathd is.
GroupName:
Here’s how it looks in main.cf:
MultiNICB csgmultinic (
UseMpathd = 1
MpathdCommand = “/usr/lib/inet/in.mpathd -a”
ConfigCheck = 0
Device = { ce0 = “”, ce4 = “” }
IgnoreLinkStatus = 0
)
How to Install another Veritas Demo License even though u have already two demo license extensions
Encountered this quite some time back, thought it would be nice to share this useful information.
i need to install an extension to the Veritas demo license even though i had already 2 Veritas demo licenses extensions.
While trying to install a Veritas demo license, encountered this error "ERROR V-21-1-19 Duplicate License key detected".
[root@myserver ~]# vxlicinst
Symantec License Manager vxlicinst utility version 3.02.16.0
Copyright (C) 1996-2006 Symantec Corporation. All rights reserved.
Enter your license key : XXXX-WDBK-ASG8-ST7O-ZZZZ-OCZW-YYY
vxlicinst ERROR V-21-1-19 Duplicate License key detected
Steps to resolve this.
Backup the current license
# cd /etc/vx/licenses/lic
# tar cfp vxlic.tar *.vxlic
Install the new license using vxlincinst again.
Do remember to verify that the demo license has been extended.
# vxlicrep
Restart VCS to verify that it can be started up using the new demo license.
Using sendmail to check fully qualified hostname and permission
root@myserver:/>/usr/lib/mail/sh/check-hostname
Hostname myserver OK: fully qualified as myserver.
root@myserver:/>/usr/lib/mail/sh/check-permissions
No unsafe directories found.
How to resolve "Unable to qualify my own domain name" error
You may have received this error in /var/adm/messages
Symptom 1:
/var/adm/messages
Mar 13 11:15:16 myserver sendmail[8420]: [ID 702911 mail.crit] My unqualified host name (myserver.) unknown; sleeping for retry
Mar 13 11:15:16 myserver sendmail[8421]: [ID 702911 mail.crit] My unqualified host name (myserver.) unknown; sleeping for retry
Mar 13 11:16:16 myserver sendmail[8421]: [ID 702911 mail.alert] unable to qualify my own domain name (myserver.) -- using short name
Mar 13 11:16:16 myserver sendmail[8420]: [ID 702911 mail.alert] unable to qualify my own domain name (myserver.) -- using short name
Symptom 2:
when you try to manually telnet localhost 25 or mailx, either you can send to a particular domain or the respond is very slow after entering the "mail from" command.
Resolution:
update /etc/hosts ( and /etc/inet/ipnodes if you are on solaris 10) to use the following.
root@myserver:/>more /etc/hosts
# internet host table
#====================
127.0.0.1 localhost
10.106.127.105 myserver. myserver
10.106.63.32 myserver-rsc
#
# Internet host table
#
::1 localhost
127.0.0.1 localhost
10.106.127.105 myserver. myserver loghost
Description:
Usually i would add the "myserver." with the dot behind to /etc/hosts but in Solaris 10, things work a little differently.
It turn out that in solaris 10, the OS will go through /etc/inet/ipnodes for IPv4 address before going to /etc/hosts.
In this case, since ldap does not have the entry, OS will go straight to /etc/inet/ipnodes.
This also mean that if you change the host IP of the solaris 10 server, please change in /etc/inet/ipnodes as well otherwise you have conflict of IP addresses.
Note that this is Solaris specific.
# man ipnodes
...
...
NOTES
IPv4 addresses can be defined in the ipnodes file or in the
hosts file. See hosts(4). The ipnodes file will be searched
for IPv4 addresses when using the getipnodebyname(3SOCKET)
API. If no matching IPv4 addresses are found in the ipnodes
file, then the hosts file will be searched. To prevent
delays in name resolution and to keep /etc/inet/ipnodes and
/etc/inet/hosts synchronized, IPv4 addresses defined in the
hosts file should be copied to the ipnodes file.
...
..
# more /etc/nsswitch.conf
...
...
# consult /etc "files" only if ldap is down.
hosts: ldap [NOTFOUND=continue] files
...
...
# Note that IPv4 addresses are searched for in all of the ipnodes databases
# before searching the hosts databases.
...
...
How to Check the Product ID / Serial Number of the HP blade
This is similar to the sneep command in solaris.
To Check on the Product ID
root@myserver:~>#dmidecode |grep 'Product ID'
String 1: Product ID: 347957-B21
To Check on the Serial Number
root@myserver:~>#dmidecode |grep 'Serial Number'
Serial Number: XXXXXXXXXX (For the Blade Server)
Serial Number: YYYYYYYYYY (For the Blade Chasis/Enclosure)
Or
root@myserver:~>#hpasmcli -s "show server" | grep 'Serial'
Serial No. : XXXXXXXXXX
If not listed, then have to check the blade label physically... :)
Remove 169.254.0.0 routing in Linux
During the setup of Linux OS, you may see the special route even though you haven't added any,
[root@myserver ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.106.2.0 * 255.255.255.0 U 0 0 0 bond0
10.106.2.0 * 255.255.255.0 U 0 0 0 eth3
169.254.0.0 * 255.255.0.0 U 0 0 0 eth3
default 10.106.2.254 0.0.0.0 UG 0 0 0 bond0
This is the zeroconf route.
Though harmless, the less route you have, the more secure i suppose.
To disable the zeroconf route during system boot, edit the /etc/sysconfig/network file and add the following NOZEROCONF value to the end of the file.
NETWORKING=YES
HOSTNAME=localhost.localdomain
NOZEROCONF=yes
How to start HP Insight Diagnostics
Using HP Offline Insight Diagnostics using the SmartStart CD
Click on http://h18023.www1.hp.com/support/files/server/us/download/24733.html
Download the file to a temporary directory and extract the ISO image to that directory. To create a CD from this ISO image, you must use a software utility capable of writing an ISO file to a recordable media such as CD-R.
Note: Simply copying the ISO file to a CD-R will NOT result in a usable CD.
Then boot off the server using the SmartStart CD and then Click On Maintain Server
Then click on Diagnose Server
The Insight Diagnostics Window Opens Up
Change the 'view level' from the default 'Overview' to 'Advanced'.
Under 'categories' drop down box, select 'All'
Save the file as default on a floppy disk
Recently, i have encountered a few of the BL25p blades that cant even boot up. Then we won't be able to use this tool at all.
Do beware that HP support may insist that you patch, the firmwares, iLO, etc, which i think is a waste of time. Insist on a on-site engineer if you have the contract else likely its hardware issue. In my case, was a faulty mainboard.
How to find Temperature Range on Sun Blade [TM] and Sun Fire[TM] systems
To find out the temperature range and the current temperature on Sun Blade Desktop or a Sun Fire entry level server, we can try this
# prtpicl -v -c temperature-sensor
This will extract the information from FRU and display on screen. DO NOTE that the info in the FRU is set at manufacturing, and can varies from CPU speed to CPU type. It should not be assumed that there is a default temp for all CPUs.
e.g.
# prtpicl -v -c temperature-sensor
CPU0_DIE_TEMPERATURE_SENSOR (temperature-sensor, 3a00000579)
:PSVC_FRU (3a000006ebTBL)
| (3a0000058dH) |
:PSVC_PHYSICAL_DEVICE (3a00000678TBL)
| (3a000005a1H) |
:PSVC_PARENT (3a000005ebTBL)
| (3a0000049dH) |
:PSVC_TS_OVERTEMP_LED (3a000005b8TBL)
| (3a000004e1H) |
:HighShutdownThreshold 95
:HighWarningThreshold 93
:LowShutdownThreshold -10
:LowWarningThreshold 0
:Temperature 59
:FaultInformation NO_FAULT
:State OK
:_class temperature-sensor
:name CPU0_DIE_TEMPERATURE_SENSOR
CPU1_DIE_TEMPERATURE_SENSOR (temperature-sensor, 3a00000583)
:PSVC_FRU (3a000006eeTBL)
| (3a00000592H) |
:PSVC_PHYSICAL_DEVICE (3a0000067bTBL)
| (3a000005a6H) |
:PSVC_PARENT (3a000005eeTBL)
| (3a000004a2H) |
:PSVC_TS_OVERTEMP_LED (3a000005bbTBL)
| (3a000004e1H) |
:HighShutdownThreshold 95
:HighWarningThreshold 93
:LowShutdownThreshold -10
:LowWarningThreshold 0
:Temperature 57
:FaultInformation NO_FAULT
:State OK
:_class temperature-sensor
:name CPU1_DIE_TEMPERATURE_SENSOR
Here, system will send out alert upon 93 deg C, shutdown upon 95 deg C. Current temperature is 57 deg C.
How to Display IML logs without going to iLO
while logged into the blade, fire up your favourite terminal and try the following.
# /sbin/cpqimlview
# /sbin/hplog -v
# /sbin/hplog -V