Veritas Volume Manager / cluster server (now called InfoScale Availability Suite) Configuration

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)

Installing and configuring VCS/VXvM two node cluster

The application example below uses IBM Informix DB server

Hardware System installation

Note: The Below example uses 2 Sun fire M5000 servers for the hardware.
For the full XSCF configuration click here.

IP and FC port Layout

Below is an example of the Network and Fiber Channel layout.

informix1 – rack1
Network – informix1-xscf

IOU#0 – right-side

Server Port Switch Port Vlan
pcie4/port-0 113/9/sw-a 20
pcie4/port-3 113/10/sw-a private
IOU#1 – left-side

Server Port Switch Port Vlan
pcie4/port-0 113/9/sw-b 20
pcie4/port-3 113/10/sw-b private
Fiber Chanel
Server Port MDF Port FC Switch Port
IOU#0/0 MDF-A/113/3 DCX 1/23
IOU#0/1 MDF-B/113/3 DCX 2/23
IOU#1/0 MDF-A/113/4 DCX 1/23
IOU#1/1 MDF-B/113/4 DCX 2/23
informix2 – rack2
Network – informix2-xscf

IOU#0 – right-side

Server Port Switch Port Vlan
pcie4/port-0 120/3/sw-a 20
pcie4/port-3 120/4/sw-a private
IOU#1 – left-side

Server Port Switch Port Vlan
pcie4/port-0 120/3/sw-b 20
pcie4/port-3 120/4/sw-b private
Fiber Chanel
Server Port MDF Port FC Switch Port
IOU#0/0 MDF-A/120/1 DCX 1/24
IOU#0/1 MDF-B/120/1 DCX 2/24
IOU#1/0 MDF-A/120/2 DCX 1/24
IOU#1/1 MDF-B/120/2 DCX 2/24

xscf login information

Next will login to the physical domain VCS will be installed (assuming xscf install was completed described here).

Install the server with jumpstart

Note: The install below still uses Solaris 10 (which uses a jumpstart server for the OS install), a similar OS install can be done in Solaris 11.x by using an AI server.

The below add_install_client options where used to install the client

After configuration was done, the fowling commend was issued to jumpstart the servers

grep informix /etc/hosts

cat rules

cat m5000_zfs

cat sysidcfg

Preparing the systems for VCS

Setup a ZFS mirrored pool system & VCS data

Create informix user and group

Create an informix directory

Installing & configuring VCS

Make available the web installer

Install and select VCS options

Note: Connect your browser to server:14161, enter root login/password

Note: Make sure to specify both nodes, the install will then auto install all pkgs on both nodes

  • Run first a pre-install check, if successful
  • Select Storage Foundation and High Availability, and follow install steps below

When done, Select Storage Foundation for Cluster file system/HA, and run the installer

Now we are ready to run the Cluster Configuration

  • The cluster Name was set to informix-cluster
  • the private interfaces was selected as nxge3 and nxge7
  • Create a cluster admin account with password when prompted
  • Add ass mail server
  • Follow the rest of the install till completed
Install the Acc library on both nodes

Install the informix agent on both nodes

Add the agent to the cluster framework – on one node only

Modify the agent configuration

The Informix agent configuration file will need to be modified with the correct values

Informix instance name
  • make sure to use the informix instance virtual name in the INFORMIXSQLHOSTS file.

Note: Use a virtual host name it ensures that if the application needs to be migrated, you are not tied down by the physical IP address given to the IBM Informix Dynamic Server.

System tuning

Next lets do some system tuning before we move on to informix VXvM configuration.

System changes



Informix volume groups – VxVM Configuration

Initialize disk groups

Create informix database disk group

Initialize Cluster fencing disk group

Create Cluster I/O fencing disk group

Configure I/O fencing

Select votingdg disk group

Configure Sub Disks

Get the the disk block count – free disk space

Create the sub disk (get the size from the last col in the free output)

Note: I used 4 sets, which will be describe below.

Create Set 1

Create sub disks and plex

Create Set 2

Create sub disks and plex

Create Set 3

Create sub disks and plex

Create Set 4

Create sub disks and plex

Configure cache volumes

Initialize and add extra disk as cache used for snapshots.

Make sure the volumes are prepared

Prepare the volumes

Create a space-optimized snapshot from s1-vol133 available as s1-vol133-snap1

Note: The size for changes are set as 775MB (meaning total change size)

Create all snaps at once

Note: If the above doesn’t work

Check volume status

Refresh all volumes

To resync/refresh all volumes, making the snaps like the source
Get a list of all the volumes to a file, then run it

Restore(rollback) from target snapshot s1-vol133-snap1 to s1-vol133

Destroy the snapshot

To create a snapshot for multiple volumes at once

Experiment helpful

Get all VxVM permeates

Problems / solutions

Cannot complete the snapshot operation.
VxVM vxassist ERROR V-5-1-6197 volume is busy in another operation
Verify from the ‘vxprint -g ‘ command if “TUTIL0 or PUTIL0” flags is set. If set, then perform the following command and then continue with the SNAP operation

Remove disk plex or volumes

removes subdik with plex

Removing a volume

To remove a disk group with the disks

Additional commends

Disable/Enable VxVM DMP

Display Veritas fencing

How to disable VCS auto monitor

Note: Once the preonline trigger is configured, you may see unexpected behavior
while manually switching or performing online operations on the Informix service
group. This behavior is a result of the control logic within the preonline trigger
that protects the Informix SDS Primary Secondary relationship. For system
maintenance, if you prefer to perform manual operations on the service groups,
you can do so by disabling the preonline trigger as follows:

Switch and check primary secondary

VXS config files

To change VCS configuration

  • Modify /etc/VRTSvcs/conf/config/

To re-read the config files restart VCS with

  • Run hastop -all -force
  • Run hastart on each node
  • VCS logs

    How to start Informix agent

    VCS/VxVM disable procedure

    Run the below commends on all cluster nodes to disable VCS and VxVM.

    Helpful links

    VCS configuration
    VxVM Cheat sheet
    VxVM layers
    VxVM volume Administration
    Oracle Cluster and zone HA
    VxVM snapshots / checkpoints

    Leave a Reply

    Notify of