Oracle AS Guard cloning - Add MT instance using Oracle ASGuard

Oracle AS Guard cloning

Please see my previous thread on how to setup the Oracle AS Guard setup.

In this thread i am posting how to install MT instance (Portal and Wireless option) and clone this newly installed MT instance to the standby instance.

Even if the primary node is unavailable (primary node is not reachable), you can still switch over to the seconday node (manual switchover). I have tested this and it perfectly works fine.OracleAS guard is more flexible.

The MT cloning took 10 min and switch over took 3 min.
In case of swithover situation,your cutover time would be approx 3-5 min.

With the help of DNS switch and/or LBR, the FO or SO option will be very transparent to the users/other applications.

Caution -
-------

Be little cautious using this option for your Oracle IAS environment as Active/Passive cluster.Without proper understanding of this process, you may corrupt the AS Guard setup.

This feature has to be completely tested before using it in a production environment.
Steps
-----
1.Keep the primary and standby instances up and running (Both instances running only Infrastructure) - ie Oracle AS Guard is setup and active for the infra instance.
2.Install MT (portal and Wireless option) on to the first instance(Primary)
3.Clone the newly added MT instance to the standby instance (DSA port should same as the primary DSA).start the asgctl from MT instance (not from infra home)
4.If this clone is successfull , veryify the topology and confirm everything is fine.
5.Now test the switchover is possible
6.Switch back to the primary node

on Primary node
===============
After the MT install, stop all the MT opmn process

connect to asgctl

connect to primary database

clone the MT instance to the standby instance.

ASGCTL> clone instance mt.ias1.ushasuji.com to rinfra
Generating default policy for this operation

IAS1:7891
Clone Instance

IAS1:7891 (home /u01/oracle/product/mt)
Running configure option in bkp_restore script.
Running node_backup prepare option in bkp_restore script. This may take a few minutes
Running node_backup image_backup option in bkp_restore script. This may take a few minutes
Please run /u01/oracle/product/mt/backup_restore/brHome12228823ba8.sh as root. Enter "Yes" after the script is completed successfully, "No" otherwise. Yes or No
yes

ias1 192.168.1.16:7891 (home /u01/oracle/product/mt)
Copying backup file "/u01/oracle/product/loha/mt/as_mt.ias1.ushasuji.com_2009-06-28_17-04-03.img" from "192.168.1.15" [192.168.1.15] to "192.168.1.16" at "/u01/oracle/product/loha/mt/as_mt.ias1.ushasuji.com_2009-06-28_17-04-03.img"
Unpacking image backup at target host. This may take a few minutes
Running node_restore sys_init option in bkp_restore script.
Please run /u01/oracle/product/mt/backup_restore/brHome122283ed9fe.sh as root. Enter "Yes" after the script is completed successfully, "No" otherwise. Yes or No
yes
Running node_restore inst_register option in bkp_restore script.
Please run /u01/oracle/product/mt/backup_restore/brHome122283ed9fe.sh as root. Enter "Yes" after the script is completed successfully, "No" otherwise. Yes or No
yes
ASGCTL>

Check everything is copied on to the standby MT home.It may prompt to run the copy script twice as root user.

This clone process will stop the MR database istance and all the Infra/MT opmn process on node1 and starts in node2.I mean the current primary node becomes standby and old standby becomes primary. So easy .... All is taken care automatically by Oracle AG Guard.

If no errors , then you are all to test the switchover.

From Primary Node
-------------------

Now switchover both infrastructure and Mid Tier applications to standby node - Rinfra

ASGCTL> switchover topology to rinfra

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com

ias1:7891 (home /u01/oracle/product/mt)
Shutting down component HTTP_Server
Shutting down component WebCache
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Running emctl command: "/u01/oracle/product/infra/bin/emctl status iasconsole".
Shutting down component OID
Shutting down component HTTP_Server
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader

IAS1:7891 (home /u01/oracle/product/mt)
Shutting down each instance in the topology
Shutting down component HTTP_Server
Shutting down component WebCache
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Running emctl command: "/u01/oracle/product/infra/bin/emctl status iasconsole".
Shutting down component OID
Shutting down component HTTP_Server
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader

IAS1:7891 (home /u01/oracle/product/mt)
Synchronizing topology
Synchronizing each instance in the topology to standby topology
Starting backup of topology ""
Backing up and copying data to the standby topology
Backing up each instance in the topology
Starting backup of instance "mt.ias1.ushasuji.com"
Configuring the backup script

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Starting backup of instance "infra.ias1"
Configuring the backup script
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Performing configuration ...
Configuration completed successfully !!!
Backing up the instance configuration files. This may take a few minutes

IAS1:7891 (home /u01/oracle/product/mt)
Backing up the instance configuration files. This may take a few minutes
Deleted directory "/u01/oracle/product/mt/dsa/backup".

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Deleted directory "/u01/oracle/product/infra/dsa/backup".
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Backing up configuration files ...
Warning(s) during backup - please check /u01/oracle/product/infra/dsa/backup/log/2009-06-28_17-46-19_config_bkp.log
Configuration backup archive is /u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-28_17-46-19.jar
Configuration backup completed with warning(s) logged in
file /u01/oracle/product/infra/dsa/backup/log/2009-06-28_17-46-19_config_bkp.log

IAS1:7891 (home /u01/oracle/product/mt)
Copying backup file "/u01/oracle/product/mt/dsa/backup/config_bkp_2009-06-28_17-46-19.jar" from "IAS1" [192.168.1.15] to "192.168.1.16" at "/u01/oracle/product/mt/dsa/backup/mt.ias1.ushasuji.com/config_bkp_2009-06-28_17-46-19.jar"

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Copying backup file "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-28_17-46-19.jar" from "IAS1" [192.168.1.15] to "192.168.1.16" at "/u01/oracle/product/infra/dsa/backup/infra.ias1/config_bkp_2009-06-28_17-46-19.jar"
Copying backup catalog file /u01/oracle/product/infra/backup_restore/data/catalog.txt from IAS1 [192.168.1.15] to 192.168.1.16

IAS1:7891 (home /u01/oracle/product/mt)
Copying backup catalog file /u01/oracle/product/mt/backup_restore/data/catalog.txt from IAS1 [192.168.1.15] to 192.168.1.16

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Completed backup of instance "infra.ias1"

IAS1:7891 (home /u01/oracle/product/mt)
Completed backup of instance "mt.ias1.ushasuji.com"
Starting restore of topology ""
Restoring data to the standby topology
Restoring each instance in the topology

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Copying backup file "/u01/oracle/product/infra/dsa/backup/infra.ias1/config_bkp_2009-06-28_17-46-19.jar" from "192.168.1.16" to "ias1" [192.168.1.16] at "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-28_17-46-19.jar"

ias1:7891 (home /u01/oracle/product/mt)
Copying backup file "/u01/oracle/product/mt/dsa/backup/mt.ias1.ushasuji.com/config_bkp_2009-06-28_17-46-19.jar" from "192.168.1.16" to "ias1" [192.168.1.16] at "/u01/oracle/product/mt/dsa/backup/config_bkp_2009-06-28_17-46-19.jar"

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Deleting backup file "/u01/oracle/product/infra/dsa/backup/infra.ias1/config_bkp_2009-06-28_17-46-19.jar"

ias1:7891 (home /u01/oracle/product/mt)
Deleting backup file "/u01/oracle/product/mt/dsa/backup/config_bkp_2009-06-28_17-46-19.jar"
Starting restore of instance "mt.ias1.ushasuji.com"

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Starting restore of instance "infra.ias1"

ias1:7891 (home /u01/oracle/product/mt)
Configuring the backup script

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Configuring the backup script

ias1:7891 (home /u01/oracle/product/mt)
Restoring the instance configuration files. This may take a few minutes

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Performing configuration ...
Configuration completed successfully !!!
Restoring the instance configuration files. This may take a few minutes
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Restoring configuration files from backup 2009-06-28_17-46-19 ...
Configuration file restore completed successfully !!!
Running opmnctl reload command: "/u01/oracle/product/infra/opmn/bin/opmnctl reload".

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
Starting backup/synchronization of database "ias.ushasuji.com"

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Starting restore/synchronization of database "ias.ushasuji.com"

IAS1:7891
Synchronizing topology completed successfully

IAS1 192.168.1.15:7890
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com
Switchover - init.

ias1 192.168.1.16:7890
Switchover - init.

IAS1 192.168.1.15:7890
Switchover - primary preparing.

ias1 192.168.1.16:7890
Switchover - standby preparing.
Stopping Job Queue Scheduler.
Stopping Advanced Queue Time Manager.
Starting managed recovery in the standby database.

IAS1 192.168.1.15:7890
Switchover - primary processing.
Active user sessions detected
The primary database is ready to switchover
Switching over the primary database to the standby role
Shutting down the old primary database
Issuing "shutdown immediate;" to shutdown the database
Starting up the old primary database as the new standby
Issuing "startup nomount;" to start the database
Old primary database now running as the new standby

ias1 192.168.1.16:7890
Switchover - standby processing.
Getting the switchover status from the database.
Switching over the standby database to the primary role.
Shutting down the new primary database.
Issuing "shutdown immediate;" to shutdown the database
Starting up the new primary database.
Issuing "startup open;" to start the database
Switchover - standby finishing.
Enabling the archive destination in the new primary database.
Starting log archiving in the new primary database.

IAS1 192.168.1.15:7890
Switchover - primary finishing.
Deferring the archive destination in the new standby database.
Enabling managed recovery in the new standby database.

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
Running opmnctl reload command: "/u01/oracle/product/infra/opmn/bin/opmnctl reload".
Starting component OID
Starting component dcm-daemon
Configuring the backup script
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Performing configuration ...
Configuration completed successfully !!!
Executing restore_config -F DCM-resyncforce option in bkp_restore.pl script
Oracle Application Server Backup/Recovery Tool 10g (10.1.2.0.2)
Copyright (c) 2004, 2005, Oracle. All rights reserved.

Resynchronizing instance with DCM repository ...
Running opmnctl reload command: "/u01/oracle/product/infra/opmn/bin/opmnctl reload".
Executing opmnctl startall command

ias1:7891 (home /u01/oracle/product/mt)
Starting component dcm-daemon
Configuring the backup script
Executing restore_config -F DCM-resyncforce option in bkp_restore.pl script
Executing opmnctl startall command

ias1:7891
HA directory exists for instance mt.ias1.ushasuji.com

ias1 192.168.1.16:7890 (home /u01/oracle/product/infra)
HA directory exists for instance infra.ias1

IAS1:7891
HA directory exists for instance mt.ias1.ushasuji.com

IAS1 192.168.1.15:7890 (home /u01/oracle/product/infra)
HA directory exists for instance infra.ias1

ias1 192.168.1.16:7890
Verifying that the topology is symmetrical in both primary and standby configuration
Switchover topology to standby host completed successfully


From node2 (new primary node)
---------------------------

try to switchover back to the original primary node

OPMN wont start up after successfull installation of Oracle Portal Mid Tier - Release 10.1.2.0.2

After successful installation of Oracle Portal , i stopped the all the IAS components using opmnctl - No errors.
But when i tried to start the opmn proces - during Portal Mid tier clonig, got into this error.

RCV: Transport endpoint is not connected Communication error with the OPMN server local port. Check the OPMN log files


I am even not able to start the OPMN process.

opmnctl start gave me the same error

While troubleshooting the issue, i found almost 35 process linked to MT opmn process.

like ,
/u01/oracle/product/mt/opmn/bin/opmn -d

After killing all these process i am able to start opmn process.


Could be another bug in Oracle iAS

Oracle Portal 10.1.2.0.2 Clonig

Will post soon

All about LVM's - Small Demo

All about LVM's
-----------------

Because of the wide technolgy today, i feel DBA should have some knowledge on hardware and OS knowledge.

I am a big fan of LVM's.I have been working on LVM's (including Veritas LVM/Volume manager and Solstice DiskSuite in Solaris ) for the past 10 years.Veritas used to be little complex those days,but any more today.

I had an opportunity to work on Veritas Volume Manager and LVM's (approx 8 years ago , for a client called Auripay at Cambridge).I used to hot swap the SCSI drives and test the VM's groups.It was so intresting to see the VM groups swaping to different nodes.Prasad putta (MD of Auripay , now he is VP at Oatsystems) used to encourage me and given me full freedom on this implementation/testing.Got a very good exposure on veritas there.

I have done all my home testing with LVM's only - not just with regular ext2 FS. Because it gives me the flexibility to add more space to the existing volumes as i needed and additionally i do see much better system performance.


Just i want to share my small Linux LVM experience (quite simple) with Other DBA's

Demo
----------

I created a small device of 8GB size

After login just find out the device name , here it is sdd.

you can use the dmesg command to find this.


SCSI device sdd: 16777216 512-byte hdwr sectors (8590 MB)
sdd: cache data unavailable
sdd: assuming drive cache: write through
SCSI device sdd: 16777216 512-byte hdwr sectors (8590 MB)
sdd: cache data unavailable
sdd: assuming drive cache: write through
sdd: unknown partition table
Attached scsi disk sdd at scsi0, channel 0, id 3, lun 0
Fusion MPT SAS Host driver 3.02.62.01rh
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: sda3: orphan cleanup on readonly fs
ext3_orphan_cleanup: deleting unreferenced inode 115623
EXT3-fs: sda3: 1 orphan inode deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
SELinux: Disabled at runtime.
SELinux: Unregistering netfilter hooks
inserting floppy driver for 2.6.9-34.EL
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
vmxnet: module license 'unspecified' taints kernel.
VMware vmxnet virtual NIC driver release 1.0.1 build-29996
ACPI: PCI interrupt 0000:00:11.0[A] -> GSI 10 (level, low) -> IRQ 10
Found vmxnet/PCI at 0x1424, irq 10.
vmxnet: numRxBuffers=(100*24) numTxBuffers=(100*64) driverDataSize=9000
divert: allocating divert_blk for eth0
eth0: vmxnet ether at 0x1424 assigned IRQ 10.
ACPI: PCI interrupt 0000:00:12.0[A] -> GSI 9 (level, low) -> IRQ 9
Found vmxnet/PCI at 0x14a4, irq 9.
vmxnet: numRxBuffers=(100*24) numTxBuffers=(100*64) driverDataSize=9000
divert: allocating divert_blk for eth1
eth1: vmxnet ether at 0x14a4 assigned IRQ 9.
ACPI: PCI interrupt 0000:00:13.0[A] -> GSI 5 (level, low) -> IRQ 5
Found vmxnet/PCI at 0x1824, irq 5.
vmxnet: numRxBuffers=(100*24) numTxBuffers=(100*64) driverDataSize=9000
divert: allocating divert_blk for eth2
eth2: vmxnet ether at 0x1824 assigned IRQ 5.
pcnet32.c:v1.31 29.04.2005 tsbogend@alpha.franken.de
ACPI: PCI interrupt 0000:00:14.0[A] -> GSI 11 (level, low) -> IRQ 11
shpchp: shpc_init : shpc_cap_offset == 0
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
USB Universal Host Controller Interface driver v2.2
ACPI: PCI interrupt 0000:00:07.2[D] -> GSI 9 (level, low) -> IRQ 9
uhci_hcd 0000:00:07.2: UHCI Host Controller
uhci_hcd 0000:00:07.2: irq 9, io base 00001060
uhci_hcd 0000:00:07.2: new USB bus registered, assigned bus number 1
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
NET: Registered protocol family 10
Disabled Privacy Extensions on device c0378f60(lo)
IPv6 over IPv4 tunneling driver
divert: not allocating divert_blk for non-ethernet device sit0
ip_tables: (C) 2000-2002 Netfilter core team
vmxnet_init_ring: offset=9000 length=9000
vmxnet_init_ring: offset=9000 length=9000
ip_tables: (C) 2000-2002 Netfilter core team
vmxnet_init_ring: offset=9000 length=9000
ACPI: AC Adapter [ACAD] (on-line)
ACPI: Power Button (FF) [PWRF]
eth0: no IPv6 routers present
eth1: no IPv6 routers present
eth2: no IPv6 routers present
EXT3 FS on sda3, internal journal
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com
cdrom: open failed.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Adding 1534196k swap on /dev/sda2. Priority:-1 extents:1
IA-32 Microcode Update Driver: v1.14
microcode: No new microdata for cpu 0
IA-32 Microcode Update Driver v1.14 unregistered
parport0: PC-style at 0x378 [PCSPP,TRISTATE]
ip_tables: (C) 2000-2002 Netfilter core team
ip_tables: (C) 2000-2002 Netfilter core team
ip_tables: (C) 2000-2002 Netfilter core team
ip_tables: (C) 2000-2002 Netfilter core team
iscsi-sfnet: Loading iscsi_sfnet version 4:0.1.11-2
iscsi-sfnet: Control device major number 254
OCFS2 Node Manager 1.2.3 Wed Jul 26 12:04:10 PDT 2006 (build 56074a7e99f767e0530 6907521c8ea25)
OCFS2 DLM 1.2.3 Wed Jul 26 12:04:10 PDT 2006 (build 05157a797e82010a31dfd4c78548 4fe9)
OCFS2 DLMFS 1.2.3 Wed Jul 26 12:04:11 PDT 2006 (build 05157a797e82010a31dfd4c785 484fe9)
OCFS2 User DLM kernel interface loaded
i2c /dev entries driver
ASM: oracleasmfs mounted with options:
ASM: maxinstances=0
SCSI device sdb: 167772160 512-byte hdwr sectors (85899 MB)
sdb: cache data unavailable
sdb: assuming drive cache: write through
sdb: unknown partition table
SCSI device sdc: 96468992 512-byte hdwr sectors (49392 MB)
sdc: cache data unavailable
sdc: assuming drive cache: write through
sdc: unknown partition table
SCSI device sdd: 16777216 512-byte hdwr sectors (8590 MB)
sdd: cache data unavailable
sdd: assuming drive cache: write through
sdd: unknown partition table
parport0: PC-style at 0x378 [PCSPP,TRISTATE]
lp0: using parport0 (polling).
lp0: console ready
ip_tables: (C) 2000-2002 Netfilter core team
vmxnet_init_ring: offset=9000 length=9000
eth0: no IPv6 routers present
[root@Apps3 ~]# fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

Now create 2 partitions using fdisk utility
-------------------------------------------

The number of cylinders for this disk is set to 1044.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdd: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1044, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1044, default 1044): +4000M

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (488-1044, default 488):
Using default value 488
Last cylinder or +size or +sizeM or +sizeK (488-1044, default 1044):
Using default value 1044

Command (m for help): p

Disk /dev/sdd: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdd1 1 487 3911796 83 Linux
/dev/sdd2 488 1044 4474102+ 83 Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

First Create a physical Volume on first partition
-------------------------------------------------

[root@Apps3 ~]# pvcreate /dev/sdd1 ---> First Partition
Physical volume "/dev/sdd1" successfully created

next create a Volume group on top of the physical volume just (/dev/sdd1) created
--------------------------------------------------------------------------------
[root@Apps3 ~]# vgcreate volgrp_1 /dev/sdd1
Volume group "volgrp_1" successfully created

See the Phyical volume structure
---------------------------------

[root@Apps3 ~]# pvdisplay /dev/sdd1
--- Physical volume ---
PV Name /dev/sdd1
VG Name volgrp_1
PV Size 3.73 GB / not usable 0
Allocatable yes
PE Size (KByte) 4096
Total PE 954
Free PE 954
Allocated PE 0
PV UUID v3tkkp-Wsa2-yk3s-lhwG-wlkn-F8g5-2GcfGS

View the Volume group structure just created
-------------------------------------------

[root@Apps3 ~]# vgdisplay volgrp_1
--- Volume group ---
VG Name volgrp_1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 3.73 GB
PE Size 4.00 MB
Total PE 954
Alloc PE / Size 0 / 0
Free PE / Size 954 / 3.73 GB
VG UUID zUAAMY-9Z7L-4CsQ-HUKK-p37P-TlvR-XzfUKW

[root@Apps3 ~]# fdisk /dev/sdd -l

Disk /dev/sdd: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdd1 1 487 3911796 83 Linux
/dev/sdd2 488 1044 4474102+ 83 Linux

To increase the VG size, just add the partition /dev/sdd2 to the existing volume group
-----------------------------------------------------------------------------------------
[root@Apps3 ~]# pvcreate /dev/sdd2 --> create this before adding
Physical volume "/dev/sdd2" successfully created
[root@Apps3 ~]# vgextend volgrp_1 /dev/sdd2
Volume group "volgrp_1" successfully extended

Now you see the Volume group size increased from 4GB to 8GB
------------------------------------------------------------
[root@Apps3 ~]# vgdisplay volgrp_1
--- Volume group ---
VG Name volgrp_1
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 7.99 GB
PE Size 4.00 MB
Total PE 2046
Alloc PE / Size 0 / 0
Free PE / Size 2046 / 7.99 GB
VG UUID zUAAMY-9Z7L-4CsQ-HUKK-p37P-TlvR-XzfUKW

Now remove the physical volume from the extended volume group
-------------------------------------------------------------
[root@Apps3 ~]# vgreduce volgrp_1 /dev/sdd2
Removed "/dev/sdd2" from volume group "volgrp_1"

See the size is reduced to 4GB now
----------------------------------

[root@Apps3 ~]# vgdisplay volgrp_1
--- Volume group ---
VG Name volgrp_1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 3.73 GB
PE Size 4.00 MB
Total PE 954
Alloc PE / Size 0 / 0
Free PE / Size 954 / 3.73 GB
VG UUID zUAAMY-9Z7L-4CsQ-HUKK-p37P-TlvR-XzfUKW

Create a logical volume now on top of the volume group - volgrp_1 size 200MB
------------------------------------------------------

[root@Apps3 ~]# lvcreate -n logvol_1 --size 200M volgrp_1
Logical volume "logvol_1" created

See the structure of LVM just created
------------------------------------

[root@Apps3 ~]# lvdisplay /dev/volgrp_1/logvol_1
--- Logical volume ---
LV Name /dev/volgrp_1/logvol_1
VG Name volgrp_1
LV UUID jwQVYp-0K5H-iJLW-JO3n-CPn4-nnyI-ibglYZ
LV Write Access read/write
LV Status available
# open 0
LV Size 200.00 MB
Current LE 50
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0

[root@Apps3 ~]# vgdisplay volgrp_1|grep "Total PE"
Total PE 954

Now you are ready to create any FS on to this LVM - here we are creating ext2 FS
---------------------------------------------------------------------------------
[root@Apps3 ~]# mkfs /dev/volgrp_1/logvol_1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
51200 inodes, 204800 blocks
10240 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67371008
25 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345, 73729

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 32 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

mount the LVM -
--------------
[root@Apps3 ~]# mount /dev/volgrp_1/logvol_1 /u01
[root@Apps3 ~]# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda3 3241580 2404772 672144 79% /
/dev/sda1 396623 13530 362612 4% /boot
none 676380 0 676380 0% /dev/shm
/dev/mapper/volgrp_1-logvol_1
198337 1550 186547 1% /u01


All set to create any Oracle installations on top of this LVM now.

Hope this helps

Oracle R12 upgrade

Will briefly cover the upgrade of EBUS R11i to R12.

Oracle R12 and Oracle IAS integration

Will post soon....

Oracle R12 Forms Customization

Will post soon...

Oracle LDAP Replication Setup - HA

Will post soon....

SAP ECC 6 - moving to RAC - HA Setup

Will post soon....

Oracle 11G Active DataGuard and Active Duplicate database setup

Will post soon...

Oracle R12 and MAA configuration

Will post soon....

Oracle R12 and RAC implementation - HA setup

Will post soon....

MS SQL Server and Oracle Architecture comparison

Will compare Oracle and MS SQL Server ....

SAP ECC 6 Cloning

Will post soon....

Oracle EBUS R12 HA Setup 2 node Apps with LBR and PCP configuration

Will post soon....

Oracle IAS DataGuard HA Implementation - Oracle IAS 10G R2 10.1.2.0.2

Oracle IAS DataGuard Implementation - Oracle IAS 10G R2 10.1.2.0.2

I was searching the google to find any help on this setup.But i couldn't find any - not even a single post.Really surprised.
Anyhow this configuration was very useful and no more worries about the IAS configuration corruption issues.

Installation overview:
---------------------

1.Set up the Virtual server name for both nodes.I used Redhat cluster suite for this purpose.
2.Install IAS (Infrastucture only) on the first node (ias1)
3.check all the OPMN components are up and running
4.Backup the entire node1 configuration
5.Copy the portlists.ini to the second node
6.Install IAS (Infrastucture only) using the portlists.ini file
7.check all the OPMN components are up and running
8.Backup the entire node2 configuration
9.start DG control utility on node1
10.connect as ias_admin and connect to the primary database (node1)
11.dump policies
12.Verify the topologies on primary and secondary instances
13.Initiate the standby topology on node2.This will backup the configuration file (using bkup_restore utility) and prepare both instances for the DG setup.Also
convert both instances into archive mode.Also the secondary shutdown and kept in mount stage (not opened). ASG does all for you.
14.BAckup the entire system node1 and node2
15.Test the switch over to node2
16.Test the switch over to node1

You have completed the IAS DG setup.

This setup includes only the Infrastucture and not any of the MID Tiers like BI or Portal/Wireless.But the setup is the same for all configurations.

NOTE - Be cautious while maintaing this environment.Virtual IP conflict may mess the startup of the OPMN components and some of OPMN process may not startup.I had some bad time on this.


On Node1
---------


ASGCTL> connect asg ias_admin/iasadmin1
Successfully connected to IAS1:7892
ASGCTL> discover topology oidhost=infra.ushasuji.com oidsslport=636 oidpassword=iasadmin1
Discovering topology on host "IAS1" with IP address "192.168.1.15"

IAS1:7892
Connecting to the OID server on host "infra.ushasuji.com" using SSL port "636" and username "orcladmin"
Getting the list of databases from OID
Gathering database information for SID "ias" from host "infra"
Getting the list of instances from OID
Gathering instance information for "infra.infra" from host "infra"
The topology has been discovered. A topology.xml file has been written to each home in the topology.

ASGCTL> dump polices

dump policies

Display the topology information

Below is an example of dump topology:

connect ASG host_foo ias_admin/pass
dump topology



ASGCTL> verify topology with rinfra
Generating default policy for this operation

IAS1:7892
HA directory exists for instance infra.infra

ias2:7892
HA directory exists for instance infra.infra

IAS1:7892
Verifying that the topology is symmetrical in both primary and standby configuration
ASGCTL> set trace on all

ASGCTL> instantiate topology to rinfra.ushasuji.com using policy /u01/oracle/product/infra/dsa/conf/verify_policy.xml

IAS1:7892
Instantiating each instance in the topology to standby topology
HA directory exists for instance infra.infra

ias2:7892
HA directory exists for instance infra.infra

IAS1:7892
Verifying that the topology is symmetrical in both primary and standby configuration

IAS1:7892 (home /u01/oracle/product/infra)
This is primary infrastructure host
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com

ias2:7892 (home /u01/oracle/product/infra)
Shutting down each instance in the topology
Shutting down component OID
Shutting down component HTTP_Server
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader
This is standby infrastructure host
Deleting the standby database ias.ushasuji.com
Shutting down the standby database ias.ushasuji.com
Issuing "shutdown immediate;" to shutdown the database

IAS1:7892 (home /u01/oracle/product/infra)
Creating a standby template

IAS1:7892
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com
Creating physical standby database - prepare phase
Setting db and log file name convert
* The prepare phase was run previously. Redoing. ***

ias2:7892
Setting db and log file name convert

IAS1:7892
Ensuring database "ias.ushasuji.com" is in ARCHIVELOG mode.
Querying primary database for data files.
Creating Standby database parameter file "/u01/oracle/product/infra/dbs/tmp_initias.ora".

ias2:7892
Creating Standby database instance "ias".
Verifying datafile location on standby host.
Updating net service entry for "ias.ushasuji.com" in tnsnames file.
Updating net service listener entry for "ias.ushasuji.com" in listener file.
Updating net service entry for "ias_remote1.ushasuji.com" in tnsnames file.

IAS1:7892
Successfully completed Prepare task for Create Physical Standby.
Creating physical standby database - copy phase
* The copy phase was run previously. Redoing. ***

ias2:7892
Checking if standby database is running

IAS1:7892
Querying primary database for data files.
Shutting down the primary database
This operation requires the database to be shutdown. Do you want to continue? Yes or No
yes
Issuing "shutdown immediate;" to shutdown the database
Issuing "startup mount ;" to start the database
Creating Standby database control file "/u01/oracle/product/infra/dbs/tmp_ias.ctl".
Shutting down the primary database
Issuing "shutdown immediate;" to shutdown the database
Copying database datafiles to the standby host
Issuing "startup open ;" to start the database
Successfully completed Copy task for Create Physical Standby.
Creating physical standby database - finish phase
Create Physical Standby:Finish - Init.

ias2:7892
Create Physical Standby:Finish - Init.

IAS1:7892
Create Physical Standby:Finish - Prepare primary.
Saving redo log information for standby server

ias2:7892
Create Physical Standby:Finish - Configure standby.
Creating directories for dump and trace
Connecting to standby database
Creating a spfile for standby database
Starting the standby database
Issuing "startup nomount ;" to start the database
Creating standby redo log
Adding log archive destination to the parameter file
Starting managed recovery
Making sure that log is being applied to standby database

IAS1:7892
Create Physical Standby:Finish - Configure primary.
Verifying access to standby database
Adding log archive destination in the parameter file
Performing a log switch

ias2:7892
Verifying log application

IAS1:7892
Successfully completed Finish task for Create Physical Standby.

IAS1:7892 (home /u01/oracle/product/infra)
Synchronizing topology
Synchronizing each instance in the topology to standby topology
Starting backup of topology ""
Backing up and copying data to the standby topology
Backing up each instance in the topology
Starting backup of instance "infra.infra"
Configuring the backup script
Backing up the instance configuration files. This may take a few minutes
Deleted directory "/u01/oracle/product/infra/dsa/backup".
Copying backup file "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_00-38-35.jar" from "IAS1" [192.168.1.15] to "192.168.1.26" at "/u01/oracle/product/infra/dsa/backup/infra.infra/config_bkp_2009-06-13_00-38-35.jar"
Copying backup catalog file /u01/oracle/product/infra/backup_restore/data/catalog.txt from IAS1 [192.168.1.15] to 192.168.1.26
Completed backup of instance "infra.infra"
Starting restore of topology ""
Restoring data to the standby topology
Restoring each instance in the topology

ias2:7892 (home /u01/oracle/product/infra)
Copying backup file "/u01/oracle/product/infra/dsa/backup/infra.infra/config_bkp_2009-06-13_00-38-35.jar" from "192.168.1.26" to "ias2" [192.168.1.16] at "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_00-38-35.jar"
Deleting backup file "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_00-38-35.jar"
Starting restore of instance "infra.infra"
Configuring the backup script
Restoring the instance configuration files. This may take a few minutes

IAS1:7892 (home /u01/oracle/product/infra)
Starting backup/synchronization of database "ias.ushasuji.com"

ias2:7892 (home /u01/oracle/product/infra)
Starting restore/synchronization of database "ias.ushasuji.com"
Synchronizing topology completed successfully

IAS1:7892
Synchronizing topology completed successfully
ASGCTL>


Switch Over to Node2
----------------------


ASGCTL> connect asg ias_admin/iasadmin1
Successfully connected to IAS1:7892
ASGCTL> set primary database sys/sys2@ias
Checking connection to database ias
ASGCTL> switchover topology to rinfra.ushasuji.com
Generating default policy for this operation

IAS1:7892
Switchover each instance in the topology to standby topology

IAS1:7892 (home /u01/oracle/product/infra)
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com

ias2:7892 (home /u01/oracle/product/infra)
Shutting down each instance in the topology
Shutting down component OID
Shutting down component HTTP_Server
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader

IAS1:7892 (home /u01/oracle/product/infra)
Shutting down each instance in the topology
Shutting down component OID
Shutting down component HTTP_Server
Shutting down component OC4J
Shutting down component dcm-daemon
Shutting down component LogLoader
Synchronizing topology
Synchronizing each instance in the topology to standby topology
Starting backup of topology ""
Backing up and copying data to the standby topology
Backing up each instance in the topology
Starting backup of instance "infra.infra"
Configuring the backup script
Backing up the instance configuration files. This may take a few minutes
Deleted directory "/u01/oracle/product/infra/dsa/backup".
Copying backup file "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_01-03-34.jar" from "IAS1" [192.168.1.15] to "192.168.1.26" at "/u01/oracle/product/infra/dsa/backup/infra.infra/config_bkp_2009-06-13_01-03-34.jar"
Copying backup catalog file /u01/oracle/product/infra/backup_restore/data/catalog.txt from IAS1 [192.168.1.15] to 192.168.1.26
Completed backup of instance "infra.infra"
Starting restore of topology ""
Restoring data to the standby topology
Restoring each instance in the topology

ias2:7892 (home /u01/oracle/product/infra)
Copying backup file "/u01/oracle/product/infra/dsa/backup/infra.infra/config_bkp_2009-06-13_01-03-34.jar" from "192.168.1.26" to "ias2" [192.168.1.16] at "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_01-03-34.jar"
Deleting backup file "/u01/oracle/product/infra/dsa/backup/config_bkp_2009-06-13_01-03-34.jar"
Starting restore of instance "infra.infra"
Configuring the backup script
Restoring the instance configuration files. This may take a few minutes

IAS1:7892 (home /u01/oracle/product/infra)
Starting backup/synchronization of database "ias.ushasuji.com"

ias2:7892 (home /u01/oracle/product/infra)
Starting restore/synchronization of database "ias.ushasuji.com"
Synchronizing topology completed successfully

IAS1:7892
Synchronizing topology completed successfully

IAS1:7892 (home /u01/oracle/product/infra)
Creating a standby template

IAS1:7892
Connecting to the primary database ias.ushasuji.com
Gathering information from the primary database ias.ushasuji.com
Switching over standby database
Switchover - init.

ias2:7892
Switchover - init.

IAS1:7892
Switchover - primary preparing.
Creating standby redo logs in the primary database

ias2:7892
Switchover - standby preparing.
Stopping Job Queue Scheduler.
Stopping Advanced Queue Time Manager.
Starting managed recovery in the standby database.

IAS1:7892
Switchover - primary processing.
Recycling the primary database because switchover status is SESSIONS ACTIVE
Shutting down the primary database ias.ushasuji.com
Issuing "shutdown immediate;" to shutdown the database
Starting the primary database ias.ushasuji.com
Issuing "startup restrict ;" to start the database
Stopping Job Queue Scheduler in the primary database
Stopping Advanced Queue Time Manager in the primary database
The primary database is ready to switchover
Switching over the primary database to the standby role
Shutting down the old primary database
Issuing "shutdown immediate;" to shutdown the database
Starting up the old primary database as the new standby
Issuing "startup nomount ;" to start the database

ias2:7892
Switchover - standby processing.
Getting the switchover status from the database.
Switching over the standby database to the primary role.
Shutting down the new primary database.
Issuing "shutdown immediate;" to shutdown the database
Starting up the new primary database.
Issuing "startup open ;" to start the database
Switchover - standby finishing.
Enabling the archive destination in the new primary database.
Starting log archiving in the new primary database.

IAS1:7892
Switchover - primary finishing.
Deferring the archive destination in the new standby database.
Enabling managed recovery in the new standby database.

ias2:7892 (home /u01/oracle/product/infra)
Starting each instance in the topology
Starting component OID
Starting component dcm-daemon
Configuring the backup script
Executing restore_config -F DCM-resyncforce option in bkp_restore.pl script
Executing opmnctl startall command

ias2:7892
HA directory exists for instance infra.infra

IAS1:7892
HA directory exists for instance infra.infra

ias2:7892
Verifying that the topology is symmetrical in both primary and standby configuration
ASGCTL>


Verify the IAS components on node 1 - All process will be down except the DSA process
-----------------------------------

[oracle@IAS1 ~]$ $ORACLE_HOME/opmn/bin/opmnctl status

Processes in Instance: infra.infra
-------------------+--------------------+---------+---------
ias-component | process-type | pid | status
-------------------+--------------------+---------+---------
DSA | DSA | 24993 | Alive
LogLoader | logloaderd | N/A | Down
dcm-daemon | dcm-daemon | N/A | Down
OC4J | OC4J_SECURITY | N/A | Down
HTTP_Server | HTTP_Server | N/A | Down
OID | OID | N/A | Down

Verify the IAS components on node 2
----------------------------------

[oracle@ias2 ~]$ /u01/oracle/product/infra/opmn/bin/opmnctl status

Processes in Instance: infra.infra
-------------------+--------------------+---------+---------
ias-component | process-type | pid | status
-------------------+--------------------+---------+---------
DSA | DSA | 31526 | Alive
LogLoader | logloaderd | N/A | Down
dcm-daemon | dcm-daemon | 12489 | Alive
OC4J | OC4J_SECURITY | 22594 | Alive
HTTP_Server | HTTP_Server | 22596 | Alive
OID | OID | 12385 | Alive


Once the switch over is complete, you can either failover or switch over back to Node1


Hope this helps.

Oracle Enterprise security installation

will be posted...

Oracle 11G - Admin Features

Will be posted soon

All about Oracle RAC HA Networking

We will see what are the available Oracle Networking options within RAC environment.

1.Client side Load Balancing (CSLB)
2.Server side Load Balacing (SSLB)
3.Client connect time Fail Over (CTFO)
4.Client side Load Balancing (CSLB)
5.TAF at client level and Server level
6.FAN


1.client side Load Balancing

Randomly the connection gets routed to the available listeners.

DisAdvantage -

Connections are completely unaware of the existing load on different
servers and a CPU-saturated server can still continue to get connections.


Example -

Oms.blue.com =
(DESCRIPTION =
(LOAD_BALANCE = yes) --> Which is default.
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1530)
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1530))
(CONNECT_DATA =
(SERVICE_NAME = oms.blue.com)
)
)

2.Server-side connection load balancing (PREFER_LEAST_LOADED_NODE) -It Improves performance.

Balance the connections among available listeners based on CPU's workload.It is at listener level.So each listener knows other instanaces workload (via dispatcher and PMON registration).This is achieved via Local_listener and remote_listener parameters.Load balancing at Dispatcher level in case if MTS is used.

PMON registers itself with listener.
PMON computes metrics every 2-3 seconds
PMON checks the listener status every 1 second

See the PMON dynamic registration between nodes.


LSNRCTL> status LISTENER_10G_rac1
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)(PORT=1530)(IP=FIRST)))
STATUS of the LISTENER
------------------------
Alias LISTENER_10G_rac1
Version TNSLSNR for Linux: Version 10.2.0.4.0
Start Date 11-JUN-2009 16:04:37
Uptime 0 days 4 hr. 53 min. 19 sec
Trace Level off
Security OFF
SNMP ON
Listener Parameter File /u01/oracle/product/10g/asm/network/admin/listener.ora
Listening Endpoints Summary...
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.132.1.95)(PORT=1530)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.132.1.96)(PORT=1530)))
Services Summary...
Service "+ASM" has 1 instance(s).
Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "+ASM1" has 1 instance(s).
Instance "+ASM1", status UNKNOWN, has 1 handler(s) for this service...
Service "+ASM_XPT" has 1 instance(s).
Instance "+ASM1", status READY, has 1 handler(s) for this service...
Service "PLSExtProc" has 1 instance(s).
Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service...
Service "OMS" has 2 instance(s).
Instance "oms1", status READY, has 2 handler(s) for this service...
Instance "oms2", status READY, has 1 handler(s) for this service...
Service "oms" has 2 instance(s).
Instance "oms1", status READY, has 1 handler(s) for this service...
Instance "oms2", status READY, has 1 handler(s) for this service...
Service "oms_XPT" has 2 instance(s).
Instance "oms1", status READY, has 2 handler(s) for this service...
Instance "oms2", status READY, has 1 handler(s) for this service...
The command completed successfully

PREFER_LEAST_LOADED_NODE_=ON/OFF - As the name says , the connection always goes to the least loaded node(CPU and connections). Please be aware that connections bursts (large number of connections initiated very quickly one after another) may not be properly balanced due to statistics update latency.


Connect time fail over (CTFO) -

This is not same as TAF. TAF is for connected sessions only (application level).It is specified via failover_mode option.CTFO is used only when the intial connection is attempted and it is specified via failover option.

3.Client side connect time fail Over - Fail Over Option for oracle clients.

It enables the clients to connect to another listener if the initial connection to the first listener fails.Address list specified in the service determines how many services (address) are tried.Without this option , only one listener is tried.

fail over is ON by default for Oracle clients.


eg.,


CTFO -

oms.blue.com=
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1530)) ==> Attempts for the first node,if it fails it goes to the next node
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1530))
)
(CONNECT_DATA =
(SERVICE_NAME = oms.blue.com)
)
)

4.client side Load Balancing (CSLB) - Oracle clients randomly connects to the nodes listed in the address list.
Now you can identify that CTFO is always sequential and CSLB is random.


oms.blue.com=
(DESCRIPTION =
(ADDRESS_LIST =
(LOAD_BALANCE = yes)
(ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1530))
(ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1530))
)
(CONNECT_DATA =
(SERVICE_NAME = oms.blue.com)
)
)




5.Transparent Application Failover (TAF) - It can be at client level or server level.

You can leave the TAF to be decided either at client or at server level.

It enables the client to reconnect to other available instances if the current connected database instance fails.

Fail over_mode Options

Backup - service name for backup connections.It should be used with preconnect option.Could be an overhead having duplicate connections in the backup server.
type - fail over type.It can be at session or select statement level.
session - automatically connect to available instances,but recovery of select statement is not possible.
select - automatically connect to available instances, recovery of select statement is possible - But Overhead
Method - Method of connection.
Basic - It can establish connection fail over connection to available instances.No overhead
Preconnect - use pre established connections.Overhead
Retries - Number of times to attempt to connect after a fail over. 5 if delay is specified.

Delay - In seconds - before it tries to re attempt connection.

Eg.,


oms.blue.com =
(DESCRIPTION=
(LOAD_BALANCE=on) --> client Load Balance
(FAILOVER=on) ---> No need to specify.It is always default.
(ADDRESS=
(PROTOCOL=tcp)
(HOST=rac1-vip)
(PORT=1521))
(ADDRESS=
(PROTOCOL=tcp)
(HOST=rac2-vip)
(PORT=1521))
(CONNECT_DATA=
(SERVICE_NAME=oms.blue.com)
(FAILOVER_MODE=
(TYPE=select) ----> Failover is at statement level
(METHOD=basic)))) ---> Connection method is simple.No reduendent connections established in the backup server.


TAF at server side -
--------------------

This option can be set at server level.If specified then no need at client level.It could be a additional overhead at server level due to the additional latency involved between servers.But the Oracle clients are restricted to use only the TAF options specified at the server level.

srvctl add service -d oms2 -s omsservice -r "rac1,rac2" -P BASIC --> Adding a service with BAISC fail over.
srvctl start service -d oms -s omsservice --> starting the omsservice.
srvctl config service -d omsservice --> checking the configuration
srvctl status service -d oms --> checking the status of database and services.


To see the TAF information at database level use the following query -

select FAILOVER_METHOD,FAILOVER_TYPE,FAILOVER_RETRIES,FAILOVER_DELAY,ENABLED,NETWORK_NAME from dba_services where service_id =< service ID>;

5. FAN

The Fast Application Notification mechanism (FAN) is a high-availability mechanism
that lets the applications know about server or node events (for example,
up and down events). It uses user-written programs or shell scripts that are
automatically executed in case of status change. The 10g version of the
JDBC driver is a subscriber of FAN (both the thin and thick driver) and will
automatically rebalance connections across existing and new nodes. To use
FAN, the application must use the JDBC Implicit Connection Cache.

Eg., of thin JDBC connection

url="jdbc:oracle:thin:@(DESCRIPTION=(LOAD_BALANCE=on)
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip) (PORT=1530))
(ADDRESS=(PROTOCOL=TCP)(HOST=rac2-vip)(PORT=1530)))
(CONNECT_DATA=(SERVICE_NAME=oms)))"
CRM=(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=rac3-vip)(PORT=1530))
(CONNECT_DATA=(SERVICE_NAME=oms)))
CRM= (DESCRIPTION=
(ADDRESS_LIST=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=rac1-vip)(PORT=1530))
(ADDRESS=(PROTOCOL=TCP)(HOST=rac2-vip)(PORT=1530))
)
(CONNECT_DATA=(SERVICE_NAME=oms)))

OPA - AERS Installation 4.5.3

OPA Application AERS (Adverse Event Reporting System) installation will be explained briefly

RH ES 4 and Oracle IAS 10G CFC setup

I will be posting the following things in this thread.

1.Set RHES 4 U3 , Cluster Suite installation.
2.Setup GFS.
3.Configure Oracle IAS 10G CFC setup with RHES 4.

Oracle Web Cache Active / Active Cluster

In this topic i will explain how Oracle Web cache can be effectively used for Load balancing Oracle products like Oracle Applications EBUS , Oracle IAS - BI (of course it cannot be effectively used with Oracle Portal).

Oracle Fail Safe installation with Oracle IAS 10G - CFO configuration

I will post this soon.

My Favorite SAP T Codes

My favorite TC on SAP
---------------------
SM04 - Displays the logged in users of the SAP system
SM12 - Displays and deletes Lock entries
SM51 - Display SAP instances of an SAP system -status and complete work process
SM37 - Monitor batch jobs
SM49 - run external OS commands
SMICM - ICM monitor
SMGW - Gateway monitor
AL12 - Buffer Monitor
ST01 - system trace
ST02 - Memory monitor
ST03N- Load monitor
ST05 - Performance analysis

Oracle Related
--------------

DBA Cockpit - system configuration and Maintenance
DB01 - Oracle Lock Monitor
DB02 - Database Performance
DB12 - Backup Logs
DB13 - DBA planning Calenar
DB16 - Database checks - for error mssgaes and warnings
DB17 - Check conditions when using BR Tools
DB26 - Database parameters
DBC0 - Database connections
ST04N- Database Performance monitor

Operating system
----------------
OSO6 and STO6 - OS Monitor
OSO2 and OSO4 - OS configuration

Oracle Data Warehousing Optimization

Will be posted soon

Oracle Pharmaceutical Application - Oracle Clinical 4.5.1 ( HA Setup)

I will post the OPA-Clinical HA implementation document soon.
This include the following Architecture design

2 Node RAC cluster - Oracle 9i -version 9.2.0.8 on Windows 2003 server with OCFS V1
2 Node OC 4.5.1 - Windows 2003 server
A Dedicated node for load balacing (windows 2003 server LBR with DNS configuration).


Please see the Oracle support on this Doc ID 815294.1
-----------------------------------------------------

Currently, Oracle Clinical has NOT been certified in a RAC or Cluster configuration, including OC 4.6.

OPA Support will attempt to assist you with problems even if your system configurations are not identical to the support matrix. Please refer to Note 180430.1 - Oracle Life Sciences Applications Supported Technology Stacks.

However, the assistance on unsupported configurations may be limited if we are unable to reproduce the problem on one of the supported configurations.

If you decide to embark in such an effort, we will support you as long as we can reproduce the error in a standard configured system.

Oracle Grid HA Set up - 10.2.0.5.0

Install RAC 11.1.0.6.0 database first (2 nodes rac1 and rac2)

This is a 4 node Grid configuration -99% HA Grid Architecture
-------------------------------------------------------------
Node1 - Rac1 - Oracle Database 11G 11.0.1.6
Node1 - Rac2 - Oracle Database 11G 11.0.1.6
Node3 - OMS1 - 10.2.0.5.0
Node4 - OMS2 - 10.2.0.5.0

Overview of Installation and Configuration
-----------------------------------------

Node OMS1

1.OMS1 Install 10.2.0.1 Grid on Linux - Software only
2.Upgrade to 10.2.0.5 software only
3.Upgrade the agent 10.2.0.5
4.Configure the Grid using ConfigureFC.pl perl script
5.Drop and Recreate the Grid repository using RepManager
6.Start the OMS process and agent.Check everything is functional
7.Stop the entire system and take a cold backup
8.start the OMS and Agent

Node OMS2
9.Install the grid 10.2.0.1 (using additional management service option)
10.Upgrade the grid 10.2.0.1 and agent
11.export the oms configuration files from OMS1 and import into OMS2 (please see below)
12.setup EMCLI
13.Configure the secure ,Agent registration,webcache ports in LBR

14.secure the OMS and agent on both nodes (OMS1 and OMS2)
15.Check the agenet upload status is fine with LBR , each OMS server is able to loopback to LBR server
16.Stop the entire system and have a backup.


issues encountered

emctl start oms - fails with the following error
-----------------------------------------------
Connection to the repository failed.Verify that the repository connection information provided is correct.

Check the emoms.log file for any errors.It could be ORA-01017 , invalid username/password.

Solution:-

alter user sysman identified by sysman123
emctl stop agent
opmnctl stopall

Change the SYSMAN password in the file emoms.properties
Backup the file emoms.properties
$ cd $ORACLE_HOME/sysman/config
$ cp emoms.properties emoms.properties.orig
Edit the file emoms.properties and change the following lines
oracle.sysman.eml.mntr.emdRepPwd=sysman123
oracle.sysman.eml.mntr.emdRepPwdEncrypted=FALSE

When you restart the OMS, the FALSE setting will change to TRUE and the password will become encrypted in the file


OMS Hearbeat unknown
--------------------
Could be a bug.The agent is not to able to upload into OMS using the HTTP protocol.
So secure the agent first and then the agent

./emctl secure oms

./emctl secure agent


Failover testing for the OMS service
-----------------------------------


[oracle@rac1 config]$ /u01/oracle/product/oms/oms10g/opmn/bin/opmnctl status

Processes in Instance: EnterpriseManager0.rac1.ushasuji.com
-------------------+--------------------+---------+---------
ias-component | process-type | pid | status
-------------------+--------------------+---------+---------
DSA | DSA | N/A | Down
HTTP_Server | HTTP_Server | 31109 | Alive
LogLoader | logloaderd | N/A | Down
dcm-daemon | dcm-daemon | N/A | Down
OC4J | home | 31110 | Alive
OC4J | OC4J_EMPROV | 31111 | Alive
OC4J | OC4J_EM | 31112 | Alive
WebCache | WebCache | 31157 | Alive
WebCache | WebCacheAdmin | 31117 | Alive

On Node2
--------

SQL> select username,machine from v$session where username is not null;

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYS rac2
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYS rac2
SYS rac1
SYS rac2
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com

17 rows selected.

SQL> select host_name from v$instance;

HOST_NAME
----------------------------------------------------------------
rac2

SQL> shutdown abort
ORACLE instance shut down.

On Node 1 - Start the database and you see the Grid process dynamically switch (for rac1 and rac2)
to node1 from node2
-------------------

[oracle@rac1 config]$ ps -ef|grep pmon
oracle 12144 10322 0 00:31 pts/6 00:00:00 grep pmon
[oracle@rac1 config]$


SQL> /

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYS rac1
SYS rac1

SQL> /

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYS rac1
SYSMAN rac1.ushasuji.com
SYS rac1
SYS rac1

SQL> /

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYS rac1
SYSMAN rac1.ushasuji.com
SYSMAN rac2.ushasuji.com
SYS rac1
SYS rac1

18 rows selected.


SQL> !hostname
rac1


Again on node 2
-------------
start the instance 2

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYS rac2
SYS rac1
SYS rac2


USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYS rac1
SYS rac1
SYSMAN rac1.ushasuji.com
SYSMAN rac1.ushasuji.com
SYS rac1
SYS rac1

18 rows selected.

Now the process are evenly distributed on nodes

USERNAME MACHINE
------------------------------ ----------------------------------------------------------------
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com
SYS rac2
SYS rac1
SYS rac2
SYSMAN rac2.ushasuji.com
SYSMAN rac2.ushasuji.com

13 rows selected.


Export the OMS configuration on rac1 node
-------------------------------------------

Enter Enterprise Manager Root (SYSMAN) Password : ExportConfig started... Exporting emoms properties... Exporting emomslogging properties... Exporting secure properties... Export has determined that the OMS is not fronted by an SLB. The local hostname was NOT exported. The exported data can be imported on any host but resecuring of all agents will be required. Please see the EM Advanced Configuration Guide for more details. Exporting port properties... Exporting emkey... Backup has been written to file: export_oms/opf_20090615_073839.bka The export file contains sensitive data. Please ensure that it is kept secure. ExportConfig completed successfully! [oracle@rac1 bin]$ cd export_oms/ [oracle@rac1 export_oms]$ ls opf_20090615_073839.bka [oracle@rac1 export_oms]$ ls -ltr
total 40 -rw-r----- 1 oracle oinstall 38060 Jun 15 07:38 opf_20090615_073839.bka [oracle@rac1 export_oms]$ scp opf_20090615_073839.bka rac2:/u01/oracle/product/oms/oms10g/bin/export_oms
opf_20090615_073839.bka 100% 37KB 37.2KB/s 00:00 [oracle@rac1 export_oms]$




import this backup oms file on to rac2 node


Enter Agent Registration Password: Pre-import OMS backup successfully stored to /u01/oracle/product/oms/oms10g/sysman/backup/opf_20090615_050013.bka Configuring emoms.properties... Configuring emomslogging.properties... Copying emkey to system... Emkey verified. Resecuring the OMS... OMS resecured!


We will be adding DataGuard service to this HA Grid environment soon.

Load Sharing with DNS

Three types of DNS load sharing techniques will be examined here. These are as follows:
1 Backup Server via Redirected Secondary DNS
2 Load Sharing with Round Robin DNS
3 Dynamic Load Balancing DNS

1.Round Robin DNS Load Balancing
--------------------------------
The in-built round-robin feature of BIND of a DNS server can be used to load balance multiple web servers. It is one of the early adopted load balancing techniques to cycle through the IP addresses corresponding to a group of servers in a cluser. The details on the implementation is discussed here.

Pros: Very simple, inexpensive and easy to implement.
Cons: The DNS server does not have any knowledge of the server availability and will continue to point to an unavailable server. It can only differentiate by IP address, but not by server port. The IP address can also be cached by other name servers and requests may not be sent to the load balancing DNS server.

2.Hardware Load Balancing
-------------------------
Hardware load balancers can route TCP/IP packets to various servers in a cluster. These types of load balancers are often found to provide a robust topology with high availability, but comes for a much higher cost.

Pros: Uses circuit level network gateway to route traffic.
Cons: Higher costs compared to software versions.

3.Software Load Balancing
-------------------------
Most commonly used load balancers are software based, and often comes as an integrated component of expensive web server and application server software packages.

Pros: Cheaper than hardware load balancers. More configurable based on requirements. Can incorporate intelligent routing based on multiple input parameters.

Cons: Need to provide additional hardware to isolate the load balancer.


Example

Load Sharing with Round Robin DNS:

One of most common implementations of DNS is the Berkeley Internet Name Domain (BIND). This allows address records (A records) to be duplicated for a specific host, with different IP addresses. The name server then alternatively rotates addresses for any one name that has multiple A records, and is known as DNS round robin.
As an example, your company has three web servers. Their real names and IP addresses are as follows:
www.yourcompany.com 128.1.1.1
www.yourcompany.com 128.1.1.2
www.yourcompany.com 128.1.1.3
You want to set up the servers so that DNS requests by clients (in this case, web server access) are round robin rotated. This is accomplished by placing multiple A records in the authoritative name server files.

For the above example, we want all clients to access our site by using www.yourcompany.com, but we want these requests to be shared between our three servers using DNS round robin. To do so, we need to place the following A records in the name server file:
www.Grid.com. IN A 192.168.1.1
www.Grid.com. IN A 192.168.1.2
www.Grid.com. IN A 192.168.1.2
Note the '.' after the name www.yourcompany.com on each A record. This is mandatory.
A Time To Live (TTL) field is often added to the A records. The TTL value is the maximum time that the information should be held to be reliable. By setting the TTL to a fairly small amount time e.g. 60 seconds, the effectiveness of the distribution can be maximized. A lower value may be specified, but this causes more DNS traffic in updates, which improves the load sharing on the web servers at the expense of increasing the local on the name server.
www.Grid.com. 60 IN A 192.168.1.1
www.Grid.com. 60 IN A 192.168.1.1
www.Grid.com. 60 IN A 192.168.1.1
When a DNS request for an IP address is received, BIND returns one of the IP addresses and makes a note of it. The next request will then return the next IP address in the file and so on until the last one, after which BIND returns to the first address again.

I am able to use this Load Balancing successfully with Oracle Products like Oracle EBUS R12, Oracle IAS , OPA products

MicroSoft Active Directory Installation

Creating the Active Directory

After you have installed Windows Server 2003 on a stand-alone server, run the Active Directory Wizard to create the new Active Directory forest or domain, and then convert the Windows Server 2003 computer into the first domain controller in the forest. To convert a Windows Server 2003 computer into the first domain controller in the forest, follow these steps:
1. Insert the Windows Server 2003 CD-ROM into your computer's CD-ROM or DVD-ROM drive.
2. Click Start, click Run, and then type dcpromo.
3. Click OK to start the Active Directory Installation Wizard, and then click Next.
4. Click Domain controller for a new domain, and then click Next.
5. Click Domain in a new forest, and then click Next.
6. Specify the full DNS name for the new domain. Note that because this procedure is for a laboratory environment and you are not integrating this environment into your existing DNS infrastructure, you can use something generic, such as mycompany.local, for this setting. Click Next.
7. Accept the default domain NetBIOS name (this is "mycompany" if you used the suggestion in step 6). Click Next.
8. Set the database and log file location to the default setting of the c:\winnt\ntds folder, and then click Next.
9. Set the Sysvol folder location to the default setting of the c:\winnt\sysvol folder, and then click Next.
10. Click Install and configure the DNS server on this computer, and then click Next.
11. Click Permissions compatible only with Windows 2000 or Windows Server 2003 servers or operating systems, and then click Next.
12. Because this is a laboratory environment, leave the password for the Directory Services Restore Mode Administrator blank. Note that in a full production environment, this password is set by using a secure password format. Click Next.
13. Review and confirm the options that you selected, and then click Next.
14. The installation of Active Directory proceeds. Note that this operation may take several minutes.
15. When you are prompted, restart the computer. After the computer restarts, confirm that the Domain Name System (DNS) service location records for the new domain controller have been created. To confirm that the DNS service location records have been created, follow these steps:
a. Click Start, point to Administrative Tools, and then click DNS to start the DNS Administrator Console.
b. Expand the server name, expand Forward Lookup Zones, and then expand the domain.
c. Verify that the _msdcs, _sites, _tcp, and _udp folders are present. These folders and the service location records they contain are critical to Active Directory and Windows Server 2003 operations.

Adding Users and Computers to the Active Directory Domain
After the new Active Directory domain is established, create a user account in that domain to use as an administrative account. When that user is added to the appropriate security groups, use that account to add computers to the domain.
1. To create a new user, follow these steps:
a. Click Start, point to Administrative Tools, and then click Active Directory Users and Computers to start the Active Directory Users and Computers console.
b. Click the domain name that you created, and then expand the contents.
c. Right-click Users, point to New, and then click User.
d. Type the first name, last name, and user logon name of the new user, and then click Next.
e. Type a new password, confirm the password, and then click to select one of the following check boxes:
 Users must change password at next logon (recommended for most users)
 User cannot change password
 Password never expires
 Account is disabled
Click Next.
f. Review the information that you provided, and if everything is correct, click Finish.
2. After you create the new user, give this user account membership in a group that permits that user to perform administrative tasks. Because this is a laboratory environment that you are in control of, you can give this user account full administrative access by making it a member of the Schema, Enterprise, and Domain administrators groups. To add the account to the Schema, Enterprise, and Domain administrators groups, follow these steps:
. On the Active Directory Users and Computers console, right-click the new account that you created, and then click Properties.
a. Click the Member Of tab, and then click Add.
b. In the Select Groups dialog box, specify a group, and then click OK to add the groups that you want to the list.
c. Repeat the selection process for each group in which the user needs account membership.
d. Click OK to finish.
3. The final step in this process is to add a member server to the domain. This process also applies to workstations. To add a computer to the domain, follow these steps:
. Log on to the computer that you want to add to the domain.
a. Right-click My Computer, and then click Properties.
b. Click the Computer Name tab, and then click Change.
c. In the Computer Name Changes dialog box, click Domain under Member Of, and then type the domain name. Click OK.
d. When you are prompted, type the user name and password of the account that you previously created, and then click OK.

A message that welcomes you to the domain is generated.
e. Click OK to return to the Computer Name tab, and then click OK to finish.
f. Restart the computer if you are prompted to do so.