Impact of improper change of VIP or VIP hostname of an Oracle Cluster

In my last post (Need to change VIP or VIP hostname of an Oracle Cluster), I discussed how to change the VIP or VIP hostname of an Oracle cluster. I recently came across a situation in which the system administrators had changed the VIP hostname and updated the respective entries in listener.ora and tnsnames.ora. But, the next time the server was rebooted, we had all sorts of problems with the database. Listed below are some of the issues that one may come across:

1) DBCA fails at the stage where it checks for the VIPs of the nodes throwing the following error:
Virtual IP addresses not configured for cluster nodes.

2) If you try to run VIPCA and check or even re-configure the VIPs, you might get the following generic error:
PRKR-1062 : Failed to find configuration for node myrac1.mydomain.com.

Take an OCR dump to check the OCR configuration:
    $ $ORA_CRS_HOME/bin/ocrdump

Search for VIP configuration in the resultant dump file:

.
.
.
[DATABASE.NODEAPPS.myrac1.VIP]
ORATEXT : myrac1_virtual_old.mydomain.com
SECURITY : {USER_PERMISSION : PROCR_ALL_ACCESS, GROUP_PERMISSION : PROCR_ALL_ACCESS, OTHER_PERMISSION : PROCR_READ, USER_NAME : root, GROUP_NAME : root}
.
.
.

Aha!! That's the problem there. The OCR is still configured with the old VIP hostname.

A logical conclusion would be to modify the nodeapps to point to the new VIP hostname:
    $ srvctl modify nodeaps -n myrac1 -o $ORA_CRS_HOME -A myrac1_virtual_new/255.255.255.0/eth0

PS: Take an OCR export before modifying anything. Better safe than sorry!!
    $ $ORA_CRS_HOME/bin/ocrconfig -export ocrexport.dmp

If this doesn't work as well, you might have to drop the nodeapps and re-add them:
1) First, make sure that you have the latest OCR dump

2) Stop all the cluster databases, ASM instances and nodeapps for all the nodes:

    $ srvctl stop database -d mydb

    $ srvctl stop asm -n myrac1

    $ srvctl stop asm -n myrac2

    $ srvctl stop nodeapps -n myrac1

    $ srvctl stop nodeapps -n myrac2

3) Verify that all the components are offline:

    $ crs_stat -t

4) Using NETCA, delete the listeners registered with all the nodes

5) Remove nodeapps from all the nodes:

    $ srvctl remove nodeapps -n myrac1

    $ srvctl remove nodeapps -n myrac2

PS: Use -f option to forcibly remove the nodeapps

6) Verify that the nodeapps are no longer listed by CRS:

    $ crs_stat -t

7) Verify the CRS parameter file points to the correct VIP hostname:

    $ $ORA_CRS_HOME/install/params.crs

8) Restart CRS for parameter changes to take effect.

    $ crsctl stop crs

    $ crsctl start crs

9) Execute VIPCA to create new nodeapps for all the nodes

10) Verify that the nodeapps are listed through crs_stat and that all the services are ONLINE:
  
    $ crs_stat -t