Below is the output of a test case for Fast Connection Failover. Maybe it is just me, but I can never seem to find "soup to nuts" examples of this type of thing.

The runOracleConnMgrFailover java class does the following (let me know if you want the source code posted):

1. Create a pool of 30 cached connections (OracleDataSource), with FCF enabled. These are connected to a two node RAC, linux1 and linux4.
   
2. Create a threaded onsSubscriber objects that will post to standard output all events received from ONS.
3. Sleep for five minutes, during which time we bounce one of the instances.
1. At this time, we should have the connections equally distributed (I verified this but didn't show it) between linux1 and linux4.
4. After the five minutes, 20 connections are created from the pool, and the instance on which they live is shown.

I chose five minutes because I wrote a simple socket server that got messages from ONS on the RAC. It appears as though the ONS software will post clients (the onsSubscriber class Oracle provides for testing ignores these messages) every two minutes (actually slightly less, about 1:58).

This gave me time to shutdown one of the instances in under two minutes and see if the cached connections reconnected to the surviving instance (as Oracle claims will happen).

While testing this, it occurred to me that ONS posts the clients when an instance dies, while not immediately, close to it!

As such, I changed five minutes to 30 seconds, and I found the connections were all on the surviving node :)

Note that the session were "stateless", in other words, they weren't doing anything when failed over. As a matter of fact, they were not "owned" by any "application" connection request until I began asking for 20 connections five minutes later.


linux3:oracle:dummy:/home/oracle>java runOracleConnMgrFailover 20
created connection manager, you can shutdown...
ONC subscriber starting
** HA event received -- Printing header:
Notification Type: database/event/service
Affected Components: null
Affected Nodes: null
Delivery Time: 1203528172772
Generating Component: database/rac/service
Generating Node: linux1
Generating Process: linux1:16358
Notification ID: linux1:1635803
Notification Creation Time:1203524376
Cluster ID: databaseClusterId
Cluster Name: databaseClusterName
Instance ID: databaseInstanceId
Instance Name: databaseInstanceName
Local Only Flag: FALSE
Cluster Only Flag: FALSE
Body: [B@10da5eb
** Body length = 97
** Event type = database/event/service
Sleep and retry.
ONC subscriber starting
ONC subscriber starting
** HA event received -- Printing header:
Notification Type: database/event/service
Affected Components: null
Affected Nodes: null
Delivery Time: 1203528174069
Generating Component: database/rac/service
Generating Node: linux1
Generating Process: linux1:16358
Notification ID: linux1:1635804
Notification Creation Time:1203524378
Cluster ID: databaseClusterId
Cluster Name: databaseClusterName
Instance ID: databaseInstanceId
Instance Name: databaseInstanceName
Local Only Flag: FALSE
Cluster Only Flag: FALSE
Body: [B@e45076
** Body length = 103
** Event type = database/event/service
Sleep and retry.
ONC subscriber starting
ONC subscriber starting
** HA event received -- Printing header:
Notification Type: database/event/service
Affected Components: null
Affected Nodes: null
Delivery Time: 1203528175107
Generating Component: database/rac/service
Generating Node: linux1
Generating Process: linux1:30339
Notification ID: linux1:3033901
Notification Creation Time:1203524379
Cluster ID: databaseClusterId
Cluster Name: databaseClusterName
Instance ID: databaseInstanceId
Instance Name: databaseInstanceName
Local Only Flag: FALSE
Cluster Only Flag: FALSE
Body: [B@15e9756
** Body length = 113
** Event type = database/event/service
Sleep and retry.
ONC subscriber starting
SID 147 started on linux4
SID 135 started on linux4
SID 152 started on linux4
SID 126 started on linux4
SID 120 started on linux4
SID 115 started on linux4
SID 133 started on linux4
SID 149 started on linux4
SID 128 started on linux4
SID 118 started on linux4
SID 122 started on linux4
SID 148 started on linux4
SID 124 started on linux4
SID 119 started on linux4
SID 132 started on linux4
SID 125 started on linux4
SID 131 started on linux4
SID 117 started on linux4
SID 121 started on linux4
SID 113 started on linux4