SRDF woes….

Ok, for about 4 months I’ve had an SRDF problem that has been kicking my ass. 

SRDF over Ethernet presents a host of new problems, unique to Ethernet. 

First, and mostly – the RE adapters want to see *ALL* of the remote adapters in the RDF group.  That means if you have four RDF/Ethernet (RE) boards in the source box and four RE boards in the target, the way the Symm does the SRDF is that it expects EACH source RE to see *ALL* target RE’s.  It allows for a nice load balancing scenario, but doesn’t do much to allow for either dual paths or direct attach connections.

The second problem I found was an undocumented feature of Solutions Enabler 6.3.1 which caused problems querying SRDF devices, and more importantly, problems running any configuration change script against an RDF device.

For instance – the config change script as follows, took about 45 minutes before it timed out and died without error.

convert dev 0db to 2-way-mir ;

Now some will recognize this as a simple “de-configuration” of an SRDF volume.  But because the two symms have to be correctly communicating with each other for this to work, the script simply failed.

I went in to work this morning determined to kick this thing once and for all.

First off I picked up a couple of 29xx series Cisco switches, configured them with ports 1-4 in a trunk leaving the 4 optical ports on each switch for the Symmetrix connections.  This allowed me to put all four connections on each Symm into one subnet.

Then I realized that that’s NOT in fact how the symm was configured – the CE left the symm configured (granted – per my suggestion) so that each RE was in a subnet with only the corresponding RE on the remote side.  My understanding being that if there was no gateway it would not try to contact the other (target) RE’s.  (bad assumption)

So I did the one thing neither Customer-me or Consultant-me should ever do.  I did a binfile change on the Symm without contacting EMC and putting it through any form of change control.  It’s not a difficult thing to do a binfile change, provided you know the passwords to get into Symmwin and have a basic understanding of how a Symm functions. 

It’s just frowned upon by EMC because they don’t like the idea that they are not the gods of the universe that they purport themselves to be.  (for binfile changes not related to initial setup they also expect to be engaged professionally, for approximately $5k per change – if I were smart, and had the time to spare, I’d start offering my services at half that price)

After the binfile change all lights as far as SRDF went were green, but my config change was still hanging.  A brief email exchange with one of the SAC’s better engineers (I’ve worked with him before) pointed me very quickly to the Solutions Enabler upgrade.

The 6.3.2 upgrade did the trick, and is now in the process of being pushed out to all servers.