I was whiteboarding a switch migration today with one of the DCR people, and it occurred to me:
If you can visualize the data’s path through the system, your life gets a lot easier.Â
If you can see the path the data is going to take through the San, I.E. from Switch1, Blade1, Port4 through an ISL to Switch2, to Switch 2, blade 2, port 16. you put yourself in a better position to shorten that path.
First, cut the ISL’s. Unless you can absolutly avoid it, don’t pass data down an ISL link. Pushing data through an ISL does two things. It creates a bottleneck where there may not be enough bandwidth to handle multiple hosts.
Second, keep the intra-switch hops to a minimum. If you can do it, plug the storage and the higest performing hosts into the same grouping of ports. Most switches use 4 port ASIC’s, and if you look at the switch you’ll see that the ports are grouped by ASIC.
However, the 32port Cisco blade is an exception, uses basically the same ASIC, but shares the bandwidth over 8 ports, so keeping that in mind it’s best to connect the lower performing hosts (Windows?) to those ports, and definately *DO NOT* connect ISL’s to the 32 port blades.
2 comments
Well, since a google search for something else brought me here, let me add another bit of wisdom for those considering MDS. If you are running blades (like IBM BladeCenter, for example), with embedded switches, and you need to ISL to your MDS core, you’ll have to fully subscribe those E ports on the MDS. That means you could render more than a few ports entirely worthless. If this sounds like an environment you’re considering, get the 12 port, fully subscribed blade for your MDS to handle your ISLs. Otherwise, you’re paying for ports you just can’t use.
Author
Yeah, we run(ran) the same deal on the switches at LTL, we ran 1 16-port blade and 1 32-port blade. By using the 16-port blades exclusively for storage connections and the 32-port blades exclusively for host connections, you mitigate the over-subscription issues. Since no single host (especially in a Windoze environment) can push the storage port to anything near the limit, you are never in danger of bottlenecking your traffic.
You mention the 12-port blades in your comment, but I thought the 16 port blades were fully subscribed as well? Am i wrong about this?