So I had a failure – Tuesday night last week, which caused me (forced me, really) to write this post:
Now in the grand scheme of things, it was probably a bit snarky, but at 4am I think I legally am not responsible for my actions.
But the bottom line is on Tuesday at about 5pm I called to open a hardware case. Hardware. The high-school kid who answers the phone routes me into the Software group. Correct me if I’m wrong, but that’s kindof the opposite of hardware. (And no, if this kid was a college graduate, please god tell me what college he graduated from so i can forbid my son from going there)
2 hour wait for the call-back, 2 hours of trading email back and forth before I convince him this is a hardware case and to please route it to the hardware people.
He never does the transfer.
In the meantime, it’s about 2am and I go fix the bloody problem myself, restart my change scripts and all is happy. I got home about 4 that morning after an 18 hour day.
Leap forward to Thursday night. Same process, different array, same failure. Now it’s 10pm and I call in and STRESS to the triage guy that this is a HARDWARE case. He routes me to hardware. 75 minutes to call-back, 2 hours to fix the problem and step the script through to the end, plus I get handy knowledge like root-cause analysis and a set of steps to ensure it doesn’t happen again.
My point is this. So many problem can be avoided if you simply LISTEN to the person who is calling in. Assess their skill level and if someone asks to be transferred to a specific group (ESPECIALLY if he knows the actual name of the group he wants to be transferred to – means he’s done this before)
Long and the short of it was on Tuesday, a failed drive on the target array was failed when the script started. This caused it to error out because it saw invalid tracks existing between the mirrors.
Same thing happened on Thursday, different drive, different symm, and out of 4,000 volumes it happened to hit a volume I was working with both times.
I should play the lottery… You’ll know if I win too because my blackberry doesn’t work on the beach in Cozumel. 😉