My manager, Philip, hired a contractor to set up a RAID array. I pointed out to him we had several such arrays already set up and that it was routine. He was nonplused but went ahead with the contractor anyway. He stated something like “We have to consider the relative value of our data…” He had obviously been reading Tech for Suits again and was impressed with an article on outsourcing. When the contractor arrived I gave him access to a UNIX server to work from. He noticed I had many non-standard scripts for various file system tasks, some in bash, some in perl, some in python. He was fascinated and emailed them all to himself. He proceeded to set up the array with frequent consultations of on-line documentation as well as man pages and a few phone calls. Really not so different from how I usually handled an unfamiliar task. So much for that.
Some time later a situation arose that I could not handle in a timely manner: A new server was giving memory errors. I told Philip we should bring in the vendor. We were having frequent slowdowns and core dumps. I realized there was a lot of work involved in pinning it down (especially not having parts to swap out) and we should set the vendor on it who had all spare parts and diagnostics ready and we were under warranty so there was no cost to us.
Philip determined that a contractor was out of the question and he couldn’t understand why we could not handle this ourselves.
I set about handling it. I analyzed all the logs and the core dumps. I Looked for hardware issues. I patched the system and the firmware.
I realised that half the RAM was being taken offline shortly after a reboot. I took out half the RAM. The problem went away (the defective DIMM was in the half I took out. That was just dumb luck.) The vendor will not replace the RAM without their man diagnosing it. So I have to replace all the RAM and it’s back to the vendor option. In any case, much time was wasted by disregarding my first diagnosis. I am blamed for not realising this sooner. Philip often hints that this outage is somehow my failure. And so this is how time is wasted and reputations tarnished. I almost suspect that the vendor option was rejected out of hand because it was my suggestion. I really don’t want to believe that.
There is no real lesson here. There is nothing I could have done differently, other than having a screaming fit. I could say that in the ideal scenario, where technicians made the technical decisions, the matter would have been expedited quickly and silently. In this case the manager made the wrong technical decision but blamed the technician for the outcome anyway. It is a small step from that to actually delegating technical decisions to the technician and then being fully justified in blaming him for the outcome.
Outsourcing
My manager, Philip, hired a contractor to set up a RAID array. I pointed out to him we had several such arrays already set up and that it was routine. He was nonplused but went ahead with the contractor anyway. He stated something like “We have to consider the relative value of our data…” He had obviously been reading Tech for Suits again and was impressed with an article on outsourcing. When the contractor arrived I gave him access to a UNIX server to work from. He noticed I had many non-standard scripts for various file system tasks, some in bash, some in perl, some in python. He was fascinated and emailed them all to himself. He proceeded to set up the array with frequent consultations of on-line documentation as well as man pages and a few phone calls. Really not so different from how I usually handled an unfamiliar task. So much for that.
Some time later a situation arose that I could not handle in a timely manner: A new server was giving memory errors. I told Philip we should bring in the vendor. We were having frequent slowdowns and core dumps. I realized there was a lot of work involved in pinning it down (especially not having parts to swap out) and we should set the vendor on it who had all spare parts and diagnostics ready and we were under warranty so there was no cost to us.
Philip determined that a contractor was out of the question and he couldn’t understand why we could not handle this ourselves.
I set about handling it. I analyzed all the logs and the core dumps. I Looked for hardware issues. I patched the system and the firmware.
I realised that half the RAM was being taken offline shortly after a reboot. I took out half the RAM. The problem went away (the defective DIMM was in the half I took out. That was just dumb luck.) The vendor will not replace the RAM without their man diagnosing it. So I have to replace all the RAM and it’s back to the vendor option. In any case, much time was wasted by disregarding my first diagnosis. I am blamed for not realising this sooner. Philip often hints that this outage is somehow my failure. And so this is how time is wasted and reputations tarnished. I almost suspect that the vendor option was rejected out of hand because it was my suggestion. I really don’t want to believe that.
There is no real lesson here. There is nothing I could have done differently, other than having a screaming fit. I could say that in the ideal scenario, where technicians made the technical decisions, the matter would have been expedited quickly and silently. In this case the manager made the wrong technical decision but blamed the technician for the outcome anyway. It is a small step from that to actually delegating technical decisions to the technician and then being fully justified in blaming him for the outcome.
Barry