• He makes pie charts.
• He is a follower.
• He follows Philip around.
• He is good at suit speak.
• He tells Philip everything.
• He bonds with Philip.
• He always agrees with Philip.
• He never overtly commits to any course of action.
• He quickly scans Philip’s body language to know if he should support or undermine an idea.
• He is very secretive about his plans, actions, and motives.

I left Wollongong with the illusion that management was about seeing the big picture and making bold decisions that defined the direction of the business.  I thought upper management communicated these priorities to middle management who in turn made tactical decisions to reinforce them.  And my role as a technician was to carry out my managers plans as best I could without second guessing.  After moving to Melbourne I realised that that was utter rubbish.

In reality, upper management communicated vague generalities which they crafted so as to be deniable.  Middle management communicated back to them how successful their own tactical decisions always were.    My managers were all cowards.  That is precisely how they survived in their jobs:  Agrandise your own accomplishments, belittle your  team,  self-preservation at all costs.  Painstakingly choose the path of least risk and most conformity.   If an underling has to be thrown under a bus, so be it.

There was no longer a plain old manager in my division.  Mine wasn’t the only department missing a manager. Apparently there had been a mad scramble to grab the new titles.

We all knew a pay freeze and hiring freeze had been mandated from HQ but they neglected to mandate a title change embargo also.  I argued that some of those title changes were coming anyway.  She looked at me pitifully and quietly told me that the payroll was now 18% higher than before the financial meltdown.  The entire increase was going to those crafty ex-managers.  They had vacated their old positions so hastily they did not give a thought to their successor.  There was a power vacuum in the middle zone.

There is a history lesson here. This is how that famous gap widens – between those that know how and those that know why. You would think everyone could have stepped up a peg but that would not have worked. Everyone would have had a pay increase. Someone somewhere would smell a rat. On the other hand,  half the managers does not amount to a visible number on paper, even though the actual cost is a 18% increase in payroll.

The situation is that there is no one ready to step into the vacant manager roles. No one was being “groomed for management.”  The policy has been to keep everyone not “in the club” at arms length from any  decision-making.  The club has done very well for itself.  I wondered if this title hopping had occurred throughout the company so that those watching the budget were also in the club and had helped themselves to a new title.

Had the executives that mandated the hiring and wage freezes purposely built in  the title-change loop-hole for themselves?

All our apps, databases and services have a planned downtime for off-line backup and other maintenance. Some services are failed over to another server (the Web never sleeps) and some aren’t (our in-house staff goes home and sleeps.) All maintenance showed up as failures.  A meeting was called to confront everyone responsible for the servers.

Item one:  Our in-house user interface was down from 3AM to 4:08AM. How could I account for this? I knew where to look:   in the crontab for that server.   I was stopping the user interface every night while the database was checked for integrity and snapshotted.  Everyone in the room had a similar story to tell:  Services were being stopped while snapshots were taken and other maintenance that required a quiet system.  As far as the public was concerned there was no outage.

The meeting was dismissed. We all left feeling like the well oiled machine of our infrastructure had survived the scrutiny and something would be done about the uptime report-making.  That is not what happened. I got an email from Philip an hour later:  “Please move the user interface outage to 3 hours earlier.”

It must be noted that all automated system administration  tasks are carefully planned in coordination with other systems that may be affected.   I realized no one had been consulted about possible conflicts. Probably no one would be in the office at midnight when Philip wanted this operation  to begin but the database would still be running as well as many other cron tasks throughout the organization (why am I even pretending this can be justified.)

Up to now the various crontabs have been executing in harmony and trouble-free. We are being asked to hastily move things around for the purpose of this pie chart.

Sure enough, when I went into Philip’s office to discuss this, there was Cecil projecting the pie chart of outages on the wall. One of the database administrators was already there and he was looking peeved. He had an email in his hand which he turned so I could read it. I recognised the short request with an AM after the time. That was all I needed to see. He was also being asked to shift his cron jobs around.

Philip was ready to explain his actions.  “If we move all these slices” he wiggled his laser pointer across the 1AM – 5AM sector “over to here,” lots of wiggling in the 0AM – 1AM sector “then all our downtime will be condensed into one hour.”

This was not satisfactory to me or the database admin. The only way this could happen is taking all servers off-line (including fail-overs) from 0AM – 1AM and that is when our American cousins are doing all their ecommerce with us (midnight in Melbourne is 10AM in New York, 3PM in London). We all pleaded to take no action until all admins could meet. We have to drag this out into meeting after meeting. It is our only hope.

I alone understand that Philip is fighting for his corporate life here. The justification for his existence comes from making changes and reporting that he was responsible for them. He wanted to meet with his boss, show the old and new pie charts side by side. His boss would be very impressed. Only the cussed sysadmins were in the way of this dynamic leveraging of  synergy.

We are still resisting but I am considering the downside of caving in to Philip’s plan and letting hell break loose with failed backups and corrupted snapshots and angry customers but that would create a lot of cleanup for all the sysadmins (a couple of us might get fired as scape-goats.) Also Philip would also heroically come to the rescue. Revert to the original regimen, and report “mission accomplished” to his boss yet again while the sysadmins struggle to hold the infrastructure together with their bare hands.

Every course of action is a lose-lose for the sysadmins and a win-win for Philip.  Cecil has started tagging along with Philip everywhere.  This is a bad synergy.

He will complain about me to my manager.  It is another example of privileged individuals who are not required to use the helpdesk.  This would have been far better handled if the helpdesk had intercepted this call and tried to follow the guidelines set out by Simon Tatham here.

In my first IT job in  Melbourne the helpdesk was taken very seriously. If a helpdesk ticket was generated you would have to respond within 15 minutes or you were in trouble. You had to talk to the user either in person or over the phone and tell them you were aware of the issue and would soon have it resolved. At this place helpdesk tickets are often ignored. We all know that no one will ever follow up. The users sense this and avoid using the helpdesk and prefer to stalk the technicians.

The people who have no clout can be made to call the helpdesk.  We just ignore them otherwise.  We will probably ignore them even after they have put in a ticket.  A lowly mail room clerk approached me to fix his printer.  He had a PC solely for the purpose of logging shipments.  His printer had not worked for 10 days.  I cleared the jam, cleared his print queue, restarted his spooler, then I went over to the helpdesk office to let them know to close the ticket.  They let me know there was no ticket.  “We don’t help them.” was the explanation.  There was some feud or disrespect that had not been settled.   How long can a company go on like this?

These events are connected by a clear failure of management.  Where I worked before we had a CIO and that is why priorities were established and rigidly enforced.  Philip is not going to enforce anything. That is akin to making waves and the successful manager knows to keep a low profile in all matters other than reporting successes. Here, CIO is an empty title that has no rank and can enforce no policies. It was bestowed on Philip to justify a rise in pay.

Barry

Melbourne

The company blossomed to 11 employees: salesmen, advertising, accounting, other office staff, but no IT people. I was being called in more and more frequently. The owner said there would be a full time job for me if I was willing to work on the office PCs. I thought “Why not?” After all Windows and the desktop PC were the things of the future.  The owner then brought in Stephan Medeiros (who is no longer in IT) to maintain the office PCs.  Stepho claimed to be an MCSE on his application but it turns out he wasn’t; he was “planning on it” so he put it down anyway.  If I hadn’t been wasting so much  time in the motor trade I would have just taken the job.  So much for being a free spirit.

I had earlier informed the owner that to have any security they would need at least Windows NT on all PCs with a domain controller.  This was cheaper than the Novell offering at the time.  The software was purchased and was waiting to be installed.   Stepho bungled it. I stepped in so that people could get some work done.

I told Stepho to install the domain controller and create 3 domain admins (him, me and the owner) and 4 domain groups (accounting, sales, advertising, IT) -  the preferred Microsoft set-up at that time.  Then do all the PCs in turn and put everyone in their group.  When I came back he had made himself  the only domain admin and everyone else a local administrator and no PCs were on the domain. At the same time he had been convincing the owner that I was an idiot and that knowing UNIX was nothing like setting up a real computer.

I turned to the owner, and told him this just wasn’t going to work.  He seemed satisfied with the setup, everyone was doing their job and it was just like before.  I told him that, in that case,  purchasing NT server for the domain controller was a huge waste of money. They should not have bothered switching over from Windows95.  Out of the corner of my eye I saw Stepho in the doorway gesticulating.  He was communicating with the owner and did not think I would see him.  I turned while he was rolling his eyes, shaking his head and waving his hands in a “don’t believe him” gesture. I said “This is your open door policy,  is it?”  I didn’t care one way or the other I had been paid for my work.  I found out later that people were sharing files by printing things out.  Also, in a panic,  the accounting PCs had been disconnected from the network.  For some reason the firewall was not protecting the office network and they suspected someone had hacked into one of them from the internet.  Later I commented to the owner “You should have disconnected all of them, solved the firewall problem, then hooked all the PCs back up.”

“Well Stepho said it was impossible to test and this was safe enough.”  It was only impossible if you didn’t know how to set up a firewall and set up a UNIX machine to monitor the traffic.  Stepho did not know any UNIX.  Who knows how much data had been sapped out of these PCs.  The one thing that endured unscathed in that office was the email system I had set up on Solaris for all the office users.  They were accessing it through Netscape.  When I found out about the difficulty with file sharing I told them to email documents to each other as a stop-gap measure until they got an admin who new what he was doing.

The lesson here is that Stepho lasted surprisingly long at that company, longer than I did.  He virtually managed the office infrastructure, however incompetently and destructively,  extraordinary as it sounds.  He had what it took to survive as a manager.  He had masqueraded as a tech and failed, then masqueraded as a manager and was able to endure in that role.  I am working on a theory of why this is so easy to do.

Barry