A (sort of) Retrospective of 2021

It is hard to really know where to start this, there have been so many significant changes in the last 5 month or so. Although there are obvious turning points, it is hard to know which one makes a good place to start for this particular chapter of my life. Looking back now it all seems so jumbled, even though I lived through all the events it is hard to believe that I have somehow ended up a System Admin with so little work experience. Sometimes I feel like I am on some sort of weird busman’s holiday and I will soon return to my old life and the endless treadmill of planning and marking, photocopying and laminating. Other days it feels like things have been this way forever and teaching was some sort of half remembered previous life. I suppose the only way to start is at the beginning, or at least some arbitrarily defined point that I will call the beginning.

I suppose, like so many things, the real beginning for me is about 6 months ago. The moment when it finally dawned on me. Not that I was no going to be an IT professional, albeit at the lowest level, but that I was no longer going to be a teacher. I watched the class I had been working with every day for the last year, through self isolation’s and lockdowns, line up to be dismissed for the final time. As they ran off to meet their parents, bags brimming with the last terms work, excited at the prospect of 6 whole weeks away from school, I was suddenly no longer a teacher. At least not in any sense that counted. I was still qualified, I hadn’t been barred for some inappropriate activity. I had chosen to walk away from the classroom. So it seemed as they ran off to meet their parents, they took with them part of me. A big part of me. Probably the only constant in my life for the last decade. I been married, divorced, moved countries 3 times, made a lost friends traveled more than would have ever though possible and in some way if not found then redefined myself. But through all of that I had always been a teacher. And now, I had to learn to be something else.

I suppose then, the logical start to this story would be the most pedestrian of events. The receipt of an email. The read something along the lines of, “you had previously expressed an interest in a vacancy at the college as an IT technician, the post has become available again and we wondered if you were still interested?”

Almost a year previously, when I had first decided that I wanted to move out of the classroom my partner had told me, too late to actually apply for it, that they were looking to take on a junior IT technician at her college to support the network manager. Although I had missed the deadline I had contacted them anyway and been informed that they had just hired an excellent candidate but that they would keep me in mind should anything come up in the future. The excellent candidate had decided to move onto bigger and better things and so something had indeed come up. I replied to say I was interested and after some back and forth I found myself sitting in an interview room, answering questions about malware attacks and disaster recovery before having to complete a technical assessment which essentially involved putting batteries in a remote control and plugging a network cable back in. It was to say the least, a disjointed recruitment process but I was offered and accepted the job.

All of this occurred approximately a week before the teaching year ended and I would have my playground epiphany about the role of my career in my identity.

For a month I lived the life of a junior IT technician. I split my time between checking things were plugged in, turning them off and on again and helping people to setup video lessons for students who were stuck at home self isolating. I spent as much time as possible out of the server room roaming round the college because the server room was not really fit for human occupation.

Everything was filthy, the cleaner would periodically knock on the door and beg to be allowed in to dust the sides at least, a request that was consistently refused on the grounds that if the room was too clean people would think that we weren’t doing any work. It was kept at approximately the same temperature as a sauna through a complex arrangement of using a massively over spec’d air conditioner to cool the server whilst using a hideously inefficient fan heater to warm the staff. And my line manager enjoyed spending most of his days ignoring the needs of the failing IT system whilst expounding on a number of conspiracy theories and the virtues of a diet that consisted entirely of cucumbers. On the last part I would have to conceded he looked remarkable for his age.

It was a week before I found myself midst of a crisis, a common refrain for the early days of working at the college. The hard disk array that ran all the virtual machines, which in turn ran all the IT services in the college, ran out of space. The servers immediately crashed and I was left running round the college whilst the network manager calmly munched on a cucumber and prodded and fiddled the various partitions to free up some space. This was an event that would repeat itself several times over the coming days until some sort of equilibrium was reached and the server scatted by with the bare minimum of free space. One really large copy and paste away from resuming their inactive state.

It is worth noting at this point that additional hard disk space was purchased but it couldn’t be used a fact I would only discover much later and that would cause my immense amounts of stress.

In this way days became weeks became month as I split my time between clearing paper jams and bearing the brunt of people’s temper when once again even the most basic IT functionality was lost. Until one day I was returning from a separate building on the campus having un-jammed the printer for the umpteenth time when I as grabbed on my way through reception, herded into the principals office and asked to wait. After 20 minutes of being sat on my own with no clue what was going on I had managed to convince myself that I was about to be fired, or possibly arrested, for what I wasn’t quite sure but it seemed the only reasonable explanation in the circumstances. Little did I know my fate was both far less dramatic and far worse.

In due course several members of the senior management team came in and told me in no uncertain terms that the current network manager was being let go and they wanted to keep me out of the way while that played out. I was then given a very nice sandwich and taken up to the server room by a visiting consultant who upgraded me to domain admin and gave me a crash course in enterprise level administration. The remainder of the day is a bit of a blur to be perfectly honest, based on this fact more than memory I imagine that I got hideously drunk (it was a Friday after all) whilst attempting to make sense of the events that had occurred.

And so, with precisely 31 days IT experience under my belt, I became the systems administrator for an IT system in such poor shape that… well I usually use the phrase that is it was a horse you would take it out and shoot it, But that isn’t even close to how bad it was, if it was a horse you would have made it comfortable for the night at let nature takes its course. A quiet death out of respect for so many years of faithful service. Letting it pass away naturally would certainly have been the most humane thing for everyone involved.

The next few months passed by in an endless cycle of meetings trying to squeeze some money out of the owner to spend on the IT, who simply couldn’t understand why people couldn’t get by with PCs that where years past end of life, attempting to bring everything up to date and make it stable when I didn’thave the first clue what most of it did and deal with bi-weekly catastrophes which would render significant portions of the infrastructure unusable. Whilst still trying to keep the printers un-jammed and help people setup video conferencing for the students stuck at home. Mostly it involved Googling, a lot of googling.

I won’t bore you with every event that I went through save a couple that stick out in my mind as the most note worthy. The first was when the entire network crashed because we accidentally unplugged the external hard disk which turned out to be a critical piece of infrastructure. The other was when I had to reprogram a switch for the first time.

The hard disk incident occurred, as these things always seem to around 3 o’clock on a Friday. We had been doing some investigation of the virtual machines that run the critical services of the college, specifically with a view to locating several hundred gigabytes of data which has some how disappeared from the file server suing one of out previous outages. After much head scratching, confused pointing and coffee drinking we came to the conclusion that we could make neither head nor tale of how things had been setup and where utterly convinced, despite evidence to the contrary that nothing was actually working

Although we didn’t dare say this too loud in case the servers, which were somehow functioning heard us and decided to give up the ghost. It is probably worth noting at this point that when I say we, I meant an IT consultant and myself and this early on, we meant he did the technical stuff and I made coffee and asked a million questions. Usually starting by saying “this might be a stupid question, but….”

For reasons that are still unclear to both of us Harpreet decided to investigate behind the server, which is where he discovered an external hard disk plugged into the back of the host controller. Following some further discussion and investigation we discovered that the hard disk was the repository of all the missing data. Excellent. Having made this discovery we decided that it would be wise to unplug the hard disk and store it safely until such time as we were able to migrate the files back to the server properly and make them available to the people who were rather upset about having lost them in the first place. Shortly after this things began spontaneously failing. People who logged out where unable to log back in, devices regain to loose connectivity. Back to crisis mode.

I won’t bore you with full details but after extensive investigation we discovered that the external hard disk not only contained a significant portion of files form the file server. It also contained the virtual hard disk for the domain controller. The primary domain controller was running from a mechanical, USB 2 drive. This made me rather nervous, it still does in fact because there is insufficient space on the SAN to migrate the virtual hard disk back. You see it turns out the additional hard disks are useless. Although that are compatible with SAN hardware, it has been so long since the SAN was updated, 9 years at the time of writing, that the firmware is incompatible.

The other significant event was the night of the great switch reprogramming. I had previously zero experience of programming managed switches and I found myself in the unfortunate situation of having to do it one evening after college had closed. This would not have been suck a major issue, had the previous network manager not kept all the passwords in his head and they had not been successfully extracted before his departure. So instead of simply being able to carry out the necessary tasks on the switch, I instead had to setup it up from scratch after a full factory reset. Picture if you will me sat at my computer with a copy of the switches manual and the CompTIA Network+ study guide feeling my way through programming a switch by trial and error. The next day I stumbled across an article with a title like ‘Programming Switches for People who don’t Understand VLANs’ which would have saved me a great deal of time had I found it the day before.

So that is more or less my journey so far. Lurching from one crisis to the next whilst trying to learn the ropes of being a system admin. I wouldn’t say it has been fun, but it certainly hasn’t been boring.

Posted by pgwinkley