Edit: It is still best to choose oil over gas when given the choice.
Originally posted November 29, 2011 on AIXchange
When I was 16, I got my driver’s license. One summer I had the opportunity to live in another state. However, my primary vehicle, a 1972 VW Bug, had to stay put. Still, I figured I needed wheels for that summer, so, after a year of busing tables at a Sizzler Steak House (and riding my bike to work), I saved enough to shell out $200 on a used station wagon.
It’s easy to understand why teenaged me made that choice. Speed and power. I seem to remember being told that the wagon had a “Buick 454.” I knew just enough about cars to understand that it had eight cylinders and that no one would be able to catch me in it. My friends’ sports cars looked better, but they couldn’t touch my beat up old station wagon. I never lost a race. (Of course it helped that none of my friends had a REAL sports car, but that old wagon was pretty fast.)
Still, as you might imagine, that purchase ended up being a very costly decision. First, I had to take it through state vehicle inspection, where issues with the muffler and exhaust system were found. I paid for the needed repairs (which, if memory serves me, cost at least as much as I’d spent on the wagon itself) and passed inspection. I thought I was ready to bask in my personal Summer of George, but my vehicle issues were only beginning.
The wagon burned oil–as much as a quart or two for every tank of gas I’d consume. And, if you weren’t already thinking it, the gas mileage on this thing was miniscule. So every few days that summer, I was buying oil and gas. I figured I had enough cash on hand to coast through those two months, but before long, I was almost tapped.
Eventually, I only had enough cash on hand for gas or oil. One, not both. As a result, my bad decision to get the wagon in the first place would be compounded by an even worse decision. I chose gas over oil. In no time at all that wagon ended up stalled on the side of the road, having thrown a rod. Then it was off to the junkyard.
Oh, and I made one other bad decision. Before my wagon was junk, I could have sold it. Someone was pestering me to buy it, and my plan was to unload it on him at summer’s end. Needless to say, that plan blew up with the motor.
I did at least learn from my mistakes. (For starters, it’s oil over gas. Always.) I guess that’s the important thing. If anything, it’s even more important now. Mistakes surely happen in IT, some even more costly than blowing an engine. Sometimes a backup doesn’t get made, and a system can’t be restored. Sometimes a backup is made but not tested, and thus doesn’t work when it’s needed. Sometimes shutdown –Fr gets run on the production box rather than the test box you thought you were logged into. Sometimes a VIO server is misconfigured and the production network goes down. Sometimes an rm command is run in the wrong directory.
These experiences lead to outages, and probably a few lost jobs. But the culprits undoubtedly learned from their mistakes.
What mistakes in IT have you made and, hopefully, learned from? I love hearing others’ IT horror stories (in part, admittedly, because they didn’t happen to me). So if you’ll indulge me, please share your learning experiences in Comments. I’m sure other readers will appreciate your stories as much as I would. And maybe by sharing you’ll keep someone else from making the same error.