Tips for Gaining Practical Systems Administrator Knowledge

Edit: Winning this contest opened doors for me.

When an individual seeks additional experience, the whole company may benefit as a result.

Originally posted April 2007 by IBM Systems Magazine

Note: As part of a collaboration between PowerAIX.org and IBM Systems Magazine, guest writers were invited to submit Tips & Techniques articles to be considered for publication. A panel decided Rob McNelly’s column, seen here, best met the contest’s criteria.

I work with an intern. He goes to school and comes to the office when he’s not studying for tests, working on homework or going to class. It’s fair to say that we subject him to some good-natured abuse. For example, we send him to the computer room to look up serial numbers or to verify that network cables are plugged in. When I ask him why he puts up with it, he tells me he’s grateful for the opportunity and will happily do anything in order to gain experience.

How else do entry-level people get their start in the industry? When I look at job listings I see plenty of opportunities for senior-level administrators with years of experience. I don’t see the same opportunities for novices that I once did. There seem to be fewer openings for people to start out on a help desk or in operations and then move their way up. It still happens, but many of those lower-skilled jobs are now being handled remotely from overseas.

Seek Practical Experience

Besides working as a paid intern, another method I’ve seen people use to gain practical experience is to get some older hardware from eBay and use that as their test lab. You don’t need the latest System p5* 595 server to learn how to get around the AIX* OS. An old machine might be slower, but it’s just fine to practice loading patches, getting used to working with the logical volume manager and learning the differences between AIX and other flavors of UNIX*. Mixing older RS/6000* machines with some older PCs running Linux* can give anyone a good understanding and exposure to UNIX without actually learning it on the job or spending a great deal of money.

People can also download Redbooks from IBM, and use those study guides to learn more. If they then get involved with a local user group, they can make connections with people who are usually willing to share their knowledge. Eventually, they have some basic knowledge, and can hope to land a position as a junior-level administrator.

This initiative to learn outside of work hours can prove invaluable. I know that if I interview someone who tells me he has little hands-on experience working in a large datacenter, but he’s shown that he’s ambitious enough to study and learn what he can on his own, I’m willing to take the chance that I can teach him the finer points of what he needs to know to do the job. Give me someone with a good attitude and a desire to learn, and he or she can usually be taught what’s necessary to be productive in my environment.

Senior-level administrators can give back by writing articles and answering questions. Personally, I’ve found some irc channels and some usenet groups that I respond to if I have time. If we want more people to learn about the benefits of using the AIX OS, then we should be willing to help them when they run into problems. Many people run Linux because it’s relatively easy to obtain and install. They know that they can go online and easily get help when they run into problems. That same type of community should be encouraged around the AIX world as well. The following are some tips that I’ve found helpful. Hopefully our intern finds them helpful as well.

Migrating Machines

When migrating machines to new hardware in the good old days, I would make a mksysb tape and take that tape over to the new server I was building. I would boot from the AIX CD, select that mksysb image and restore it to my new machine. As time goes on, I find it less common to see newer hardware equipped with tape drives. Much of my server cloning these days occurs using the network. Two tools that I rely on are NIM and Storix. I create my mksysb and move it to my NIM server or use the Storix Network Administrator GUI and create a backup image of my machine to my backup server. In either case, I just boot the machine that I want to overwrite, set up the correct network settings and install the image over the network. This can be a problem in a disaster-recovery situation if you haven’t made sure that these backup images are available offsite, but for day-to-day system imaging I’ve found both methods to be useful.

Sorting Slots

I know that some people have issues when looking at the back of a p595 server. It can be a chore when you want to know which slot is which. This can be important when creating several LPARs on the machine. You want to keep track of which Fibre card and which network card goes with which LPAR. Anything looks complicated until someone shows you how it works.

First, find the serial number of the drawers on the machine, as this is the information that’s displayed on the hardware-management console (HMC) and we’re trying to correlate the physical slot to what’s displayed on the HMC. I used a flashlight and looked at the left side of the front of my example machine. It has two drawers, in this case 9920C6A and 9920C7V.

When you go to the back of the machine, start counting your I/O cards from left to right. There will be four cards, a card you ignore, then six cards. These will be your first 10 slots on the drawer’s left side. There are four more cards, a card you ignore, and six more cards, making up the 10 slots on the drawer’s right side.

These slot numbers correspond with the slots you see when you select required and desired I/O components from the HMC. This I/O drawer had the following selections that I could choose from on the HMC (P1 is the left side of the top drawer, or the first 10 slots. P2 is the right side of the top drawer, or the second 10 slots.):

  • 9920C6A-P1
  • 9920C6A-P2

When I looked at the drawer, going from left to right, I wrote down:

  • C01 is an Ethernet card
  • C02 is a Fibre card
  • C03 and C04 are empty
  • C05 is a Fibre card
  • C06 is an Ethernet card
  • C07 is empty
  • C08 is a Fibre card
  • C09 is empty
  • C10 is a SCSI bus controller

So, I assigned C01, C02 and C05 from 9920C6A-P1 to this LPAR. If I continue the exercise and go to the right side of the top drawer, I start over with C01 and note which type of cards were in which slot. I then continue to do the same thing on the bottom drawer. In this way, I know exactly which cards are in which slot, and it’s simple to assign them to the particular LPAR in which you want them. For redundancy, I’ve heard recommendations state that you take one Fibre card from your top drawer and another Fibre card from your bottom drawer. This way you will still have a path to the SAN if you were to lose one of the drawers.

A Fresh Perspective

Another thing I like to do when I bring in new employees is to have them look for what we’re doing wrong. The new employee has fresh eyes. They don’t know that “this is the way we always do things around here.” They see a piece of documentation, tool or a process, and can question why things are done.

In some cases, there are perfectly good reasons why things are being done a certain way and you can explain them. In other cases, there’s no good reason, other than it’s the way things have always been done. Instead of trying to get them up to speed and make them do it the company way, let them ask you to defend why the company does things this way in the first place. Maybe their previous employer had a much better method that they used to get things done. This is a great time to learn from each other to improve the environment.

We can try to help make a difference in a newer AIX administrator’s career. However, that doesn’t mean we’re the fountain of all information. I’ve found a time or two when an intern has asked why we do things a certain way, and I didn’t have a good answer. I told him to figure out a better way, and come back and inform the group. This helps him with his knowledge of where to look for information and it has helped us all think about processes and procedures that we’ve taken for granted.

The intern’s learning is an example of the boon a little practical knowledge can make. When an individual seeks additional experience, the whole company may benefit as a result.