Category: AIXchange archives

Edit: AIXChange 2007-2020 RIP

Originally posted December 15, 2020 on AIXchange

In Rob McNelly’s final AIXChange post, we revisit some of his other posts from over the years.

By now I assume you’ve seen this IBM statement:
 
“… we will be retiring IBM Systems magazine for IBM Z and Power Systems at the end of 2020. We want to thank the team of publishers, editors and writers at MSP TechMedia for the years of great partnership and collaboration in producing the magazine on our behalf.”
 
Personally, this means the end of my run as a writer, blogger and technical editor for IBM Systems magazine.
 
I started writing periodically for IBM Systems in the early 2000s. This bit on VMware and this write-up of vnc are from 2004.
 
This April 2007 article, about the challenges newcomers to tech face in gaining practical knowledge, actually stemmed from an initiative between IBM Systems Magazine and PowerAIX.org. A contest was held for guest writers to submit their work for publication, and of all the entries, mine was chosen as the winner. As I noted on my archive, that opened doors for me. It’s no coincidence that the AIXchange blog debuted just a few weeks later.
 
This blog has been a huge part of my life. When AIXchange started, my kids were young. Now one’s left the nest while another is ready to graduate high school. At times the blog has been personal for me, like when I’ve written about staying in shape. A quick update on that: The diet I found has worked for me for several years now. I use a heart rate monitor and app to “game-ify” my workouts. I also engage in virtual contests with friends. Seriously, if you’d like to know more, please reach out. I’m eager to help anyone who has a goal of getting healthier. I understand everyone’s mileage may vary, but I also know now that it can be done.
 
No one knows what the future holds. I would like to find another outlet for my writings. Ultimately I could post some original work on what is now my article archive, or I could explore other ventures. But doing this blog has been a fantastic experience. I believe it was definitely a factor in me being named a Lifetime IBM Champion in 2019.
 
A lot goes on behind the scenes. Several people helped make and maintain AIXchange. Those who posted my articles online, the editors who organized my thoughts and helped me look good—I’m grateful for all of you.
 
And of course I’m grateful for you, the readers. Over the years I’ve gotten to know a number of you through email banter or from get-togethers at conferences. It’s led to some real-world friendships. And so many of you have been willing to share information and help solve problems when I get stuck. I’m grateful for all of that. I’m grateful for everything.

Edit: Some powerful tools for your HMC

Originally posted December 8, 2020 on AIXchange

Rob McNelly notes the availability of scripts that are designed to simplify the HMC command line interface.

If you find navigating the HMC command line interface to be a bit challenging, you should consider ezh.

Easy HMC Command Line Interface

EZH is a script for the IBM HMC console to provide an alternate, easier to use, command line interface for many common commands and the goal of the project is to make the HMC command line interface easier to use for day to day administration tasks. Starting with version 0.6 it also has an interactive menu to make it even easier to use (“ezh” command)

For more, check out this video overview/demo.

There’s also an enhanced version of ezh that’s being maintained by Matthew Opoka.

Enhanced Easy HMC Command Line Interface (eezh)

Enhanced EZH script is a fork of EZH by Brian Smith for the IBM HMC console to provide an alternate, simpler command structure which makes the HMC command line interface easier to use for day to day administration tasks. The vision for this project is to enhance and bring EZH commands forward and inline with technologies provided in the newer HMC code base…

The eezh page features installation instructions:

To install, download the “eezh” script.
Then use scp to copy the script to your home directory on the HMC (i.e. scp eezh hscroot@hmc_hostname:~)
SSH to the HMC and login as your account. Then type “source eezh”. You will need to run this command each time you login to the HMC for the EZH commands to be available.

Change Log
Version 2.0 (2020-09-29)
Changed name from ezh to eezh; Enhanced EZH.
Enhancements:
Most commands now sort output by frame name then LPAR name. Input listings for frame and LPAR names are sorted as well.
LPAR power on, shutdown, restart commands to list only running or powered off LPARs depending on selected operations.
framelsfirm now displays the Update Access Key (UAK) expiration date for POWER8 or later hardware.
lparlscpu no longer returns “null” for dedicated CPU LPARs.
New commands:
sriovlsfirm – list SRIOV firmware, driver and frame update availability.
Commands in the works:
sriovlsadapter (99%)
sriovlsphysport (50%) Which fields should we use?
sriovlslogport (0%)

Have you been using ezh in your environment? Did you already know about eezh?

Edit: Yet another post lost to the great purge

Originally posted December 1, 2020 on AIXchange

Rob McNelly offers additional insight with this post about displaying UDID/UUID information.

Awhile back I was asked how one could display the UDID/UUID of a LUN. As is often the case, I found the answer through a simple web search. This search turned up a post from My AIX Cheatsheet.

… I just discovered an interesting (but not new) option for lspv command: -u ! (available for AIX 6.1 TL07 and AIX 7.1 TL01), which displays the UDID and UUID of the physical device directly without scripting headaches:

[The -u tag lists] all the physical volumes in the system along with the following information:

• Physical disk name.
• Physical volume identifiers (PVIDs).
• The volume group (if any) or label (if any) that the physical volume belongs to and that is locked with the lkdev command.
• The state of the volume group.
  • Active: When the volume group is varied on.
  • Concurrent: When the volume group is varied on in the concurrent mode.
  • Locked: When the physical volume is locked with the lkdev command.
• Unique device identifier (UDID).
• Universally Unique Identifier (UUID).

root@lpar:/root# lspv -u

hdisk0             00f68936d157faf1                  rootvg              active
1D06670CF209SYMMETRIX03EMCfcp
302abcdf-d558-58fe-a102-7c3eb11460c9

hdisk1             00f68917df5f3909                  datavg             active
1D06670CF309SYMMETRIX03EMCfcp
c8ddd1f3-557d-689a-5e46-5bc8c037d7ca
hdisk2             00f68916dh5fabf0                  app2vg            active
1D06670CF409SYMMETRIX03EMCfcp
ed4a46c6-0c2f-2230-0967-5207df205184

Small reminder:

• The Unique device identifier (UDID) is the unique_id attribute from the ODM CuAt ObjectClass.
• The UDID is present for PowerPath, AIX MPIO, and some other devices.
• The Universally Unique Identifier (UUID) is present for AIX MPIO device.

The AIX Cheatsheet blogger credits an IBMer named Urban Biel, who wrote about this topic in detail on IBM developerWorks. Unfortunately with the dW site being retired earlier this year, Urban’s original post appears to be lost.

Edit: Always love learning about new things

Originally posted November 24, 2020 on AIXchange

Learn about the modernization of N_Port ID Virtualization by enabling NPIV Multiple-Queues.

On Twitter, Chris Gibson cited this IBM Knowledge Center document on NPIV multiple queue support.

Learn about the modernization of N_Port ID Virtualization (NPIV) by enabling multiple-queues, which is commonly known as NPIV Multiple-Queue (MQ).

Currently, Fibre Channel (FC) adapters with high bandwidth, such as 16 GB or 32 GB FC adapters support multiple-queue pairs for storage I/O communication. Multiple-queue pairs in the physical FC stack significantly improve the input/output requests per second (IOPS) due to the ability to drive the I/Os in parallel through the FC adapter. The objective of the NPIV Multiple-Queue is to add similar Multiple-Queue support to all components such as the client operating system (OS), POWER® Hypervisor (PHYP), and the Virtual I/O Server (VIOS). The NPIV VIOS stack and the PHYP are updated to allow client LPARs to access multiple-queues. The Multiple-Queue feature is supported only on AIX^® client logical partitions and on VIOS Version 3.1.2, or later.

NPIV scaling improvements through Multiple-Queue provides the following benefits:

• Efficient utilization of available Multiple-Queue FC adapters bandwidth when mapped to a single or multiple LPARs.
• Enable and drive multiple logical units (LUN) level I/O traffic in parallel through FC adapter queues.
• Storage I/O performance improvement due to increased input/output requests per second (IOPS).

Toward the end of the doc, there’s information about client tunable attributes:

The number of queues that the NPIV client uses depends on several factors such as FC adapter, FW level, VIOS level, and tunable attributes of the VFC host adapter. During the initial configuration, the VFC client negotiates the number of queues with the VFC host and configures the minimum value of num_io_queues attribute and the number of queues that are reported by the VFC host.

After the initial configuration, the negotiated number is the maximum number of channels that the VFC client can enable. If the VFC host renegotiates more channels after operations (such as remap, VIOS restart, and so on), the number of channels remains the same as the initially negotiated number. However, if the VFC host renegotiates with fewer channels, the VFC client reduces its configured channels to this new lower number.

For example, if the initial negotiated number of channels between the VFC client and VFC host is 8, and later if the VFC host renegotiates the number of channels as 16, the VFC client continues to run with 8 channels. If the VFC host renegotiates the number of channels as 4 channels, the VFC client adjusts its number of configured channels to 4. However, if the VFC host renegotiates the number of channels as 8 channels, which result in increasing the number of configured channels to 8, the VFC client must be reconfigured to renegotiate the number of channels from the client side.

This doc has much more information than I’ve included here, so be sure to read the whole thing. Also take a moment to view Chris’s actual tweet, which includes an image showing output from the lsdev –dev viosnpiv0 –attr command; viosnpiv0 is a new pseudo device you’ll find on your vio server.

Edit: Did you watch the replays?

Originally posted November 17, 2020 on AIXchange

For those who missed the recent virtual IBM Technical University, some free sessions are available on YouTube.

For those who missed the recent virtual IBM Technical University, some free sessions are available on YouTube.

Nigel Griffiths: njmon, InfluxDB & Grafana 7.1 First Graphs Demo.

Jaqui Lynch has a number of sessions: Cloud 101, NIM 201, and VIO care and feeding part 1, part 2, and part 3.

To gain access to most everything that went on—more than 600 sessions in all, including 130 in Portuguese or Spanish—purchase an on demand package by Dec. 30. All online content will remain available through January. (Note: Lab Sessions and some NDA sessions are excluded.)

For details, check out the agenda and options for covering the $400 tuition.

$400 may seem pricey, but compared to a typical in-person conference where you or your employer also foot the bill for airfare, lodging, and food, Virtual TechU is a solid value.

Edit: Have you tried this yet?

Originally posted November 10, 2020 on AIXchange

Rob McNelly provides perspective on a new Redbook and its relationship with IBM Private Cloud.

IBMer Gareth Coates recently tweeted about a new Redbook, “IBM Power Systems^TM Private Cloud with Shared Utility Capacity featuring Power Enterprise Pools 2.0.”
 
This draft publication, which is compiled by several authors, including Coates, consists of 132 pages covering planning, configuration and management and migrating from Private Cloud 1.0 to 2.0:
 
“As more enterprises look at ways to reduce costs while maintaining the scalability required to stay competitive in today’s fast-paced business environment, pay-as-you-go or consumption-based infrastructure models have grown increasingly popular. These purchase models enable clients to quickly scale their IT infrastructure up or down to optimize costs while quickly adapting to dynamic business environments.
 
“The IBM Power Systems Private Cloud Solution expands the breadth and availability of Shared Utility Capacity to enable a wider range of Power clients to take advantage of cloud agility and economics provided by Power Enterprise Pools 2.0 on the flagship Power E980. With Shared Utility Capacity, the concept of Power Enterprise Pools 2.0 is expanded to the Power Server E950 as well as to the Power Servers S922 (9009-22G) and S924 (9009-42G). In addition, minimum activation requirements were lowered to 1 Base Processor Activation and 256 GB Base Memory Activation when moving systems into a pool. This can substantially lower the initial system acquisition price for clients who want to minimize their short-term capital expenses and/or tailor their system configuration to their specific deployment plans.”
 
Here are some details about IBM’s CoD offerings:
 
Capacity upgrade on Demand: You can permanently activate inactive processor cores and memory units by purchasing an activation feature and entering the provided activation code on the Hardware Management Console. You can do this without restarting your server or interrupting your business
 
Elastic Capacity on Demand: You can activate processor cores or memory units for a number of days by using the Hardware Management Console to activate resources on a temporary basis. Activations are available in 24 hour increments.
 
Utility Capacity on Demand: Utility CoD is used when you have unpredictable, short workload spikes. You can automatically provide additional processor capacity on a temporary basis within the shared processor pool of a single system. Usage is measured in processor minute increments.
 
Trial Capacity on Demand: You can evaluate the use of inactive processor cores, memory, or both, at no charge using Trial CoD. After it is started, the trial period is available for 30 power-on days.
 
Power Enterprise Pool: A Power Enterprise Pool is a group of systems that can share Mobile Capacity on Demand (CoD) processor resources and memory resources. Mobile resources can be assigned to individual servers in the pool from the Hardware Management Console.
 
IBM Power Systems Private Cloud with Shared Utility Capacity: In a Shared Utility Capacity Pool, all systems in a pool share their activated base capacity without any requirement for user intervention. The Pool’s usage is tracked by the minute on the Cloud Management Console. Capacity above the pool’s base capacity can be automatically used and is metered and charged by the minute.
 
It’s always worth the time to keep abreast of the changes that are taking place with Power Systems.

Edit: Hopefully you are not running into this issue.

Originally posted November 3, 2020 on AIXchange

IBM Support Q&A resolves a dilemma when a client ran into issues installing the PowerVM® Virtual Input/Output Server.

A client attempted to install the PowerVM^® Virtual Input/Output Server (VIOS). However, when going through the installation menus, none of the internal disks were visible.

A search turned up this IBM Support Q&A:

Here’s a list of common reasons why the installation menu doesn’t display internal disks:

1. VIOS partition does not have disk I/O controller.
Recommendation: Login to the VIOS management GUI (i.e. HMC, IVM, FSM, etc) and verify the VIOS has the expected disk I/O controller allocated in the VIOS partition’s profile. For assistance determining what resources are allocated to the VIOS partition, contact your local GUI management support representative.

2. The disk isn’t properly formatted for AIX/VIOS OS.
Recommendation: If the environment involves IBM i or was previously used for IBM i, it is possible that the disk may be formatted for IBM i OS. Ensure the disk is formatted for AIX/VIOS OS.

(This explanation references the IBM Support doc, How to Format i Drives to AIX Format for the Purpose of Installing VIOS.)

3. VIOS Install level is not compatible with the POWER system.
Recommendation: Verify the machine type/model supports the VIOS level you are trying to install. For details, refer to the system to VIOS maps.

4. Downlevel microcode.
Recommendation: Verify that the system, adapter and disk microcode are up to date.

It turned out that we needed to format the disks. Once we did so, we were able to install VIOS as expected.

Learn more about VIOS in Why You Should be Running VIOS 3.x.

Edit: What about when you upgrade your phone? Do you notice the same thing?

Originally posted October 27, 2020 on AIXchange

Rob McNelly gets a new laptop and relearns an important lesson: While you’d expect technology to progress, it’s still worth noticing.

I’ve been using ThinkPad laptops for more than 20 years, and each generation is an improvement upon the previous one. While you’d expect that sort of progression, it’s still worth noticing.

Honestly, I wasn’t planning to get a new laptop. My old one was running great and I just got it—or so I thought. It took an email reminder from the corporate help desk to make me realize that I’d had my previous laptop for three years. It actually was time for a refresh.

We all intuitively understand that technology advances significantly over the course of three years, but it’s easier to appreciate when you get your hands on something new. So how much better is a new Lenovo T490 than an old T460?

For starters, the drive capacity is better. The battery life is better. It’s a bit slimmer, which is also better. The CPU is a quad core rather than a dual core. It has a larger maximum memory capacity and a faster memory controller, and it uses DDR4 memory.

An interesting change is the drive and how it’s attached. In the old days with smaller capacity drives, I could carve out extra storage on my ThinkPad by putting a second drive into an adapter. Then I’d put that adapter into the DVD-ROM slot. My new hardware doesn’t support DVD-ROM, but of course I don’t need it now; there’s plenty of space.

I’d been accustomed to SSD drives connecting via SATA, but now the drive is an NVMe SSD. That’s a completely different form factor.

On the previous version, the docking station had VGA output. On the new one the video uses DisplayPort, so I need different cables and adapters to connect external monitors. (Luckily, my Model M still works.)

My verdict? This new machine screams. The T490 runs circles around the previous version, and is leaps and bounds better than the one I had before that. It’s funny. I was perfectly happy with the old laptop. Those three years flew by. But in an instant with the latest and greatest I could see the benefits of moving forward.

So maybe you need a new laptop. Or maybe it’s time to look at your infrastructure and consider upgrading your hardware and software. Again, I was fine chugging along as I was, but now with the snappier performance, I realize what I was missing out on. I also can’t count the number of times I’ve seen IBM Power Systems^TM users react similarly to the performance of new hardware. Yes, everyone has budgets, but what is it worth to your organization when response times are better, jobs complete faster and more work is being done with fewer cores?

It’s not easy to quantify the value of faster boot times or faster-running applications, but I believe those small improvements add up.

Edit: What are you doing to collect performance data?

Originally posted October 20, 2020 on AIXchange

IBM has discontinued the Performance Management for Power Systems™ service.

As noted by Nigel Griffiths, IBM has discontinued the Performance Management for Power Systems™ service.
 
PM for Power was a way for clients to share their data with IBM so it could be analyzed. As I mentioned in this piece on options for graphing AIX^® performance data, the service consisted of a no-charge basic version and a fee-based full-featured product.
 
If you used PM for Power, you should disconnect the service. This IBM Support FAQ tells you how to stop data collection and transmission.
 
An option going forward is the njmon performance stats tool. I touch on it in the article above; the njmon site has YouTube videos and instructions on how to get it working. You can also access replays of the monthly njmon/nmon user meetings.

Edit: Are you able to survive for very long without power?

Originally posted October 13, 2020 on AIXchange

Rob McNelly reflects on how an event this summer reminded him to always be prepared.

I live in Phoenix, which is always hot but was hotter than ever this summer. This year Phoenix had a record number of days with highs above 110F degrees; 13 of those days were at least 115F. Despite the constant heat and the occasional heavy thunderstorm (though none this year), power outages are relatively rare. However, I had one this summer.
 
A transformer blew, leaving 33 houses in my neighborhood without power. Luckily temps were “only” in the mid-90s at the time. At the moment the power went out, my distance-learning high school son and remote worker wife were forced offline. Both use desktop computers without any type of UPS. I was fortunate to have a laptop with a battery. While my docking station and external monitors were useless, I could at least use a cellular hot spot, manage the power settings to conserve power on my laptop, and suspend the machine when necessary.
 
That outage lasted a few hours. But with hurricanes and fires knocking out power and disrupting lives around the country these days, it’s smart to consider your options should you find yourself in this situation. What would you do if your whole city or region loses power for an extended period? Running a desktop may not be possible, but smaller devices like phones, tablets and laptops could still be managed, assuming of course that other pieces of infrastructure like cell towers are still functional.
 
Obviously work may not be your top priority during an outage, but having the capability to communicate is vital regardless. When the power goes out, you’re looking for ways to generate and store electricity, and deliver that electricity to your devices. The easiest way to get juice is to use a simple power inverter that can be plugged into a cigarette lighter outlet in your vehicle. I’ve been on camping trips where I was able to log in and remotely solve urgent client issues because I brought my laptop and had good cellular coverage. A DC charger that bypasses the loss that comes from the inefficiency of converting to AC power via the inverter could also come in handy. With either solution, be sure to maintain the vehicle battery via the alternator. You don’t want to drain your starter.
 
There are other power options. RV users are familiar with the concept of deep cycle house batteries, be they lead acid, gel, AGM, or lithium ION. These longer-lasting batteries can also be used as a stand-alone power supply. Of course they still require periodic recharging; that can be accomplished with a vehicle alternator, a gas burning generator, or even solar panels.
 
On that note, portable solar generators are becoming popular. They’re essentially battery banks (from the likes of Jackery. Bluetti, or Yeti) that can be recharged from a wall or vehicle outlet, or solar panels.
 
If you’re thinking big picture, there are ways to power your entire home, or at least your larger appliances. Whole house generators are one solution, or you could consider something like Tesla’s Powerwall mixed with solar panels. It all comes down to the problems you’re trying to address, the risks you face from not having electricity, and of course, your budget.
 
I generally take my electric utility for granted, but my recent experience reminds me of the value of emergency planning. Knowing that alternative power sources are available, whether I’m working from home or out camping, provides peace of mind.

Edit: Still great information

Originally posted October 6, 2020 on AIXchange

IBMer Gareth Coates maintains a blog covering topics from NIC speed in AIX to processor compatibility modes on POWER9, scripts like sumcol, duvg and dfvg and much more.

IBMer Gareth Coates maintains a blog called Tricks of the Power Masters.

As of this writing, Gareth’s blog has 25 entries covering everything from NIC speed in AIX^® to processor compatibility modes on POWER9^® systems to scripts like sumcol, duvg and dfvg and much more.

A recent post explains what to do if the HMC displays the wrong OS level (in an LPAR) after an upgrade.

After upgrading the operating system in an LPAR on an IBM Power System, the HMC does not always notice that there has been a change. This page will show you how to correct this and also has a script to find the O/S versions of all LPARs on all systems managed by an HMC.

Note: a Resource Monitoring and Control (RMC) connection between the HMC and the LPAR is required to be able to see the O/S version.

Read on and you’ll learn about the –osrefresh flag on the lssyscfg command:

hscroot@hmc16:~> lssyscfg -m P9-S922-amber -r lpar -F os_version –osrefresh –filter lpar_names=vm220-a74b3c63-0000002e

AIX 7.2 7200-04-02-2028

The HMC has refreshed its data and outputs the new version.

The next time that the page is refreshed, the GUI will now show the updated version too.

He then provides a script that updates the OS that the HMC displays:

hmcuser=hscroot@hmc16              

ssh $hmcuser ‘            

lssyscfg -r sys -F name | sort | while read server            

do            

lssyscfg -m ${server} -r lpar –osrefresh -F name,os_version| sed -e s/^/${server},/            

echo            

done            

‘

Take a moment and read the whole thing; it makes more sense in its entirety. Then spend some time going through Gareth’s other entries. I’m sure you’ll learn something new.

Edit: Have you ever used this site?

Originally posted September 29, 2020 on AIXchange

For a new resource guide, check out the logically and literally named Linux on Power Developer Portal.

If you’re interested in developing in Linux on Power environments, check out the logically and literally named Linux on Power Developer Portal.
 
“Learn about developing on the IBM Power architecture, find open source packages, get access to cloud resources, discover the tools and technologies you need to build applications, and connect with your peers and other developers, data scientists, and academics.”
 
The site features detailed and regularly updated documentation on everything from porting tips and techniques to development tools and utilities, and much more. For example, dig into this quick list of recommendations if you’re considering a switch from x86 to Linux on POWER.
 
The Open Source POWER Availability Tool (OSPAT) can also be accessed from the portal. I wrote about OSPAT in 2016, but many of the IBM developerWorks links in that post no longer work. The good news is some material now on the portal appears to have been re-homed from developerWorks. I’m always happy to see good technical content preserved.

Edit: Did any of you run into this issue?

Originally posted September 22, 2020 on AIXchange

Rob McNelly reveals the commonality that some problematic tasks share.

Chris Gibson alerted me to this information about commands and tasks that may hang indefinitely on the HMC.

Per IBM, “certain task executions can hang indefinitely with no progress for hours.” This problem is specific to the 7063-CR1 POWER HRM platform V9R1. The problematic tasks are:

• Installing a PTF with updhmc or through the Install Corrective Service task
• Performing a LIC Code Update or Code Upgrade with updlic or through the Change Licensed Internal Code wizard
• Collecting pedbg

The commonality with these tasks and commands is that each “calls the underlying Linux® rpm command for execution. The rpm commands hang and prevent the tasks or command from continuing to completion.”

Typically I’d ask you check the document itself, but since this one is relatively short I’ll share the bulk of it here:

Diagnosing The Problem

Each of the tasks stops at a different message specific to the individual task and not proceed further:

            HMC installation > Installing Updates …
            LIC Code Update to the managed system > Retrieving Updates …
            pedbg collection > Collecting PTF History …

Resolving The Problem

Terminate the task that is hung either with Ctrl c from CLI or if executed from the GUI by navigating to:

Users and Security > Users and Roles > Manage Users and Tasks
Find the task and select it then “Terminate”

If the HMC is at V9.R1.930 and is updated with the following PTF, or higher:

PTF MH01825 HMC V9 R1.930.1–for 7042 Machine Types or vHMC for x86 hypervisors (5765-HMW)
PTF MH01826 HMC V9 R1.930.1–for 7063 Machine Types or vHMC for PowerVM (5765-HMB)

Then the following command can be run as user hscpe: #runsig -s 700

The runsig command will kill all rpm processes, remove the rpm database, and then rebuild it.

This will allow the previous tasks/commands to be performed and now complete successfully. Until a fix is available the RPM command could hang again in the future….

The document was last modified on Aug. 19, so as of this writing there’s no fix yet. Hopefully one will come in time, but for now you’ll need to rely on runsig.

I believe it’s helpful to circulate this type of information. Awareness of a problem makes it easier to deal with should you actually encounter this issue. After all, knowing is half the battle.

Edit: What will things look like 40 years from now.

Originally posted September 15, 2020 on AIXchange

Rob McNelly dips into 80s nostalgia and discovers how amazing it is to have a terminal and a PC on one desk.

Once I saw this, I had to try it:
 
cool-retro-term is a terminal emulator which mimics the look and feel of the old cathode tube screens. It has been designed to be eye-candy, customizable, and reasonably lightweight.
 
It uses the QML port of qtermwidget (Konsole) developed by me.
 
This terminal emulator works under Linux and macOS and requires Qt 5.2 or higher.
 
Settings such as colors, fonts and effects can be accessed via context menu.
 
Check out the screen shots. Those of you of a certain age will likely feel nostalgic when you see them. While I wouldn’t say this emulator is practical for my day to day system management needs, installing it on a Linux desktop and running through the different colors and computer types and fonts was pretty easy.
 
And by the way, other emulators are also bringing back that distinct 80s look. I’m definitely going to try Guake.
 
When I started in this industry, my only computer access was a single console on a desk. I’m not being flippant when I say that advancing to the point where I had both a terminal and a PC on the same desk was an amazing day. It’s safe to say that the computing access and network ubiquity I currently enjoy would be unfathomable to me in the days when amber and green screens were all I knew.

Edit: Still good information

Originally posted September 8, 2020 on AIXchange

Rob McNelly discovers another hack. This IBM Redpaper increased bandwidth and faster data transfers.

This IBM Redpaper examines recent updates to POWER9™-based servers: namely the S924, S922 and S914 G models. It also covers the performance impact that PCIe Gen 4 slots bring to this hardware. Short version: increased bandwidth and faster data transfers.

The Redpaper covers a lot of ground in just two chapters, and I encourage you to download the document. To give you an idea of the content, I’ll highlight the basic speeds and feeds for the updated systems:

The Power S922 (9009-22G) server is a 1-2 socket server that includes up to 22 activated cores. If only one socket is populated at the time of the order, the second socket can be populated later.

The Power S922 server supports a maximum of 32 DDR4 registered DIMM (RDIMM) slots. If only one processor socket is populated, then only 16 RDIMMs can be used. Supported memory features are 16 GB, 32 GB, 64 GB and 128 GB. The S922 runs at speeds of 1600, 2133 or 2666 MHz and offers a maximum system memory of 2 TB if one socket is single-chip populated and 4 TB with both sockets populated.

The Power S914 (9009-41G) server includes up to eight activated cores. A one-socket system with a 4- or 6-core processor is available in either rack (19-inch rack-mount 4U) or tower configurations. The higher performance 8-core higher system is available only in a rack configuration.

The Power S914 server supports a maximum of 16 DDR4 error-correcting code (ECC) RDIMM slots. Supported memory features are 16 GB, 32 GB and 64 GB. The S914 runs at speeds of 2133 or 2666 MHz and offers maximum system memory of 1 TB.

The Power S924 (9009-42G) server is a 1-2 socket server that includes up to 24 activated cores. This server supports two processor sockets, offering 8 or 16 cores at 3.8-4.0 GHz, 10 or 20 cores at 3.5-3.9 GHz, 11 or 22 cores at 3.45-3.9 GHz, or the maximum core configuration with 12 or 24 cores at 3.4-3.9 GHz. The systems are in a 19-inch rack-mount, 4U drawer configuration. All the cores are active.

The Power S924 server supports a maximum of 32 DDR4 RDIMM slots per processor socket that is populated. Supported memory features are 16 GB, 32 GB, 64 GB and 128 GB. Maximum system memory is 4 TB.

Learn more about these systems by listening to the Power Systems VUG July 30 meeting replay. (Slides here.)

Also be sure to check out the recently updated IBM Power Systems Performance Report, which provides the latest rperf and CPW numbers for POWER9 hardware.

Edit: Still good tips

Originally posted September 1, 2020 on AIXchange

Rob McNelly makes the point that if you run into an issue, in many cases the solution to your problem–or at least a solution to a problem similar to yours–is only a web search away.

Here’s an interesting question: Suppose you make a copy of your datavg using the savevg command. Can you restore that backup to another machine with a different number and/or size of LUNs available? Or do you need to restore your system with a like for like number and size of disks?

Sometimes in a disaster recovery (DR) or cloning situation, we don’t have access to the same sizes or numbers of disks that our source machines had. What to do then? Use mkvgdata and restvg.

It is useful to run the following commands before you create your (at least) weekly mksysb image:

            # lsvg -o | xargs -i mkvgdata {}
            # tar -cvf /sysadm/vgdata.tar /tmp/vgdata

Add these commands to your mksysb script, just before running the mksysb command. What this does is to run the mkvgdata command for each online volume group. This will generate output for a volume group in /tmp/vgdata. The resulting output is then tar’d and stored in the /sysadm folder or file system. This allows information regarding your volume groups, logical volumes, and file systems to be included in your mksysb image.

To recreate the volume groups, logical volumes and file systems:

Run: # tar -xvf /sysadm/vgdata.tar
Now edit /tmp/vgdata/{volume group name}/{volume group name}.data file and look for the line with “VG_SOURCE_DISK_LIST=”. Change the line to have the hdisks, vpaths or hdiskpowers as needed.
Run: # restvg -r -d /tmp/vgdata/{volume group name}/{volume group name}.data

Make sure to remove file systems with the rmfs command before running restvg, or it will not run correctly. Or, you can just run it once, run the exportvg command for the same volume group, and run the restvg command again. There is also a “-s” flag for restvg that lets you shrink the file system to its minimum size needed, but depending on when the vgdata was created, you could run out of space, when restoring the contents of the file system…

This older post deals with the same concept, though the subject is using a script to automate the process with mirrored mksysb images.

The below script does the following:

1. Modifies the source_disk_data stanza to a single entry.
2. Modifies the VG_SOURCE_DISK_LIST stanza to the disk set by the -d paramater.
3. Modifies the LV_SOURCE_DISK_LIST stanza to the disk set by the -d paramater.
4. Modifies the COPIES stanza to 1.
5. Modifies the PP stanza to match the LP’s value.

The script takes two values, the resulting hdisk on the other node (In my case, hdisk0 for rootvg, and hdisk1 for datavg), and the location of the image.data or vg.data file.

A lot of what we need to do may be new to us, but chances are it’s been done before. It certainly makes things easier when you can do a quick search and find proven solutions. And frequently, even if you can’t find an explanation of your specific problem, there’s some tip out there that can still be readily applied to your situation.

Edit: Still something that needs to be addressed, how to hobbyists get better with this hardware if they do not have access to it?

Originally posted August 25, 2020 on AIXchange

Rob McNelly explains why he’s always been an advocate of using test systems.

I’ve long been an advocate of using test systems:

“I think every IT pro understands that there’s a huge difference between reading about something and actually doing it. A test box is like a classroom that’s always open and available.”

“What do you need a test box for? What don’t you need it for? When a test box is available, programmer/administrator mistakes are learning opportunities rather than lost uptime. Test boxes are where we learn, where we validate, where we get comfortable with the technology. If you ask around, I think you’ll find that the people who excel at their jobs generally have spent considerable time on test systems. Certainly access to test hardware makes for more confident admins.”

“Now, if your employer absolutely won’t pay for one, there are other ways to access a test system. IBM has a virtual loaner program available to business partners. With this you can at least logon to the command line of a remote AIX systems. Of course this isn’t the same as having a test box onsite, available whenever you want to play around with it.”

For years IBM had what was known as the virtual loaner program (later called the IBM Power Development Cloud), which provided business partners with access to remote access to test environments. While that program was discontinued, now IBM offers a new option for accessing test systems: the Client Experience Centers Cloud (CECC).

The CECC is available at no charge to IBM PartnerWorld members worldwide. Available IBM hardware includes POWER9™ and POWER8® processor-based servers on AIX®, IBM i and Linux operating systems. The CentOS, RHEL, SUSE & Ubuntu Linux distributions can also be accessed. As noted on the CECC main page:

“CECC is intended for Development, Porting and Functional testing only. For any Performance or Scalability testing, users are requested to approach local Client Experience Centers.”

I should add that this program is limited in scope. For instance, it’s not possible to do things like tinker with the HMC or VIO server, or do anything at the virtualization layer. But if you’re simply looking to access an AIX test environment and play around, this is a viable possibility.

Edit: Do you ever play around with these?

Originally posted August 18, 2020 on AIXchange

Rob McNelly shares a script that can make life easier when working with multiple VIO servers.

I’ve posted a few Dean Rowswell scripts over the years, but it’s been awhile. He emailed me recently though, so I’m sharing the following with his permission:

There is a really nice -client flag on the VIOS padmin fcstat command that displays virtual FC adapter stats and I wanted to improve on the output and to work with multiple VIO servers so I made a script.
Here is the usage of it:

USAGE: get_vio_fcstat [ -v, -V, -h, -u, -l, -n ]

  -v or -V will print out the version and exit
 -h VIOS hostname(s) or IP address(es) COMMA SEPARATED to use
 -u VIOS userid to use (only required if padmin not used)
 -l will sort the report by LPAR/HOST name
 -n only include specific HOSTS or VIOS

EXAMPLE: get_vio_fcstat -h vios1,vios2

 get_vio_fcstat -n lpar

 get_vio_fcstat -h vios1,vios2 -n “lpar|vio” -l

You will need to change the VIOS_LIST variable at the top of the script and then have password-less ssh setup from the server running it on to the target VIO(S).

I usually run this script from a NIM server for this purpose.

The Script

#!/bin/ksh
# Created by Dean Rowswell, July 27, 2020
# Modified by Dean Rowswell, July 28, 2020
#  V1.1 – change report column order + add flag to display the report sorted by LPAR/HOST name
# Modified by Dean Rowswell, July 30, 2020
#  V1.2 – calculate the total physical and virtual statistics + add flag to only include specific LPAR/HOST names in the report
#
# This script runs the command fcstat -client on Virtual I/O Servers to display the physical and virtual Fibre Channel statistics
# The FC read and FC write statistics are totaled and the default report is sorted from the highest to the lowest
# The VIOS physical adapters are labeled “PHYS”

VIOS_LIST=”vios1 vios2″
VIOS_USER=”padmin”
VER=”1.2″

# Parameter checks
if [ ${#*} -ne 0 ]
then
  while getopts :vVh:u:ln: PARMS
  do
    case $PARMS in
      v|V) echo “This is $0 version: $VER” ; exit ;;
      h)   VIOS_LIST=`echo ${OPTARG} | tr ‘,’ ‘ ‘` ;;
      u)   VIOS_USER=${OPTARG} ;;
      n)   LPAR_NAME=${OPTARG} ;;
      l)    LPAR_SORT=1 ;;
      ?)   echo “\nUSAGE:\t$0 [ -v, -V, -h, -u, -l, -n ]” ; echo “\t-v or -V will print out the version and exit” ; echo “\t-h VIOS hostname(s) or IP address(es) COMMA SEPARATED to use”
           echo “\t-u VIOS userid to use (only required if padmin not used)” ; echo “\t-l will sort the report by LPAR/HOST name”
          echo “\t-n only include specific HOSTS or VIOS\n” ; echo “EXAMPLE: $0 -h vios1,vios2\n\t $0 -n lpar\n\t $0 -h vios1,vios2 -n \”lpar|vio\” -l\n” ; exit ;;
    esac
  done
fi

printf “\n%-24s %-12s %-17s %-24s %-6s %14s %11s %12s %12s %11s %12s\n” HOSTNAME DEVICE WWPN VIOS PHYS TOTAL_R+W_GiB “(    TiB   ) =” READS_GiB “(    TiB   )  +” WRITES_GiB “(    TiB   )”
echo “———————— ———— —————– ———————— ——  ————- ————    ———– ————    ———– ————“
VIOS=””

GET_VIO_FCSTAT () {
for VIO in ${VIOS_LIST}
do
  ssh -q ${VIOS_USER}@${VIO} ioscli ioslevel >/dev/null 2>/dev/null
  if [ $? -ne 0 ]
  then
    echo “Password-less SSH access to VIOS ${VIO} with user ${VIOS_USER} is not setup\n”
    continue
  fi
  ssh -q ${VIOS_USER}@${VIO} “ioscli fcstat -client” 2>/dev/null | awk -v VIOS=”$VIOS” -v NAME=”$LPAR_NAME” ‘{
  if (NF == 11 && $1 != “hostname”) {
    if (VIOS == “”) { VIOS=$1; PHYS=$2 } ; HOST=$1 ; DEV=$2 ; WWPN=$3 ; READ=$7 ; WRITE=$8 ; TOTAL=$7+$8
    if (HOST == VIOS) { DEV=DEV”[PHYS]” }
    if ($1 ~ NAME) { printf “%-24s %-12s %-17s %-24s %-6s %14.3f   %8.3f   %14.3f   %8.3f   %14.3f   %8.3f\n”, HOST, DEV, substr(WWPN,3,16), VIOS, PHYS, TOTAL/1073741824, TOTAL/1099511627776, READ/1073741824, READ/1099511627776, WRITE/1073741824, WRITE/1099511627776}}
  if (NF == 0) { VIOS = “” }
  }’
done
}

CALC_TOTALS () {
awk ‘$2 ~ /[PHYS]/ { P_RW_GB += $6 ; P_RW_TB += $7 ; P_R_GB +=$8 ; P_R_TB += $9 ; P_W_GB += $10 ; P_W_TB += $11 ; print $0 ; next }  { V_RW_GB += $6 ; V_RW_TB += $7 ; V_R_GB +=$8 ; V_R_TB += $9 ; V_W_GB += $10 ; V_W_TB += $11 ; print $0 } END { print ”                                                                                —————  ————- ————    ———– ————    ———– ————” ;printf “%87s %14.3f ( %8.3f ) %14.3f ( %8.3f ) %14.3f ( %8.3f )\n”, “PHYSICAL total:”, P_RW_GB, P_RW_TB, P_R_GB, P_R_TB, P_W_GB, P_W_TB ; printf “%87s %14.3f ( %8.3f ) %14.3f ( %8.3f ) %14.3f ( %8.3f )\n”, “VIRTUAL total:”, V_RW_GB, V_RW_TB, V_R_GB, V_R_TB, V_W_GB, V_W_TB }’
}

if [ ${LPAR_SORT:=0} -eq 1 ]
then
  GET_VIO_FCSTAT | sort | CALC_TOTALS
else
  GET_VIO_FCSTAT | sort -nrk6 | CALC_TOTALS
fi

Edit: This is still pretty neat

Originally posted August 11, 2020 on AIXchange

Rob McNelly has a hack for accessing AIX information and IBM solutions without needing to rely on a search engine.

If you’re looking for lifecycle information for AIX® or any number of related IBM solutions, you don’t have to resort to a search engine. If you have curl loaded on your system, it’s simply a matter of downloading and installing a shell script.

Which script? Armin Schmidt has the details.

How often have you searched the lifecycle information (start of general availability and end of support) for AIX, VIOS, PowerHA or HMC via Google?

This information can also be found easily using the command line of the IBM FLRT API.

All you need is:
* a direct internet connection or connection via an HTTP proxy
* an installed version of curl

Sample output is provided:

 $ show_life_cycle VERSION GA EOSPS UPDATE UPGRADE 7200-04-02 2020.05.15 2022.11.30 – – 7200-04-01 2019.11.15 2022.11.30 – – 7200-04-00 2019.11.15 2022.11.30 7200-04-01 – 7200-03-04 2020.02.14 2021.09.30 – 7200-04-01 7200-03-03 2019.05.10 2021.09.30 7200-03-04 7200-04-01 7200-03-02 2018.11.16 2021.09.30 7200-03-04 7200-04-01

Download the tool here. Search for show_life_cycle.

PowerCampus lists a number of other tools and corresponding presentations (in English and German). For instance if you’re interested in the LPAR-Tool, you may also wish to download the LPAR-Tool PDF. Additionally, I recommend this article on Inventory Scout, an IBM tool for monitoring firmware and microcode levels.

I love learning of new (to me) tools and scripts and sharing this information with you. And I appreciate those who take the time to share their finds with me.

Edit: How do you avoid running into avoidable problems?

Originally posted August 04, 2020 on AIXchange

This post provides instructions for updating HMC Version 9.1.0 with the latest fixes for 9.1.0.

Here’s a rundown of HMC V9R1 updates:

This document provides instructions for updating HMC Version 9.1.0 with the latest fixes for 9.1.0.

1. Verify the HMC is at Version 9 Release 1.0
If the HMC is not at 9.1.0, you must first upgrade to 9.1.0. To upgrade the HMC to Version 9.1.0, refer to document N1022499 , Model 7042/x86: Upgrading the HMC from Version 8.8.6 or Later to Version 9 or N1022504, Model 7063/ppc: Upgrading the HMC from Version 8.8.7.0 to V9R1

2. Determine the type of HMC (x86 versus ppc)
Starting with V8R8.7 all HMC Recovery images, network images, iFixes, and Service Packs are published in two formats, x86 and ppc. There is a unique PTF number/download for each. Determine which PTF/download is need based on the HMC model or vHMC component.

x86 This term is used to reference HMCs running on 7042 Machine type or vHMC hosted by x86/Intel/AMD hardware and hypervisors (KVM, VMWare, Xen).

ppc Term is used to reference HMCs running on 7063 Machine type or vHMC hosted by Power servers.

Step 5 covers various update methods, including:

.iso images: Instructions for downloading the fixes and installing them from media
Ordering/Using DVDs: Instructions for ordering the media from IBM and installing from optical media.
Network: Instructions for installing the fixes directly from the IBM FTP site (or alternately a local FTP server).

At the end, you’ll find instructions for backing up your HMC. Part of me wonders why that isn’t the first item, but perhaps IBM hopes you’ll take the time to read the whole thing before getting started:

This task backs up (or archives) the data that is stored on your HMC hard disk that is critical to support HMC operations.

Back up the HMC data after changes have been made to the HMC or information associated with logical partitions.

The HMC data stored on the HMC hard drive can be saved to a remote system mounted to the HMC file system (such as NFS) or sent to a remote site using File Transfer Protocol (FTP).

By using the HMC, you can back up all important data, such as the following data:
* User-preference files
* User information
* HMC platform-configuration files
* HMC log files
* HMC updates through Install Corrective Service.

Finally, keep your hardware firmware, HMCs, VIO servers, and AIX® servers at recommended levels so you don’t run into avoidable problems.

Edit: Hopefully you have upgraded by now.

Originally posted July 28, 2020 on AIXchange

VIOS 2.6 is going out of support—and every prior 2.x version is already out of support. It’s time to upgrade to 3.x.

VIOS 2.6 is going out of support—and every prior 2.x version is already out of support. It’s time to upgrade to 3.x.

Nigel Griffiths’s comprehensive look at VIOS 3.1 features good information about upgrading your VIO servers. You’ll also find presentation slides and links to IBM documentation.

On that note, you should be aware of potential issues surrounding specific VIOS versions when you do an update or a 2.x to 3.x upgrade:

HIPER APAR IJ25390

USERS AFFECTED:
Systems running the AIX 7200-04-02-2015 or 7200-04-02-2016 Technology Level or VIOS 3.1.1.20 or 3.1.1.21 with devices.vdevice.IBM.l-lan.rte below the 7.2.4.3 level.

ERROR DESCRIPTION:
The Virtual Ethernet Adapter can cause a significant degradation in TCP performance for largesend packets.

This issue can be seen with VIOS (serving any AIX®, IBM i, or Linux client LPAR) running the affected VIOS levels below. It can also be seen with any AIX client LPAR running the affected AIX levels below (regardless of the VIOS level). This issue only occurs when the client LPARs are configured for largesend traffic or VIOS is configured for largereceive traffic.

RECOMMENDATION:
Install APAR IJ25390.
Prior to fix availability, an interim fix is available from either:

ftp://aix.software.ibm.com/aix/ifixes/ij25390/
https://aix.software.ibm.com/aix/ifixes/ij25390/

The iFix can be installed using Live Update (LU). If LU is not used, installation of the iFix requires a reboot.

Local fix
Disable PLSO:

Either disable mtu_bypass:
# chdev -l enX -a mtu_bypass=off

or disable PLSO on the VEA or trunk VEA:
# chdev -l entX -a platform_lso=no

Thanks to Chris Gibson, who gave me a heads up about this issue some weeks back.

Edit: I still prefer working from home.

Originally posted July 21, 2020 on AIXchange

The technology, tips and tricks Rob McNelly has gained under his sleeve while working remotely all these years.

Unlike most people, I haven’t had to adapt to working at home. That’s because I’ve worked at home for years.

For me, working at home was an easy choice. I love being able to get going first thing in the morning while I’m fresh and alert. On weekdays at least, I’ll typically awaken thinking of work anyway. So rather than sitting through a commute, worrying about losing time, I can get right to it. I suppose being an introvert plays a part as well. Being around others all day can be draining; I use my alone time to recharge.

For others who may have never previously worked remotely—particularly extroverts who are energized by daily interactions—I can see where this experience would be a struggle. Of course some people simply weren’t set up to work from home. Until a few months ago lots of folks never had the need for a laptop, or for superior cell phone connections and network connectivity, outside of the office.

In my case I further benefit from the fact that my children are grown and my dedicated office workspace has long been in place. I have fast internet, a full-size multi-montior setup and my cherished old school tools: an actual landline and a vintage Model M keyboard.

Of course, 2020 has still been an adjustment for me. In a typical year, I’ll travel to work at client sites as required. If the budget allows it, I’ll also attend a couple of conferences.

From these experiences I do understand the unique value of face to face interactions in business settings. Developing relationships online is certainly doable. I have numerous friends that I only know in the virtual world. However, something can be lost when interactions take place exclusively via email, phone, IM, Webex, or what have you. Taking time for meals, chit chat in the halls, and less formal interactions are often the glue that holds relationships together with those you only see infrequently. For this reason, I’ve always tried to meet with remote team members. I believe that being able to put faces with names and get to know one another better is well worth the effort and expense.

And as we’ve seen, there’s still essential work that can only be done onsite. Data centers still need hands on attention, new hardware still needs to be installed in racks, cables must be pulled, parts must be replaced, etc.

So at this point, I am ready to go back on the road. But I’ll always appreciate being able to work remotely.

Edit: I did not make it 13 more years.

Originally posted July 14, 2020 on AIXchange

While Rob McNelly understands the thinking behind decisions to archive content, he can’t help but dwell on how much we’ve lost over the years as archives across the web have gone away.

Thirteen years ago this week, my first AIXchange post went live for IBM Systems Magazine.
 
You may notice that the above link takes you to RobMcNelly.com, my personal website and archive. Early this year, around the same time IBM developerWorks shut down, IBM Systems magazine redesigned its website.
 
The decision was made to scale back the magazine archives, in part because the numbers showed few views for most of the old articles. Though I understand the thinking behind these decisions—as well as the ever-changing nature of the web itself—I can’t help but dwell on how much we’ve lost over the years as archives across the web have gone away.
 
In my case, I decided to re-home my content, which includes almost a decade’s worth of AIXchange posts from July 2007 through December 2016. And in case you’re wondering, it’s no small task; I’m still in the process of loading my archives.
 
On that note, I’m continually amazed by the number of broken links I find in my old articles. I typically reference related, more detailed technical information, and these supporting docs also get moved around or removed from the web. While I update these links whenever I can, a lot of it cannot be recovered. Incidentally this is why I generally quote from sources I reference; if the supporting link goes away, at least you have an idea of what was written.
 
As forward-thinking as the world of tech is, I do feel a little nostalgic for the words and ideas that have disappeared from the web. And I wonder how often we’re forced to reinvent wheels and solve problems that were already solved because the solutions put forth in old but still relevant articles are no longer online. Sure, things I wrote two, five or 13 years ago may not be read much these days, but I believe they’re still worth preserving. Collectively these short weekly posts represent part of my life’s work, so for that reason alone archiving is worth the effort. Plus, as I’ve mentioned previously, sometimes when trying to resolve a system issue I’ll find the answer in one of my old posts. Seriously, it’s happened more times than I can count.
 
The AIXchange anniversary is important to me. I use the occasion not just to look back but to look ahead. Wherever you are in your career, take a minute to imagine where you plan to be, and envision what computing might look like then. Me? My plan is to keep writing, linking, and archiving. My hope is I’ll still be doing this 13 years from now, and that you’ll still be reading.

Edit: The loss of old information is tragic.

Originally posted July 7, 2020 on AIXchange

While a lot of information is dated, it’s still relevant. Old documentation and archives actually hold answers to a lot of our current questions.

A while ago I was asked about implementing system accounting software:

“The system accounting utility allows you to collect and report on individual and group use of various system resources.

“The accounting system utility allows you to collect and report on individual, group, and Workload Manager (WLM) class use of various system resources.

“This accounting information can be used to bill users for the system resources they utilize, and to monitor selected aspects of the system operation. To assist with billing, the accounting system provides the resource-usage totals defined by members of the adm group, and, if the chargefee command is included, factors in the billing fee.

“The accounting system also provides data to assess the adequacy of current resource assignments, set resource limits and quotas, forecast future needs, and order supplies for printers and other devices.”

Along with that introduction from IBM Knowledge Center, check out this primer, this IBM Redbook, this discussion thread on AIX® auditing, and this lengthy doc on monitoring user activity.

Yes, a lot of that information is quite dated, but it’s still relevant to this topic. Old documentation and archives actually hold answers to a lot of our current questions. Personally I can’t count the number of times I stumbled onto some old post that had precisely the information I was searching for.

Edit: I love reading about this stuff.

Originally posted June 30, 2020 on AIXchange

Rob McNelly makes a toast to early GUI decisions and designs, and the people behind them.

As a fan of computing history—as well as someone who played with a Macintosh in his college computer lab in the late 1980s—I found this 2019 article on Susan Kare, designer of the first Macintosh icons, to be pretty entertaining.

If it wasn’t for needlepoint, the computer graphics we have come to know and love today might have looked a lot different. Pioneering designer Susan Kare was taught by her mother how to do counted-thread embroidery, which gave her the basic knowledge she needed to create the first icons for the Apple Macintosh 35 years ago.

“It just so happened that I had small black and white grids to work with,” she says. “The process reminded me of working needlepoint, knitting patterns or mosaics. I was lucky to have had a mother who enjoyed crafts.”

Kare’s breakthrough designs for the Macintosh, which included the smiling computer at startup, trash can for recycling and a computer disk for saving files, are now commonplace in the digital era. They are so recognizable that they are legendary.

Although Macintosh wasn’t the first GUI, most would argue it was the first time mainstream users used a GUI. Like a lot of you who are my age, I was already comfortable with command line interfaces, so learning to use a mouse was interesting. Single click? Double Click? Drag and drop? If I recall, there were tutorials and programs to help us along. On that note, Susan Kare also designed the card deck for Microsoft’s Solitaire program, which was intended to be a learning aid for the computer mouse.

Here’s a bit more from the Kare article. Be sure to read the whole thing:

Kare devised various ideas and concepts to translate basic commands and procedures into visual cues for users. Thus emerged the trash can, computer disk and document with turned-up page corner—all of which are, in one form or fashion, omnipresent icons for computer functions.

Using graphics on computers was not new but Apple wanted to demystify the operating system so average people would understand intuitively what they needed to do. Early computers tended to be complicated behemoths that were developed for mathematically inclined scientists and engineers.

Honestly, it’s hard to even remember what using a computer was like before the Macintosh. How else would you try to visualize your interface? How could you differentiate the array of functions and file systems to simplify the desktop experience for non-technical users? Can you remember marking up documents in WordPerfect, and then transitioning to Microsoft Word? I remember the first time I used WYSIWYG publishing software with a laser printer; the results were astounding. 

It may be taken for granted, but we owe quite a bit to those early decisions and designs, and the people behind them.

Edit: Have you watched these?

Originally posted June 23, 2020 on AIXchange

Get to know the series of YouTube videos from Nigel Griffiths that he calls AIX in Focus.

You may be interested in this series of YouTube videos from Nigel Griffiths that he calls AIX in Focus.

I love his description:
AIX the best OS in the world. Fast, built to make the best use of POWER based computer with:

• 192 CPU cores and 8192 core threads
• 32 TB of memory
• 64 Gen 4 PCIe adapter slots 

For the system admin guys the commands to make life simple and flexible are second to none and commands are stable—not changing the tools and command every couple of years.

There is also a plenty of hot open source tools recompiled for AIX the tookbox. Plus AIX is the birthplace of nmon and now njmon performance stats tools pulling data from the excellent “perfstat” library.

I choose my bank based on them hosting my bank account on AIX 🙂

This series covers topics like the LVM, smitty, JFS2, JFS2 snapshots, JFS2 with NO log, NIM servers, and much more.

Nigel has made many other vids as well. Check them out on his YouTube main page, and get ready to do a ton of learning.

I suppose videos aren’t for everyone, but I find them very valuable. Not that I rely on any one educational/training platform; I still read plenty, including IBM Redbooks and other documents and articles. Of course nothing beats actual machine time, but beyond that, there’s truly no one way to learn.

Edit: Hopefully this will help someone if they run across the same issue.

Originally posted June 16, 2020 on AIXchange

Having a spare internal disk is handy in general but that doesn’t come without navigating some challenges. Rob McNelly gives the breakdown on both.

When I build a new system, I prefer to have the VIO servers run from internal disks. Sure it’s slower, but it’s a trade-off I’m willing to make. This way I know if I lose the SAN, I can still boot VIOS to conduct some troubleshooting.
 
If the customer is willing, I like to have some spare SAS disks available in the frame as well. That takes the urgency out of replacing a failed disk. We just immediately migrate our data to the currently inactive disk we have available, and then swap out the bad one at our leisure after IBM ships the new disk.
 
Having a spare internal disk is pretty handy in general. During the early stages of racking and stacking systems, while the SAN team is creating and mapping LUNs, an available disk can be used to load a test LPAR or be a place to stage a NIM server, among many other things.
 
For instance, during a recent POWER9 model S922 install, I created a temporary NIM server on an internal disk that I mapped to a client LPAR. My plan was to migrate it to a SAN LUN once they were available. While I’ve done this many times without a hitch, in this instance, running extendvg rootvg hdisk1 (where hdisk1 is my new SAN LUN) triggered this error:
 
#extendvg rootvg hdisk1
0516-1980 extendvg: Block size of all disks in the volume group must be the same. Cannot mix disks with different block sizes. 0516-792 extendvg: Unable to extend volume group.
 
A search for the message brought up this IBM Support doc:
 
“Such an error means you cannot mix a physical volume (PV) of 4 KB block size with PV blocks of other sizes. The block size of all PVs in the volume group must be the same.
 
“This is explicitly mentioned in the man page for the extendvg command.
 
“Unfortunately, there is no way to fix that issue, because AIX, at the present, only supports 4K block sizes on sissas drives. AIX does not support 4K block sizes on fibre-attached storage. There is no option for block-size translation for SAS disks in firmware nor in AIX kernel.
 
“Naturally, that means that you will not be able to include both sissas drive and fibre-attached drives within the same volume group, since the volume group requires all disks within the volume group to utilize the same disk block size.”
 
I was left with a few choices. I could backup my system to a mksysb and restore it from a NIM server using that mksysb. Of course that was problematic in this case, since this was the NIM server I was trying to move from internal disk in the first place. The better option was to bail myself out using the alt_disk_copy command.
  
“The alt_disk_copy command allows users to copy the current rootvg to an alternate disk and to update the operating system to the next maintenance or technology level, without taking the machine down for an extended period of time and mitigating outage risk. This can be done by creating a copy of the current rootvg on an alternate disk and simultaneously applying software updates. If needed, the bootlist command can be run after the new disk has been booted, and the bootlist can be changed to boot back to the older maintenance or technology level of the operating system.”
 
I wasn’t looking to update the software; I only wanted to copy rootvg to my new disk. I ran:
 
            #alt_disk_copy  -B -V -d hdisk1
 
After running a bosboot, I modified my bootlist so it pointed to the new disk and rebooted from the SAN LUN. Once this was done and I was satisfied everything was running as expected on the new LUN, I cleaned up the original rootvg by running:
 
#alt_rootvg_op -X old_rootvg
 
Sometimes in these situations I don’t immediately consider the power we have with AIX®  and IBM Power Systems™ hardware. With alt_disk_copy, I don’t always have to do backup/restore operations, or even migratepv. Of course it may require a reboot, but in this instance it was well worth the time spent.

Edit: Have you run into this issue?

Originally posted June 9, 2020 on AIXchange

What happens when you try to use migratepv to move rootvg to the new LUN and remove the copy of rootvg from the original LUN?

My client added a LUN to an LPAR on the fly using cfgmgr. When we tried to use migratepv to move rootvg to the new LUN and remove the copy of rootvg from the original LUN, we got this error:

0516-1259 mirrorvg: None of the available disks support booting on this specific system or the boot logical volume already exists on the specified disk. If these are new bootable disks and you have not rebooted since configuring them. You may try exporting LVM_HOTSWAP_BOOTDISK=1 and run this command again to override this condition. 0516-1200 mirrorvg: Failed to mirror the volume group.

A web search turned up the following.

On POWER machine systems, a disk is found to be bootable by the system firmware, at IPL time. Each LPAR runs its own private version of the firmware, which probes for all available hardware for that partition, and builds a device tree in NVRAM. When AIX® boots it reads this device tree and runs cfgmgr to configure the devices in AIX.

If a disk is added AFTER boot time, it has usually not been configured correctly by the firmware to allow AIX to see that it is bootable.

Resolving The Problem
1) Run the command below:
# export LVM_HOTSWAP_BOOTDISK=1
# bootinfo -B hdisk1

Then, please confirm the output is 1 then process to mirror the root volume group again.

2) [Restart] the firmware by shutting down the LPAR and powering it off, then reactivating it and rebooting will allow the firmware to probe the adapter and disks again to and mark them as bootable.

If the same issue persists, please contact the IBM AIX Software Support Team.

We tried exporting the LVM_HOTSWAP_BOOTDISK=1 and running the mirrorvg, but it failed again. But once we shut off the LPAR and restarted it, it worked as expected.

Disclaimer: I like sharing these tips with you, the reader, but my future self also appreciates having these out there. That’s because it’s entirely possible I’ll run into a similar dilemma down the line.

Edit: Did you already know about this?

Originally posted June 2, 2020 on AIXchange

A concise, easy to understand explanation of MTUs.

While I wasn’t obsessively desiring to learn about MTUs, I genuinely appreciate how this brief blog post clearly and thoroughly covers an obscure topic:

“The MTU (Maximum Transmission Unit) states how big a single packet can be. Generally speaking, when you are talking to devices on your own LAN the MTU will be around 1500 bytes and the internet runs almost universally on 1500 as well. However, this does not mean that these link layer technologies can’t transmit bigger packets.

“For example, 802.11 (better known as WiFi) has a MTU of 2304 bytes, or if your network is using FDDI then you have a MTU around 4352 bytes. Ethernet itself has the concept of ‘jumbo frames,’ where the MTU can be set up to 9000 bytes (on supporting NICs, Switches and Routers).

“However, almost none of this matters on the internet. Since the backbone of the internet is now mostly made up of Ethernet links, the de facto maximum size of a packet is now unofficially set to 1500 bytes to avoid packets being fragmented down links.

“On the face of it 1500 is a weird number, we would normally expect a lot of constants in computing to be based around mathematical constants, like powers of 2. 1500, however fits none of those.

“So where did 1500 come from, and why are we still using it? The ‘Ethernet: Distributed Packet Switching for Local Computer Networks’ paper from 1980 is an early note of the efficiency cost analysis of larger packets on a network. This being especially important to Ethernet at the time, since Ethernet networks would ether be sharing the same coax cable between all systems, or there would be Ethernet hubs that would only allow one packet at a time to be transmitted around all members of the Ethernet segment.

“A number had to be picked that would mean that transmission latency on these shared (sometimes busy) segments would not be too high, but also that packet header overhead would not be too much…. It would seem at best that the engineers at the time picked 1500 bytes, or around 12000 bits as the best ‘safe’ value.

“Since then various other transmission systems have come and gone, but the lowest MTU value of them has still been ethernet at 1500 bytes. Going bigger than lowest MTU on a network will either result in IP fragmentation, or the need to do path MTU detection. Both of which have their own sets of problems. Even if sometimes large OS vendors dropped the default MTU to even lower at times.”

Such a concise, easy to understand explanation. Read the whole thing.

Edit: To be clear the lppsource and spot were from the new AIX level, you do not use a spot from the mksysb in order to get this method to work. And it still works great with 7.1 and 7.2 if you need to have a more recent version of AIX to get it to run on POWER9.

Originally posted May 26, 2020 on AIXchange

The system software map comes in handy when a customer that was running AIX 6.1 on a POWER5 server wanted to migrate their LPAR to POWER9.

Recently a customer that was running AIX 6.1 on a POWER5 server wanted to migrate their LPAR to POWER9*. They were running 6100-09-09-1717 and rather than perform an update on the source LPAR, they preferred to take a mksysb and simply run it on their POWER9 system.

They consulted the system software map to learn the minimum levels needed to run AIX 6.1 on POWER9:

Technology Level: 6100-09
Base Level: 6100-09-11
Recommended Level: 6100-09-12-1846
Latest Level: 6100-09-12-1846

So how could we take their mksysb from 6100-09-09-1717 to the recommended/latest level? I reached out to IBMer Chris Gibson, who said they could migrate their mksysb on the fly.

You can restore directly the mksysb to the latest TL, if you have a nim server use the command below:

nim -o bos_inst -a source=mksysb -a lpp_source=<lpp_source> -a spot=<SPOT> -a mksysb=<mksysb> -a image_data= mksysb_image_data -a accept_licenses=yes client_hostname

My customer wisely has extended support from IBM for AIX 6.1. This provides access to download media from Entitled Systems Support (ESS).

The last level available for AIX 6.1 base media was 6100-09-12-1837, so they downloaded that and made it available on their NIM server as the lppsource and the spot. Then they ran the command, which restored and then updated their mksysb on the fly. Once it booted up, we were able to download the latest fixes and update it to 6100-09-12-1846.

It’s quite a testament to AIX that an OS version that GAed in 2007 can run on the latest hardware—not to mention that the whole migration/updating process can be done on the fly.

You might not run into this sort of thing every day, but if you ever need to do something similar, hopefully you’ll recall this post.

Edit: Did any of you contribute spare cycles?

Originally posted May 19, 2020 on AIXchange

Alternatives to running SETI@home.

Many years ago as a young-ish administrator, I ran SETI@home. There were clients for different operating systems, including AIX. It was easy to see how different processors affected the speed of completing work units.
 
As of March 30, SETI@home no longer sends data for clients to work on, but if you still enjoy using your spare machines, or have considered setting up a spare machine for distributed computing like this, they suggest some alternatives:
 
“… we encourage you to continue donate computing power to science research—in particular, research on the COVID-19 virus. The best way to do this is to join Science United and check the Biology and Medicine box.”
 
As noted on the Science United site:
 
“BOINC is the preeminent platform for volunteer computing (VC). It is used by most VC projects, including SETI@home, Einstein@home, Climateprediction.net, IBM World Community Grid, and Rosetta@home.
 
“Science United is a new way to participate in BOINC. With Science United, computer owners volunteer for science areas rather than for specific projects. Science United assigns computers to appropriate projects; these assignments may change over time.
 
“We call this the coordinated model for VC. It has the advantage that new projects can get computing power without having to do their own publicity and volunteer recruitment. The goal is to provide the power of VC to thousands of scientists, rather than a few dozen as was previously the case. …
 
“The user interface of Science United is designed to appeal to a wider audience than the current BOINC user base, which is mostly male and tech-savvy. For example, Science United has no leader boards. …
 
“Science United is also intended to serve as a unified “brand” for VC, so that it can be marketed more effectively.”
 
Another project, Folding@home, does similar work with spare cycles:
 
“What is distributed computing?… the calculations we wanted to do would take about a million days on a fast processor. So it seemed natural that we might be able to get it done in 10 days if we had access to 100,000 processors. By using distributed computing, we can split up the simulation, run each piece through a computer, and then combine them together afterwards. This really sped up our results.”
 
If you’ve ever wondered what you can do to help, this is something to consider.

Edit: Still good stuff

Originally posted May 12, 2020 on AIXchange

Tips for technical documentation like IBM Redbooks, Power Systems Virtual User Group replays. There’s always something to learn, or relearn.

I’m always interested in hearing how others configure and maintain their systems. Sure, many lessons can be learned by trial and error in the test lab, but through my years in tech, I’ve gained the most by listening to those around me. I try to soak up all information I can, whether it’s random scripts and tips and anecdotes, or technical documentation like IBM Redbooks, Power Systems Virtual User Group replays and the perspectives of IBM Champions and AIX experts on Twitter. There’s always something to learn, or relearn.

With this in mind, I want to note that the latest version of the IBM POWER Virtualization Best Practices Guide (version 4.0) came out in March. While it’s a fairly short document, it’s a dense read—but it’s definitely worth your time.

Here’s the table of contents:

1 INTRODUCTION
2 VIRTUAL PROCESSORS
            2.1 Sizing/configuring virtual processors
            2.2 Entitlement vs. Virtual processors
            2.3 Matching entitlement of a LPAR close to its average utilization for better performance
            2.4 When to add additional virtual processors
            2.5 How to estimate the number of virtual processors per uncapped shared LPAR
3 AIX VIRTUAL PROCESSOR MANAGEMENT -PROCESSOR FOLDING
            3.1 VPM folding example
            3.2 Tuning Virtual Processor Management Folding
            3.3 POWER7/POWER7+/POWER8/POWER9 Folding
            3.4 Relationship between VPM Folding and PHYP dispatching
4 AIX PERFORMANCE MODE TUNING
            4.1 Processor Bindings in Shared LPAR
5 LPAR PAGE TABLE SIZE CONSIDERATIONS
6 ASSIGNMENT OF RESOURCES BY THE POWERVM HYPERVISOR
            6.1 PowerVM Resource assignment ordering
            6.2 Overview of PowerVM Hypervisor Resource Assignment
            6.3 How to determine if a LPAR is contained within a chip or drawer/Dual Chip Module (DCM)
            6.3.1 Displaying Resource Assignments for AIX
            6.3.2 Displaying Resource Assignments for Linux
            6.3.3 Displaying Resource Assignments for IBM i
            6.4 Optimizing Resource allocation for affinity
            6.5 Optimizing Resource Assignment –Dynamic Platform Optimizer
            6.6 Affinity Groups
            6.7 LPAR_placement=2
            6.8 LPAR_placement considerations for failover/Disaster recovery
            6.9 Server Evacuation Using Partition Mobility
            6.10 PowerVM Resource Consumption for Capacity Planning Considerations
            6.11 Licensing resources (COD)
7 ENERGY SCALE
8 PROCESSOR COMPATIBILITY MODE
9 CONCLUSION
10 REFERENCES

The references consist of a number of links to various articles and documents, including a piece I wrote on setting up LPARs. Note that the link in the doc no longer works, but that article (“An LPAR Review”) is preserved in my archive. Even though it was originally published in 2009, the information is still relevant. But be sure to pull up the doc and check out all the references, because you’ll find lots of good information.

Edit: Does this interest you?

Originally posted May 5, 2020 on AIXchange

On Twitter, IBM’s David Spurway announced information for IBM Visual Insights V1.2.

On Twitter, IBM’s David Spurway posted announcement information for IBM Visual Insights V1.2:
 
IBM Visual Insights V1.2, previously called IBM PowerAI Vision, extends support to GPU-accelerated AI software on x86-based servers.
 
To accommodate the diversity of infrastructures used for AI solutions, IBM has expanded support of its award-winning software beyond POWER architectures to include Intel platforms. To avoid confusion in the marketplace, IBM PowerAI Vision has been renamed IBM Visual Insights.
 
In addition to its current functions, IBM Visual Insights V1.2 now offers:
 
* Support on x86-based servers with GPUs for training and inference
* Availability on IBM Cloud as a client-managed service
 
The announcement letter also includes specific hardware and software requirements. This solution runs in POWER9 and POWER8 environments:
 
            * Power System S822LC (8335-GTB) for HPC servers
            * Power System AC922 (8335-GTG and 8335-GTH) for HPC servers
            * Power System IC922 (9183-22X)
            * x86-based servers
            * Minimum system RAM 128 GB
 
            * Red Hat Enterprise Linux 7.6-Alt (POWER9)
            * Red Hat Enterprise Linux 7.7 (POWER8 and x86)
            * Ubuntu Server 18.04
            * NVIDIA GPU driver

Edit: Did you attend any of these?

Originally posted April 28, 2020 on AIXchange

In light of the current worldwide circumstances, IBM and other vendors are moving their conferences and events to the digital realm.

In light of the current worldwide circumstances, IBM and other vendors are moving their conferences and events to the digital realm. Considering the registration and traveling expenses that are typically required to attend these events, you may find it well worth your time to check them out online.
 
For instance, IBM Think 2020 is now a digital event, with sessions running May 5-6. Register for a “free pass.”
 
And IBM’s TechU spring sessions are already underway, and continue till July 2. Choose from one of three educational tracks: IBM Power Systems, IBM Storage and IBM Z/LinuxONE. Through replay sessions and presentations decks, you can catch up on any session that’s already taken place.
 
Here are some other events of interest:
 
* Red Hat Summit 2020, April 28-29
 
* Cisco Live, June 2-3
 
* VMware Empower (online events for various geographic regions run from May-July)
 
If I left anything out, please let everyone know in the comments.

Edit: This is still the only way to do HMC upgrades.

Originally posted April 21, 2020 on AIXchange

A guide for remote upgrades on the HMC.

I love that it’s possible to do remote upgrades on the HMC. This isn’t anything new of course, so imagine my shock when I was recently in a conversation with someone who didn’t realize that remote HMC upgrades can be done via the network.
 
Happily, there’s a lot of good documentation about remote upgrades, and HMC upgrades in general. Let’s start with IBM Support. Here’s how to upgrade to Version 8.8.7, and here’s how to upgrade x86 or Model 7042 versions to Version 9. Both docs provide detailed information about various methods, so they’re helpful whether you’re using the network or some DVDs.
 
Here are directions for getting updates for your HMC. The IBM Knowledge Center has a deeper dive into this same topic.
 
Finally, here are a couple of things from my archive. In 2011 I wrote about remote upgrades. Spoiler: I loved them back when, too—so much so that I revisited the topic back in 2013.
 
Yes, a lot of this information is dated, but it remains applicable. Plus a refresher never hurts.

Edit: Still good to know.

Originally posted April 14, 2020 on AIXchange

What to do when your AIX system has an altinst_rootvg.

Awhile back this interesting question came up: If you’re using alt disk to clone and upgrade your AIX system and later on you log into that system and see it has an altinst_rootvg, how do you determine the AIX version that’s on the inactive disk? What if you want to look at old data or information on that disk?
 
The answer is explained here:
 
Test2:/# oslevel -s
7100-01-04-1141
Test2:/#
 
Test2:/# lspv
hdisk0          00d342e7131c6b47                rootvg        active
hdisk1          00d342d637j21a59                 altinst_rootvg
Test2:/#
 
Test2:/# alt_rootvg_op -W -d hdisk1
Waking up altinst_rootvg volume group …
 
Test2:/#
 
Now the altinst_rootvg is in Active state also alt filesystems are in mounted state on the server
 
Test2:/# lspv
hdisk0         00d342e7131c6b47                 rootvg        active
hdisk1         00d342d637j21a59                  altinst_rootvg  active
Test2:/#
 
Test2:/# df
Filesystem    512-blocks      Free %Used  Iused %Iused Mounted on
/dev/hd4           524288      380024 28%    3091     3% /
/dev/hd2         3801088      396856 90%    34020    8% /usr
/dev/hd9var    2621440     2279336 14%    3560     2% /var
/dev/hd3          524288      499272    5%     105      1% /tmp
/dev/hd1          524288      507336    4%     102      1% /home
/proc                –              –    –          –       –  /proc
/dev/hd10opt    524288        278872    47%       3370     6% /opt
/dev/alt_hd4     524288        365552    31%       3871     3% /alt_inst
/dev/alt_hd1     524288        507336    4%         104      1% /alt_inst/home
/dev/alt_hd10opt 1310720   562888   58%        5694    4% /alt_inst/opt
/dev/alt_hd3     524288        499120    5%         116      1% /alt_inst/tmp
/dev/alt_hd2     5636096      184120   97%     103336   15% /alt_inst/usr
/dev/alt_hd9var 2621440    1835656  30%       6632      3% /alt_inst/var
 
We need to start the chroot shell within the alternate rootvg to identify the OS level/TL/SP information.
 
Test2:/# chroot /alt_inst /usr/bin/ksh
Test2:/# oslevel -s
7100-01-01-1216
Test2:/#
Test2:/# exit
 
You can return to the rootvg environment by exit from the alt shell.
 
Now it is really really important to put the cloned rootvg back to sleep.
 
Test2:/# alt_rootvg_op -S altinst_rootvg
Putting volume group altinst_rootvg to sleep …
forced unmount of /alt_inst/var
forced unmount of /alt_inst/usr
forced unmount of /alt_inst/tmp
forced unmount of /alt_inst/opt
forced unmount of /alt_inst/home
forced unmount of /alt_inst
Fixing LV control blocks…
Fixing file system superblocks…
 
Related: The IBM Knowledge Center explains how to access data between the original rootvg and the new alternate disk and provides this intro to the alt_rootvg_op command. And in an AIXchange post from August 2019, I shared Chris Gibson’s tips on managing multiple instances of alt_rootvg.

Edit: When was the last time you had to worry about this?

Originally posted April 7, 2020 on AIXchange

Rob McNelly walks through customer troubleshooting from an IBM Redbook.

A client was looking to repurpose and virtualize an old TS3200 tape library. The idea was to connect it to a VIO server and then assign it to their IBM i clients.

However, errors came up when running cfgdev:

            cfgmgr: 0514-621 WARNING: The following device packages are required for device support but are not currently installed.
            devices.sas.changer

After downloading atape drivers from IBM Fix Central and loading them on the LPAR, the errors went away. An scm0 and some rmt* devices could be seen.

            # lsdev -Cc tape
            rmt0 Available 06-00-00 IBM 3580 Ultrium Tape Drive (SAS)
            rmt1 Available 06-00-00 IBM 3580 Ultrium Tape Drive (SAS)
            rmt2 Available 06-00-00 IBM 3580 Ultrium Tape Drive (SAS)
            rmt3 Available 06-00-00 IBM 3580 Ultrium Tape Drive (SAS)
            smc0 Available 06-00-00 IBM 3573 Library Medium Changer (SAS)

However, more errors emerged with an attempt to use mkvdev to assign the drive to the client LPAR. It took some searching to sort it all out.

First, this tip (scroll down):

You can virtualize SAS tape drives, but only per-LPAR. Meaning, on VIOS it shows Available, then you “mkvdev -vdev rmt0 -vadapter vhostx -dev vrmt0” as padmin ID. This will work ONLY if no atape is installed on VIOS.

Then this reminder from IBM Support:

Note 1: The VIOS cannot use Atape drive, such as the 3580, when attempting to virtualize a SAS tape drive to AIX client.

And finally, another forum comment:

… You cannot virtualize the 3100. In fact, you cannot virtualize any medium changer. You have to get rid of the virtual layer and assign a dedicated adapter to the LPAR and connect the 3100. Then you will have rmt0 and smc0.

Also, do not install Atape drivers on VIOS. This causes havoc on your virtual layer. If Atape is installed, then you will not be able to share the tape drive by mapping it to LPARs.

For completeness, refer to this IBM Redbook.

In this case, once they removed the devices, removed the drivers, reran cfgdev and then reran the mkvdev commands, it worked as expected.

I realize only a handful of people will ever deal with this issue, but I believe whenever a problem is solved, the solution is worth sharing.

Edit: I still recommend following them.

Originally posted March 31, 2020 on AIXchange

Today’s post features a trio of valuable technical insights, courtesy of IBMers on Twitter.

Chris Gibson alerted me to this IBM technical bulletin (excerpt below) that impacts those running VIO server 3.1.1 and using N_Port ID virtualization (NPIV).

“This is only an issue using NPIV with VIOS 3.1.1. Client LPARs can experience I/O hangs and timeouts under heavy I/O load. We have seen this issue primarily during database backups, but can occur with any large sequential I/Os at high rates.

Affected VIOS Levels and Recommended Fixes
Minimum Affected Level: VIOS 3.1.1.0 devices.vdevice.IBM.vfc-server.rte 7.2.4.0
Maximum Affected Level: VIOS 3.1.1.10 devices.vdevice.IBM.vfc-server.rte 7.2.4.0
Fixing Level: VIOS 3.1.1.20 IJ23222

Interim Fix: iFix

Note: Installation of these fixes requires a reboot.”

On Twitter, Chris noted the summ tool (excerpt below), which is used to summarize and decode AIX I/O error messages:

“DESCRIPTION
“summ” is an AIX only diagnostic tool used to decode fibre-channel and SCSI disk AIX error report entries. It is an invaluable tool that can aid in diagnosing storage array or SAN fabric related problems providing the source of the error.

The script generates single line error messages enhancing the readability of the AIX error report. This is the same tool used by IBM Support worldwide and is considered safe to run in a production environment.

USAGE
The summ command can process results of the AIX error log from a file or from standard input…”

Also from Twitter, Gareth Coates cited his article (excerpt below) that lays out the merits and caveats of connecting baseboard management controller (BMC) systems to your HMCs:

“Summary

Some IBM Power Systems have a flexible service processor (FSP) and these systems are generally connected to, and managed by one or two Hardware Management Consoles (HMCs).

Other Power Systems have a Baseboard Management Controller (BMC) and can be controlled in a number of ways, including the Intelligent Platform Management Interface (IPMI) and the web-based graphical user interface (GUI). It is also possible to connect the system to an HMC. This article discusses the merits of doing so.”

Read the articles, and if you’re on Twitter, follow Chris and Gareth. Both consistently provide valuable technical insights.

Edit: I still wonder if I was the only one that did not know.

Originally posted March 24, 2020 on AIXchange

Want remote access to important customer-based service information? It’s actually just a QR code away.

I had to laugh at Kiran Tripath’s Tweet about quick response (QR) code locations. I can’t believe I’ve never actually tried to use those stickers to look up information.

The codes themselves enable remote access to customer-based service information, including details about problem analysis, parts removal and replacement, error codes, firmware license keys and service videos. In all seriousness, the stickers are quite useful.

Here’s a summary of QR code locations:

• For codes 8335-GTC, 8335-GTG, 8335-GTH, 8335-GTW and 8335-GTX, the QR code label is located on the right flange located on the front of the EIA rack unit.
• For codes 9006-12P, 9006-22C and 9006-22P, the QR code label is located on the top service cover. The system must be in the service position to view the label.
• For codes 9008-22L, 9009-22A, 9009-41A and 9009-42A, the QR code label is located on the top of the server on the upper-right corner.
• For codes 9223-22H and 9223-42H, the QR code label is located on the top of the server on the upper-right corner.
• For code 9080-M9S, the QR code label is located on the front service card.
• For code 9040-MR9, the QR code label is located on the right flange located on the front of the EIA rack unit.
• For ESLL and ESLS storage enclosures, the QR code label is located on the left side at the rear of the system.
• For the EMX0 PCIe Gen3 I/O expansion drawer, the QR code label is located on the front service card.

The IBM Knowledge Center has a list of QR code locations in table form.

Assuming you’re on the raised floor and have phone service, you can simply take a picture of the QR sticker that is physically on the server and learn what you need to know in an instant. It’s just one more option for gaining access to relevant information.

So have you all known about this for years and I’m just late to the party?

Edit: There are still people that do not know about these changes.

Originally posted March 17, 2020 on AIXchange

Reminder: Support will soon end for x86-based HMC devices. Now is the time to migrate your data.

A coworker alerted me to this end of service notice for x86-based HMC devices (excerpt below):

“This is the last release to support the 7042 machine type. HMC V9R2 will support the 7063 machine type and Virtual HMC Apliances (x86/ppc64le) only.

“Note: iFixes and Service packs on top of V9 R1 M940 will be supported on 7042 machine types.”

I’ve previously discussed the big changes to the HMC, but as more of you move to POWER9 hardware, here’s a reminder: It’s time to migrate from the older HMCs that still reside in your environments.

While we’re on this topic, I was recently asked about adding a user ssh key to an HMC. The process is detailed in this IBM Knowledge Center doc (excerpt below):

“To enable scripts to run unattended between an SSH client and an HMC, complete the following steps:

1. Enable remote command execution….
2. On the client’s operating system, run the SSH protocol key generator. To run the SSH protocol key generator, complete the following steps:

a. To store the keys, create a directory that is named $HOME/.ssh (either RSA or DSA keys can be used).
b. To generate public and private keys, run the following command:
ssh-keygen -t rsa

The following files are created in the $HOME/.ssh directory:
private key: id_rsa
public key: id_rsa.pub

The write bits for both group and other are turned off. Ensure that the private key has a permission of 600.

On the client’s operating system, use ssh and run the mkauthkeys command to update the HMC user’s authorized_keys2 file on the HMC by using the following command:

ssh hmcuser@hmchostname mkauthkeys -–add <the contents of $HOME/.ssh/id_rsa.pub>

Note: Double quotes (“) are used in commands to ensure that the remote shell can properly process the command. For example:

ssh “mkauthkeys hscuser@somehmchost –add ‘ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABAQDa+Zc8+hn1+TjEXu640LqnVNB+UsixIE3c649Cgj20gaVWnFKTjcpWVahK/duCLac/zteMtVAfCx7/ae2g5RTPu7FudF2xjs4r+NadVXhoIqmA53aNjE4GILpfe5vOF25xkBdG9wxigGtJyOKeJHzgnElP7RlEeOBijJDKo5gGE12NVfBxboChm6LtKnDxLi9ahhOYtLlFehJr6pV/lMAEuLhd6ax1hWvwrhf/h5Ym6J8JbLVL3EeKbCsuG9E4iN1z4HrPkT5OQLqtvC1Ajch1ravsaQqYloMTWNFzM4Qo5O3fZbLc6RuJjtJv8C5t4/SZUGHZxSPnQmkuii1z9hxt hscpe@vhmccloudvm179′”

To delete the key from the HMC, you can use the following command:
ssh hmcuser@hmchostname mkauthkeys –remove joe@somehost

To enable passwords that prompts for all hosts that access the HMC through SSH, use the scp command to copy the key file from the HMC: scp
hmcuser@hmchostname:.ssh/authorized_keys2 authorized_keys2

Edit the authorized_keys2 file and remove all lines in this file and then, copy it back to the HMC: scp authorized_keys2 hmcuser@hmchostname:.ssh/authorized_keys2″

There’s also supporting information about enabling remote command execution and using the HMC remote command line (excerpt below):

“You can use the command line interface in the following situations:

• When consistent results are required. If you have to administer several managed systems, you can achieve consistent results by using the command line interface. The command sequence can be stored in scripts and run remotely.
• When automated operations are required. After you have developed a consistent way to manage the managed systems, you can automate the operations by invoking the scripts from batch-processing applications, such as the cron daemon, from other systems.”

Edit: Have you been patching without rebooting?

Originally posted March 10, 2020 on AIXchange

Both operating systems offer ways to do limited patching without having to reboot the system.

AIX admins often manage Linux servers as well. If this fits your job description, you should know that each operating system has an option that allows limited patching to be done without a reboot. The AIX version is called Live Update; the Red Hat version is called Live Patching.

From the AIX site:
“Starting with AIX Version 7.2, the AIX operating system provides the AIX Live Update function that eliminates the workload downtime that is associated with AIX system restart that is required by previous AIX releases when fixes to the AIX kernel are deployed. The workloads on the system are not stopped in a Live Update operation, yet the workloads can use the interim fixes after the Live Update operation.

“IBM delivers kernel fixes in the form of interim fixes to resolve issues that are reported by customers. If a fix changes the AIX kernel or loaded kernel extensions that cannot be unloaded, the host logical partition (LPAR) must be restarted. To address this issue, AIX Version 7.1, and earlier, provided concurrent update-enabled interim fixes that allow deployment of some limited kernel fixes to a running LPAR. All fixes cannot be delivered as concurrent update-enabled interim fixes. Starting with AIX Version 7.2, you can use the Live Update function to eliminate downtime that is associated with the AIX kernel update operation. This solution is not constrained by the same limitations as in the case of concurrent update enabled interim fixes.”

This is from Red Hat:
“RHEL 8.1 marks the first release of RHEL 8 that will receive live kernel patches for critical and selected important CVEs, and no premium subscription is required. They will be delivered via the regular content stream and can be consumed via Yum updates. (Previously, these were on request for premium subscription customers and “hand delivered.”) The goal of the program is to minimize the need to reboot systems in order to get the latest critical security updates.”

For more, check out this Red Hat video and this discussion of AIX Live Update methodology. Chris Gibson has a best practices guide and presentation slides, and on March 25 you can take in his Power VUG session on Live Update best practices. 

Of course these are different tools for different operating systems, but take a moment to consider them in tandem: We continue to advance in a direction where more fixes can be applied on the fly. While I don’t imagine we’ll ever see a world completely free of reboots, this is welcome progress.

Edit: Did you get up to speed on the server?

Originally posted March 3, 2020 on AIXchange

IBM unveiled the Power Systems IC922 server last month. Here’s a handy list of where to learn more about it.

IBM unveiled the Power Systems IC922 server last month. For those who haven’t had time to dig into the details, here’s an information roundup.

Videos

* Nigel Griffiths and Gareth Coates webinar
* First look with Nigel Griffiths
* Five fast facts

Announcement Presentations and Information

* Slides: Part one and part two. Note: These slides track with the webinar video.

* Data sheet (excerpt below):

“IBM Power System IC922 provides the compute-intensive and low-latency infrastructure needed to unlock business insights from trained AI models. POWER9-based, Power IC922 provides advanced interconnects (PCIe Gen4, OpenCAPI) to support faster data throughput and decreased latency. Accelerated, the Power IC922 supports up to six NVIDIA T4 GPUs.”

* Announcement letter (excerpt below):

“The IBM Power System IC922 (9183-22X) server is a powerful new two-socket server designed for high-performance data analytics and computing workloads to help meet your accelerated computing needs.

“The Power System IC922 server offers:
* Two IBM POWER9 processor single-chip module (SCM) devices that provide high performance with 24, 32, or 40 fully activated cores and 2 TB maximum memory TPM2.0 for secure and trusted boot
* Up to 24 2.5-inch SAS3/SATA drives via 3 optional drive backplanes (each supporting 8 drives) and several optional SAS3/SATA adapters
* Up to six NVIDIA T4 GPU accelerators
* Two integrated USB 3.0 ports in rear and one USB 3.0 in front
* Two hot-swap and redundant power supplies: 2000 W 220 V AC with C20 receptacle
* Six hot-swap and redundant fans
* 19-inch rack-mount hardware with full rails and cable management arm included
….

“The Power System IC922 server supports two processor sockets, offering 12-core (2.8 GHz base frequency), 16-core (3.35 GHz base frequency), and 20-core (2.9 GHz base frequency) POWER9 technology-based system configurations in a 19-inch rack-mount with a 2U (EIA units) drawer configuration.

“The Power System IC922 server provides two hot-swap and redundant power supplies and 32 DIMM memory slots. Sixteen GB (#EM62), thirty-two GB (#EM63), and sixty-four GB (#EM64) supported memory features allow for a maximum system memory of 2 TB. Memory bandwidth is maximized with 16 DIMM memory slots populated.

Multiple I/O options in the system, including:
* Two PCIe x16 3.0 FHFL slots (supports double-wide accelerator)
* Two PCIe x16 4.0 LP slot
* Two PCIe x8 3.0 FHFL slots (physically x16)
* Two PCIe x8 3.0 FHFL slots
* Two PCIe x16 3.0 LP slots
* Up to 24 2.5-inch SAS3/SATA drives via 3 optional drive backplanes (each supporting 8 drives) and several optional SAS3/SATA adapters.”

Additional Coverage

* IBM Redbook: IBM Power System IC922―Technical Overview and Introduction
* IBM IT Infrastructure blog: “Complete your AI puzzle with inference”
* IBM Marketplace IC922 summary (excerpt below):

“Engineered for AI inference, the IBM Power System IC922 provides the compute-intensive and low-latency infrastructure you need to unlock business insights from trained AI models. The POWER9-based Power IC922 provides advanced interconnects (PCIe Gen4, OpenCAPI) to support faster data throughput and decreased latency. Accelerated, the Power IC922 supports up to six NVIDIA T4 Tensor Core GPU accelerators.”

Take the time to go through this material and you’ll be an IC922 expert in no time.

Edit: Have you signed up yet?

Originally posted February 25, 2020 on AIXchange

Learn what’s new and on the horizon in the world of storage through videos produced by the IBM Washington Systems Center.

Are you familiar with the IBM Washington Systems Center? This group produces video presentations that cover a wide range of storage topics.

Check out their YouTube page. And―for now―you’ll find presentation abstracts on the developerWorks Connections Platform (excerpt below):

“Welcome to the IBM North America Washington Systems Center-Storage Accelerate with IBM Storage program. This blog is for our program of technical webinars covering a variety of storage topics. Please subscribe to this blog or join our mailing list to hear about upcoming technical webinars. To join our mailing list please send an email to accelerate-join@hursley.ibm.com.”

Here are abstracts from the most recent meetings (also excerpted from that page):

[Jan. 21] Cisco/IBM c-type SAN Analytics
As Storage Area Networks continue to evolve so does the need to understand how and what is causing issues. This session will help introduce the newest Analytics features of IBM C-type. This technology as included in the 32Gbps platform provides the visualization that can answers to many of the questions that administrators are forced to answer every day. This SAN Analytics data is driven to provide visibility of the next generation high speed connectivity deployments which include 32Gbps Fibre Channel carrying both SCSI and NVMe traffic. This presentation will explain how to leverage the inbuilt always on technology of C-type SAN Analytics. We will discuss how to architect, deploy and use the data to solve real world issues that every administrator faces from differing IO sizes, Exchange Completion times, storage device response times, and slow drain information as well.

[Jan. 14] Brocade/IBM b-type SAN Modernization
Do you need to modernize your Storage Network? Be Ready for what’s New and Beyond. Storage technologies are now based on silicon … not spinning disks … and massive changes are underway (NVMe over Fibre Channel being just the first step).

These changes in storage will place pressure squarely on the storage network so you must be ready with a modern SAN infrastructure to ensure optimum performance from your applications. If not, then bottlenecks move into the network, slowing your workloads. IBM b-type Gen 6 SANs are the proven modern infrastructure ready for storage today and as it evolves. Learn about the products and powerful tools that will assess and prepare your network for what is coming over the next decade.”

Although most of us concentrate on servers and operating systems, all of our servers connect to storage. That’s incentive enough to learn about new function and capabilities, and discover what’s on the horizon.

Edit: Have you tried this yet?

Originally posted February 18, 2020 on AIXchange

A new PowerVM capability called Virtual Persistent Memory (vPMEM) provides a fast restart of workloads during outages in SAP HANA environments.

Jay Kruemcke recently examined a new PowerVM capability called Virtual Persistent Memory (excerpt below):

“One of the drawbacks of in-memory databases is the amount of time required to load the data from disk into memory after a restart. In one test with an 8TB data file representing 16TB database, it took only six minutes to shut down SAP HANA, but took over 40 minutes to completely load that database back into memory. Although the system was up and responsive much more quickly, it took a long time for all data to be staged back into memory….

“In October 2019, IBM announced Virtual Persistent Memory (vPMEM) with PowerVM. Virtual Persistent Memory isn’t a new type of physical memory but is a way to use the PowerVM hypervisor to create persistent memory volumes out of the DRAM that is already installed on the system. vPMEM is included at no additional charge with PowerVM.

“The data in memory is persistent as long as the physical Power server is not powered off. By maintaining data persistence across application and partition restarts, it allows customers to leverage fast restart of a workload using persistent memory. An added benefit is that there is no difference in application performance when using vPMEM because the underlying DRAM technology is the same as for the non-persistent
memory.

“Although vPMEM does not preserve memory across a server power down, most IBM Power customers seldom power off their systems because of the many reliability features that are built into Power hardware. vPMEM provides for fast restart in the vast majority of planned maintenance and unplanned outages without compromising the performance of HANA during normal use.”

The excerpt below comes from IBM’s Asim Khan, program director, offering management for SAP HANA on POWER:

“The feedback we received from our HANA on POWER clients was that Power Systems clients do not face the challenge of frequent hardware-related outages. In fact, one large SAP HANA client told us they haven’t had to reboot their Power System for 30 months straight due to any hardware outages. The Forrester TEI Study of IBM Power Systems for SAP HANA determined an average reduction of 48 hours of annual downtime after customers move from non-POWER environment to Power Systems. What POWER clients say they want is a solution that helps to fast restart the environment when there’s a software-related planned (patch the OS or SAP environment) or unplanned outage in their SAP HANA environment.”

As you evaluate SAP HANA, hopefully this technology factors into your decision-making calculus. And if you want to get deeper into vPMEM, check out this IBM Techdocs Library whitepaper by Jim Nugen and Olaf Rutz. Related docs are also available.

Edit: What a loss to the community.

Originally posted February 11, 2020 on AIXchange

Today’s post is a friendly (and important) reminder that the developerWorks Connections platform is set to permanently shut down on March 31.

You’re probably aware that IBM is sunsetting the developerWorks Connections platform.

Hopefully you’ve bookmarked the new locations of your favorite AIX content. In my case, that includes Nigel Griffiths, Gareth Coates, Chris Gibson and the AIX Virtual User Group.

While many of the old links still work, the Connections platform is set to permanently shut down on March 31, 2020. If all this is news to you, be sure to read the IBM FAQ (excerpted below):

NEW: On January 2, 2020, the majority of the developerWorks Connections platform will be taken offline and made unavailable. If your Connections group is not active, no further action is needed.

A small number of communities that have requested a sunset extension will be kept online and placed in read-only mode. You will receive an error message if you try to save new content (wikis, posts, forum messages) after that date. Affected pages will have a banner at the top identifying it as a target for final removal on March 31, 2020. This removal includes all community blogs, wikis, forums, activities and files….

Q. Why are these Connections pages going away?
A. IBM is consolidating major content portals to improve the customer experience.

Q. What specific content of mine will be impacted and removed?
A. Your community and its app will no longer be available including: Activities, blogs, files, forums and wikis.

All URLs starting with https://ibm.com/developerworks/community/ and including:

• https://www.ibm.com/developerworks/community/groups/*
• https://www.ibm.com/developerworks/community/wikis/*
• https://www.ibm.com/developerworks/community/activities/*
• https://www.ibm.com/developerworks/community/blogs/*
• https://www.ibm.com/developerworks/mydeveloperworks/blogs/*
• https://www.ibm.com/developerworks/community/files/*
• https://www.ibm.com/developerworks/community/forums/*
• https://www.ibm.com/developerworks/community/profiles/*
• https://www.ibm.com/developerworks/community/homepage/*

Q. What will happen to the information that is currently published on developerWorks Connections? Where will I be able to find it?
A. The majority of existing content and posts will be reviewed by the content owners and moved to the appropriate IBM website… Backups of all Connections content will be made prior to January 2 as a precaution. Select URL redirects will be created to help customers find content in its new home if it has been moved.

Edit: You should be able to bypass the registration

Originally posted February 4, 2020 on AIXchange

Published last month, the new IBM Redbooks publication covers security and networking enhancements, virtualization and cloud capabilities, PowerVM features—plus plenty more.

Glad to have come across this current Redbooks pulication, titled “IBM AIX Enhancements and Modernization”:

This publication, which was finalized in January, is a reassuring reminder of the ongoing work that’s bringing significant new and enhanced features and function to my favorite operating system.

The Redbooks publication covers security and networking enhancements, virtualization and cloud capabilities, AIX and PowerVM features, disaster recovery and high availability and plenty more, including a handy chapter on AIX fundamentals.

For some specifics, Chapter 1, General Enhancements, features these topics: Live Update; Server Flash Caching; Multipath I/O; iSCSI software initiator; Network Installation Manager; Logical Volume Manager; JFS2; Multiple alternative disk clones; Active Memory Expansion; nmon current processor frequency reporting; National language support; and AIX Toolbox for Linux Applications.

The Security Enhancements chapter describes recent IBM AIX security enhancements, including AIX trusted execution, AIX secure boot, multi-factor authentication, “Cryptolibs” and address space layout randomization.

Finally, the AIX Fundamentals chapter covers things like JFS2, RBAC, EFS, AIXpert and MultiBOS. You get the idea. There’s much more of course, so download it or order a hard copy. (Registration with IBM is required.)

Edit: I assume you are not still seeing this issue

Originally posted January 28, 2020 on AIXchange

Rob finds a simple solution to a customer’s PowerVC install fail.

A while back I helped out on a PowerVC 1.4.3.0 installation. The customer was getting this error:

ERROR powervc_oslo.config.data_utils [-] End Of File (EOF) in read_nonblocking().

Exception style platform.

<pexpect.spawn object at 0x7f625b1ce990>

version: 2.3 ($Revision: 399 $)

command: /usr/bin/mysqladmin

args: [‘/usr/bin/mysqladmin’, ‘password’]

searcher: searcher_re:

    0: re.compile(“New password:”)

buffer (last 100 chars):

before (last 100 chars): .sock’ (2)’

Check that mysqld is running and that the socket: ‘/var/lib/mysql/mysql.sock’ exists!

As is often the case when I encounter a problem, I started with a web search—I should say, I started and finished. Seriously, I’m not sure I’ve ever found such a simple answer so quickly. The exact error came up on the first return, along with this solution (excerpt below):

“Resolving The Problem
Please proceed with the steps below:
Please run
ln -s /usr/bin/resolveip /usr/libexec/resolveip

Then attempt this command

/root/powervc-1.4.3.0/lib/config.sh –interface yourinterface –ipversion 4 –ipoption default –firewall yes –silent 0 –hostaddress yourfqdn –operation configure –logfile /opt/ibm/powervc/log/yourlastinstalllogfilename

* where yourinterface is the name of the Network adapter used to configure your PowerVC server.
* where yourlastinstalllogfilename is the name of the powervc install log from the current failed install
* where yourfqdn is the fully qualified domain name of your PowerVC server
Once it completes with successful then run the below command: powervc-restore.”

Hopefully you won’t run into this issue, but if you do, it is easily resolved.

Edit: Still good information

Originally posted January 21, 2020 on AIXchange

Courtesy of Russell Adams, here’s some useful information on performance monitoring tool nmon and how to more securely verify IBM downloads.

Recently I found some interesting information on Russell Adams’ blog. This is about nmon, the performance monitoring tool (excerpt below):

“Recently I was learning about Nigel’s new efforts with JSON and web based graphing, and came across his nmonchart tool. This new tool has dynamically zooming graphs via javascript directly in your browser from a single file! I had to try it, and I’m very impressed.

Running it against a single file was trivial and the resulting HTML loaded into a browser without issues to view the data. However when I wanted to view several days of data in separate files there wasn’t an option.

A few minutes later and some awk magic result an awk script to combine data files for reading into nmonchart.”

He includes a sample script and some files, so be sure to read the whole thing.

In another blog entry, he covers an option for verifying IBM downloads made to the NIM server (excerpt below):

“IBM doesn’t typically publish a simple text file of checksums with any of their POWER or AIX downloads. They do include an XML file for Download Director.

They do make an attempt to allow customers to validate the download using that XML file in VIO downloads by providing a file called ck_sum.bff. The customer is instructed to execute ck_sum.bff against the directory of downloads to confirm the downloads.

This raises many red flags for me for violating security best practices. I should never execute untrusted code from any source on my systems! The typical place this would run is on a NIM system or AIX box as root! I strongly advise against using this method.

Given IBM does have the checksums in an XML file, we can extract them for validation without using untrusted code. I accomplished this on a Linux box with XSLTPROC, but I believe this tool may be available for AIX as well.”

Again, there’s a script and sample output with this post.

Edit: Hopefully you are on top of this with your machines.

Originally posted January 14, 2020 on AIXchange

As the initial warranty periods on POWER8 systems start to expire, update access keys will need to be replaced. Here’s how.

A customer recently experienced issues with POWER8 firmware access keys (page excerpt below):

“POWER8 (and later) servers include an “update access key” that is checked when system firmware updates are applied to the system. Update access keys include an expiration date. As these update access keys expire, they need to be replaced using either the Hardware Management Console (HMC), the Advanced Management Interface (ASMI) on the service processor, or directly using the update_flash command.”

To get a new update access key, follow the steps below (excerpted from the page):

“An expired update access key can be acquired by visiting the following website:
http://www.ibm.com/servers/eserver/ess/index.wss

Step 1: Gather Data—You will need to know your server’s machine type, serial number, and the country of installation in order to obtain a new update access key.

Step 2: Login to the “Entitled System Support” web page using your IBM WebID.
http://www.ibm.com/servers/eserver/ess/index.wss

Step 3: Navigate to the “My Entitled Hardware” section and select “View, download and request update access keys.”

Step 4: Enter your server’s machine type, serial number and country of installation. You may enter each server machine type and serial number individually, or you can upload a list of servers.

Step 5: On the “Update Access Key Request Confirmation” page, you’ll see the result of your request(s). You’ll have the option of viewing your server’s new update access key directly, downloading the key via a file or sending it to your IBM Web ID via email.

Note: If you were not provided with a new update access key for your server, click on “contacts” on the right side of the screen to engage with IBM.

Step 6: After retrieving your server’s new update access key, return to your server and enter the key on your HMC (using the “Enter CoD Code” feature) or via the ASMI (using the “CoD Activation” feature).”

My client was getting reminders on their HMC that it was time to input a code, but after going to the IBM Entitled Systems Support (ESS) site and selecting the update access key option, the dates and the keys did not change. This IBM Support Page explains why:

“The initial Update Access Key is loaded during the manufacturing process and is dated to align with the expiration of the initial warranty period. The term of any subsequent agreement (e.g. IBM hardware maintenance service agreement or special bid agreement) may vary. Therefore, replacement Update Access Keys are issued with a duration of 180 days.”

In short, the initial warranty hadn’t expired. When they purchased their system, the date of expiration was a few years out. Once that date was reached and the new maintenance agreement kicked in, they could update the keys with no problem.

Going forward, they’ll need to update keys every 180 days. This will become more common as the initial warranty periods of these POWER8 systems run out, so keep it in mind if you haven’t reached that point with yours.

Edit: Have you tried this?

Originally posted January 7, 2020 on AIXchange

The latest version of the IBM Entitled Systems Support mobile app puts asset management at your fingertips.

As a business partner, customers often ask me for help with downloading software, updating user access keys, etc. Typically I’d go to ibm.com/eserver/ess to help with these requests. Recently though I received an interesting email about the latest version of the IBM Entitled Systems Support mobile app, which provides another way to get this information. Below is an excerpt from the email:

“ESS mobile: new version 5 available now
We are excited to announce a new major release for the ESS mobile app.

Our team has worked tirelessly and rebuilt the ESS mobile app from the grounds up, greatly improving multiple components and creating a starting point for fast updates going forward.

Highlights of version 5 release include:

• login is greatly improved and will work more consistently (for real this time)
• enhanced speed and fluidity of screen actions and transitions
• purchase of Elastic Capacity on Demand is back
• push notifications for your Elastic CoD orders and generated codes on your mobile device
• share your generated Elastic codes or any other available keys & codes directly with your email or messaging apps
• use the mobile app in your browser; it is available as a Progressive Web Application (PWA) under https://www.ibm.com/eserver/ess/mobile/
• more information about PWA technology can be found here: https://en.wikipedia.org/wiki/Progressive_web_applications
• PWA apps work in Google Chrome, Apple Safari and Firefox (non-private mode)
• PWA apps do not work in Microsoft Edge and Internet Explorer. Other webkit-based browsers may work (not tested)
• a quick launch icon for PWA version is available on the right side of the ESS website”

And to recap the basics about ESS mobile:

“IBM ESS mobile is asset management at your fingertips. Quickly view and access all your Power systems, available keys and codes, run reports and purchase and assign Elastic Capacity on Demand: any time, all the time. Your IBMid profile is fully synchronized across both the website and mobile app: access, registration and all activities are always shared.”

To test the app without downloading it to your phone, go here (registration required).

This appears to be the same information I can get from my phone. I have these options in the Reports section:

SWMA Inventory report: Active
SWMA Inventory report: Expires within three months
SWMA Inventory report: Inactive
Electronic Proof of Entitlement (ePoE) for Power report
Key Version/Release Inventory report: Active keys
SWMA Inventory report: All
Permanent key installation commands
ESS software update orders report.
Capacity Reset report
ESS downloadable product list

If I’m not at my desk, it’s great to have a way to access this information. How about you? Does this app interest you, or do you exclusively use a desktop?

Edit: I mention Ragnar and triathlons in this piece, I was not able to do either in 2020, but that did not stop me from running and staying active and being ready for the time when I am once again able to compete. And yes, I am still doing my Duolingo lessons, and I still have a long way to go.

Originally posted December 17, 2019 on AIXchange

Doing just a little bit on a regular basis can have a lasting impact on your career (and life).

This post is almost three years old, but these equations at the heart of it have stuck with me:

1.01 ^ 365 = 37.8
0.99 ^ 365 = 0.03

Below is an excerpt from that post:

“One of my favorite concepts of finance is compounding. Compounding is simply when we make an investment and reinvest the return on that investment. So, if you invest in a house that you rent out to other people, then if you take that rent and buy new houses, you are compounding your investments.

In other words, compounding is a process where the value of an investment is consistently reinvested over time. This is the most powerful concept when it comes to maximizing growth on investments. That is why I love this poster.

In the first case, we are multiplying 1.01 with 1.01 for 365 days. In the other 0.99 is multiplied with 0.99 for 365 days. There is just this tiny difference but over a year, these kinds of differences add up to crazy differences.”

As I wrote about a year ago, I’ve been making small adjustments to my lifestyle. I’ve been eating better, I’ve been more active, and I’m seeing results. I have been competing in mini triathlons for a few years now, and I ran my first sprint triathlon in 2018. I was satisfied with my time, but I entered the same event and completed the same course this year, in 14 fewer minutes.

I run most days at this point. I have more endurance. I have more strength. I like the beach. The closest one to me here in Phoenix is in Puerto Penasco, Mexico, about a three-and-a-half-hour drive. Since I’m there on a somewhat regular basis, learning some Spanish seemed appropriate. As with my fitness regimen, a little adds up when it comes to taking on a new language. I spend about 10-15 minutes a day doing Duolingo lessons on my phone. That may not sound like much, but I find I now understand rudimentary words and phrases. If I tune to Spanish language radio or watch Univision, I notice common words and phrases.

You can apply this to anything. If you’re looking to pick up AIX, or Linux, or a new software package, diving in and drinking from a fire hose might work; for the sake of your career, it may even be an essential course of action. But small, consistent effort over time does produce results. Cramming the information into your brain might have got you through that college term paper, but I’d argue that doing just a bit on a regular basis over the long term offers a lasting impact as far as actual learning and/or development.

I know I’ll continue to keep it slow and steady as I prepare for my next Ragnar event.

Edit: Is there anyone who has not updated their NIM servers yet?

Originally posted December 10, 2019 on AIXchange

A recent change in OS versioning requires NIM servers to run on the latest versions of AIX.

AIX Network Installation Management (NIM) can be used to build out VIO servers. If you rely on this valuable tool, take note of a recent change in OS versioning that requires the NIM server to run on the latest versions of AIX. As of VIOS 3.1.x, AIX 7.2 is now running under the covers (prior versions ran AIX 6.1). Because your NIM server’s AIX version must be at least equal to the AIX version that’s managing NIM clients, you should update to 7.2 sooner rather than later.

Which specific AIX version do you need? Chris Gibson recently provided a link on Twitter to the minimum NIM master levels for VIOS clients, which should clear up any confusion:

If using NIM to backup, install or update a VIOS partition, the NIM master must be greater than or equal to the levels shown below.

VIOS Release	VIOS Level Minimum	NIM master level
VIOS 3.1.0	VIOS 3.1.0.21	AIX 7200-03-03
	VIOS 3.1.0.10	AIX 7200-03-02
VIOS 2.2.6	VIOS 2.2.6.41	7200-03-03
	VIOS 2.2.6.32	7200-03-02
	VIOS 2.2.6.31	7200-03-01
	VIOS 2.2.6.21	7200-02-02
	VIOS 2.2.6.10	7200-02-01
	VIOS 2.2.6.0	7100-02-00
VIOS 2.2.5	VIOS 2.2.5.60	7200-01-06
		7200-02-04
		7200-03-04
		7200-04-01
	VIOS 2.2.5.50	7200-03-01
	VIOS 2.2.5.40	7200-02-02
	VIOS 2.2.5.30	7200-02-01
	VIOS 2.2.5.20	7200-01-02
	VIOS 2.2.5.10	7200-01-01
	VIOS 2.2.5.0	7100-04-03

The complete chart found with the link also includes information for earlier VIOS versions as well AIX 6.1 and AIX 7.1. But again, going forward, to install VIOS 3.1.x you must be at AIX 7.2, so these recommendations are the ones that matter. If your NIM and VIO servers aren’t yet updated, get moving.

Edit: vSCSI or NPIV both work for LPM

Originally posted December 3, 2019 on AIXchange

Moving to POWER9? Live Partition Mobility (LPM) can be used to virtualize storage on LPARs connected to VIO servers via vSCSI.

Recently I was asked if it’s possible to perform Live Partition Mobility (LPM) operations on LPARs that are connected to VIO servers via vSCSI. Since N_Port ID virtualization (NPIV) is so common now, I actually had to think about it, but the answer is yes, you can virtualize your storage either way. In fact originally, LPM was delivered and worked exclusively with vSCSI.

This means that shops that are preparing for the arrival of new POWER9 hardware should be able to migrate with no downtime, assuming their older systems are virtualized and that the SAN is zoned properly. Below is an excerpt from the IBM Developer page on LPM:

“You must have a minimum of two machines, a source and a destination, on POWER6 or higher with the Advanced Power Virtualization Feature enabled. The operating system and application must reside on a shared external storage (Storage Area Network). In addition to these hardware requirements, you must have:

• One hardware management console (optional) or IVM.
• Target system must have sufficient resources, like CPU and memory.
• LPAR should not have physical adapters.

Your virtual I/O servers (VIOS) must have a Shared Ethernet Adapter (SEA) configured to bridge to the same Ethernet network which the mobile partition uses. It must be capable of providing virtual access to all the disk resources which the mobile partition uses (NPIV or vSCSI). If you are using vSCSI, then the virtual target devices must be physical disks (not logical volumes).”

Also note the limitations of LPM, as Jaqui Lynch explains (excerpted below):

“LPM is not a replacement for disaster recovery or high availability solutions. It’s designed to move a running LPAR or a properly shutdown LPAR to a different server. It cannot move an LPAR with a crashed kernel or from a failed machine. If all the prerequisites are met, LPM can be used to move an AIX, IBM i or Linux LPAR from one LPM-capable POWER server to another compatible server. By compatible I mean that it has to meet the requirements for the Power Systems server, the management console, the VIOS (PowerVM) and the LPAR itself.”

Finally, this reminder. When you purchase POWER9:

“Each system will have PowerVM Enterprise Edition built in, and IBM is helping customers migrate by providing 60-day temporary licenses for existing machines that don’t already have PowerVM Enterprise Edition. This will allow you to use live partition mobility to migrate running workloads from existing POWER7 or POWER8 machines to your new POWER9 machine.”

Edit: This is still a great tool

Originally posted November 26, 2019 on AIXchange

The IBM Interactive Product Tour Catalog allows you virtually preview 3D representations of new hardware.

Are you familiar with the IBM Interactive Product Tour Catalog?

Just go here, select Systems and Servers, and then click on IBM Power Systems.

From there, choose Enterprise or Scale-out, and IBM POWER9 or POWER8.

Finally, choose an actual system.

It may take a few seconds for the page to load, but it’s worth the wait. For your system of choice you can see different views of the system and get some basic information.

Of course there’s important and useful information in standard IBM presentations (like this one), but the way the catalog allows you to visualize the new hardware makes a much stronger impression in my opinion. There’s just something about being able to look at the 3D image. It’s like being there on the raised floor, ready to open the box and install it in the rack. You can (virtually) take the lid off and see where the parts go, where the I/O cards go, etc. If you’re ordering a new system sight unseen, this sneak peek provides a unique perspective.

Edit: I still like to watch a demo or video

Originally posted November 19, 2019 on AIXchange

This short demo video breaks down the process of using Cloud Automation Manager to create templates and deploy virtual machines.

We are always looking for ways to simplify our lives, but simplification is more elusive in some situations than others.

Consider a small environment where it makes sense to build LPARs by hand. The plan is to keep machines on site, and new builds are infrequent enough that extensive automation isn’t a pressing need.

In other environments where builds are automated, using AIX Network Installation Management (NIM) with some homegrown scripts to build new LPARs makes perfect sense.

Then there are those sites that are either drowning in new deploys or allowing users to build their own LPARs on demand. These types of installations can benefit from tools like PowerVC and IBM Cloud Automation Manager.

This video gives you a taste of what it takes to deploy existing VMs with Cloud Automation Manager and add AIX, IBM i and Linux applications to a cloud private catalog:

Here is an excerpt of the video’s description:

“Joe Cropper demonstrates how to configure IBM Cloud Private and Cloud Automation Manager to create an Oracle database self-service catalog entry within IBM Cloud Private. When the database is deployed, it provisions the underlying resources through IBM PowerVC.”

Demos usually resonate with me when I can see actual interaction with products. As valuable as articles and presentations can be, a picture—or, in this case, 9:32 of video—really is worth a thousand words. This video nicely breaks down the whole process of using this tool to create templates and deploy virtual machines.

Edit: Always good to nominate yourself or others, and take the AIX surveys

Originally posted November 14, 2019 on AIXchange

Technical editor Rob McNelly highlights upcoming deadlines for Champions nominations and to complete an AIX survey.

A quick reminder about two rapidly approaching deadlines.

First, the nomination period for IBM Champions ends on Friday, Nov. 22. Consult these tips and tricks if you plan to make a submission. Also consider this information, which comes from an internal newsletter for IBM Champions:

We are particularly looking for IBM Champions who advocate for and have expertise in Enterprise AI, Cloud, and Enterprise Linux. Nominate yourself and those who already advocate for our brand and story through speaking at events, blogging, customer references, videos, leading user groups, etc.

As noted, you can nominate yourself or someone else. As a Lifetime IBM Champion, I encourage you to get involved in the process.

Also, I want to amplify the message about the 2020 AIX survey. The deadline for that is also this Friday, Nov. 22:

Last year’s survey provided valuable insight into the needs and plans of AIX users around the world. We hope you’ll take a part in shaping the insights of this year’s survey. Watch for the results in early 2020.

Take the AIX survey here. Apologies for not providing more advanced notice. I’ll try to get further out in front of these things next time around.

Edit: I will miss sharing these tips and tricks

Originally posted November 5, 2019 on AIXchange

IBM Support troubleshoots an NPIV client boot hanging with Error Code 554.

On Twitter Chris Gibson highlighted an IBM Support document that troubleshoots an NPIV client
boot hanging with Error Code 554, which I’ve excerpted below:

“AIX NPIV Client Boot Hanging with Error Code 554 due to max_xfer_size

Cause
This is cause [sic] by client’s max_xfer_size setted too high compared to VIOS Fiber Physical Adapter

Environment
AIX & NPIV

Diagnosing The Problem 
To confirm the boot failure is due to max_xfer_size’s value too high on AIX Client, perform a Boot Debug on AIX Client.

To enable this Boot Debug follow guidance:
http://www-01.ibm.com/support/docview.wss?uid=isg3T1019037

Then trap the following message: “open of /dev/fscsi0 returned 61“

Resolving The Problem
Boot AIX client partition in Maintenance Mode, put back max_xfer_size value equal or lower to the one configured on the VIOS… .

Then you can set this value in AIX Client ODM (Maintenance Mode) with chdev command and perform a normal Boot.

Another way to address this would be to change “max_xfer_size” on Physical HBA to match Client’s one, but this change require VIOS’ reboot. This way seems more risky if one of LPARs is not fully operational from Multi-Pathing perspective.”

Be sure to read the entire thing.

I love sharing these tips for a couple of reasons. One is self-interest. Many times when I need information on an issue, my web search will return one of my long-forgotten AIXchange posts. And hopefully by raising these issues here maybe you’ll think of this post and/or that document, or you’ll simply remember to avoid making the max_xfer_size too high in the first place.

Edit: There is not a day that goes by that I do not click on an article that takes me to a landing page that tells me that information is lost forever. Still heartbreaking to lose so much knowledge.

Originally posted October 29, 2019 on AIXchange

Big news: IBM is sunsetting its entire developerWorks Connections platform.

I’m a long-time reader of IBM developerWorks, a vast collection of blogs, groups, forums and much more that’s maintained by some accomplished IBM technologists. Chris Gibson’s blog on AIX and PowerVM, The AIX Virtual User Group and the AIXpert blog are just a few of my most-clicked bookmarks.

So I was surprised by the recent announcement that IBM is sunsetting its entire developerWorks Connections platform on Jan. 1, 2020. Below is an excerpt of that announcement:

“As part of the overall IBM effort to reduce the number of duplicate client interaction portals and simplify the user experience with the digital side of IBM, we are sunsetting the developerWorks Connections platform which includes all URLs starting with https://ibm.com/developerworks/community/.

Affected pages will have a banner at the top identifying it as a target for removal on December 31, 2019. This removal includes all community blogs, wikis, forums, activities and files. Please send any questions or comments on our Support form.

On January 1, 2020, the developerWorks Connections platform and its apps will no longer be available.

Q. Why are these Connections pages going away?
A. IBM is consolidating major content portals to improve the customer experience.

Q. What specific content of mine will be impacted and removed?
A. Your community and its apps will no longer be available including: Activities, blogs, files, forums and wikis…

Q. Can I receive a copy of all of my content I’ve published?
A. Unfortunately due to technical constraints of this product, we are unable to provide that service.”

If you want to leave feedback, you can do so here.
https://developer.ibm.com/code/dw-connections-sunset-support/

As someone who has blogged for 12 years, I understand that a great deal of technical information, even if goes back several years, remains relevant and useful for today’s IT pros. If you’re a regular reader or even if you only occasionally peruse the developerWorks community pages, I encourage you to register your opinion with IBM. My hope is that if they hear from enough of us, they’ll at least put off the sunsetting, giving us more time to figure out how to archive this valuable content.

Edit: This is quite a bit to digest.

Originally posted October 22, 2019 on AIXchange

Here’s what AIX/IBM Power Systems users should know about the latest announcements.

IBM has come out with a bunch of announcements this month.

Here are some details on a few announcements and updates that should be of particular interest to AIX/IBM Power Systems users:

PowerHA SystemMirror V7.2.4 for AIX enhancements

• Cross-cluster verification: Compare the test cluster and validate that the new production cluster has the same configuration parameters as the preproduction test cluster.
• Availability metrics updates: With the availability metrics feature, PowerHA SystemMirror clients can see a timeline of events that have happened in the cluster, including the duration of each step. This release of the tool adds reports on configuration bottlenecks and historical averages.
• Smart Assist currency updates:
  • Adds support for the latest version of the Smart Assist application.
  • Enables support for the latest version of Spectrum Protect (IBM Tivoli Storage Manager) with PowerHA.
  • Federated Security is enhanced to enable PowerHA SystemMirror commands that are not already supported for role-based access control (RBAC). Now the clmgr command returns a proper error if an unprivileged user performs an action that is not allowed. Commands currently enabled for RBAC can be tested for proper behavior on the basis of different PowerHA SystemMirror supported roles. Any deviation from proper behavior of RBAC-enabled commands will be fixed. An option is provided for users to list the commands that they are authorized to perform.
  • IBM Db2: With PowerHA SystemMirror, users can configure IBM Db2 with multiple databases and create a new monitor for each database, enabling them to handle individual DB failures without impacting the other DBs in the multiple parallel DB configuration.
• Support for IBM SAN Volume Controller (SVC) role-based access: With this enhancement, users can configure a user name while adding an SVC cluster in PowerHA SystemMirror. The user name configured using the smit or clmgr command for an SVC cluster is used for all the SVC operations carried out for that particular SVC cluster.
• In addition, PowerHA V7.2.4 supports SVC 8.1. PowerHA SystemMirror V7.2.4 for AIX encrypts the various passwords stored in the configuration database. This ensures that passwords at rest are encrypted for security compliance requirements.
• Increases the maximum number of resource groups from 64 to 256.
• IBM WebSphere MQ listener support: Clients can configure PowerHA Smart Assist for WebSphere MQ to start, stop, and monitor WebSphere MQ listeners.

AIX 7.2 TL4
The IBM AIX operating system provides clients with an enterprise-class IT infrastructure that delivers the reliability, availability, security, performance, and scale that is required to be successful in the global economy. IBM AIX 7.2 TL4 provides the following enhancements:

• New levels of workload scalability
• New levels of OS security
• Enhanced scope and usability for AIX Live Update (high availability)
• New I/O features and storage enhancements
• File systems improvement

Additionally, the AIX Toolbox (across AIX 6.1, AIX 7.1, and AIX 7.2) is enhanced.

PowerSC Standard Edition V1.3
IBM Power SC provides a security and compliance solution that is optimized for virtualized environments on IBM Power Systems servers. Security control and compliance are key components needed to defend virtualized data centers and cloud infrastructure against evolving threats.

PowerSC Standard Edition V1.3 enhancements include:

• New built-in compliance profiles
  • An SAP AIX hardening profile
  • A US Department of Defense-Security Technical Implementation Guides for AIX 7 profile
  • A Center for Internet Security (CIS) for AIX profile
• IBM i integration through a new IBM i compliance automation profile
• Support for alt-disk updates
• Improvements within the Patch Management Component
• Improvements within the reporting section
• Multifactor authentication enablement

IBM PowerVM V3.1.1, IBM PowerVC V1.4.4, IBM Virtual HMC (vHMC) 9.1.940, and the IBM Cloud Management Console (CMC) Monthly Term offering
IBM PowerVM V3.1.1, which delivers industrial-strength enterprise virtualization for IBM AIX, IBM i, and Linux environments on IBM POWER processor-based systems, has expanded function and new management capabilities.

• IBM PowerVM Hypervisor has been updated with new features that include:
• Support for DRAM-backed persistent memory for faster VM restarts
• Enhanced SR-IOV and vNIC support
• LPM performance improvement

IBM PowerVM V3.1.1 has been updated with significant I/O performance, efficiency, usability, and scaling enhancements, including:

• An increased number of NPIVs allowed per Fibre Channel port on the 32 GB Fibre Channel HBA. The current limit is 64, and more NPIVs per port will provide better utilization, density, and efficiency.
• NPIV multiqueue (VIOS server) for improved performance and scalability.
• Fibre Channel device labels for improved manageability.
• iSCSI performance enhancements.
• Improved VIOS Upgrade Tool capabilities.
• SSP network resiliency improvements.

IBM PowerVC V1.4.4, which is designed to simplify the management of virtual resources in IBM Power Systems environments, has been updated with several enhancements to storage integration, usability, and performance, including:

• IBM i license key injection
• Hitachi GAD support
• Initiator storage tagging
• Live Partition Mobility (LPM) VMs to the original system after evacuation
• Ability to pin VMs to a specific host
• Ability to dynamically add PowerVC created VM to an HMC user resource role
• Inactive partition migration
• Image sharing between projects
• New restricted administrator assistance role, with no deletion access
• NovaLink support for multivolume attachment
• FlexVolume driver support for Red Hat OpenShift

IBM Virtual HMC (vHMC) 9.1.940, released in conjunction with firmware FW940, which gives clients the ability to use their own hardware and server virtualization to host the IBM-supplied HMC virtual appliance, has been updated with the following enhancements:

• User mode NX accelerator enablement
• Support for SR-IOV logical ports in IBM i restricted I/O mode
• Injection of IBM i license keys
• Improved LPM error messages
• Ability to manage resource roles automatically in HMC
• Progress indicators for specific HMC commands
• Email notifications for scheduled operations
• IBM PowerSC Multi-Factor Authentication (MFA) in-band support for HMC

IBM Cloud Management Console (CMC) Monthly Term offering is a cloud-delivered monitoring service that connects to multiple HMCs and gives administrators an aggregate view of their entire Power infrastructure. CMC has been updated with the following enhancements:

• Support for Enterprise Pools 2.0
• Cloud host options that include North America and Europe

Edit: It is always good to go back to basics now and again

Originally posted October 15, 2019 on AIXchange

The IBM Knowledge Center’s overview of AIX file systems is handy as both a refresher and introduction.

On Twitter, Soumya Menon (@soumya_159) linked to this IBM Knowledge Center overview of AIX file systems, excerpted below:

“A file system is a hierarchical structure (file tree) of files and directories.

This type of structure resembles an inverted tree with the roots at the top and branches at the bottom. This file tree uses directories to organize data and programs into groups, allowing the management of several directories and files at one time.

A file system resides on a single logical volume. Every file and directory belongs to a file system within a logical volume. Because of its structure, some tasks are performed more efficiently on a file system than on each directory within the file system. For example, you can back up, move, or secure an entire file system. You can make an point-in-time image of a JFS file system or a JFS2 file system, called a snapshot.

… To be accessible, a file system must be mounted onto a directory mount point. When multiple file systems are mounted, a directory structure is created that presents the image of a single file system. It is a hierarchical structure with a single root. This structure includes the base file systems and any file systems you create. You can access both local and remote file systems using the mount command. This makes the file system available for read and write access from your system.

… Some of the most important system management tasks have to do with file systems, specifically:

• Allocating space for file systems on logical volumes
• Creating file systems
• Making file system space available to system users
• Monitoring file system space usage
• Backing up file systems to guard against data loss if the system fails
• Maintaining file systems in a consistent state”

There’s much more, of course. Read the whole thing, and maybe check out some of the related overviews―including docs on managing, configuring and maintaining file systems―that are linked at the bottom of the page. Also be sure to open the table of contents (its appearance may vary depending on the browser being used) for links to a host of other topics:

Workload manager
Device nodes
Device location codes
Device drivers
Setting up an iSCSI offload adapter
PCI hot plug management
Multiple Path I/O
Targeted device configuration
Tape drives
USB device support
Caching storage data
Login names, system IDs, and passwords
Common Desktop Environment
Live Partition Mobility with Host Ethernet Adapters
Relocating an adapter for DLPAR
Loopback device
Files Reference
IBM Hyperconverged Systems
AIX on KVM
Installing
Networking
Operating system management
Performance tuning
Printing
Programming for AIX
Security
System management
Technical reference
Troubleshooting
IBM Workload Partitions for AIX

We may take the basics for granted, but it never hurts to get a refresher on these topics.

Edit: Some links no longer work.

Originally posted October 8, 2019 on AIXchange

A handy, consolidated spreadsheet lets you quickly compare rPerfs and CPW across different hardware generations.

Have you ever wondered how much faster POWER9 systems benchmark against earlier versions of the hardware? Well there’s an easy way to find out. Go to the IBM Lab Services UKI (United Kingdom and Ireland) POWER team web page to download a spreadsheet with this information.

As you can see, the most recent update came in August:

Date of Change	Description
13-Aug-2019	Added 6 Core E980 servers
7-Sept-2018	Added POWER9 servers (E950 and E980)
27-Feb-2018	Added POWER9 servers (S922, S914 and S924) and updated POWER8 servers to reflect Spectre and Meltdown
12-Sep-2017	Added S812 POWER8 servers and updated S822 POWER8 servers
8-Nov-2016	Added C Model Power8 Servers (E850C, E870C and E880C)

To download the actual data, do what the large red text says and click here for consolidated spreadsheet.

I’m sure I’m not the only one who enjoys being able to compare rperfs and CPW across different hardware generations. While the POWER Facts and Features guides also have this information, this spreadsheet puts everything at your fingertips. Check it out.

Edit: I still recommend customers get enhanced support.

Originally posted October 1, 2019 on AIXchange

IBM’s Proactive Support is now included by default for all mid-range and enterprise POWER9 Systems.

For those who create hardware orders, IBM’s configuration tool now includes Proactive Support by default for all E980 and E950 configurations.

Under Proactive Support, customers are assigned an IBM account manager. Through frequent contact, including regularly scheduled calls, a strong working relationship develops. For IBM, the collaboration provides knowledge of the individual environment, making it possible to respond more quickly when a ticket is opened or when new vulnerabilities arise that require action.

Having worked with teams that have benefited from this type of enhanced relationship with IBM Support, I recommend you look into this offering.

This is from the Aug. 9 announcement letter:

“Clients have found significant value in IBM’s Proactive Support offerings on mission-critical systems, as this provides personalized support, proactive recommendations, and accelerated response times versus standard support. As a result, IBM is including the IBM Proactive Support in the default configuration for all Mid-Range and Enterprise IBM POWER9 Systems – for IBM AIX, IBM i, Linux, and SAP HANA workloads. Other configurations are also available.”

I saw a slide deck that highlights these points:

“IBM Proactive Support can streamline management, reduce costs and increase availability. This solution is designed to:

• Provide a single source of customized support for hardware and software from multiple vendors, helping you improve the return on your IT investment.
• Balance high availability with improved affordability to help maintain converged, virtualized and cloud-based IT environments.
• Increase your IT staff’s productivity and free it to focus on more strategic initiatives.
• Supply an optimized support model with global delivery through worldwide IBM technical centers.
• Shorten the response time to failures and avoid a major impact on revenue, cost, customer satisfaction, reputation and more.”

Edit: Customers will not buy what they do not know about. It looks like the videos have been removed.

Originally posted September 24, 2019 on AIXchange

Ads for IBM Power Systems must convey both technical and marketing information—undoubtedly quite the needle to thread.

A while back I discovered this on the Twitter feed of IBM’s Paulo Carvao (@pcarvao). It’s the best Power Systems ad I’ve seen yet. Check it out; it only takes a couple minutes.

The video not only highlights the world’s fastest supercomputers that run on POWER9 processors, it also mentions both AIX (“the No. 1 OS in the UNIX market”) and IBM i (“double-digit growth in 2018”). Also cited are Linux, SAP HANA running on POWER, and much more.

The tagline at the end states: “the time to bet your tomorrow on IBM Power Systems is now.”

If you’re suitably inspired by that, you may want to watch another video: “IBM POWER9: Let’s put smart to work.“

This particular video came out in early 2018. See if you can spot (at the 1:10 mark) a tiny picture of me among many other IBM Champions.

What do you think of these videos? If it was your call, what would you like to see highlighted? I have my own ideas, but I recognize the challenges of making these sorts of ads. The trick is balancing technical and marketing information while making the whole thing visually interesting for non-technical viewers. Undoubtedly that’s quite the needle to thread.

Edit: Hopefully you took care of this

Originally posted September 17, 2019 on AIXchange

Those using legacy versions of Elastic Capacity on Demand (CoD) will soon need to upgrade their codes.

In July, IBM’s David Spurway tweeted about Elastic Capacity on Demand (CoD). The enablement codes used with legacy versions of Elastic CoD will soon be withdrawn, so if you’re on an older version, you’ll need to get new codes in short order.

Below is an excerpt from the IBM announcement letter:

“Hardware withdrawal: Power System Legacy Elastic CoD features

As we expand Elastic CoD via Entitled System Support Worldwide for flexible provisioning and usage on the web in minutes, we are withdrawing our legacy Elastic CoD offerings.

For additional support on how to administer these new features, see the Entitled System Support website.

Effective October 31, 2019 and April 30, 2020, IBM will withdraw from marketing the selected IBM Power Systems features listed in the Withdrawn products section. On or after the effective dates of withdrawal, you can no longer order these features directly from IBM. Field-installed features associated with the machine types not listed in this announcement continue to be available.

If a client continues to have prepaid Elastic CoD credits regarding this withdrawal, contact the ECoD project office to do a one-time migration to Elastic CoD via Entitled System Support.

For new orders, the client-requested arrival date (CRAD) can be no later than November 29, 2019 and May 29, 2020.”

Edit: Have you noticed any disruptions to either company?

Originally posted September 10, 2019 on AIXchange

Red Hat’s acquisition by IBM is bound to take us in some interesting directions.

In July, Red Hat announced its acquisition by IBM. Senior vice president and CTO Chris Wright also posted this brief Q&A. I’ll excerpt a few key points:

“Q: Will the acquisition change the way Red Hat contributes to upstream projects?
A: No, Red Hat will continue to contribute to and participate in open source projects as we do today.

Q: Will the work that Fedora does, including all of the Editions, Spins, and Labs, change as a result of the acquisition?
A: Fedora’s build products will not be affected. All changes will continue to be driven by the Fedora Project.

Q: Are Red Hatters still free to work on open source projects outside of Red Hat?
A: Yes. Red Hat associates can contribute to and participate in open source projects outside of Red Hat as they do today.

Q: Will community projects be forced to support specific software or hardware?
A: No. Any inclusion of software and hardware support will continue to be driven by the community.

Q: Will community projects or Red Hat contributors be forced to use specific technologies?
A: No.

Q: Will the logos of Red Hat-sponsored projects change as a result of the IBM acquisition?
A: No logos of Red Hat-sponsored projects will change as a result of the acquisition.”

From the IBM side, check out this joint interview with IBM and Red Hat officials, this developer perspective, and this interview that covers IBM Cloud Paks and more. And a host of additional information can be found here.

The IBM-Red Hat union is bound to take us in some interesting directions.

Edit: It is good to know AIX will be around for years to come

Originally posted September 3, 2019 on AIXchange

IBM’s recently published guide lays out the company’s long-term commitment to AIX.

IBM recently published what it calls “an executive guide to the strategy and roadmap for the AIX operating system for IBM Power Systems.”

Don’t let the dry title fool you. This document has some good information about IBM’s long-term commitment to AIX. Since registration is required to access this 15-page PDF, I thought I would give you a taste.

This is from the executive letter:

“AIX has been the foundation of mission-critical workloads for a large and dedicated client community for more than thirty years. AIX has evolved to help drive cloud and enterprise AI initiatives for thousands of enterprise businesses and organizations around the world. And now, the team behind AIX have developed a forward-looking strategy and roadmap.”

This is from the introduction:

“As IBM Power Systems expands its portfolio to deliver value-driven offerings for the emerging Enterprise AI workload market, we remain committed to delivering a roadmap of innovation for both Power Systems hardware and AIX. The strategy focuses on supporting workload growth for the POWER architecture and solidifies an investment stream and market relevance for the AIX platform. Power Systems with AIX is the foundation for many core business applications and database environments.

AIX is deployed across a variety of industries such as finance, manufacturing, retail, telecommunications, healthcare, travel and government, along with many others. Today, it’s no secret businesses are experiencing growth as it relates to data. Fortunately, AIX is and will continue to be built to meet such growing demands for its community.”

The executive guide mentions hybrid multi-cloud, public cloud, PowerVM and PowerVC and more. A chart notes that most top companies—”most” as in 80-90 percent—run their businesses on IBM Power Systems, and they experience the lowest percentage of unplanned server downtime.

And as indicated in that long title, there is an AIX roadmap. It goes into the next decade… and the one after that. As of now, the plan for AIX extends to 2030.

Check out the guide’s closing statement:

“With thirty years of release engineering practices, AIX has a proven model for delivering new hardware support and software innovation through TLs. This approachminimizes disruption for AIX clients and ISVs by enabling them to easily adopt new capabilities because we are able to introduce all-new features via TLs. Experience has shown that new major AIX releases require additional qualification activities by clients and create a dependency on ISV certification and support statements before clients can adopt the new releases. TL’s minimize client disruption and the possible need for ISV’s to recompile, re-test and re-certify their software.

As IBM enhances AIX and plans updates, the following factors are considered. AIX has a very strong commitment to binary compatibility for APIs and command-line outputs across TL releases. Even across AIX major releases where compatibility impacts may be considered, this compatibility is an important goal. Binary compatibility changes are very carefully reviewed with new major releases. If new technology innovation in AIX were to challenge binary compatibility in a significant manner, a new major AIX release would be considered.”

AIX definitely isn’t going away. Take a moment to register and read the entire guide for yourself.

Edit: Do you use this?

Originally posted August 27, 2019 on AIXchange

Server Message Block (SMB) 2.1 support is now available for AIX 7.2.

On Twitter, IBM’s Petra Bührer noted the availability of Server Message Block (SMB) 2.1 support for AIX 7.2. She followed that up by tweeting several IBM Knowledge Center links that explain how to use SMB.

On the SMB client filesystem:

The SMB server is a server that runs Windows Server 2012 or Windows Server 2016 server operating system. In each of these server operating system types, a directory can be exported as a share. This share can then be mounted on an AIX logical partition by using the SMB client file system. By using the SMB client file system, you can access the shares on SMB servers as local file systems on the AIX logical partition. You can use the SMB client file system to create, delete, read, and write files and directories on the SMB server and also to modify the access duration to these files and directories. However, you cannot change the owner or access permission of these files and directories.

These docs cover the mount command, the smbcd daemon, the smbcstat command, the smbctune.conf file, and the smbctune command.

This is welcome news for those working in an environment with UNIX and Windows servers. Typically separate IT groups are managing these servers, so getting this working requires some coordination. But having the option to share files between these systems can be beneficial.

Edit: I still use it

Originally posted August 20, 2019 on AIXchange

The IBM Power Systems Quick Reference Guide is available in iOS and Android formats.

In the span of a few days, I received an email and came across this tweet touting the availability of the IBM Power Systems Quick Reference Guide in iOS and Android formats:

GREAT NEW tool now available for POWER Systems sellers! Actually for ANYONE—partners and customers!

This app provides the information from the traditional PDF “POWER Quick Reference Guide” plus more, now in app format. Available for both iOS and Android.

IBM POWER Quick Reference is a mobile application for IBM POWER Server Reference guide.

Reference details available for POWER8 and POWER9 servers. It is a simple offline application which will help to check POWER server basic details quickly. It will act as a POWER server quick reference guide. The key benefits are to view POWER servers details like Processor, Memory, Disk etc.

Once application is installed, application will work even in offline. Application will be updated as and when new product launches.I was intrigued enough to visit the Google Play Store and install the app. For years I’ve been looking up the PDF version of the reference guide, but being able to do this no matter where I am is definitely a plus. The menus are simple and intuitive, and now all this valuable information is at my fingertips.

What do you think? Is this something you plan to use?

Edit: Still good information

Originally posted August 13, 2019 on AIXchange

Chris Gibson recently tweeted this IBM Support technote on how to manage multiple instances of altinst_rootvg on AIX.

Earlier this summer Chris Gibson tweeted this IBM Support technote that explains how to manage multiple instances of altinst_rootvg on AIX.

Here’s the gist of it:

Question
What is the recommended way of managing multiple instances of altinst_rootvg and applying interim fixes to each of the clones?

Answer
This document describes the recommended preparation and process when managing multiple instances of altinst_rootvg. We will discuss the usage of the ‘-v’ flag for the ‘alt_rootvg_op’ command, applying interim fixes when cloning the rootvg and finally post-checks and FAQ.

What will this document cover?
· Creating of multiple instances of altinst_rootvg and renaming them for convenience and easy management by:
– cloning of the rootvg using the ‘alt_disk_copy’ command
– cloning and updating the rootvg using ‘alt_’ commands
– migrating the rootvg using the ‘nimadm’ command
– renaming of volume group(s) by using the ‘alt_rootvg_op’ command
· Waking up and putting back to sleep cloned/migrated disk(s)
· Applying interim fixes to the migrated and cloned disk(s)

What this document will not cover?
· Most flags used with ‘alt_’ commands
· Alternate disk clone and nimadm pre-checks and requirements
· Any NIM related operations in conjunction with ‘alt_disk_copy’
· The phases of the nimadm process
· Most flags used with the ‘nimadm’ command
· Requirements for nimadm
· Limitations for nimadm

Frequently asked questions

Q: Can I wake up 2 rootvgs at the same time and perform operations on them?
A: No. Only one rootvg can be woken up at a time.

Q: Is it possible to wake up a rootvg that is at a higher oslevel than the rootvg used to initiate the wake up?
A: No. The running system’s operating system must be a version greater than or equal to the operating system version of the volume group that undergoes the wake up.

Q: Is there a limitation of maximum clones supported?
A: No there is no limitation.

Q: Can I update one of the clones in the future to a new Service Pack or Technology level without having to boot in it?
A: Yes. First you would wake up the rootvg undergoing the update and run the following:
#alt_rootvg_op -W -d <hdisk#>
#alt_rootvg_op -C -b update_all -l /<path to updates>

Read the whole thing, then try it out on a text box. And keep in mind that supporting alt disk operations like this is another unique benefit of AIX.

Edit: Some links no longer work

Originally posted August 6, 2019 on AIXchange

In this AIXchange blog, Technical Editor Rob McNelly recaps a presentation by IBMer Nigel Griffiths.

Nigel Griffiths recently gave a presentation on the long-term viability of IBM Power Systems hardware and the POWER processor. He then summarized his points here:

1. AIX is the UNIX winner in Technology and Performance—IBM having out Researched and out Developed all the other UNIX servers from other vendors over this time.
2. Linux on POWER performance lead—POWER9 2+ times faster than Intel per core
3. Accelerated AI and HPC Workloads on the AC922 give world class and the current number 1 and number 2 in the Top500 HPC computers List

This is only a small part of his post, but obviously the idea that IBM servers represent an end-to-end solution is compelling.
 
Of course, processors don’t make a server. With Power Systems IBM makes the servers too and we make sure new POWER processors are designed to make powerful servers by enhancing everything else too: memory, disks, adapters and all the interconnecting buses.
 
Finally, he lays out the investment IBM—and just a couple of its largest customers—have made in this technology:
 

Is IBM going to ditch POWER?

1. Power Systems is a large successful business. I’m not allowed to detail the numbers of specific parts of IBM (some details are in the IBM Annual Statement) but shall we say it is in the many billions of dollars scale. IBM is not going to shut that down on a whim.
2. [About] four years ago, I was asked by a worldwide banking client to audit a particular workload on POWER7+ as it was about to move to POWER8. It was only ~ 25 CPU cores but lots of RAM and was running nicely but pretty busy.

• I was to predict how it would run on POWER8 and any recommended tuning changes.
• At lunchtime, the client visited my cubical to explain this was an important workload.
• If the workload server failed for an hour, it would seriously impact the bank’s reputation.
• If it was not available for a whole day the world economy might not recover!!!
• “No pressure,” she said as she walks back down the corridor laughing loudly!
• I started triple checking all my figures!
• This is the sort of workload clients insist on running POWER servers—it has to fast and it has to be reliable.
• The server was actually running in a quadruple backup server arrangement and across countries.

3. I am now working on a soak test for a High-Performance Cluster at a racing team (can’t say more). The soak test involves taking the server to 100% busy for 10 days with a 2 day cooling down period in the middle. This, I think, is due to unreliable previous hardware. The client is moving to POWER9 for high performance and super high RAS = Reliability, Availability, Serviceability. It does not breakdown; if in the unlikely case of a break down it stay up running (disabling the fault components); then the faulty components can be replaced online or at a later convenient time). With Live Partition Mobility (LPM) running workloads can be moved while running to a different server. This is all normal for Power Systems.

 While I’ve previously explained why AIX isn’t going away, I want to make you aware of Nigel’s thoughts and some of the reasoning behind them. He covers much more ground, so be sure to read the whole post.

Edit: The first post on the new website

Originally posted July 30, 2019 on AIXchange

In this AIXchange blog post, Technical Editor Rob McNelly explains network attributes

This Technote—authored by Darshan Patel, Perry Smith and Saania Khanna—has some important information about VIO server configuration.

I’ll start with their conclusions:

• Largesend and jumbo frame does two different actions. These actions are mutually independent.
• Largesend decides who does the segmentation
• Jumbo frame decides the size of the segmented packet that goes on the wire

This is in response to this query: “I am configuring VIO Servers and AIX VIO Clients on Power system. There are many network attributes related to laregsend and jumbo frame such as largesend, large_send, mtu_bypass, jumbo_frame, use_jumbo_frame. I am confused about these attributes. I want to understand how largesend and jumbo_frame work and what is the difference between two.”

The document includes illustrations and outlines various configuration scenarios:

• In case 1, VIO server has largesend on, jumbo frames off, VIO client has MBP off, and MTU of 1500. In this case, messages are segmented to MTU size 1500 by TCP Kernel stack in AIX.
• In case 2, VIO server has largesend on, jumbo frames off, VIO client has MBP on, and MTU of 1500. In this case messages are segmented to MTU size 1500 by Real Ethernet Adapter.
• In case 3, VIO server has largesend on, jumbo frames on, VIO client has MBP off, and MTU of 9000. In this case messages are segmented to MTU size 9000 by TCP Kernel stack in AIX.
• In case 4, VIO server has largesend on, jumbo frames on, VIO client has MBP on and MTU of 9000. In this case, messages are segmented to MTU size 9000 by Real Ethernet Adapter.

 This Technote should help you understand the different settings and give you an idea of the results you should expect to see based on how you configure your environment. Be sure to read the whole thing.

Edit: I still love this document.

Originally posted July 23, 2019 on AIXchange

The Power Systems Best Practices document was recently updated. This is the presentation that IBMer Fredrik Lundholm maintains. 

In his email notification, Fredrik requested reader feedback:

I would really like some comments and feedback on VIOS rules, and if we can automate a few more parameters.

I changed into a new presentation format, I hope you like it and please forward any feedback as usual.

Dated June 19, this is version 2.1. As I’ve not written about this for a couple of years, here’s a summary of all the changes from the current and previous updates:

• Changes for 2.1:New Layout, POWER9 Enterprise, VIOS, AIX recommendation VIOS rules, recommended network and SAN adapters, cleaned out obsolete content.
• Changes for 1.20: Rootvg failure monitoring in PowerHA 7.2, Default Processor mode.
• Changes for 1.19: 2018 April Update, POWER9 enablement, Spectrum Scale 4.2 certified with Oracle RAC 12c.
• Changes for 1.18: 2017 September Update, new AIX default multipathing for SVC.
• Changes for 1.17: 2017 Update, VIOS 2.2.5, poll_uplinkclarification (edit).

This disclaimer is from Fredrik’s introduction:

While this presentation lists the expected best practices, all customer engagements are unique. It is acceptable to adapt and make implementation deviations after a mandatory review with the responsible architect (not only engaging the customer) and properly documenting these.

Administrators should appreciate this message. Best practices are of course valuable, but rigidly adhering to standards 100 percent of the time won’t cut it. 

Fredrik does a great service, and, as I’ve noted, he has a following. But if you’re not already familiar with this document, take the time to review it. Odds are you’ll learn something.

Edit: I did not make it another 12 years.

Originally posted July 16, 2019 on AIXchange

After 12 years I guess it’s inevitable, but this blog is no longer the only AIXchange in town. 

Occasionally I do web searches on “aixchange” to locate my old articles. But over the past year or so, these searches have led me to a number of other things. There was the 2018 conference on AI, a tour agency offering excursions to Europe and India, the Facebook group for residents of AIX-en-Provence, France, and whatever this company is.

I suppose it’s not surprising that the name would pop up elsewhere online. But on a symbolic level at least, I like to think it means that this particular AIXchange, the blog I’ve authored since July 2007, was in fact a good idea. 

I’d already been contributing articles to IBM Systems Magazine for about three years when Evelyn Hoover, the publication’s content director, approached me about doing a weekly blog. While I don’t precisely recall how we settled on the AIXchange name, I found some old emails where this was discussed. Rejected names include AdminExchange, IT Calibrate and AIXexchange. Gee, that last one is kind of a tongue-twister. 

At first I was pretty apprehensive about writing every week. I wondered how I could continually feed the beast, as it were. But I soon realized that coming up with ideas isn’t that hard. Going in depth on IBM announcements or sharing my experiences helping resolve client issues is natural fodder for this blog. I’ll also get personal from time to time. 

Mostly though I just keep note of interesting information, whether it involves AIX, IBM hardware or really, anything with technology. At any given time my inbox will contain several article drafts. Eventually I’ll finish them up and send them to my editor. 

On that note, I should add that this has always been a team effort. I’m grateful that Evelyn thought of me in the first place. And my editor, Neil Tardy, whose feature articles can be found elsewhere on this site, has been with me from week one. After all this time we’ve developed a shorthand that makes the editing process pretty simple. While we have our occasional back and forths, I generally go with his suggestions because each week, he makes me look good. 

One reason I love doing this is it forces me to pay attention to new products, announcements, tips and tricks, etc. This blog allows me to document things for me to rediscover later, and it allows me to connect with many of you as you seek clarification and/or voice your opinions about my writings. 

So thanks to everyone. Here’s to another 12 years—at least.

Edit: Some links no longer work

Originally posted July 9, 2019 on AIXchange

I’ve written about the IBM Systems Technical University (TechU) conferences many times, most recently following the event in Atlanta in May. As most of our audience is in the United States, I’ll note the fall conference is in October in Las Vegas, but TechU events are scheduled worldwide, so keep this on your radar.

One thing I appreciate about TechU is that they get the little things right. For instance, you can download slides from any session once you get home. Of course presenters typically include their contact info on their presentations, and I’ve found most are willing to respond to follow-up queries. 

For an idea of what went on at the Atlanta conference I attended, see this. Or check out this Twitter graph that analyzed tweets with the #IBMTechU hastag:

The graph represents a network of 493 Twitter users whose recent tweets contained “#IBMTechU”, or who were replied to or mentioned in those tweets, taken from a data set limited to a maximum of 18,000 tweets. The network was obtained from Twitter on Wednesday, 08 May 2019 at 17:12 UTC.

The tweets in the network were tweeted over the 9-day, 1-hour, 44-minute period from Monday, 29 April 2019 at 14:18 UTC to Wednesday, 08 May 2019 at 16:03 UTC.

Additional tweets that were mentioned in this data set were also collected from prior time periods. These tweets may expand the complete time period of the data.

This provides a handy list of people to follow during the conference. I like knowing which sessions are well attended and what conference-goers are interested in, and I join in those conversations. Even if I wasn’t there, I’d still like to know what I’m missing out on. 

Believe me, TechU is worth your time and effort.

Edit: Still useful to know about

Originally posted June 25, 2019 on AIXchange

Those of you who are running E980 servers should be aware of this announcement:

IBM Power Enterprise Pools 2.0 is a new IBM Power E980 offering designed to deliver enhanced multisystem resource sharing and by-the-minute consumption of on-premises Power E980 compute resources to clients deploying and managing a private cloud infrastructure.

All installed processors and memory on servers in a Power Enterprise Pool (2.0) are activated and made available for immediate use when a pool is started. There is no need to reallocate mobile resources from server to server.

System capacity may be seamlessly made available when it is needed without requiring human awareness or intervention.

Permanent, Capacity Upgrade on Demand processor and memory activations (Base Capacity) and corresponding license entitlements are purchased on each Power E980 system. These Base Processor and Memory Activation resources are then aggregated across a pool.

Unpurchased (inactive upon shipment) processor and memory capacity in the pool is activated when a pool is started and can subsequently be used on a pay-as-you-go basis (Metered Capacity) from Capacity Credits purchased from IBM or an authorized IBM Business Partner.

Processor resource within a pool is tracked by the minute, based on actual consumption by shared processor partitions.

Memory resource within a pool is tracked by the minute, based on the assignment of resources to partitions, not based on operating system usage of the memory.

Metered Capacity consumption on one system may be offset by idle Base Capacity elsewhere in the pool during the same period.

A single Power Enterprise Pool (2.0) may support up to 500 shared processor partitions across up to 16 Power E980 systems within a single enterprise, within a single country.

Each Power Enterprise Pool (2.0) is monitored and managed from a Cloud Management Console in the IBM Cloud.

Capacity Credits may be purchased from IBM, an authorized IBM Business Partner, or online through the IBM Entitled System Support website, where available.

Clients may more easily identify capacity usage and trends across their Power E980 systems in a pool by viewing web-accessible aggregated data without spreadsheets or custom analysis tools.

So rather than manually activate resources and dynamically move LPARs or CPU or memory entitlements around the physical machines in your environment, capacity is simply available. And you use it. 

Depending on which systems and LPARs are busy, you could theoretically establish offsetting usage patterns—e.g., one machine is busy during the day and one handles the off hours. If you exceed your base resources across your whole pool—say, if all systems are busy at once—you’d be billed on a metered basis for the resources you temporarily use. 

Power Enterprise Pools is an interesting solution that’s worth considering for your environment.