Edit: I do not know that anyone would argue about the overhead, but you never know. I would probably call Earl Jew and Nigel Griffiths and let them hash it out. I did not edit the links, I will leave it as an exercise for the reader to google for more information if it is desired.
Originally posted July 22, 2008 on AIXchange
During a class I recently attended, an AIX systems internals guy argued that nmon and topas add (minimal) overhead to any machine that you might need to analyze. He also said that the tools’ granularity is such that they could miss some things. The intervals these tools use are measured in seconds, while events on the machine occur at the millisecond level.
His recommendation: If you really want to analyze a machine, use trace, and then use curt or trcrpt to analyze the information trace generates. Events on your machine are being collected all the time. Using trace just logs the information that’s already being collected. While the argument could then be made that logging this information creates some overhead, I think we’re just being pedantic at that point.
I use trace all the time. To get started, run this command:
trace -a -o /tmp/trc.out
Make sure you let the trace run for a reasonable period of time (hopefully long enough for the behavior that you’re trying to detect to present itself).
To end trace, run trcstop. Then you’re ready to run curt:
curt -i /tmp/trc.out | more
With curt, it obviously takes some knowledge to make sense of the information that’s generated. If you need help, IBM is an option here–I’ve seen IBM support use trace when helping cusomers with their performance problems.
Information on curt is available from the Information Center.
This Information Center link introduces trcrpt, which you may also be familiar with. This tool analyzes captured trace data.
To look at the trace information collected with trcrpt, run:
trcrpt /tmp/trc.out
curt provides a clean summary report of all the information in the trace output, but if I’m looking at something specific, I may use trcrpt.
I won’t tell you which tool you should use, but getting down to the trace level can be a good next step when you’re working on an issue.