Anyone who has used the Work with System Activity (WRKSYSACT) command can tell you how useful it is to identify jobs in real time that are using too many system resources, usually CPU. What you may not know is that you can also run WRKSYSACT in batch to collect data about how well your system is performing.
When you run WRKSYSACT in batch, data is collected to a default file, or one that you specify, and later you can run another command called Print Activity Report (PRTACTRPT) against the collected data. There are a variety of options in the PRTACTRPT command that allow you to view the data a number of ways so that you can find jobs that may not be performing optimally.
You can run PRTACTRPT repeatedly over the same data collection to view the data in different ways. For example, maybe the job that is causing you trouble is not CPU bound, but rather I/O bound. You would be unlikely to see the job if you sorted the results by CPU, but when sorted by total number of I/O's it may stand out like a sore thumb.
Running WRKSYSACT in batch is often a very good place to start when you suspect you have one or more jobs that are misbehaving. It can be especially useful if you have a performance slowdown at a specific time of the day that occurs regularly. You can schedule a run of WRKSYSACT from the job scheduler on the system to capture data during the slowdown even if it is off shift. Later, you can run the PRTACTRPT command to view the data and find those troublemaker jobs. One warning here, you will need to run PRTACTRPT between successive job scheduler runs of WRKSYSACT because WRKSYSACT erases any data in the specified output file when it starts.
To run WRKSYSACT in batch, type in WRKSYSACT at a command line and press F4 to prompt and then F9 to show all parameters. You will see a screen like the one shown in Figure 1.
Work with System Activity (WRKSYSACT)
Type choices, press Enter.
Output . . . . . . . . . . . . . * *, *FILE, *BOTH
Interval length . . . . . . . . 60 1 - 900 seconds
Sequence . . . . . . . . . . . . *CPU *CPU, *DBCPU, *IO, *STGALC...
Type of information . . . . . . *ALL *ALL, *JOBS, *TASKS
Number of intervals . . . . . . 10 Number
Member . . . . . . . . . . . . . QAITMON Name
Library . . . . . . . . . . . . QPFRDATA Name
Bottom
F3=Exit F4=Prompt F5=Refresh F12=Cancel F13=How to use this display
F24=More keys
F24=More keys
Figure 1: The WRKSYSACT command
To have WRKSYSACT submitted to batch, specify the output as *FILE. The interval length is the length of time in seconds that WRKSYSACT will run to capture data, i.e. the length of the collection interval. The number of intervals says how many 'collections' you want to make. The two combined tell you how long the WRKSYSACT command will run.
In the example in Figure 1, I have set the interval length to 1 minute and the number of intervals to 10, thus the command will run for 10 minutes and collect 10 different one-minute sets of data. You can leave the 'Sequence' parameter set to *CPU (you can sort the data later with PRTACTRPT) and the 'Type of information parameter' set to *ALL.
Note that you can use WRKSYSACT to only display licensed internal code tasks by setting the Type of Information parameter to *TASKS, in either batch or interactive mode, which can be handy if you suspect your problem is being caused by IBM code and not your application or, more likely, that your application code is driving some LIC tasks very hard. Tasks in the interactive WRKSYSACT display do not have a user profile.
Only one instance of WRKSYSACT can be run at any given time due to the fact that the command runs at a very high system priority. This rule is observed whether you run WRKSYSACT interactively or in batch, so if you specify a very long batch run, you need to remember that you and other users will be locked out of the command for the duration of that run. If WRKSYSACT is already in use, it will send a message with the userid of the person or job currently running it.
Once you have collected data for one or more intervals, you can run the PRTACTRPT. If you type PRTACTRPT at a command line and hit F4 and then F9, you will see a screen similar to that in Figure 2.
Print Activity Report (PRTACTRPT)
Type choices, press Enter.
Member . . . . . . . . . . . . . QAITMON Name
Library . . . . . . . . . . . . QPFRDATA Name
Title . . . . . . . . . . . . . *BLANK
Report type . . . . . . . . . . *SUMMARY *SUMMARY, *DETAIL, *ALL
Time period for report:
Starting time . . . . . . . . *FIRST Time, *FIRST
Starting date . . . . . . . . *FIRST Date, *FIRST
Ending time . . . . . . . . . *LAST Time, *LAST
Ending date . . . . . . . . . *LAST Date, *LAST
Sequence . . . . . . . . . . . . *CPU *JOBTASK, *USER, *PTY...
Number of jobs . . . . . . . . . 10 1-32767, *ALL
Job name . . . . . . . . . . . . PRTACTRPT Name, PRTACTRPT, *MBR
More...
F3=Exit F4=Prompt F5=Refresh F12=Cancel F13=How to use this display
F24=More keys
Figure 2: The PRTACTRPT command
The Report Type and Sequence parameters are the most important. In Report Type, *DETAIL will give you the type of output that is shown in Figure 3, which shows the detail for one 60-second interval, sorted by *CPU. In the Sequence parameter, specify the way in which you want the data to be sorted. Use the help for the Sequence parameter to see what each value means. Most times you will want to sort by either *CPU or *TOTALIO, but there are several more choices.
The part of the report that is of most interest is the first section, which in Figure 3 shows the top 10 CPU-consuming jobs on the system for the first collection interval (other collection intervals have been removed from the report to simplify the example, but they would be similar in structure to the interval shown). Note that BAD_JOB used 22% of the available CPU during this interval and did very little I/O. This would probably be a job worth investigating, especially if it showed up as the worst offender in multiple intervals.
System Activity Report 2/25/05 21:08:2
Page
Member . . . . : QAITMON Report Type . . . . : DETAIL Version . . . . : 5 Started . . . . : 02/25/05 21:07:0
Library . . . . : QPFRDATA Sequence . . . . . : CPU Release . . . . : 2.0 Stopped . . . . : 02/25/05 21:08:0
System name . . : SYS1
Time . . . . . . . . . . . . . . . . . . . : 15:07:05 Average CPU Utilization . . . . . . . . . . : 28.9
Number of CPUs . . . . . . . . . . . . . . : 2 CPU 1 Utilization . . . . . . . . . . . . . : 31.3
Current processing capacyty . . . . . . . . : 2.00 CPU 2 Utilization . . . . . . . . . . . . . : 29.4
Total Total -------Synchronous I/O----- ------Asynchronous I/O------ DB
Job or CPU Sync Async Total DB DB Non-DB Non-DB DB DB Non-DB Non-DB CP
Task User Number Thread Pty Util I/O I/O Wait Read Write Read Write Read Write Read Write Uti
---------- ---------- ------ -------- --- ---- ----- ----- ----- ----- ----- ------ ------ ----- ----- ------ ------ ---
BAD_JOB BADUSER 478614 00000004 15 22.6 0 9 100.7 0 0 0 0 0 9 0 0 .
MONJOB USERTWO 478094 00000001 20 1.2 307 4952 125.4 17 134 109 47 6 910 4036 0 .
JOBFIVE USER5 481265 00000015 50 1.1 1983 136 .0 60 4 1814 105 36 9 19 72 .
CFINT01 0 1.1 0 0 .0 0 0 0 0 0 0 0 0 .
OBJRTV BKUSER 480073 00000004 51 .9 3306 859 101.8 115 0 203 2988 9 14 0 836 .
OBJSND BKUSER 479558 00000001 51 .9 151 7 99.0 1 0 12 138 7 0 0 0 .
OBJSND BKUSER 480072 00000005 51 .8 563 13 98.6 119 0 378 66 13 0 0 0 .
SERVERJOB SERVER1 478200 00000001 19 .5 288 198 171.5 48 68 96 76 11 186 0 1 .
SERVERJOB SERVER2 478111 00000001 19 .4 171 159 66.2 3 68 27 73 2 155 0 2 .
SERVERJOB SERVER3 478208 00000001 19 .4 239 84 303.6 24 52 105 58 2 79 0 3 .
Job or ---------------Storage---------------
Task User Number Thread Allocated Deallocated Net
---------- ---------- ------ -------- ----------- ----------- -----------
JOBFIVE USER5 481265 00000015 9,740 2,584 7,156
JOBFIVE USER5 481268 00000009 6,383 234 6,149
JOBFIVE USER5 481266 00000005 6,383 234 6,149
JOBFIVE USER5 481264 00000007 6,383 234 6,149
BRANCH1 USER62 478504 00000003 5,512 403 5,109
LOGJOB LOGUSER 478543 00000005 2,048 0 2,048
SPL1S2 QSPLJOB 481269 00000009 10,050 8,205 1,845
WRKSTNJOB USER77 481267 0000000F 1,872 58 1,814
SERVERJOB SERVER1 478197 00000001 2,380 1,356 1,024
SERVERJOB SERVER1 478159 00000001 1,672 1,416 256
Job or Task -- Job or task name
User -- User profile associated with the job
Number -- Job number
Thread -- Thread identifier
Pty -- Job or task priority
CPU Util -- Percent of CPU used by the job or task
Total Sync I/O -- Total number of synchronous I/O operations
Total Async I/O -- Total number of asynchronous I/O operations
Total Wait -- Total percent of time spent by the job or task in waiting state
Synchronous I/O --
DB Read -- Number of synchronous database reads
DB Write -- Number of synchronous database writes
Non-DB Read -- Number of synchronous non-database reads
Non-DB Write -- Number of synchronous non-database writes
Asynchronous I/O --
DB Read -- Number of asynchronous database reads
DB Write -- Number of asynchronous database writes
Non-DB Read -- Number of asynchronous non-database reads
Non-DB Write -- Number of asynchronous non-database writes
DB CPU Util -- Percentage of CPU used to perform database processing
Storage --
Allocated -- The amount of storage (in pages) assigned to the job or task
Deallocated -- The amount of storage (in pages) released by the job or task
Net -- The difference between allocated and deallocated storage
Figure 3: A sample report from PRTACTRPT
Try WRKSYSACT in batch; you may just discover some troublesome jobs that you didn't know about.
-----------------------------------
About the author: Dan Reusche is a senior systems administrator at Think Federal Credit Union in Rochester, Minn. He has worked with the IBM AS/400 and iSeries platform since 1988, when he worked at the IBM Rochester Development Lab and support of AS/400 systems used within IBM. You may contact him at dreusche@chartermi.net.
