This is an analysis of the data contained in the file sar15. The data was collected on 2007/05/15 (yyyy/mm/dd), from 08:00:00 to 18:00:00, from the system 'paxhp1'. There were 31 data records used to produce this analysis. The operating system used to produce the sar report was HP-UX Release B.11.11. 1 processor is present. 2048 megabytes of memory are present.
Data collected by the ps -elf command on 2007/05/15 from 08:00:00 to 17:40:00, and stored in the file 20070515, will also be analyzed.
Table of Contents
A change to at least one tunable parameter has been recommended. When the data was collected, no CPU bottleneck could be detected. No significant I/O bottleneck was seen. The system showed signs of occasional memory pressure, but not enough to suggest a problem.
Limits to future growth have been noted in the Capacity Planning section.
All recommendations contained in this report are based solely on the conditions which were present when the performance data was collected. It is possible that conditions which were not present at that time may cause some of these recommendations to result in worse performance. To minimize this risk, analyze data from several different days, implement only regularly occurring recommendations, and implement them one at a time.
Change the value of 'dbc_max_pct' from 15 to 18. This recommendation has been made because the buffer cache statistics indicate that a larger buffer cache might improve performance.
Change the value of 'dbc_min_pct' from 5 to 6. This recommendation has been made because the buffer cache statistics indicate that a larger buffer cache might improve performance.
Change the value of 'nfile' from 5000 to 5910. The parameter 'nfile' sets the size of the file descriptor table, which determines the total number of files which can be simultaneously open on the system.
Use the System Administration Manager (SAM) to change the values of tunable parameters. More information on the SAM utility and relinking the kernel is available in the System Administration Tasks manual.
Average CPU utilization was only 4.8 percent. This indicates that spare CPU capacity exists. If any performance problems were seen during the entire monitoring period, they were not caused by a lack of CPU power. User CPU as measured by the %usr column in the sar -u data averaged 2.48 percent and system CPU (%sys) averaged 2.35 percent. The sys/usr ratio averaged 0.95 : 1. The ratio of %sys to %usr activity did not indicate any excessive overhead caused by system calls. CPU utilization peaked at 11 percent during multiple time intervals.
The CPU was waiting for I/O an average of 6.0 percent of the time. This statistic does not indicate the presence of an I/O bottleneck. In at least one quarter of the samples, more than 7 percent of the CPU's time was spent waiting for disk I/O. This infers the possibility of an intermittent I/O bottleneck. The time that the system was waiting for I/O peaked at 18 percent from 13:40:01 to 14:00:00. Disk statistics indicate that some intermittent bottlenecks may have been present.
The CPU was idle (neither busy nor waiting for I/O) and had nothing to do an average of 89.1 percent of the time. If overall performance was good, this means that on average, the CPU was lightly loaded. If performance was generally unacceptable, the bottleneck may have been caused by remote file I/O which cannot be directly measured with sar and therefore cannot be considered by SarCheck.
The run queue had an average depth of 13.6 which indicates that a CPU bottleneck was present and performance degradation was likely. The run queue was usually occupied, which is to be expected on a system with a significant CPU queue depth. Average run queue depth (when occupied) peaked at 45.9 from 08:40:00 to 09:00:00. During that interval, the queue was occupied 100 percent of the time. Peak resource utilization statistics can be used to help understand performance problems. If performance was worst during the period of peak CPU queuing, then a performance bottleneck may be the CPU. Consistently high values normally indicate a CPU-bound environment but the sar -u statistics did not confirm this.
The peak run queue occupancy seen was 100 percent from 07:40:00 to 18:00:00. The following graph shows both the run queue length and occupancy. The occupancy is shown as %runocc/10, where a run queue occupied 100 percent of the time would be shown a vertical line reaching a height of 10.0.
The average context switching rate was 2232.1 per second. This works out to an average of one context switch every 0.45 milliseconds. No recommendations have been made to the timeslice parameter because no problems were seen with the context switching rate.
No unusual configurable parameter values were seen in those parameters which relate to the process accounting system. The current values of acctsuspend and acctresume are unlikely to have an impact on system performance.
Large systems, such as this one, may experience system panics due to a lack of 'equivalently mapped memory'. The current value of eqmemsize, the parameter which controls the size of equivalently mapped memory, is 15 and should be increased if system failures occur. There is not enough information to recommend a specific value for eqmemsize.
The average rate of System V semaphore calls was 0.140 per second. No problems have been seen, and no changes have been recommended for System V semaphore parameters. Note that SarCheck only checks these parameter's relationships to each other since semaphore usage data is not available. Algorithms used by SarCheck to check these relationships are available in the help text of SAM.
No System V message activity was seen. No problems have been seen, and no changes have been recommended for System V message parameters. Note that SarCheck only checks these parameter's relationships to each other since message usage data is not available. Algorithms used by SarCheck to check these relationships are available in the help text of SAM, and in the file /usr/include/sys/msg.h.
The ratio of exec to fork system calls was 1.00. This indicates that PATH variables are efficient.
The syncer daemon used 0.006 percent of the CPU during the monitoring period. The syncer is responsible for writing data from the buffer cache to disk. It's activity indicates that it is not so active as to cause a problem.
This system's buffer cache is dynamic, meaning that its size is determined by the amount of free memory on the system. The average cache hit ratio of logical reads was only 60.8 percent. Based on the current values of dbc_min_pct and dbc_max_pct, the buffer cache can range in size from 102 to 307 megabytes of memory. The actual size of the dynamic buffer cache ranged from 279.2 to 297.8 megabytes of memory. After implementation of the changes described in the Recommendations Section, the size of the dynamic buffer cache will range from 122.9 to 368.6 megabytes of memory.
The following graph shows that the actual size of the dynamic buffer cache did not fall far below the amount allowed by the dbc_max_pct parameter. This is common on systems with little or no memory pressure.
The swap out rate was always 1.00 per second. This does not necessarily indicate a problem and is not an unusual phenomenon on HP-UX systems.
Data collected with ps -elf shows that the swapper daemon used an average of 0.325 percent of the CPU. This indicates a possible memory shortage.
At least 2690 pages of memory were always free. The value of lotsfree was 8192 pages and the value of gpgslim never changed, indicating a lack of memory pressure. The value of desfree was also 1024 pages and the fact that gpgslim always equalled desfree does not indicate a memory-poor condition. The following graph illustrates the fact that freemem was always less than the value of lotsfree, indicating that this system usually needed to free up some memory.
The inode cache did not fill up during the monitoring period. This indicates that the cache is larger than necessary, but no changes have been recommended because no significant memory bottleneck was detected and the cache was not large enough to degrade performance. Peak inode cache usage statistics (max used/cache size) as reported by sar: 1039/4438.
During part of the monitoring period, the file table was nearly full. Specific recommendations for increasing the size of this table have been made in the recommendations section. Peak table usage statistics (max used/final table size) as reported by sar: Process table: 352/4020. Open file table: 4673/5000.
The fs_async flag is set to 0. This may result in reduced disk performance, but keeps filesystem data structures consistent in the event of a system crash. This option is currently in the state recommended for production systems. Since no disk I/O bottleneck was seen on this system, setting the fs_async flag would be unlikely to provide enough of an improvement to justify the additional risk.
There were 4 volume groups seen and the maxvgs parameter was set to 10. This leaves plenty of room for growth and no changes to maxvgs have been recommended.
The volume group /dev/vg00 contained 2 physical volumes and 8 logical volumes. All of the physical volumes were active and all of the logical volumes were open. The size of the group was 67.81 gigabytes, of which 72.86 percent was allocated and 27.14 percent was free.
The volume group /dev/vg01 contained 1 physical volume and 3 logical volumes. All of the logical volumes were open. The size of the group was 101.73 gigabytes, of which 100.00 percent was allocated.
The volume group /dev/vg02 contained 1 physical volume and 1 logical volume. All of the physical volumes were active and all of the logical volumes were open. The size of the group was 33.91 gigabytes, of which 100.00 percent was allocated.
The volume group /dev/vg03 contained 2 physical volumes and 1 logical volume. All of the physical volumes were active. The size of the group was 135.65 gigabytes, of which 100.00 percent was allocated.
The average system-wide local I/O rate as measured by the r+w/s column in sar -d was 40.4 per second. This I/O rate peaked at 143 from 13:40:01 to 14:00:00.
The following graph shows the average and peak percent busy and service time for 3 disks, sorted by percent busy.
The -dbusy switch has been used to sort the disk analysis by the average percent of time the disk was busy. The -dtoo switch has been used to format disk statistics into the following table.
| Disk Device Statistics Results sorted by Average Percent Busy | |||||||
|---|---|---|---|---|---|---|---|
| Disk Device (vol group) | Avg Pct Busy | Peak Pct Busy | Avg Queue Depth | Avg Svc Time | Disk Size | Pct Allocated | Fragmented LVs? |
| c6t0d0 (/dev/vg01) | 7.62 % | 30.24 % | 1.2 | 3.8 ms | 101.73 GB 13022 blocks | 100.00 % | No |
| c1t15d0 (/dev/vg00) | 1.13 % | 2.25 % | 0.8 | 9.1 ms | 33.91 GB 4340 blocks | 72.86 % | Yes |
| c3t15d0 (/dev/vg00) | 0.79 % | 1.46 % | 0.9 | 9.0 ms | 33.91 GB 4340 blocks | 72.86 % | Yes |
The disk device c6t0d0 was busy an average of 7.62 percent of the time and had an average queue depth of 1.2 (when occupied). This indicates that the device is not a performance bottleneck. The average service time reported for this device and its accompanying disk subsystem was 3.8 milliseconds. This is indicative of a very fast disk or a disk controller with cache. Service time is the delay between the time a request was sent to a device and the time that the device signaled completion of the request. The disk device c6t0d0 was reported by pvdisplay as being a 101.73 gigabyte disk. All of the space on this disk device has been allocated. This disk device was a part of volume group /dev/vg01 and contained 3 logical volumes. Each logical volume on this disk resided in a contiguous block of physical extents. This is the most efficient layout for a logical volume.
The disk device c1t15d0 was busy an average of 1.13 percent of the time and had an average queue depth of 0.8 (when occupied). This indicates that the device is not a performance bottleneck. The average service time reported for this device and its accompanying disk subsystem was 9.1 milliseconds. This is relatively fast. The disk device c1t15d0 was reported by pvdisplay as being a 33.91 gigabyte disk. 9424 megabytes of space was reported as being free and 25296 megabytes have been allocated. This disk device was a part of volume group /dev/vg00 and contained 8 logical volumes. At least one logical volume occupied noncontiguous physical extents on the disk. Performance will suffer when logical volumes are busy and not mirrored because the disk's read/write heads are likely to travel back and forth in an inefficient manner.
The disk device c3t15d0 was busy an average of 0.79 percent of the time and had an average queue depth of 0.9 (when occupied). This indicates that the device is not a performance bottleneck. The average service time reported for this device and its accompanying disk subsystem was 9.0 milliseconds. This is relatively fast. The disk device c3t15d0 was reported by pvdisplay as being a 33.91 gigabyte disk. 9424 megabytes of space was reported as being free and 25296 megabytes have been allocated. This disk device was a part of volume group /dev/vg00 and contained 8 logical volumes. At least one logical volume occupied noncontiguous physical extents on the disk.
At 15:00:00 ps -elf data indicated that there were 351 processes present. This was the largest number of processes seen with ps -elf but it is not likely to be the absolute peak because the operating system does not store the true "high-water mark" for this statistic. There were an average of 341.6 processes present.
No runaway processes, memory leaks, or suspiciously large processes were detected in the data contained in file 20070515. No table was generated because no unusual resource utilization was seen in the ps -elf data.
This section is designed to provide the user with a rudimentary linear capacity planning model and should be used for rough approximations only. These estimates assume that an increase in workload will affect the usage of all resources equally. These estimates should be used on days when the load is heaviest to determine approximately how much spare capacity remains at peak times.
Based on the limited data available in this single sar report, the system cannot support an increase in workload at peak times without some loss of performance or reliability, and the bottleneck is likely to be the size of the file table (nfile). Implementation of some of the suggestions in the recommendations section may help to increase the system's capacity.
The CPU can support an increase in workload of at least 100 percent at peak times. The busiest disk can support a workload increase of at least 100 percent at peak times. For more information on peak CPU and disk utilization, refer to the Resource Analysis section of this report.
The process table, controlled by the parameter 'nproc', can support at least a 100 percent increase in the number of entries. The file table, controlled by the parameter 'nfile', can support approximately a 0 percent increase in the number of entries.
The default MLTIME threshold was changed in the sarcheck_parms file from 7195 to 1500 seconds. This value is likely to compromise the accuracy of the analysis.
The default DCLP threshold was changed in the sarcheck_parms file from 10 to 5. This keyword controls the number of suspiciously large processes which are reported.
The default DCML threshold was changed in the sarcheck_parms file from 10 to 15. This keyword controls the number of processes with possible memory leaks which are reported.
The default DCRP threshold was changed in the sarcheck_parms file from 10 to 5. This keyword controls the number of possible runaway processes which are reported.
The default TEXTCOLOR value was changed in the sarcheck_parms file from black to blue.
The date format used in this report is yyyy/mm/dd. It was set in the sarcheck_parms file.
Please note: In no event can Aptitune Corporation be held responsible for any damages, including incidental or consequent damages, in connection with or arising out of the use or inability to use this software. All trademarks belong to their respective owners. Evaluation copy for: Your Company. This software expires on 2007/08/08 (yyyy/mm/dd). SC9000 Code version: 6.02.00. Serial number: 56172654.
Thank you for trying this evaluation copy of SarCheck. To order a licensed version of this software, just type 'analyze9000 -o' at the prompt to produce the order form, and follow the instructions.
(c) copyright 1995-2007 by Aptitune Corporation, Plaistow NH, USA, All Rights Reserved. http://www.sarcheck.com
| Statistics for system, paxhp1 | ||||
|---|---|---|---|---|
| Start of peak interval | End of peak interval | Date of peak interval | ||
| System model number is, | 9000/785/C360 | |||
| Statistics collected on, | 2007/05/15 | |||
| Average CPU utilization, | 4.8% | |||
| Peak CPU utilization, | 11% | Multiple peaks | Multiple peaks | |
| Average user CPU utilization, | 2.5% | |||
| Average sys CPU utilization, | 2.4% | |||
| Average waiting for I/O, | 6.0% | |||
| Average run queue depth, | 13.6 | |||
| Peak run queue depth, | 45.9 | 08:40:00 | 09:00:00 | 2007/05/15 |
| Average swap queue occupancy, | 0.0% | |||
| Average swap out rate, | 1.00/sec | |||
| Average cache read hit ratio, | 60.8% | |||
| Average cache write hit ratio, | 75.9% | |||
| Disk device w/highest peak, | c6t0d0 | |||
| Avg pct busy for that disk, | 7.62% | |||
| Peak pct busy for that disk, | 30.24% | 17:40:00 | 18:00:00 | 2007/05/15 |
| Avg number of processes seen by ps, | 341.6 | |||
| Max number of processes seen by ps, | 351 | 15:00:00 | 2007/05/15 | |
| Percent of process tbl used, | 8.8% | |||
| Process table overflows, | No | |||
| Percent of file table used, | 93.5% | |||
| File table overflows, | No | |||
| Inode cache pct of time full, | 0.0% | |||
| Inode cache overflows, | No | |||
| Approx CPU capacity remaining, | 100%+ | |||
| Approx I/O bandwidth remaining, | 100%+ | |||
| Remaining process tbl capacity, | 100%+ | |||
| Remaining file table capacity, | 0.0% | |||
| Can memory support add'l load, | Yes | |||