SarCheck(TM): Automated Analysis of Solaris sar and ps data

(English text version s4.00)

This is an analysis of the data contained in the file /tmp/rpt. The data was collected on 04/06/2001, from 09:20:01 to 16:40:00, from system 'drew'. There were 22 sar data records used to produce this analysis. Operating system is Solaris 2.7. 1 processor is present. 64 megabytes of memory are present.

Data collected by the ps -elf command on 04/06/2001 from 09:20:01 to 16:40:01, and stored in the file /opt/sarcheck/ps/20010406, will also be analyzed.

SUMMARY

When the data was collected, no CPU bottleneck could be detected. No significant memory bottleneck was seen. No significant I/O bottleneck was seen. A change has been recommended to at least one tunable parameter. Limits to future growth have been noted in the Capacity Planning section.

RECOMMENDATIONS SECTION

All recommendations contained in this report are based solely on the conditions which were present when the performance data was collected. It is possible that conditions which were not present at that time may cause some of these recommendations to result in worse performance. To minimize this risk, analyze data from several different days, implement only regularly occurring recommendations, and implement them one at a time.

A CPU upgrade is not recommended because the current CPU had significant unused capacity.

Change the value of priority paging from 0 to 1. This parameter can be changed by adding the following line to the /etc/system file: 'set priority_paging = 1'. NOTE: Don't forget to check /etc/system first to see if there's already a set command modifying this tunable parameter. If there is, modify that command instead of adding another one.

Change the value of ncsize from 4236 to 8472. This change is recommended because the DNLC hit ratio, as calculated from kernel statistics, is only 59.97 percent. Kernel statistics have been used instead of sar -a data because the values seen in the sar data were too low to calculate the DNLC hit ratio accurately. The low values seen in the sar -a data mean that the degree of improvement realized by implementing this recommendation is likely to be small. This parameter can be changed by adding the following line to the /etc/system file: 'set ncsize = 8472'. NOTE: Don't forget to check /etc/system first to see if there's already a set command modifying this tunable parameter. If there is, modify that command instead of adding another one. Once this change has been implemented, changes to the maxusers parameter will no longer affect ncsize.

Change the value of ufs_ninode from 4236 to 8472. This recommendation will increase the value of ufs_ninode to match the value of ncsize. Whenever a recommendation is made to increase ncsize, this program will also recommend increasing ufs_ninode to the same value. For more information, see page 309 of the second edition of Sun Performance and Tuning by Adrian Cockcroft. This parameter can be changed by adding the following line to the /etc/system file: 'set ufs_ninode = 8472'. NOTE: Don't forget to check /etc/system first to see if there's already a set command modifying this tunable parameter. If there is, modify that command instead of adding another one. Once this change has been implemented, changes to the maxusers parameter will no longer affect ufs_ninode.

We do not recommend implementing all of these changes at once. More information on how to change tunable parameters is available in the System Administration Guide. We recommend making a copy of /etc/system before making changes, and understanding how to use boot -a in case your changes to /etc/system create an unbootable system.

RESOURCE ANALYSIS SECTION

Average CPU utilization was only 0.1 percent. This indicates that spare capacity exists within the CPU. If any performance problems were seen during the monitoring period, they were not caused by a lack of CPU power.

The CPU was idle (neither busy nor waiting for I/O) and apparently had nothing to do an average of 99.9 percent of the time. If overall performance was good, this means that on average, the CPU was lightly loaded. If performance was generally unacceptable, the bottleneck may have been caused by remote file I/O which cannot be directly measured with sar and cannot be considered by SarCheck.

The CPU was waiting for I/O an average of 0.0 percent of the time. This confirms the lack of a regularly occurring I/O bottleneck.

The average cache hit ratio of logical reads was 98.1 percent, and the average cache hit ratio of logical writes was 73.9 percent. These statistics, and the lack of any significant memory bottleneck, indicate that there is little to gain by changing the value of bufhwm.

The fsflush daemon used only 0.00 percent of the CPU. This indicates that it is probably not using enough of the CPU to cause a problem.

In the event of a system crash, an average of 32 seconds worth of data will be lost because it will not have been written to disk. This is controlled by the autoup and tune_t_fsflushr parameters. This statistic has been calculated using the formula: autoup + (tune_t_fsflushr / 2).

The ratio of exec to fork system calls was 1.02. This indicates that PATH variables are efficient.

The DNLC hit ratio, as calculated from the sar -a statistics, was 100.00 percent. Due to the presence of low numbers in the iget/s or namei/s fields of the sar -a report, this hit ratio is likely to be inaccurate. As a result, the current DNLC hit ratio has been calculated using real time kernel statistics, and the current hit ratio is 59.97 percent. As a rule, if the DNLC ratio is much less than 90 percent, an increase in the value of ncsize will be recommended. The disadvantage of using kernel statistics is that they represent real time data, not data collected at the same time as the sar and ps -elf statistics.

The value of maxuprc is 917 and the size of the process table as reported by sar was 922. There is no reason to change the value of maxuprc or max_nprocs based on this data.

The number of active inodes fit in the inode cache. This indicates that the inode cache was large enough to efficiently meet the needs of the system. Peak used/max statistics for the inode cache during the monitoring period were 2274/4236. The value of %ufs_ipf was always zero, but an increase in the size of the DNLC cache was recommended. Whenever a recommendation is made to increase ncsize, this program will also recommend increasing ufs_ninode to the same value. For more information, see page 309 of the second edition of Sun Performance and Tuning by Adrian Cockcroft.

At least 6.5 percent of the system's memory, or 4.2 megabytes, was always unused during sar sampling. This indicates that while the system is not in need of memory, there isn't an unusually large quantity of physical memory that remains unused. Please note that this value is not a true high-water mark of memory usage and only reflects what was happening when sar sampled system activity.

The average page scanning rate was 0.04 per second. Page scanning peaked at 0.65 per second from 09:40:00 to 10:00:00. Peak resource utilization statistics can be used to help understand performance problems. If performance was worst during the period of high scanning activity, then a performance bottleneck may be caused by heavy I/O or the lack of memory available.

We recommended a change to priority paging from 0 to 1. Priority paging stops random or non 8k filesystem I/O from slowing the system. The pager only frees application pages when there is a real memory shortage.

The average system-wide local I/O rate as measured by the r+w/s column in the sar -d data was 0.05 per second. This I/O rate peaked at 1 per second from 12:20:00 to 12:40:00.

The -dtoo swich has been used to format disk statistics into the following table.

The device dad0 was busy an average of 0.1 percent of the time and had an average queue depth of 0.0 (when occupied). The average service time reported for this device and its accompanying disk subsystem was 13.9 milliseconds. This is relatively fast considering that queuing time is included in this statistic. Service time is the delay between the time a request was sent to a device and the time that the device signaled completion of the request.

No runaway processes, memory leaks, or suspiciously large processes were detected in the data contained in file /opt/sarcheck/ps/20010406. No table was generated because no unusual resource utilization was seen in the ps -elf data.

More information on performance analysis and tuning can be found in The System Administration Guide Volumes 1 & 2, and in Adrian Cockcroft's Sun Performance and Tuning.

CAPACITY PLANNING SECTION

The section is designed to provide the user with a rudimentary linear capacity planning model and should be used for rough approximations only. These estimates assume that an increase in workload will affect the usage of all resources equally. These estimates should be used on days when the load is heaviest to determine approximately how much spare capacity remains at peak times.

Based on the data available in this single sar report, the system should be able to support a moderate increase in workload at peak times, and memory is likely to be the first resource bottleneck. See the following paragraphs for additional information.

The CPU can support an increase in workload of at least 100 percent at peak times. Because some swap space was used and significant page scanning or swapping statistics were not seen, the amount of memory present can probably handle a moderate increase in workload. The busiest disk can support a workload increase of at least 100 percent at peak times. For more information on peak CPU and disk utilization, refer to the Resource Analysis section of this report.

The process table, measured by sar -v, can hold at least twice as many entries as were seen.

Please note: In no event can Aurora Software Inc. be held responsible for any damages, including incidental or consequent damages, in connection with or arising out of the use or inability to use this software. All trademarks belong to their respective owners. Evaluation copy for: Your Company. This software expires on 06/30/2001 (mm/dd/yyyy). Code version: 4.00 for Solaris SPARC 64-bit. Serial number: 00012345.

Thank you for trying this evaluation copy of SarCheck. To order a licensed version of this software, just type 'analyze -o' at the prompt to produce the order form and follow the instructions.

**SPECIAL PROMOTION** Order a SarCheck license by 06/30/2001 and we'll send you one of our famous propeller hats! See the order form (analyze -o) for details.

(c) copyright 1994-2001 by Aurora Software Inc., Plaistow NH 03865, USA, All Rights Reserved. http://www.sarcheck.com/