#Triple D Consulting #Network Attached Device Diagnostic System #README File Agent Name: diskload Agent Version: Version 3.0 Agent Author: David Blizard Agent Date: 2003-May-24 Agent Purpose: Agent monitors multiple hard disks using iostat command for heavy usage. Agent How Does It work: Diskload agent runs iostat using intervals and count to take a sample of disk stats and monitors this data for user defined max levels for transfers per second, blocks read and blocks written per second. Diskload agent sends BCNU error if any of these max levels are occuring in the sample data. Works fine with multiple disks. Interval is the amount of time(seconds) of each report generated by iostat, count is the number of reports to generate. Max warn is the maximum number of warning alerts to send before sending an error alert.(to avoid error alerts from disk spikes). Agent History: I have revamped all the code and it now no longer sends multiple messages per run. In order to accomplish this I had problems with variable scope because my while read was spawning a shell so variable changes where not accessible outside the loop so I was forced to add a file to store this variable. Also in order to work properly I added another if statement to the while loop which adds overhead but works ;). Originally I was planning on allowing multiple disks to have different alert paramaters but as of now I think one set of paramaters for all disks per host is fine. Agent Installation Instructions: Place agent in the agents directory within the $BCNUHOME directory. Add entry to agents file. agent:diskload:::::max_transfers_per_second=max_blocks_read_per_second=max_blocks_written_per_second=interval=count=max_warn: Agent call example: agent:diskload:::::max_transfers_per_second=max_blocks_read_per_second=max_blocks_written_per_second=interval=count=max_warn: eg. agent:diskload:0-24:10:10:enabled:1000=1000=1000=1=10=1: Note: If no paramaters are set the following default values are used - 1000=1000=1000=1=10=1 If not all paramaters are set defaults are used(warning: paramaters are read from left to right) Agent Operating Dependancies: Linux (Tested on Redhat 7.3 Linux Kernel 2.4.18-3) Agent Application Dependancies: iostat command (Tested with Redhat sysstat-4.0.3-2.i386.rpm) BCNU v1.22 Features To Add: - fix variable scope issue if possible(makes code faster) -----------------TESTING-------------------------- Tested by: Dave and Dustin Date Tested: April 27, 2003 Testing platform: Redhat 7.3 Linux Kernel 2.4.18-3, sysstat-4.0.3-2.i386.rpm, BCNU v1.22 Testing methodology: Tested by setting max values for tps, blkrd/sec and blkwtn/sec very low so they trigger alerts. Also accessing large files like movies were used to test this agent. Warn condtions - Expected response: "warning - Disk Load Warning Alert" BCNU message with Device Status and iostat output in the agent details. Actual response: "warning - Disk Load Warning Alert" BCNU message with Device Status and iostat output in the agent details. Sample output: "warning - Disk Load Warning Alert" BCNU message |====Device Status====| Device: dev3-0 Status: WARNING 1 Transfers Per Second errors 0 Blocks Read Per Second errors 1 Blocks Written Per Second errors |====iostat output - Mon May 26 22:44:05 PDT 2003====| Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 48.00 0.00 1248.00 0 1248 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 Successful conditions - Expected response: ok - Disks OK with supplied paramaters and disk status and output from iostat command Actual response: ok - Disks OK with supplied paramaters and disk status and output from iostat commmand Sample output: "ok - Disks OK" BCNU message |====Device Status====| Device: dev3-0 Status: OK |====iostat output - Mon May 26 23:02:20 PDT 2003====| Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 10.00 0.00 408.00 0 408 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 Error conditions - Expected response:"error - Disk Load Error Alert" BCNU message and device status and output from iostat Actual response:"error - Disk Load Error Alert" BCNU message and device status and output from iostat Sample output: "error - Disk Load Error Alert" BCNU message |====Device Status====| Device: dev3-0 Status: ERROR |====iostat output - Mon May 26 23:12:20 PDT 2003====| Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn dev3-0 65.00 1024.00 1800.00 1024 1800 dev3-0 6.00 0.00 192.00 0 192 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 4.00 0.00 88.00 0 88 dev3-0 9.00 1024.00 0.00 1024 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 0.00 0.00 0.00 0 0 dev3-0 9.00 1024.00 0.00 1024 0 dev3-0 2.00 0.00 24.00 0 24