Sun 3510FC array long checkpoints resolved
Posted in 2007
I am posting a FYI incase anyone else has strange performance issues with Sun's 3510 disk arrays. About twice a month I was experiencing very long checkpoints (100+ seconds) for about 3-4 hours. Normally our checkpoints are 0. The arrays are hardware mirrored and striped using Veritas for two 3510FC arrays with 15k disks. The problem happened again this week at our worst time 3:30pm. As usual I could not find any apps that were causing any huge I/O. This time I decided to focus on the hardware. I telneted into the raid array. To my surprise there was a "WT" on the console screen that I never noticed before. It turns out the 3510 array cycles it's battery on a monthly basis and during the cycle a trigger is set to disable the write cache when the battery is low. I forced the write cache back on and bingo the long checkpoints went down to zero seconds. I made a call to Sun and they confirmed the write cache is disabled during this battery maintenance. There is an option in the array to disable that trigger which I did. I reviewed the event log and sure enough every battery cycle coincided with my online.log file with the long checkpoints. Gary Quiring