MySQL database repairs itself in Scrutinizer v7

A common occurrence and cause for data loss in Scrutinizer prior to version 7 has been crashed database tables.  It has also been the subject of many calls to our technical support group. Due to the large amount of NetFlow data received by NetFlow collectors, corrupted database tables can cause a large amount of data loss in a short period of time.

To minimize data loss and support time required to repair the corrupted database tables, in version 7 of our NetFlow and sFlow analyzer, we have introduced the ‘self-healing’ database.

A MySQL database check and repair is run on a regularly scheduled basis, once an hour.  If the database check finds any corrupted database tables, it will attempt the repair.  If it is unable to repair, an alarm will be generated to Scrutinizer to alert you of the corrupted table.

Also, the Server Health LED (right-most of the four system LEDs in top right of the screen) will turn red if database corruption is detected.  Another note on the Server Health LED – If the servers disk space drops below 2 GB or available memory is less than 128 MB, this LED will turn yellow.

If the Server Health LED is red, clicking on that LED will display what database corruption exists.  For example, in the image below, the plixer.fa_threats and plixer.task_results database repairs failed, but the repair for the plixer.fa_topdomainresolve database table was successful.

An alarm is also generated when Scrutinizer is unable to successfully repair a corrupted database table and can be viewed by going to the Alarms tab and selecting the Database Health selection from the dropdown list then clicking the Search button.  If you have a syslog server configured in Admin->Settings->Syslog Server, then you can also configure your syslog server to alert you when a database repair is unsuccessful.  Thus minimizing data loss.

In the example documented in this blog, the auto-repair failed the first time, but by the time I was able to run a manual check and repair, the auto-repair had successfully repaired the database table.  In other words, there are few cases of database corruption that will require manual intervention.  But if the corruption is persistent, please contact technical support for assistance.

Or if you have not yet upgraded to Scrutinizer v7 NetFlow and sFlow Analyzer, please contact us for instructions and/or assistance with the upgrade so you, too, can benefit from the many new features we have introduced in version 7 of our network analysis application.