Sending Alerts when replication breaks

For complex topics that regular users would not be interested in. For power users and database administrators.

Sending Alerts when replication breaks

Postby Manny Ramirez » Thu Nov 05, 2015 8:44 am

My RMM system allows me to create automated tasks and send me alerts. It would be nice if I could create a script, add it the automated tasks, and monitor when replication breaks and send me an alert. I know that there is a query that the replication monitor sends out to the database to check if replication is running. Is there an OD documentation on replication monitor or queries?

As a side note: I wonder if there is something that can be done with the Percona Toolkit. They seem to have a very robust set of utilities to monitor MYSQL replication and integrity. Any thoughts?
Manny Ramirez
Senior Network Engineer
E-ssential Networks LLC
User avatar
Manny Ramirez
 
Posts: 129
Joined: Tue Mar 06, 2012 4:07 pm
Location: Miami/Puerto Rico

Re: Sending Alerts when replication breaks

Postby jsalmon » Thu Nov 05, 2015 10:14 am

We built a replication monitor into Open Dental. It will actively tell all Open Dental workstations when a failure occurs so you'll be notified from any workstation you are currently at. Is this sufficient enough?
http://www.opendental.com/manual/replicationmonitor.html
The best thing about a boolean is even if you are wrong, you are only off by a bit.

Jason Salmon
Open Dental Software
http://www.opendental.com
User avatar
jsalmon
 
Posts: 1486
Joined: Tue Nov 30, 2010 1:33 pm

Re: Sending Alerts when replication breaks

Postby Manny Ramirez » Thu Nov 05, 2015 4:05 pm

I read the post. The output of the query goes to the slave monitor and the slave monitor evaluates it. Based on the results it will send the alert. We know the process and we are all aware of what to do. What I would like to create is a script to do what the slave monitor is programmed to do and have it sent to my RMM system instead. That would be great for us in the field because we will be able to act before they call us 30 minutes later.
Manny Ramirez
Senior Network Engineer
E-ssential Networks LLC
User avatar
Manny Ramirez
 
Posts: 129
Joined: Tue Mar 06, 2012 4:07 pm
Location: Miami/Puerto Rico

Re: Sending Alerts when replication breaks

Postby jsalmon » Thu Nov 05, 2015 6:34 pm

The easiest script I could think of would be to monitor Open Dental's preference called "ReplicationFailureAtServer_id" and have your RMM get notified if that pref is ever set to anything higher that 0 (we store it as a string but its value represents an integer). That will let Open Dental do the heavy lifting for you and all you have to do is monitor a singular value periodically.

Otherwise you have to run the MySQL query
Code: Select all
SHOW SLAVE STATUS;

And process it accordingly by evaluating the results. E.g. looking at Slave_SQL_Running, Last_Errno, Last_Error, etc.

If you are interested in seeing what we do exactly, check out our source code and look at FormOpenDental.timerReplicationMonitor_Tick()
The best thing about a boolean is even if you are wrong, you are only off by a bit.

Jason Salmon
Open Dental Software
http://www.opendental.com
User avatar
jsalmon
 
Posts: 1486
Joined: Tue Nov 30, 2010 1:33 pm


Return to Advanced Topics

Who is online

Users browsing this forum: No registered users and 1 guest