Sending Alerts when replication breaks

For complex topics that regular users would not be interested in. For power users and database administrators.
Post Reply
User avatar
Manny Ramirez
Posts: 129
Joined: Tue Mar 06, 2012 3:07 pm
Location: Miami/Puerto Rico
Contact:

Sending Alerts when replication breaks

Post by Manny Ramirez » Thu Nov 05, 2015 7:44 am

My RMM system allows me to create automated tasks and send me alerts. It would be nice if I could create a script, add it the automated tasks, and monitor when replication breaks and send me an alert. I know that there is a query that the replication monitor sends out to the database to check if replication is running. Is there an OD documentation on replication monitor or queries?

As a side note: I wonder if there is something that can be done with the Percona Toolkit. They seem to have a very robust set of utilities to monitor MYSQL replication and integrity. Any thoughts?
Manny Ramirez
Senior Network Engineer
E-ssential Networks LLC

User avatar
jsalmon
Posts: 1551
Joined: Tue Nov 30, 2010 12:33 pm
Contact:

Re: Sending Alerts when replication breaks

Post by jsalmon » Thu Nov 05, 2015 9:14 am

We built a replication monitor into Open Dental. It will actively tell all Open Dental workstations when a failure occurs so you'll be notified from any workstation you are currently at. Is this sufficient enough?
http://www.opendental.com/manual/replic ... nitor.html
The best thing about a boolean is even if you are wrong, you are only off by a bit.

Jason Salmon
Open Dental Software
http://www.opendental.com

User avatar
Manny Ramirez
Posts: 129
Joined: Tue Mar 06, 2012 3:07 pm
Location: Miami/Puerto Rico
Contact:

Re: Sending Alerts when replication breaks

Post by Manny Ramirez » Thu Nov 05, 2015 3:05 pm

I read the post. The output of the query goes to the slave monitor and the slave monitor evaluates it. Based on the results it will send the alert. We know the process and we are all aware of what to do. What I would like to create is a script to do what the slave monitor is programmed to do and have it sent to my RMM system instead. That would be great for us in the field because we will be able to act before they call us 30 minutes later.
Manny Ramirez
Senior Network Engineer
E-ssential Networks LLC

User avatar
jsalmon
Posts: 1551
Joined: Tue Nov 30, 2010 12:33 pm
Contact:

Re: Sending Alerts when replication breaks

Post by jsalmon » Thu Nov 05, 2015 5:34 pm

The easiest script I could think of would be to monitor Open Dental's preference called "ReplicationFailureAtServer_id" and have your RMM get notified if that pref is ever set to anything higher that 0 (we store it as a string but its value represents an integer). That will let Open Dental do the heavy lifting for you and all you have to do is monitor a singular value periodically.

Otherwise you have to run the MySQL query

Code: Select all

SHOW SLAVE STATUS;
And process it accordingly by evaluating the results. E.g. looking at Slave_SQL_Running, Last_Errno, Last_Error, etc.

If you are interested in seeing what we do exactly, check out our source code and look at FormOpenDental.timerReplicationMonitor_Tick()
The best thing about a boolean is even if you are wrong, you are only off by a bit.

Jason Salmon
Open Dental Software
http://www.opendental.com

Post Reply