API Events Question

[mandmdiet]

For the API Events functionality, specifically Database Events, in the Open Dental API, it says that

"Open Dental and the API Service generate these events by polling the database at the frequency you specify."

My question is, does it do this by checking a timestamp or other date field in the table being polled and see if that field has been updated to a newer value since the last time it polled? Is this API only going to send me a web hook when a record is created or updated and not when a record has been deleted?

Thanks

[justine]

For the API Events functionality, specifically Database Events, in the Open Dental API, it says that

"Open Dental and the API Service generate these events by polling the database at the frequency you specify."

My question is, does it do this by checking a timestamp or other date field in the table being polled and see if that field has been updated to a newer value since the last time it polled? Is this API only going to send me a web hook when a record is created or updated and not when a record has been deleted?

Thanks

You are correct, this is essentially how it functions. However, AppointmentsDeleted will return data when an appointment has been deleted.

Thanks!

[mandmdiet]

You are correct, this is essentially how it functions. However, AppointmentsDeleted will return data when an appointment has been deleted.

Is that done as an event pushed from the software? I don't see a way looking at the appointment table to know if or when it was deleted unless DateTimeDismissed is a deleted date but I wouldn't think so.

[justine]

You are correct, this is essentially how it functions. However, AppointmentsDeleted will return data when an appointment has been deleted.

Is that done as an event pushed from the software? I don't see a way looking at the appointment table to know if or when it was deleted unless DateTimeDismissed is a deleted date but I wouldn't think so.

Yes, it is an event pushed from the software. No, DateTimeDismissed is not used in that way.

The AppointmentDeleted Event relies on the HistAppointment table. If you'd like to see the lifespan of any appointment, take a look at HistAppointments GET.

Thanks!

[mandmdiet]

The AppointmentDeleted Event relies on the HistAppointment table. If you'd like to see the lifespan of any appointment, take a look at HistAppointments GET.

Awesome, thank you.

Incidentally, is there any way to quickly see with a query when a claim has been deleted? We have several tables we'd like to know when a record is deleted but I don't think we'll be able to swing that.

[justine]

The AppointmentDeleted Event relies on the HistAppointment table. If you'd like to see the lifespan of any appointment, take a look at HistAppointments GET.

Awesome, thank you.

Incidentally, is there any way to quickly see with a query when a claim has been deleted? We have several tables we'd like to know when a record is deleted but I don't think we'll be able to swing that.

No, there is not. However, if you have a ClaimNum that was previously valid, but Claims GET is returning 'NotFound', it has likely been deleted. Additionally, if a claim was deleted via OD API, a securitylog was created. You could then use SecurityLogs GET with PermType=ClaimDelete, which would return all deleted claims and their associated PatNums.

Thanks!

[mandmdiet]

No, there is not. However, if you have a ClaimNum that was previously valid, but Claims GET is returning 'NotFound', it has likely been deleted. Additionally, if a claim was deleted via OD API, a securitylog was created. You could then use SecurityLogs GET with PermType=ClaimDelete, which would return all deleted claims and their associated PatNums.

OK, thank you. Our plan, then, will be to get a list of claims we have record of and use those ClaimNums in a query against the OD API to see if they still exist or not.

[mandmdiet]

I remember someone mentioning to us in an email at some point that the hard limit for requests to the Open Dental API is 1 request per second per customer API Key averaged across the day, so essentially 86,400 requests per day per customer API Key. Is that correct? So, if we want to check every 5 seconds or 20 seconds or 30 seconds to see if a claim has been deleted (for example) that wouldn't be of concern would it?

[justine]

I remember someone mentioning to us in an email at some point that the hard limit for requests to the Open Dental API is 1 request per second per customer API Key averaged across the day, so essentially 86,400 requests per day per customer API Key. Is that correct? So, if we want to check every 5 seconds or 20 seconds or 30 seconds to see if a claim has been deleted (for example) that wouldn't be of concern would it?

Good morning mandmdiet,

You are correct, throttling is rated at approximately 1 request per second, per customer API key, in remote mode. However, I question the need for up-to-the-second claim resolution.

Paging through every claim possible, nonstop, for every customer, seems wildly unnecessary. This would likely result in a lot of duplicated claim data and would not scale well. If you were to extend this idea to multiple endpoints, that quantity of requests could impact you or other API developers.

However, API requests made via Local or Service mode, do not experience throttling and return 1000 or 10,000 items, depending on if an API developer has the APIEnterprise permission.

Additionally, starting in 23.3.11, it is possible to get a single claim. If the result is 'NotFound' you know the claim is not valid. (Incorrect PK, or likely deleted.)

If you already have a list of 'good' ClaimNums, and all you want to know if a claim has been deleted, (PK missing in dataset) you could run a ShortQuery comparing your list to every ClaimNum in your customer's database. I imagine running such a query during off-hours every night would suffice. Running a query every 5-30 seconds seems excessive.

[mandmdiet]

Thank you for your reply justine

I understand, we wouldn't want to do that many queries but it's important for me to understand our throughput allowances so I can make sure whatever we design doesn't run afoul of any limits Open Dental has set as we expand.

Here's what I'm thinking at the moment, I'd appreciate your feedback on this.

We have users who, when using our software, will see a list of claims that can be submitted to insurance for collection. If our software is made aware of a change to one of these claims in Open Dental we update our claim data so that the user knows that claim has changed. Then, when the claim is submitted from our software, the claim has the most up-to-date information on it. If a claim is deleted, the user would likewise want to see the claim removed from our system so as not to submit a claim for collection that is no longer valid.

Given this scenario, we'd probably want to do something like this:

1. On a regular cadence, poll the tables using either the OD local API or Short Query API to check for changes to claims. The query would be something like SELECT ClaimNum FROM Claim WHERE SecDateTEdit > LastPollTimestamp. This would give us a list of records that have changed in the Claim table since the last time we polled. So, if we polled every 2 minutes (for example) we'd only get results for 2 minutes worth of changes to that table. Querying every 2 minutes would be 720 requests per day, every minute would be 1,440 and every 30 seconds would be 2,880.
2. On a regular cadence, poll tables for deleted records. Something like SELECT ClaimNum FROM Claim WHERE ClaimNum IN (12345,12346, ...). This would give us a list of claims that haven't been deleted and by process of elimination we can figure out those that were deleted.

We have more than just the claim table to poll, but we could poll more than one table in a single request if there is a concern about the number of requests going through the API. This would work similar to how your Events API works where you can configure it to poll every N seconds.

So doing this kind of thing would put us well under the 86,400 limit, which, ostensibly, seems like it would be fine, right? It seems like the minimum we could do to support this feature. One last tidbit to add is that we plan on having up to hundreds of Open Dental instances on our software sometime this year.

[justine]

Thank you for your reply justine

I understand, we wouldn't want to do that many queries but it's important for me to understand our throughput allowances so I can make sure whatever we design doesn't run afoul of any limits Open Dental has set as we expand.

Here's what I'm thinking at the moment, I'd appreciate your feedback on this.

We have users who, when using our software, will see a list of claims that can be submitted to insurance for collection. If our software is made aware of a change to one of these claims in Open Dental we update our claim data so that the user knows that claim has changed. Then, when the claim is submitted from our software, the claim has the most up-to-date information on it. If a claim is deleted, the user would likewise want to see the claim removed from our system so as not to submit a claim for collection that is no longer valid.

Given this scenario, we'd probably want to do something like this:

1. On a regular cadence, poll the tables using either the OD local API or Short Query API to check for changes to claims. The query would be something like SELECT ClaimNum FROM Claim WHERE SecDateTEdit > LastPollTimestamp. This would give us a list of records that have changed in the Claim table since the last time we polled. So, if we polled every 2 minutes (for example) we'd only get results for 2 minutes worth of changes to that table. Querying every 2 minutes would be 720 requests per day, every minute would be 1,440 and every 30 seconds would be 2,880.
2. On a regular cadence, poll tables for deleted records. Something like SELECT ClaimNum FROM Claim WHERE ClaimNum IN (12345,12346, ...). This would give us a list of claims that haven't been deleted and by process of elimination we can figure out those that were deleted.

We have more than just the claim table to poll, but we could poll more than one table in a single request if there is a concern about the number of requests going through the API. This would work similar to how your Events API works where you can configure it to poll every N seconds.

So doing this kind of thing would put us well under the 86,400 limit, which, ostensibly, seems like it would be fine, right? It seems like the minimum we could do to support this feature. One last tidbit to add is that we plan on having up to hundreds of Open Dental instances on our software sometime this year.

Good afternoon mandmdiet,

Firstly, we would like you to know how much we appreciate your thoughtful responses. It really helps the API team understand your workflow and in turn, helps us make meaningful recommendations.

As to your question about the quantity of API calls yes, our servers can handle it. However, the server at each office processing those requests may or may not handle it. That load is entirely dependent on each individual office and how powerful their server/workstation may be at processing MySQL.

For example, we had a different API developer absolutely hammering each endpoint with repeated requests. They were collecting as much data from each API endpoint as they could. In turn, individual offices called in concerned with how slow Open Dental was acting, resulting in them turning off their 3rd party API developer's keys. The recommendation, for the API developer's initial backfill of data, would be to collect that data in a different way, or collect it only during off-hours. The last thing anyone wants is the customer suffering through a less than ideal user experience.

If your polling paradigm were to occur in Local or Service mode, you would skip all throttling restrictions, essentially allowing you to make as many requests as that local server can handle. Again, too many requests will result in a poor Open Dental user experience and (possibly) alienated customers.

Thanks!

[mandmdiet]

Great, thank you as well justine. We will be very careful to craft queries that don't put too much load on the practice's servers and build in a mechanism to rachet up or down the frequency of requests on a per-practice basis so practices with lower-end hardware can be given more breathing room if needed.

Incidentally, does the Local API solution also work if they're using Open Dental's cloud service?

[justine]

Great, thank you as well justine. We will be very careful to craft queries that don't put too much load on the practice's servers and build in a mechanism to rachet up or down the frequency of requests on a per-practice basis so practices with lower-end hardware can be given more breathing room if needed.

Incidentally, does the Local API solution also work if they're using Open Dental's cloud service?

Hello mandmdiet,

We need to be careful here because there is a technical difference between Local API and the API Service. Specifically, API for Cloud users is supported, and relies on the API Service. Open Dental does not need to be running for the API Service, so you'd want to use that for Cloud users.

Thanks!

[mandmdiet]

OK, so there's the Local API, the API Service and the Cloud API.

• We're using the Cloud API whenever we hit api.opendental.com/api. It is hosted by Open Dental.
• The Local API is one that is hosted by an instance of Open Dental whenever it is open and is available on any workstation or server where Open Dental is running.
• The API Service is hosted on the same machine as the eConnector and can be used by someone on the workstation or another machine on the same local network.

If a user is using Open Dental Cloud (has their database hosted by Open Dental), the only API we can use is the Cloud API. Is that right?

[justine]

OK, so there's the Local API, the API Service and the Cloud API.

• We're using the Cloud API whenever we hit api.opendental.com/api. It is hosted by Open Dental.
• The Local API is one that is hosted by an instance of Open Dental whenever it is open and is available on any workstation or server where Open Dental is running.
• The API Service is hosted on the same machine as the eConnector and can be used by someone on the workstation or another machine on the same local network.

If a user is using Open Dental Cloud (has their database hosted by Open Dental), the only API we can use is the Cloud API. Is that right?

Almost, but not quite. 'Cloud API' is just another term for the API Service that works with OD Cloud. What you're describing as the 'Cloud API' is actually the Remote API. OD Cloud users cannot use the Local API, because you cannot install executables on the VM. However, you could install your .exe on the local office workstation and communicate with the OD Cloud VM's eConnector (remote API) or port 30223 (API Service).

I suggest you take a look at the API Modes documentation critically. Every alternative API mode is described there in detail.

Thanks!

[mandmdiet]

OK, Thank you again justine.

[mandmdiet]

OK, I've re-read that API documentation and have a better understanding.

Remote API is the API that we are using now found at https://api.opendental.com/api/v1

I have a few questions about Cloud API.

1. Since Cloud API connects directly to the API Service running on the Open Dental Cloud VM, what is the advantage of using the Cloud API over the Remote API? Does connecting to the Cloud API allow us to use the Open Dental API without putting load on the Remote API infrastructure? Would there be as much concern over the number of requests made if we go through that channel of communication*?
2. Do we have to connect to the Cloud API FROM the practice network or could we connect to it from an AWS cloud environment?

*In order to keep claim information up to date in our software we end up monitoring 32 different OD tables. Some change less frequently than others, like Provider, but when something like a Provider's name changes we need to include a different name on the outgoing claim. Similarly if something on a procedure changes or in the patient record or a patient's insurance changes, that affects the outgoing claim, so we monitor those tables for changes. We can still keep the total requests down by monitoring some tables more frequently than others and using a single query to detect changes for more than one table at a time, but it is still a concern that we have.

[justine]

OK, I've re-read that API documentation and have a better understanding.

Remote API is the API that we are using now found at https://api.opendental.com/api/v1

I have a few questions about Cloud API.

1. Since Cloud API connects directly to the API Service running on the Open Dental Cloud VM, what is the advantage of using the Cloud API over the Remote API? Does connecting to the Cloud API allow us to use the Open Dental API without putting load on the Remote API infrastructure? Would there be as much concern over the number of requests made if we go through that channel of communication*?
2. Do we have to connect to the Cloud API FROM the practice network or could we connect to it from an AWS cloud environment?

*In order to keep claim information up to date in our software we end up monitoring 32 different OD tables. Some change less frequently than others, like Provider, but when something like a Provider's name changes we need to include a different name on the outgoing claim. Similarly if something on a procedure changes or in the patient record or a patient's insurance changes, that affects the outgoing claim, so we monitor those tables for changes. We can still keep the total requests down by monitoring some tables more frequently than others and using a single query to detect changes for more than one table at a time, but it is still a concern that we have.

Good morning!

1. API Service in OD Cloud ("Cloud API") exists for API developers that have designed applications that rely on the API Service. If a 3rd party OD API developer designed an application and wanted to support an Open Dental Cloud customer, their application works.

You are correct, if you use the API service with OD Cloud ("Cloud API"), API requests are not routed through Remote API infrastructure. These requests are not subjected to throttling, and their response size is increased to 1000 elements, up from 100. All of the API requests are processed in that VM and are passed back to the program installed on the customer's workstation, almost as if they were on the same network. The load is essentially between the customer's workstation and the Cloud VM. Due to this, the concern for load quantity is greatly reduced.

2. You connect to the API Service ("Cloud API") from the practice network, where your program is installed.

Thanks!

[mandmdiet]

API Service in OD Cloud ("Cloud API") exists for API developers that have designed applications that rely on the API Service. If a 3rd party OD API developer designed an application and wanted to support an Open Dental Cloud customer, their application works.

Ah, OK, that makes sense. All of our code is currently using the Remote API, we'd rather keep using that as long as we won't run into issues with request counts since that allows us to keep our deployments in our backend environment.

With the Cloud API and API Service is the cost for using the API the same as it is with Remote API?

Also, is the API itself the same? Meaning it uses the same models, calls and format between API Service and Remote API?

[justine]

API Service in OD Cloud ("Cloud API") exists for API developers that have designed applications that rely on the API Service. If a 3rd party OD API developer designed an application and wanted to support an Open Dental Cloud customer, their application works.

Ah, OK, that makes sense. All of our code is currently using the Remote API, we'd rather keep using that as long as we won't run into issues with request counts since that allows us to keep our deployments in our backend environment.

With the Cloud API and API Service is the cost for using the API the same as it is with Remote API?

Also, is the API itself the same? Meaning it uses the same models, calls and format between API Service and Remote API?

Hello mandmdiet,

Yes, the cost is the same. In practice, I have seen API developers move to Local/Service mode for the increase in result size and overall API execution times.

Yes, the API is the same. That's the whole point! : )

Thanks!

[mandmdiet]

Great! Thank you!