eConnector Services starts failing making several requests close together

For requests or help with our API
Post Reply
mandmdiet
Posts: 113
Joined: Tue Aug 17, 2021 9:37 am

eConnector Services starts failing making several requests close together

Post by mandmdiet » Wed Jun 01, 2022 4:00 pm

Just wanted to make a quick inquiry about an issue we encountered in the eConnector Service where it started failing. Each request would return either a 400 Bad Request or a 504 Gateway Timeout. This occurred on a test box while we were doing new development. The machine didn't appear to be taxed on resources (CPU, Memory, Hard Disk) so we were surprised to see this happen. In order to get the eConnector working again we had to restart the eConnector.

I am curious as to what kind of a load the eConnector is expected to be able to handle if the specs of the machine or network connection weren't a factor. Also, in a similar vein, I assume the OD API itself is set up to scale to handle large amounts of traffic and that our only bottleneck concern will be at the eConnector level. Is that right?

Thanks

SLeon
Posts: 476
Joined: Mon Mar 01, 2021 10:00 am

Re: eConnector Services starts failing making several requests close together

Post by SLeon » Thu Jun 02, 2022 8:56 am

The API has been carefully designed to scale and handle an extremely high volume of traffic. At the dental office level, the eConnector is the only theoretical limitation on throughput. That being said, the eConnector is designed to handle such a high level of traffic that customers should never be effected in real world conditions. We monitor the eConnector closely and it has been stress tested.

We have had a developer report a 504 error when making a large number of requests in a short period of time in their development environment. Their issue was in how they were calling our API. Specifically, the Python library they were using defaulted to setting TCP connections as 'KeepAlive', and therefore, tried to continuously reuse the same connection for subsequent requests. Our API servers will eventually close these connections after a prolonged period of time, which would yield these error codes.

mandmdiet
Posts: 113
Joined: Tue Aug 17, 2021 9:37 am

Re: eConnector Services starts failing making several requests close together

Post by mandmdiet » Thu Jun 02, 2022 9:34 am

SLeon wrote:
Thu Jun 02, 2022 8:56 am
The API has been carefully designed to scale and handle an extremely high volume of traffic. At the dental office level, the eConnector is the only theoretical limitation on throughput. That being said, the eConnector is designed to handle such a high level of traffic that customers should never be effected in real world conditions. We monitor the eConnector closely and it has been stress tested.
OK that is very helpful to know. With that, we can expect that if the resources on the box itself aren't being taxed, then, within reason, we shouldn't have to limit the amount of requests or frequency of requests we are making. Right?

I think in our test we did about 200 requests within 5 seconds although we weren't measuring, so the amount of time might be off by a few seconds.

For what it's worth we are developing to a lower minimum standard of required resources by the OD machine and so we are refactoring to significantly reduce this volume, but given that this was something we experienced I wanted to ask about it.

SLeon
Posts: 476
Joined: Mon Mar 01, 2021 10:00 am

Re: eConnector Services starts failing making several requests close together

Post by SLeon » Thu Jun 02, 2022 10:12 am

That is correct, you should not have to limit the quantity or frequency of requests at any point.

If you are utilizing the remote API endpoint (https://api.opendental.com) you may bump into our throttle rate, but throttled requests are simply delayed not failed. It is worth noting that our Local API does not have a throttle limitation.

mandmdiet
Posts: 113
Joined: Tue Aug 17, 2021 9:37 am

Re: eConnector Services starts failing making several requests close together

Post by mandmdiet » Wed Jun 29, 2022 3:39 pm

I am concerned about overwhelming an Open Dental eConnector machine. In some cases the machine hardware might be old and there might not be a lot of available resources. In other cases they may have poor internet bandwidth or a cheap router. With the throttle rate you're talking about in your last post, is that a throttle between the Open Dental API and an eConnector instance to keep this from happening? Or is this a throttle between us and the Open Dental API? If it is the latter, do you also have a throttle between Open Dental and each eConnector meant to prevent too many messages at a time being sent to individual eConnector instances?

SLeon
Posts: 476
Joined: Mon Mar 01, 2021 10:00 am

Re: eConnector Services starts failing making several requests close together

Post by SLeon » Thu Jun 30, 2022 12:53 pm

The throttle rate I mentioned in my previous post occurs entirely on Open Dental API servers' side. The request is verified upon arrival and throttled at that point. This therefore translates into the response being throttled downstream. The eConnector's activity will be quite light even for older machines, should have very little impact.

Another option that you may consider exploring, depending on your intended use, is either the Local API or the API Service (https://www.opendental.com/site/apilocal.html). The Local API launches on a workstation when Open Dental is launched, and API requests can be made to that localhost. A single internet call is made at the start of the session to confirm credentials, but otherwise do not require the eConnector to function. Similarly, the API Service (currently in beta) runs on the same server as the eConnector, but other than the initial credential verification, does not require it for subsequent requests. The API Service can be used by any/every workstation on the network. Both options are available for all API customers.

mandmdiet
Posts: 113
Joined: Tue Aug 17, 2021 9:37 am

Re: eConnector Services starts failing making several requests close together

Post by mandmdiet » Thu Jun 30, 2022 3:50 pm

Oh, that's wonderful. Then the throttling you're referring to is meant to protect the eConnector from being overwhelmed with requests? If that is the case then we don't have any concerns actually because we can effectively hand off all of our traffic to the Open Dental API and no matter the frequency or number of requests, the client will continue to operate normally.

I have that right, yes?

SLeon
Posts: 476
Joined: Mon Mar 01, 2021 10:00 am

Re: eConnector Services starts failing making several requests close together

Post by SLeon » Tue Jul 05, 2022 9:53 am

Yes, you have that right. The eConnector would not be overwhelmed with requests. You can hand over any number of API requests without worrying about the traffic's impact.

Post Reply