Potential Downtime- Read this one!!

Hey folks,

Rough day today here! I'm sorry for the multiple outages that you may have experienced. It was a stumper to figure out what was happening.

There was a query that became "stuck" in one of our database servers, and it just slowly kept grinding things slower and slower, until it became un-responsive. I've re-built that query now and several others that are similar.

In addition, I have built a new monitor that will let me know of any queries that are running for awhile and have the potential to become "stuck". I've never seen this before, so just in case, I feel like this is a good thing to do and monitor. This new monitor simply looks around at all the queries that are running and says "Hey, anyone here been running for more than a few minutes?" It's pretty simple, will run on every database periodically, and shoots me a notification via PapertrailApp if a bad query is found.

I think we had three instances of downtime- looking at the logs it didn't hit everyone, but for those who did, I apologize about that and I really hope it didn't cause too much frustration with your customers.

Be well,

Wayne

Kiva Logic systems are back to 100%! We will continue to monitor to make sure the issue has been resolved and will follow up with any updates if need be.

Kiva Logic is currently slow or unavailable. We have detected an issue and are investigating now.