Resolved -
This incident has been resolved
Oct 13, 16:16 IDT
Investigating -
The webhook service is currently experiencing timeouts when connecting to the brokers again. We are investigating the issue.
Oct 12, 21:47 IDT
Monitoring -
After applying the fix, it appears that the system is stabilizing. We will continue to monitor the situation closely.
Oct 12, 20:17 IDT
Update -
We have applied a fix to minimize the issue. Webhooks are now being registered, but there may still be a delay in ingestion.
Oct 12, 19:51 IDT
Update -
We are continuing to work on a fix for this issue.
Oct 12, 18:06 IDT
Identified -
We have identified the root cause of the webhook processing failures, which was a connection timeout to our Kafka brokers.
Our team is actively investigating possible solutions to resolve the issue.
Please note that this may also be causing a lag in the scorecards, as it is experiencing timeouts as well.
Oct 12, 18:06 IDT
Investigating -
We are currently experiencing an issue with our incoming generic webhook service, which is causing a significant number of webhook requests to fail with a status code 500.
Users may experience failed webhook requests from third party applications into Port.
Oct 12, 17:16 IDT