The webhook service is currently experiencing timeouts when connecting to the brokers again. We are investigating the issue.
Posted Oct 12, 2024 - 21:47 IDT
Monitoring
After applying the fix, it appears that the system is stabilizing. We will continue to monitor the situation closely.
Posted Oct 12, 2024 - 20:17 IDT
Update
We have applied a fix to minimize the issue. Webhooks are now being registered, but there may still be a delay in ingestion.
Posted Oct 12, 2024 - 19:51 IDT
Update
We are continuing to work on a fix for this issue.
Posted Oct 12, 2024 - 18:06 IDT
Identified
We have identified the root cause of the webhook processing failures, which was a connection timeout to our Kafka brokers. Our team is actively investigating possible solutions to resolve the issue.
Please note that this may also be causing a lag in the scorecards, as it is experiencing timeouts as well.
Posted Oct 12, 2024 - 18:06 IDT
Investigating
We are currently experiencing an issue with our incoming generic webhook service, which is causing a significant number of webhook requests to fail with a status code 500.
Users may experience failed webhook requests from third party applications into Port.