Insights degraded performance issue.
Incident Report for Acquia DAM
Postmortem

Overview

On November 25, 2020, at 4:36 AM CT (11:30 AM CET) we were alerted of increased errors from an upstream service provider that delivers analytics events to the Collective suite used in the Insights application and began investigating with our service provider.  At this time these errors were not causing issues for the Collective suite.  After further investigation, we’ve found that between November 25, 2020, at 9:30 AM CT (4:30 PM CET) and 10:30 PM CT (5:30 AM CET on November 26, 2020) intermittent analytics events had failed.  We were able to recover some analytics events and our engineers are currently investigating the feasibility of recovering additional events.  These additional analytics events include portal views, video activity from portals, logins to the Assets application, and uploads via the new upload experience & API v2 that occurred during this incident.

Root Cause

Increased API Error Rates from an upstream service provider.

What We’re Doing?

Our engineers were able to restore analytics events for user activity within the Assets application and we are continuing to review the feasibility of recovering additional events.

Posted Dec 03, 2020 - 16:46 CST

Resolved
We’ve resolved the issue causing a delay for new analytics events in Insights. Contact Customer Support for help on Widen Connect if you continue to experience any ongoing issues.
Posted Nov 26, 2020 - 17:11 CST
Monitoring
We’ve resolved the issue causing a delay for new analytics events in Insights and are monitoring to ensure stability.
Posted Nov 26, 2020 - 07:25 CST
Identified
We've identified the issue causing a delay for new analytics events in Insights and are actively working with our upstream provider to resolve this issue. We will continue to provide updates as we have them.
Posted Nov 25, 2020 - 13:24 CST
Investigating
We're aware of an issue causing a delay for new analytics events in Insights and are actively investigating.
Posted Nov 25, 2020 - 12:31 CST
This incident affected: API (REST v2), Assets & Templates, Portals, and Workflow.