Is Google Cloud Media CDN down?
Current Google Cloud Media CDN status is UP

Summary: Cloud Audit Logging misconfigured for prod dedicated environment
Description: Mitigation work is currently underway by our engineering team.
We do not have an ETA for mitigation at this point.
We will provide more information by Monday, 2023-08-14 10:00 US/Pacific.
Diagnosis: None at this time.
Workaround: None at this time.
Summary: Chronicle customers in all regions using the SENTINEL_EDR default parser (product source: "SentinelOne EDR") may have incorrect process enrichment results.
Description: As communicated previously, we have segmented our repair of this incident into two phases:
a) Stopping erroneous enrichments (Completed on 2 August 2023)
b) Repairing historically impacted events that used these erroneous enrichments. (Revised ETA - 18 August 2023)
We will provide an update by Friday, 2023-08-18 10:30 US/Pacific with current details.
Diagnosis: Fields set by process aliasing for SENTINEL_EDR (including UDM principal.process, src.process, and target.process) may be incorrect.
Workaround: None at this time.

Summary: Media CDN increased rates of 5xx errors
Description: We are experiencing an issue with Media CDN beginning at Friday, 2023-08-11 13:30 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Friday, 2023-08-11 18:30 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers may experience increased rates of 5xx errors or failed connections for Media CDN requests.
Workaround: None at this time.
Summary: Cloud Audit Logging misconfigured for prod dedicated environment
Description: Mitigation work is currently underway by our engineering team.
We do not have an ETA for mitigation at this point.
We will provide more information by Monday, 2023-08-14 10:00 US/Pacific.
Diagnosis: None at this time.
Workaround: None at this time.
Summary: Chronicle customers in all regions using the SENTINEL_EDR default parser (product source: "SentinelOne EDR") may have incorrect process enrichment results.
Description: As communicated previously, we have segmented our repair of this incident into two phases:
a) Stopping erroneous enrichments (Completed on 2 August 2023)
b) Repairing historically impacted events that used these erroneous enrichments. (Revised ETA - 18 August 2023)
We will provide an update by Friday, 2023-08-18 10:30 US/Pacific with current details.
Diagnosis: Fields set by process aliasing for SENTINEL_EDR (including UDM principal.process, src.process, and target.process) may be incorrect.
Workaround: None at this time.

Summary: Cloud Data Fusion instance update failure.
Description: Mitigation work is still underway by our engineering team. Currently our engineers have mitigated us-east5 and us-south1 and are actively working on mitigating other regions.
We will provide more information by Thursday, 2023-07-20 21:30 US/Pacific.
Diagnosis: Cloud Data Fusion instances created before June 14, 2023 failing during instance upgrade in a few regions. However, the instance usage will not be impacted.
Workaround: None at this time.

Summary: Global: Media CDN experiencing elevated playback failures
Description: We are experiencing an issue with Media CDN beginning at Wednesday, 2023-07-12 00:00 US/Pacific.
Our engineering team continues to investigate the issue.
We will provide an update by Tuesday, 2023-07-18 13:07 US/Pacific with current details.
We apologize to all who are affected by the disruption.
Diagnosis: Customers may see a small percentage of connections being reset, which may manifest as playback failures.
Workaround: None at this time.
Summary: Change notifications for Healthcare API services (FHIR, DICOM, HL7v2) in the us-meta region are being delayed. This impacts Pub/Sub notifications, streaming to BigQuery, streaming de-identification, and could delay search indexing
Description: The team mitigated any further dropped notifications and resolved the delays as of July 15, 2023 9:41 AM Pacific Time. The issue is fully mitigated for ongoing traffic.
The engineering team is working to replay notifications that were not processed, which impacted less than 5% of traffic between 2023-07-11 08:30 US/Pacific and 2023-07-13 14:00 US/Pacific.
We do not have an ETA for replay of dropped notifications at this point.
We will provide more information by Tuesday, 2023-07-18 15:00 US/Pacific.
Diagnosis: Data is being successfully written to the Healthcare API but may be delayed or missing from streaming and notifications to other services. Search results within the Healthcare API may be stale.
Workaround: None at this time.

Summary: Customers may see unexpected additional messages in GKE cluster logs.
Description: Mitigation work is currently underway by our engineering team.
We have identified a fix, and are still working to test and roll it out.
We do not have an ETA for rollout completion at this time, however we will continue to provide updates on any status changes.
Customers may see unexpected additional messages in GKE cluster logs, for GKE clusters on version 1.24 and later. These additional messages should not have been added to the logs by GKE and can be safely ignored. For very large workloads (i.e., using thousands of nodes per project) these logs may exceed the Logging API quota and thus cause other log messages to be dropped.
The workloads processed by GKE clusters are not impacted.
We will provide more information by Monday, 2023-07-10 17:00 US/Pacific.
Diagnosis: Customers may see unexpected additional log messages in GKE cluster logs.
Error/Warning log entries:
1. Failed to get record: decoder: failed to decode payload: EOF
2. cannot parse '0609 05:31:59.491654' after %L
3. invalid time format %m%d %H:%M:%S.%L%z for '0609 05:31:59.490647'
Workaround: To avoid hitting Logging API quota customers can request quota Logging API increase.
To avoid storing these additional messages (that provides impact on logging billing) customers can apply Log exclusions by using the following query:
resource.labels.container_name = "fluentbit-gke" OR LOG_ID("fluentbit")
(jsonPayload.message =~ "^Failed to get record: decoder: failed to decode payload: EOF" OR jsonPayload.message =~ "^cannot parse .* after %L$" OR (jsonPayload.message =~ "^invalid time format" AND jsonPayload.plugin = "parser:glog"))