Root Cause Analysis (RCA):

Database storage dropped due to a large transaction at between 10:23 AM IST to 10:40 AM IST

The primary root cause was storage exhaustion caused by the interaction of two concurrent high-I/O operations on the RDS instance:

• Large transaction: A single large transaction (bulk DML operation) generated a large volume of transaction log data that consumed storage faster than anticipated.
• Concurrent automated backup: This transaction overlapped with the scheduled automated backup window. RDS automated backups consume locks the modification of database which blocks the storage auto scaling for that duration.

Resolution:
- Manually increased allocated storage via AWS CLI once backup completed
- Killed the large transaction to free up log space
- Verified database came back online and application connectivity restored.

APIs - Order creation / AWB Registration recovered.

Tracking Page - personalised 1 and Tracking Page - personalised 2 recovered.

Tracking page - backend application recovered.

Dashboard - B2C [Backend] and Returns [Backend] and Tracking Page - personalised 3 recovered.

Tracking Page - personalised 4 and Returns and Exchanges - Customer Portal recovered.

Returns and Exchanges - Customer Portal went down.

Tracking Page - personalised 4 went down.

Tracking Page - personalised 3 went down.

APIs - Order creation / AWB Registration went down and Tracking Page - personalised 3 recovered.

Tracking Page - personalised 4 recovered.

Tracking Page - personalised 4 went down.

Dashboard - B2C [Backend] and Returns [Backend] and Tracking Page - personalised 3 went down.

Tracking Page - personalised 3 recovered.

Tracking Page - personalised 4 recovered.

Tracking Page - personalised 3 went down.

Tracking Page - personalised 1, Tracking Page - personalised 2, and Tracking Page - personalised 4 went down.

Tracking page - backend application went down.