Due to an issue during a database migration on 2019-02-14, Drip Campaign activations failed to activate during a 27 hour period from 2019-02-14 12:50 PST to 2019-02-15 15:36 PST. Any Drip Campaign emails scheduled to send before the incident and any deactivations made with the API or automation triggers continued to work as normal.
We apologize for any disruption this may have caused for our customers and their business. We have already notified all account administrators for effected accounts.
Our infrastructure team executed a planned data migration of the database responsible for Drip Campaigns. During this time all new Drip Campaign activations were to be made redundant across both the old and new datasets. An error in our code to configure this redundancy caused loss of data during this migration period. This ultimately prevented any new Drip Campaigns from properly activating.
During the incident window:
- Drip Campaign stopped activating via the API
- Drip Campaign stopped activating via automation triggers
- No new Drip Campaigns could be created
Any accounts that activated a Drip Campaign during the period above would have been affected. This includes Drip Campaign activations through the API and through Automation Triggers. If this was the case, your account administrator An email has been sent to your Sendwithus administrators if your account was affected.
While do mistakes happen, it's most important to us that they're caught early so they can be swiftly corrected. To mitigate the likelihood of this happening in the future we will be:
- Adjusting our alert thresholds for this particular issue.
- Adding more thorough migration testing processes