Downtime Update: April 29, 2015

Written by Formstack on April 29, 2015

Posted in Formstack Updates

We wanted to make our customers aware of downtime we experienced on Wednesday, April 29. Our site and forms were inaccessible from approximately 4 p.m. – 5:30 p.m. EST.

UPDATE THURSDAY, APRIL 30 AT 9:30 AM EST

Formstack’s Dev Team has provided this overview of yesterday’s downtime:

At 3:53 EST on Wednesday, April 29, our engineers were alerted that Formstack (our website, form builder, and forms) went offline. We quickly diagnosed the outage as network related. Working with our hosting provider, we discovered that our firewall was unresponsive because it had run out of sessions.

After a few attempts of bringing it back online failed, we rebooted the firewall. Once the firewall came back online, service was restored shortly before 5:30 PM EST. Our monitoring systems show we were down for an hour and a half.

Engineers are now monitoring traffic at the firewall and load balancer level to ensure that our systems have stabilized.

UPDATE 6:18 PM EST

We were able to get our service back online at 5:30 PM EST. We experienced an issue with our firewall, and it has been restarted and our systems are back online. We will provide greater detail on why this happened and what we are doing to prevent it in the future as we continue to diagnose the problem.

 

UPDATE 5:03 PM EST

Around 3:58 EST on Wednesday, April 29, 2015, we began to encounter issues with our servers. Since that time, we have been working with our hosting provider to understand the problem and find a fix. We have escalated the situation with our provider and are working as fast as we can towards a resolution.

We will continue to update our Twitter and Facebook with real-time information as soon as we have it. We will also post an update here on the blog when we have more information and a solution.