After a long hard fight tonight, we got the server back online using backups. Discourse (the forum software we use) required a couple upgrades due to changes with Docker (the platform it runs on). This was compounded by the fact that the server was out of disk space (really?!) so upgrading required a little bit of server shuffling. Needless to say, it was frustrating but we're just glad to have the forum back.
We are making the following changes to ensure further future proofing:
- Setting up monitoring on the site for any issues with the site going down.
- Doing a daily check on the S3 databackups (a few days were missing, but the most recent day was available, thank goodness).
- Upped the size of the server one notch (we use Digital Ocean)
What does this means for you?
It's possible that a few replies or new topics were lost between when the backup happened and when the site was restored. If this is the case, I apologize. There's very little we can do about that. If you have an email notification about a response that you no longer see on the server, please get in touch with whomever posted it so they can have the exact text to repost.
We know this has been a daily resource for many users and many of you keep coming back to share your knowledge with others. We appreciate that! We'll continue to work to keep the server running in top shape. Many thanks to the good folks at Discourse for offering this platform for free.