05:07
Apr 25 2024
Show Posts
31
General Discussion / Harddrive crash and recovery
« on: April 27, 2013, 08:42:35 pm »
Just wanted to make an announcement about what happened with our harddrive crash last night.
At roughly midnight last night our host informed us that our primary harddrive on our web and database server had completely failed. Following this, I made an announcement on our Steam group stating that we'd be encountering some downtime.
We had some pretty critical services on that drive (such as the donator database, fastdl), so the downtime had also affected our game servers in an indirect way.
Fast forward to right now: I've been working pretty much around the clock since then to get all of our services back up. So far I've (mostly) restored everything except our MVM server, which was also hosted on that drive.
We kept database backups of everything for exactly this kind of scenario, but unfortunately we had no backups of the actual forums website and the MVM server.
For the forums, I have attempted to the best of my ability to restore everything, but there are some issues that need to be resolved:
Our last issue is our MVM server.
I'm still on the fence about what I want to do here. For one, I'd ideally like to keep non-essential servers off of our web server, which means that (ideally) we would not be bringing back the MVM server. I weigh this decision as a cost-benefit analysis: It will take a considerable effort to recreate the MVM server to the state it was before we lost it.
I know this would make some players unhappy, so I welcome any input on what should be done. If enough people want the MVM server back, I guess I can sit down and find some time to recreate it.
At roughly midnight last night our host informed us that our primary harddrive on our web and database server had completely failed. Following this, I made an announcement on our Steam group stating that we'd be encountering some downtime.
We had some pretty critical services on that drive (such as the donator database, fastdl), so the downtime had also affected our game servers in an indirect way.
Fast forward to right now: I've been working pretty much around the clock since then to get all of our services back up. So far I've (mostly) restored everything except our MVM server, which was also hosted on that drive.
We kept database backups of everything for exactly this kind of scenario, but unfortunately we had no backups of the actual forums website and the MVM server.
For the forums, I have attempted to the best of my ability to restore everything, but there are some issues that need to be resolved:
- All user avatars and attachments were lost, so you will need to reupload them through the profile page.
- Steam profiles are currently missing, but I'll get around to fixing this later.
- The website themeing may look odd in some places.
Our last issue is our MVM server.
I'm still on the fence about what I want to do here. For one, I'd ideally like to keep non-essential servers off of our web server, which means that (ideally) we would not be bringing back the MVM server. I weigh this decision as a cost-benefit analysis: It will take a considerable effort to recreate the MVM server to the state it was before we lost it.
I know this would make some players unhappy, so I welcome any input on what should be done. If enough people want the MVM server back, I guess I can sit down and find some time to recreate it.