Hey, everyone. Invariel here.
As you may or may not have noticed, 2024 has been a pretty turbulent time for the website. We've had a couple of several hour long outages, and only vague posts on the Discord about them after they've happened for a couple of hours. This is an unfortunate side effect of trying to keep things modern, while also being in a state of burnout and recovery.
We've had several temporary outages due to our certificates being expired; I finally got that process properly automated and then the version of the software being used to maintain our certs updated itself, bringing down the website because it removed a fix I had to make, which caused it to stop updating our certificates. That's been dealt with, thankfully, and until that software is updated again, our site is certified secure. Thank you, Let's Encrypt, for existing, and making it easy for people to secure their websites.
On the backend, we've modernized to using .NET 8 as our development framework. Keeping things up to date is a Herculean effort, and everyone involved in site development is greatly appreciated. Frustratingly, updating our server to use .NET 8 required an OS update. That OS update stalled a couple of times, bringing the site down for far longer than was expected. The OS update also required updating our database package, and a couple of hours were spent trying (and failing repeatedly) to update it accordingly. Thankfully, we just needed the latest version to be installed, we do not have to actually use it, which is great because every attempt to migrate from old to new failed in frustratingly informationless fashion. Once those issues were resolved, the site was able to be deployed without issue, and we were back.
We then had an issue where our automated process for restarting the website failed, and stopped giving us relevant output. I suspect this is due to upgrading the OS, but the ultimate effect of this was the inability to bring the website back up; resolving this took a few hours of investigation and reworking, and our stop and start scripts are again playing nicely.
This past weekend, we deployed a new version of the website, and somewhere in the process an important line of code moved to a point after when it was required, which caused configuration information to be unavailable at the point it was required. Thankfully, this issue is resolved now, and ultimately stems from again modernizing our software. Being completely technical for a second, the .NET framework has changed the way the starting point of the code works, drastically altering how the file looks, and this transition nicely hid the issue that was ultimately resolved earlier today.
With all of that said, when things work, they really work -- deploying the website brings literal seconds of downtime when everything is working properly. It's rarely noticeable. The website is also constantly up, even when being worked on regularly, and our ability to deploy quickly means that fixes can be deployed and issues can be resolved in a timely manner. There are still a few things to do: the database version does need to be updated to stay modern, I'd like to make sure that the "previous" version of the website is available more readily in case there are deployment issues, and now that we have all of this deployment infrastructure and stability it's worth looking into setting up a development instance of the website to ensure that features look proper on the host machine rather than hoping for the best.
These things take time -- we're all volunteers here -- but we're all committed to a working, maintainable website that the community can work on and develop.
As always, if you want to contribute, the website code is available
on GitHub, and our list of open issues is
here.
Thank you for your time, patience, and understanding.
