Forums
As the single-digit Farenheight temperatures creep across the northeast United States, one's mind drifts off to holidays- specifically summer holidays where it isn't so cold that it hurts to breathe.
Luciano M works in Italy, where August 15th is a national holiday, but also August is the traditional time of year for everyone to take off, leaving the country mostly shut down for the month.
A long time ago, Luciano worked for a small company, along with some friends. This was long enough that you didn't rent compute from a cloud provider, but instead ran most of your intranet services off of a private server in your network closet somewhere.
This particular server ran mostly everything: private git hosting, VPN, email, and an internal Jabber server for chat. Given that it ran most services in the company, one might think that they were backing it up regularly- and you'd be right. One might also think that they had some sort of failover setup, and that's where you'd be wrong.
Late August 12th, the hard drive on their server decided it was time to start its own holiday. The main reason everyone noticed when it happened wasn't due to some alert that got triggered, but as mentioned, Luciano was friends with the team, which meant they used the Jabber server to chat with each other about non-work stuff.
Because half the country was already closed for August, getting replacements delivered was a dubious proposition, at best. Especially with the 15th looming, which not only made shipping delays worse, but this particular year was on a Friday, marking a 3-day weekend. Unless they wanted to spend the better part of a week out of commission, they needed to find an alternative.
The only silver lining was that "shipping is delayed" is the kind of problem which can be solved by spending money. By the time it was all said and done, they paid more for shipping than they paid for the drive itself, but the drive arrived by the 14th, and by the end of the day, they had the server back up and running, restored from backup.
And everything was happy, until August 12th, the following year, when the new hard drive decided to die the exact same way as the previous one, and the entire cycle repeated itself.
And on the third year, a hard drive also failed on August 12th. At least, by that point, they were so used to the problem that they kept spare drives in inventory. Eventually, someone upgraded them to a RAID, which at least kept the downtime at a minimum.
Luciano has long since moved on to a new job, but the date of August 12th is his own personal holiday: an unpleasant one.