How we backup

I woke up to the terrible news that our good friends on another instance had lost their database during a software upgrade. Godspeed and good luck in bringing it back online. We’re pulling for you!

The Free Radical site backs itself up hourly to a private S3 bucket, and keeps a month’s worth of these snapshots. It’s configured to upload all media files to S3 and serve them from there. In the event of a complete server failure, I could – assuming all goes well – re-deploy the software on a new server and restore from backup without losing more than just users and posts created since the last hour’s backup.

S3 URLs have changed; update your Content-Security-Policy

I’m serving Free Radical’s images etc. from S3. When I updated to Mastodon v2.1.0, I noticed that all the page’s images were missing. Safari’s Show JavaScript Console menu revealed a lot of errors like:

[Error] Refused to load https://s3-us-west-2.amazonaws.com/freeradical-system/accounts/avatars/000/014/309/static/91f9782fad3f6284.png because it does not appear in the img-src directive of the Content Security Policy.

Turns out that some time between the releases of v2.0.0 and v2.1.0, the Mastodon switched from generating S3 URLs like:

https://freeradical-system.s3-us-west-2.amazonaws.com/...

to

https://s3-us-west-2.amazonaws.com/freeradical-system/...

Because I’d jumped through the hoops of setting up a Content-Security-Policy header, Safari wasn’t allowing those images to render. I had to change my CSP header in Nginx from:

add_header Content-Security-Policy "default-src 'self'; img-src 'self' https://freeradical-system.s3-us-west-2.amazonaws.com/ data:; style-src 'self' 'unsafe-inline'; connect-src 'self' wss://freeradical.zone/; media-src 'self' https://freeradical-system.s3-us-west-2.amazonaws.com/";

to

add_header Content-Security-Policy "default-src 'self'; img-src 'self' https://freeradical-system.s3-us-west-2.amazonaws.com/ https://s3-us-west-2.amazonaws.com/freeradical-system/ data:; style-src 'self' 'unsafe-inline'; connect-src 'self' wss://freeradical.zone/; media-src 'self' https://freeradical-system.s3-us-west-2.amazonaws.com/ https://s3-us-west-2.amazonaws.com/freeradical-system/";

so that both the old and new S3 URLs are permitted.

Costs of hosting a small server

I had no idea how much it would cost to host a Mastodon instance. This is for everyone like I was who’s thinking about running their own server but wants to know what to expect.

Free Radical currently has 62 users. The numbers here are what I pay our small little instance and would certainly jump up if we grew by a factor of 100x. Continue reading Costs of hosting a small server