So I’ve been running the mastodon.technology instance of Mastodon since April, and I’ve written about different things I’ve learned while administering a server and contributing to the project. One thing I’ve only recent learned about is log rotation: an automatic task to prevent log files from filling up a disk.

This relates to a problem I had a month ago. My primary disk filled up, prevent media uploads from working for a few hours (the database was unaffected since it uses a separate disk). The problem was exacerbated by an issue I’ve been having with Sidekiq log file sizes, but the root of the issue is that I wasn’t rotating Mastodon log files at all. I was just logging in about once a week to do restart the docker containers (thereby clearing the logs). But I forgot one week.

I asked some colleagues for advice and they told me about log rotation. So I added the following new, named /etc/logrotate.d/docker-container (see this helpful post).

/var/lib/docker/containers/*/*.log {
  rotate 7
  daily
  compress
  size=1M
  missingok
  delaycompress
  copytruncate
}

This moves the log contents to a new file and compresses previously moved logs, so I get a disk use increase of about 50MB/day instead of 1GB/day.

Obviously the logs are still growing, but the instance could now run for months without me restarting docker containers and not run out of disk space. Nice.

And I think Docker has this kind of functionality built in now, but this is working so I’m running with it.

So yeah, logrotate.d is another minor skill I’ve added to my tool belt that’s come from working on and administering Mastodon.