Huge breakthrough: I’ve finally discovered how to get my database passwords out of plaintext .env files and into an actual password manager.

The industry way of doing this is to use a key management system (KMS), and then have an orchestration system inject sensitive values into a machine’s environment variables. This is handy as the sensitive data never has to hit the disk.

You can do this same thing with 1Password, with fewer moving parts, which is ideal for someone like me who runs a few hobby services.

The steps for getting these values into a Docker container are:

  1. Create a new 1Password Vault. I called mine ‘Basement Credentials’, as it’ll hold the passwords that my ‘Basement’ machine will need to hold onto.
  2. Create a new 1Password Service Account and give it read access to only our new vault.
  3. Install 1Password CLI.
  4. Set OP_SERVICE_ACCOUNT_TOKEN on the user who is going to be running the docker container. Usually you’d do this with a ~/.bashrc change. (See: ‘Robbing Peter to pay Paul’ below.)
  5. Create a new Secure Note in the new Vault, and name it after the application hosted in Basement. Like, Forgejo.
  6. For each secret bit of data in your application that’s hanging around in your .env file, create a new field: ‘Add more’ on the 1Password Secure Note window. Choose ‘Password’, though this doesn’t matter. Name the new field after the env var, if you can.

  7. Update your .env file with placeholders for 1Password to update.

    # .env
RAILS_ENV=production
SECRET_KEY_BASE="op://Basement Credentials/Forgejo/SECRET_KEY_BASE"

  8. Update your docker-compose.yaml to read envs from environment variables. You likely want both environment: and env_file. (See ‘Disappointing complexity’ below.)

    web:
    image: forgejo:latest
    environment:
        SECRET_KEY_BASE="${SECRET_KEY_BASE}"
        [... etc, but not RAILS_ENV, which will be loaded via .env]
    env_file: .env

  9. Load docker compose via op run.

    op run --env-file=.env -- docker compose up

And then you’re good to go. No secret details held on disk anywhere. They can’t be accidentally committed, and you can commit your .env file, which was a key requirement for me.

The way this works is that op run creates a shell session with those env vars available. Docker-compose will then interpolate those via the ${FOOBAR} syntax used in the docker-compose.yaml.

I’m reasonably happy with this solution for the moment.

Disappointing complexity

I’m saddened by the need to add both environment: and env_file: attributes.

This works because of a (documented and intended) quirk of docker compose where environment takes precedence and is merged on top of whatever is in env_file.

This means we can still put non-secret values in .env for configuration (where I’m used to looking) without polluting the yaml file too much with a huge list.

I wish it were enough for op run to process the .env file and figure out what I want, but that isn’t the case: the .env file that docker compose sees is still the one that’s on disk (the file on disk has not changed). So, it ends up seeing it with the 1Password placeholder strings.

Robbing Peter to pay Paul

That’s a very good point you’ve made: this whole thing hasn’t changed the security concerns if someone gains access to the server. The secrets aren’t on disk any more, but that 1Password vault key is right in that .bashrc and it can output the lot.

This isn’t the problem being solved. Every security system will have this issue.

The aim here is to move the secrets out of the app configuration folders so the whole lot can be safely committed to git. That Git repo is distributed to many systems, including CI, whenever I update the configuration. They don’t all need to know the password for Basement.

Well, you shouldn’t be committing .env files anyway. There are two reasons this is very good advice for almost every other project. First, because they usually have secrets in them. Mine no longer does. Second, because .env files are for configuration and other machines will need different values, so why commit this? It’s like committing your VSCode config: we don’t want to mess up another dev’s machine. However, this particular repo is entirely configuration for this one server. The application itself is in another repo, following this ‘do not commit .env’ rule.