Rails 8: Multi-db with Solid Queue, Solid Cache and Kamal

Guillaume Briday

Paris.rb - January 7, 2025

The plan for today

Why bother?
Concepts behind of Solid Queue & Solid Cache
How to implement it?
Deploy it with Kamal

Lead dev at

Open source contributor

✌️ @guillaumebriday 👉 https://guillaumebriday.fr

A little bit of context

Kamal: Why and how you should leave the cloud? // Paris.rb at @algolia

Self-Hosting Revolution and Kamal 2 // Double Slash Podcast

Why bother?

What's Solid Cache and Solid Queue?

Why bother?

Donal McBreen - Solid Cache: A disk backed Rails cache - Rails World 2023

Rosa Gutiérrez - Solid Queue internals, externals and all the things in between - Rails World 2024

Cost, Simplicity and performance

Why bother?

Rails 8 only requires a database as a dependency.

And it can be SQLite.

Why bother?

Toolchain free
Importmaps and No Build
Solid Queue through Puma (like async)
Solid Cache & Solid Cable
Interactions with Hotwire

The concept behind the Solid Trifecta

It replaces the traditional RAM-based solutions like Redis with efficient, cost-effective database-backed tools.

RAM-based solution

db-backed solution

Many advantages

One less component to manage
Cheaper
Way more space
Fast enough, maybe even faster
We can use SQL to query data
Works out of the box

Introducing Solid Cache

Active Support cache store
Battle tested
Cache expiry
Encryption
Sharding

Solid Cache trade-off

	Size	Price	Performance
Redis	🤏	💵💵💵	🏎️🏎️🏎️
Database	🌌	💵	🏎️🏎️

Solid Cache cost example

	Size	Price
Heroku	500MB	$120/m
Scalingo	4GB	€115/m
Scaleway	8c/16t 64GB 2TB NVMe	€120/m

Solid Cache trade-off

A balance between speed and cache miss rate.

PostgreSQL Optimizations

Made to run on NVMe
Disable Write Ahead Logging (WAL)
Increase the buffer cache
Use pg_prewarm extension
Solid Cache for Rails and PostgreSQL by Andrew Atkinson

Introducing Solid Queue

Exactly the same as Solid Cache
Delayed jobs
Concurrency controls
Recurring jobs (Cron)
Bulk enqueuing (perform_all_later)
Trying out Solid Queue and Mission Control with PostgreSQL by Andrew Atkinson

How to implement them?

config/database.yml

  
production:
  primary:
    <<: *default
    host: <%= ENV["POSTGRES_HOST"] %>

  cache:
    <<: *default
    host: <%= ENV["CACHE_POSTGRES_HOST"] %>
    migrations_paths: db/cache_migrate

  queue:
    <<: *default
    host: <%= ENV["QUEUE_POSTGRES_HOST"] %>
    migrations_paths: db/queue_migrate

config/environments/production.rb

  
# Replace the default in-process memory cache store with a durable alternative.
config.cache_store = :solid_cache_store

# Replace the default in-process and non-durable queuing backend for Active Job.
config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }

Kamal Ansible Manager

kamal-ansible-manager/roles/scaleway/vars/main.yml

            
---
scaleway_image: "69f43bac-da9b-49e3-b4fd-b826bef91e22"
scaleway_organization: "..."
scaleway_api_token: "..."
scaleway_commercial_type: "DEV1-L"
scaleway_region: "par1"
scaleway_public_ip: "dynamic"
scaleway_state: "running"
scaleway_instances:
  - app-1
  - worker-1
  - primary-db
  - queue-db
  - cache-db

Terminal

            
              $ ansible-playbook scaleway.yml

Kamal Ansible Manager

Deploy it with Kamal

config/deploy.yml

            
servers:
  web:
    - 192.168.0.1

  job:
    hosts:
      - 192.168.0.2
    cmd: bin/jobs

env:
  clear:
    POSTGRES_HOST: "192.168.0.3"
    QUEUE_POSTGRES_HOST: "192.168.0.4"
    CACHE_POSTGRES_HOST: "192.168.0.5"
  secret:
    - POSTGRES_USER
    - POSTGRES_DB
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:16
    host: 192.168.0.3
    port: 5432
    env:
      secret:
        - POSTGRES_USER
        - POSTGRES_DB
        - POSTGRES_PASSWORD

  queue-db:
    image: postgres:16
    host: 192.168.0.4
    port: 5432
    env:
      secret:
        - POSTGRES_USER
        - POSTGRES_DB
        - POSTGRES_PASSWORD

  cache-db:
    image: postgres:16
    host: 192.168.0.5
    port: 5432
    env:
      secret:
        - POSTGRES_USER
        - POSTGRES_DB
        - POSTGRES_PASSWORD

Deploy it with Kamal

Terminal

            
$ kamal setup # or kamal deploy

Deploy it with Kamal

Let’s Wrap Up

Keep it simple!
It just works
Hardware is more powerful than you think
Less headache
Mission Control Jobs is cool
Absolutely worth a try

Thanks! 🙏

https://guillaumebriday.fr/talks