Rails 8: Multi-db with Solid Queue, Solid Cache and Kamal
Geneva.rb - March 12, 2025
The plan for today
- Why bother?
- Concepts behind of Solid Queue & Solid Cache
- How to implement it?
- Deploy it with Kamal
Open source contributor
A little bit of context
Why bother?
What's Solid Cache and Solid Queue?Why bother?
Cost, Simplicity and performance
Why bother?
Rails 8 only requires a database as a dependency.
And it can be SQLite.Why bother?
- Toolchain free
- Importmaps and No Build
- Solid Queue through Puma (like async)
- Solid Cache & Solid Cable
- Interactions with Hotwire
The concept behind the Solid Trifecta
It replaces the traditional RAM-based solutions like Redis with efficient, cost-effective database-backed tools.
RAM-based solution
db-backed solution
Many advantages
- One less component to manage
- Cheaper
- Way more space
- Fast enough, maybe even faster
- We can use SQL to query data
- Works out of the box
Introducing Solid Cache
- Active Support cache store
- Battle tested
- Cache expiry
- Encryption
- Sharding
Solid Cache trade-off
| Size | Price | Performance | |
|---|---|---|---|
| Redis | 🤏 | 💵💵💵 | 🏎️🏎️🏎️ |
| Database | 🌌 | 💵 | 🏎️🏎️ |
Solid Cache cost example
| Size | Price | |
|---|---|---|
| Heroku | 500MB | $120/m |
| Scalingo | 4GB | €115/m |
| Scaleway | 8c/16t 64GB 2TB NVMe | €120/m |
Solid Cache trade-off
A balance between speed and cache miss rate.
PostgreSQL Optimizations
- Made to run on NVMe
- Disable Write Ahead Logging (WAL)
- Increase the buffer cache
- Use pg_prewarm extension
- Solid Cache for Rails and PostgreSQL by Andrew Atkinson
Introducing Solid Queue
- Exactly the same as Solid Cache
- Delayed jobs
- Concurrency controls
- Recurring jobs (Cron)
- Bulk enqueuing (perform_all_later)
- Trying out Solid Queue and Mission Control with PostgreSQL by Andrew Atkinson
How to implement them?
config/database.yml
production: primary: <<: *default host: <%= ENV["POSTGRES_HOST"] %> cache: <<: *default host: <%= ENV["CACHE_POSTGRES_HOST"] %> migrations_paths: db/cache_migrate queue: <<: *default host: <%= ENV["QUEUE_POSTGRES_HOST"] %> migrations_paths: db/queue_migrate
config/environments/production.rb
# Replace the default in-process memory cache store with a durable alternative.
config.cache_store = :solid_cache_store
# Replace the default in-process and non-durable queuing backend for Active Job.
config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }
Kamal Ansible Manager
Kamal Ansible Manager
kamal-ansible-manager/roles/scaleway/vars/main.yml
---
scaleway_image: "69f43bac-da9b-49e3-b4fd-b826bef91e22"
scaleway_organization: "..."
scaleway_api_token: "..."
scaleway_commercial_type: "DEV1-L"
scaleway_region: "par1"
scaleway_public_ip: "dynamic"
scaleway_state: "running"
scaleway_instances:
- app-1
- worker-1
- primary-db
- queue-db
- cache-db
Terminal
$ ansible-playbook scaleway.yml
Kamal Ansible Manager
Deploy it with Kamal
config/deploy.yml
servers:
web:
- 192.168.0.1
job:
hosts:
- 192.168.0.2
cmd: bin/jobs
env:
clear:
POSTGRES_HOST: "192.168.0.3"
QUEUE_POSTGRES_HOST: "192.168.0.4"
CACHE_POSTGRES_HOST: "192.168.0.5"
secret:
- POSTGRES_USER
- POSTGRES_DB
- POSTGRES_PASSWORD
accessories:
db:
image: postgres:16
host: 192.168.0.3
port: 5432
env:
secret:
- POSTGRES_USER
- POSTGRES_DB
- POSTGRES_PASSWORD
queue-db:
image: postgres:16
host: 192.168.0.4
port: 5432
env:
secret:
- POSTGRES_USER
- POSTGRES_DB
- POSTGRES_PASSWORD
cache-db:
image: postgres:16
host: 192.168.0.5
port: 5432
env:
secret:
- POSTGRES_USER
- POSTGRES_DB
- POSTGRES_PASSWORD
Deploy it with Kamal
Terminal
$ kamal setup # or kamal deploy
Deploy it with Kamal
Let’s Wrap Up
- Keep it simple!
- It just works
- Hardware is more powerful than you think
- Less headache
- Mission Control Jobs is cool
- Absolutely worth a try
Thanks! 🙏
https://guillaumebriday.fr/talksRails 8: Multi-db with Solid Queue,
Solid Cache and Kamal Guillaume Briday Geneva.rb - March 12, 2025