Houseplant: Database Migrations for ClickHouse

Originally published on the June blog.

At June, we use ClickHouse as our analytics database to process billions of user events and generate real-time insights for our customers. While ClickHouse is incredibly powerful for analytical workloads, managing database schema changes has been a pain in our dev workflow.

Unlike traditional relational databases that have mature migration tools (like ActiveRecord for Rails), ClickHouse lacks a robust solution for managing schema migrations. We'd accumulated a lot of migrations and rake tasks to manage the schema of our ClickHouse cluster, but over time, it became hard for one person to make sense of all of it, let alone four.

We love working with ActiveRecord and how easy it makes checking the database schema, generating, applying, and rolling back migrations (if needed!). We wanted something similar for ClickHouse, so we built it ourselves.

Today, we're excited to announce Houseplant, a Python-based CLI tool that brings ActiveRecord-style migrations to ClickHouse!

Why Houseplant?

• Schema Management: Houseplant automatically tracks and manages your ClickHouse schema changes, making it easy to evolve your data model over time
• Developer Experience: Write migrations in YAML format, making them easy to read, review, and maintain
• Environment Support: Different configurations for development, testing, and production environments
• Rich CLI: Comes with an intuitive command-line interface for all migration operations

Getting Started

You can install Houseplant using pip:

$ pip install houseplant

Then you can initialize your project, and use it to generate and apply migrations just like how you would do it with the rails db:migrate command.

You just need to set up the correct environment variables which you can find in the project README.

Open Source

Houseplant is open source and available on GitHub. We believe in the power of community-driven development and welcome contributions from anyone who wants to improve ClickHouse schema management.

Check out the README to get started, and feel free to submit issues or pull requests!

We're excited to share Houseplant with the ClickHouse community and hope it makes your database management as enjoyable as tending to your houseplants! 🪴