Full Stack Hanami, Part 2

hanami rom-rb

Damian C. Rossney

Not dogma, just what I'm learning and thinking about right now.
Comments and feedback are welcome on Mastodon.

"If you're thinking without writing, then you just think you're thinking."
—Leslie Lamport

Welcome to Part 2 of Full Stack Hanami, where we will explore using ROM-rb in our Hanami app. This is Part 2 of a three part series on adding persistence to Hanami 2.1.X. These three articles cover the following.

Part 1 - Adding ROM. Covers the gems we need and how to get them working with Hanami 2.
Part 2 - Working with ROM. Covers relations, repos, and database chores (creation, migrations, etc.).
Part 3 - Testing. Set up tooling to use your database in RSpec.

These guides are derived from the examples contained in the Hanami docs, Tim Riley’s Decaf Sucks project, and Brooke Kuhlmann’s Hemo demo application. I am grateful to those authors for providing such excellent examples to follow!

1. Migrations

In the previous article, we added a bin/hanami script to allow us to perform various database tasks from the command line. The first thing we did was to create our database using bin/hanami db setup. As discussed in that article, the setup command is the equivalent of running create and migrate in succession. However, since we don’t have any migrations yet, we still don’t have any tables to work with in our database. Let’s look at how we use migrations to create tables in our database.

To generate a migration file, we will use the db create_migration task from bin/hanami. This task expects to receive the name of a migration in snake case format. The task will then prepend a timestamp to the name provided and generate a migration file with that name in db/migrate. Note that neither of these directories exist in a new Hanami app, so the create_migration task will also create them for you. Let’s see what this process looks like.

If we run bin/hanami db create_migration create_books from the command line, we should then see the following file in db/migrate.

# db/migrate/20231109231807_create_books.rb

# frozen_string_literal: true

ROM::SQL.migration do
  change do
  end
end

We’ll then edit this file to create the structure of our Books table. A complete tutorial on migrations is beyond the scope of this article, but the instructions provided in the Hanami 1.3 guides are still applicable, I believe, as are the instruction provided in the ROM-rb documentation and the Sequel documentation. As an example, we could edit the migration file as follows.

# frozen_string_literal: true

ROM::SQL.migration do
  change do
    create_table :books do
      primary_key :id

      column :title, String, null: false
      column :author, String, null: false
    end
  end
end

The above migration creates a table named “books” with three columns:

id The primary key (auto-incrementing).
title A text field to hold the title of each book.
author A text field to hold the author’s name.

To apply this migration, and any other pending migrations, we use bin/hanami db migrate from the command line. The pending migrations will be applied in the order they were created, based upon the timestamp in their name.

2. Other Database Chores

As we already learned, we can use bin/hanami db setup to create and migrate the database. The counterpart to delete the database is bin/hanami db drop. You can use bin/hanami db reset to perform all three commands (drop, create, and migrate), and start with a fresh, empty database.

We can create a Ruby script to load the database with sample data. The Hanami CLI tasks provide two ways to do this, although I can’t see any difference between the two. You can create a script at db/sample_data.rb and invoke it with bin/hanami db sample_data. Or you can create a script at db/seeds.rb and invoke it with bin/hanami db seed. In either case, the script should use the relations and repositories we create below to feed data into the database. Since we haven’t covered relations and repositories, let’s hold off on a seed script example until the end of this article.

3. Relations

In Hanami 1.3, relations did not exist as a framework concept (they were there, but remained behind the scenes). However, they are a central concept in ROM-rb. You can find the ROM-rb documentation on relations here.

The purpose of relations is to provide an abstraction layer between ROM and the actual storage technology (Postgres in our case). Remember that ROM is designed to work with multiple–potentially unlimited–data sources, not just Postgres, and not just relational databases. As such, ROM relations have a number of responsibilities.

The first responsibility of a relation is to specify the adapter to use for the intended data source. For this article, that will be the rom-sql adapter used for relational databases. Next, the relation must inform ROM of the structure, or schema, of the data source it will use. Luckily, ROM is able to infer the structure of our tables through introspection, similar to ActiveRecord in Rails.

The simple relation implementation below is likely to be representative of the majority of relations you create. However, it’s good to know that relations retain the ability to act as a translation layer. We can use this, for example, to alias table and column names in a legacy database that we are not free to alter. We also have the ability to modify the inferences used to translate between database data types and Ruby data types when either reading or writing data. This is powerful stuff. Consult the docs linked above for more info.

Finally, relations define any references to associated tables in our database. Let’s see how this works. If we wanted to create a simple relation based on the Books example table above, it should look like this:

# app/relations/books.rb

module Fullstack
  module Relations
    class Books < ROM::Relation[:sql]
      schema(:books, infer: true)
    end
  end
end

Note that the :sql symbol passed in the class declaration tells ROM to use the rom-sql adapter for this relation. We pass the symbol :books to indicate our table name. If your actual table name is different than the class name you use, it should be passed as the first argument to schema in the form of a symbol, like this:

schema(:libres, infer: true)

As a final example, let’s look at relations that include associations between tables. In the real world, we know that books can have many authors, and authors can write many books, but for this example we will keep it simple and only model the case of books having many authors (that is, each of the authors in our database will be linked to a single book). With that in mind, let’s delete the migration we created above and create two new migrations for both books and authors.

> bin/hanami db create_migration create_books

# create_books migration

ROM::SQL.migration do
  change do
    create_table :books do
      primary_key :id

      column :title, String, null: false
    end
  end
end

> bin/hanami db create_migration create_authors

# create_authors migration

ROM::SQL.migration do
  change do
    create_table :authors do
      primary_key :id
      foreign_key :book_id, :books, null: false

      column :name, String, null: false
    end
  end
end

At this point, we have two new migrations and a database that was created based on a migration that no longer exists. To fix this, run bin/hanami db reset from the command line to drop the database, create a new one, and run all existing migrations. Just like that, we’re back in business!

As you can see, we no longer have an author column in Books. Instead, we will keep our authors in a separate table. Finally, note the convention of naming the foreign key with a symbol of the style :<foreign_table>_id.

To inform ROM of this association, we add association references in both our Books and Authors relation classes. We do this by passing a block to the schema declaration. These relations should look something like this (create the app/relations/authors.rb file now).

# app/relations/books.rb

module Fullstack
  module Relations
    class Books < ROM::Relation[:sql]
      schema(:books, infer: true) do
        associations { has_many :authors }
      end
    end
  end
end

# app/relations/authors.rb

module Fullstack
  module Relations
    class Authors < ROM::Relation[:sql]
      schema(:authors, infer: true) do
        associations { belongs_to :book }
      end
    end
  end
end

Note that in the Books relation, we passed the “symbolized” version of Authors: :authors. But in the Authors relation, the associations DSL requires us to pass the singularized symbol for the Books table: :book. By following these conventions, ROM is able to infer both the associated table names, and the foreign key column name contained in Authors. Now, if we invoke the Hanami console, boot our app, and examine Hanami.app.keys, we’ll see both relations.books and relations.authors present there. We’ll use these relations in the repositories we create in the next section.

That’s all we’ll cover in relations in this introductory tutorial, but I encourage you to consult the ROM docs cited above for more information on the power of relations.

4. Repositories

Just as relations form an abstraction layer between ROM and the underlying database, repositories form an abstraction layer between our app and ROM. By explicitly defining every intended interaction with the database as a method in a repository class, we can isolate our app from an explicit dependency on ROM or the underlying database technology we choose. It also makes it easier to test our application objects that depend on the repositories we create by defining the expected interface of the repository. We can then create “plain old Ruby objects” (POROs) in our tests that mock out the expected interface, and use these POROs to test our application objects without actually touching the database–a big win for test speed and isolation.

In brief, the repository provides us with an object to use to interact with the database. All of the “queries” (to retrieve data) and “commands” (to create, change, or destroy data) exist as methods on the repository object. Unlike relations, where there is a 1:1 mapping of database tables to relation classes, you will typically have fewer repositories than you do database tables. This is because related child tables will commonly be accessed through the parent table’s corresponding repository rather than have a repository class of their own.

A Repository Base Class

Per the ROM docs, “Repositories must be instantiated with a ROM container passed to the constructor, this gives access to all components within the container,” like so:

user_repo = UserRepo.new(rom-container)

However, rather than deal with this every time we use a repository, instead we’ll create a repository base class to use throughout our app. We’ll call the file repository.rb and we’ll place it directly in our app directory. From the example in Decaf Sucks, the file should look like this:

# app/repository.rb

# auto_register: false
# frozen_string_literal: true

require "rom-repository"

module Fullstack
  class Repository < ROM::Repository::Root
    include Deps[container: "persistence.rom"]
  end
end

Naming Conventions

While there are no official conventions for the naming of relations and repositories (that I am aware of so far), I follow the convention of using plural names for relations (and database tables), and the corresponding singular name for repositories. When instantiating a repository, I append “_repo” to the instantiated object’s name to distinguish the repository object from instances of the underlying records. For example, I want to distinguish between a “book_repo” instantiated in my code and any particular instance of a “book” that I might be working with.

The Book Repository

With the base class in place, we can now create repository classes as needed. We’ll start by creating a book repository, named book.rb, in a new repositories directory under /app. When creating our repository, we’ll specify the corresponding relation by passing the symbol :books in the repository class declaration.

# app/repositories/book.rb

# frozen_string_literal: true

module Fullstack
  module Repositories
    class Book < Fullstack::Repository[:books]
      def all
        book.sorder(:title).to_a
      end

      def find_by_id(id)
        books.by_pk(id).one
      end
    end
  end
end

Instantiating a Repository

To utilize a repository in your code, you will typically use Hanami’s Deps mixin (refresher here). As discussed above, we will specify an alias for our repositories, like book_repo for the above Book repository:

include Deps[ book_repo: "repositories.book" ]

Using the Book Repository to Read Data

In the above example, Book has two methods #all and #find_by_id. The #all method returns an array of all books in the books table (in alphabetical order by title), while #find_by_id returns either the book with the matching id, or null if there is no match. In all of these cases, the objects returned will be instances of ROM::Struct, with attr_readers for each record attribute, as defined in the database or aliased in your relation class. You can read more about this and additional methods for querying data in the ROM docs here and here.

You could invoke these methods as follows:

book_repo = Hanami.app["repositories.book"]

# array of all books
book_repo.all

# single author by id
book_repo.find_by_id(id)

Using the Book Repository to Change Data

Let’s round out the basic CRUD operations by adding methods to create, update, and delete records. Since these are common tasks, ROM provides “macros” to avoid boilerplate code. Following is our book repository implementing the create, update, and delete macros. Note that since update and delete operations require us to identify an existing record, we will do this by passing the :by_pk symbol to these macros. This tells the update and delete commands to use the #by_pk method to locate the target record. The #by_pk method is provided by the rom-sql adapter.

# app/repositories/book.rb

# frozen_string_literal: true

module Fullstack
  module Repositories
    class Book < Fullstack::Repository[:books]
      commands :create, update: :by_pk, delete: :by_pk

      def all
        books.order(:title).to_a
      end

      def find_by_id(id)
        books.by_pk(id).one
      end
    end
  end
end

Now we can create a new record by passing keyword arguments representing required fields to #create. Note that #create will return a ROM::Struct representing the newly created record. We can use this return value, for example, to retrieve the id of the new record, if needed. Let’s try some of this in the Hanami console.

> bundle exec hanami console

fullstack[development]> Hanami.app.boot
=> Fullstack::App

fullstack[development]> book_repo = Fullstack::App["repositories.book"]
=> #<Fullstack::Repositories::Book struct_namespace=ROM::Struct auto_struct=true>

fullstack[development]> book_repo.create(title: "POODR")
=> #<ROM::Struct::Book id=2 title="POODR">

fullstack[development]> book_repo.create(title: "Extreme Programming Explained")
=> #<ROM::Struct::Book id=3 title="Extreme Programming Explained">

fullstack[development]> book_repo.all
=> [#<ROM::Struct::Book id=3 title="Extreme Programming Explained">,
#<ROM::Struct::Book id=2 title="POODR">]

fullstack[development]> book = book_repo.find_by_id(2)
=> #<ROM::Struct::Book id=2 title="POODR">

fullstack[development]> book_repo.update(book.id, title: "Practical Object-Oriented Design in Ruby")
=> #<ROM::Struct::Book id=2 title="Practical Object-Oriented Design in Ruby">

fullstack[development]> book_repo.all
=> [#<ROM::Struct::Book id=3 title="Extreme Programming Explained">,
#<ROM::Struct::Book id=2 title="Practical Object-Oriented Design in Ruby">]

fullstack[development]> book_repo.delete(3)
=> #<ROM::Struct::Book id=3 title="Extreme Programming Explained">

fullstack[development]> book_repo.all
=> [#<ROM::Struct::Book id=2 title="Practical Object-Oriented Design in Ruby">]

Books with Authors

That was fun. ;-) Let’s move on to working with books and authors together. First, we’ll refactor our Book repository to work with authors. Let’s make it look like this.

# app/repositories/books.rb

# frozen_string_literal: true

module Fullstack
  module Repositories
    class Book < Fullstack::Repository[:books]
      def create(book)
        books_with_authors.command(:create).call(book)
      end

      def all
        books_with_authors.order(:title).to_a
      end

      def find_by_id(id)
        books_with_authors.by_pk(id).one
      end

      private

      def books_with_authors
        books.combine(:authors)
      end
    end
  end
end

Now, #all and #find_by_id will return a book struct that includes a collection of author structs under the key :authors. Assuming the following data exists in the database, an example usage could look like this:

book_repo = Hanami.app["repositories.book"]

book_repo.find_by_id(1)

# => #<ROM::Struct[Book] id=1 name="Practical Object Oriented Design"
authors=[#<ROM::Struct[Author] id=1 book_id=1 name="Metz, Sandi">]>

The Author Repository

With that in place, let’s create our Author repository.

# app/repositories/author.rb

# frozen_string_literal: true

module Fullstack
  module Repositories
    class Author < Fullstack::Repository[:authors]
      commands :create, update: :by_pk, delete: :by_pk

      def all
        authors.order(:name).to_a
      end

      def find_by_id(id)
        authors.by_pk(id).one
      end
    end
  end
end

Finally, we’ll use the next section on seed scripts to demonstrate how to work with related (aggregated) data.

5. Seed Scripts

This example will be a bit contrived, because you will likely seed your database from pre-configured data, maybe from prepared CSV files, or using generators like Factory Bot or similar. For our purposes, we will just specify a few records for illustrative purposes and load them into the database. In order to create these records, we’ve added a #create method to our Books repository. This method expects to receive keyword arguments representing the attributes of a book, that is :title (a string), and :authors (an array of authors). Let’s see how we can use #create in a seed script. We’ll start by creating a seeds.rb file in /db.

# db/seeds.rb

# frozen_string_literal: true

BOOKS = [
  {title: "POODR", authors: [{name: "Metz, Sandi"}]},
  {title: "Domain-Driven Design", authors: [{name: "Evans, Eric"}]},
  {title: "Test-Driven Development", authors: [{name: "Beck, Kent"}]},
  {title: "Hypermedia Systems", authors: [
    {name: "Gross, Carson"},
    {name: "Stepinski, Adam"},
    {name: "Askimsek, Deniz"}
    ]}
  ]

book_repo = Hanami.app["repositories.book"]

BOOKS.each { |book| book_repo.create(book) }

Usage might look like this:

❯ bin/hanami db reset
=> database fullstack_development dropped
=> database fullstack_development created
=> database fullstack_development migrated in 0.3924s

❯ bin/hanami db seed
=> seed data loaded from db/seeds.rb in 0.3735s

❯ bundle exec hanami console

fullstack[development]> app = Hanami.app.boot
=> Fullstack::App

fullstack[development]> book_repo = app["repositories.book"]
=> #<Fullstack::Repositories::Book struct_namespace=ROM::Struct auto_struct=true>

fullstack[development]> book_repo.all
=> [#<ROM::Struct::Book id=2 title="Domain-Driven Design"
authors=[#<ROM::Struct::Author id=2 book_id=2 name="Evans, Eric">]>,
 #<ROM::Struct::Book id=4 title="Hypermedia Systems"
 authors=[#<ROM::Struct::Author id=4 book_id=4 name="Gross, Carson">,
 #<ROM::Struct::Author id=5 book_id=4 name="Stepinski, Adam">,
 #<ROM::Struct::Author id=6 book_id=4 name="Askimsek, Deniz">]>,
 #<ROM::Struct::Book id=1 title="POODR"
 authors=[#<ROM::Struct::Author id=1 book_id=1 name="Metz, Sandi">]>,
 #<ROM::Struct::Book id=3 title="Test-Driven Development"
 authors=[#<ROM::Struct::Author id=3 book_id=3 name="Beck, Kent">]>]

This example is nice because everything is right there in front of you. The seed data is structured in a readable way, and the usage of the book repository is clear and familiar. But what if we want to seed a lot of data? The above method would become unwieldy very quickly.

One strategy is to seed using randomly generated data. You can check out this episode of Hanami Mastery to see this approach in action. However, the random strategy falls short when you need to create specific scenarios in your data. In this case it is common to save the seed data in some structured format, such as CSV or YAML, and then read these files in with the seed script and write them to the database.

Let’s see how this approach might work with the same data from the previous example. For simplicity’s sake, we won’t read from data files, but will simulate their structure as constants in the seed script. To make things more interesting, we’ll bypass our repositories and use relations to access the database. This will allow us to use a built-in ROM relation method to save multiple rows at once.

# db/seeds.rb

# frozen_string_literal: true

BOOKS = [
  ["id", "title"],
  [1, "POODR"],
  [2, "Domain-Driven Design"],
  [3, "Test-Driven Development"],
  [4, "Hypermedia Systems"]
]

AUTHORS = [
  ["book_id", "name"],
  [1, "Metz, Sandi"],
  [2, "Evans, Eric"],
  [3, "Beck, Kent"],
  [4, "Gross, Carson"],
  [4, "Stepinski, Adam"],
  [4, "Askimsek, Deniz"]
]

rom = Hanami.app["persistence.rom"]
books = rom.relations["books"]
authors = rom.relations["authors"]

# labels start as strings to simulate a CSV file
book_labels = BOOKS.first.map(&:to_sym)
book_tuples = BOOKS[1..].map { |tuple| book_labels.zip(tuple).to_h }

books.multi_insert(book_tuples)

author_labels = AUTHORS.first.map(&:to_sym)
author_tuples = AUTHORS[1..].map { |tuple| author_labels.zip(tuple).to_h }

authors.multi_insert(author_tuples)

You would invoke this script in the same way as the previous one:

> bin/hanami db reset
> bin/hanami db seed

This script is written to simulate CSV input (with the field labels as strings in the first row of each data array, for example). To load prepared CSV files, you could just require 'csv' at the top of the seed script and load however many CSV files you have in place of the constants provided here. You will note that I had to specify the id fields for the book records. This is because we needed to specify them in the author records for referential integrity.

That’s it for the second article in this series! We can now move on to Part 3, using the database in testing.