Turbo Streaming is going to work best with ActiveRecord Models. The Hacker News model uses Item models, which can be stories, comments, jobs, polls, or poll options. In the first implementation, the Item was modeled to be as generic as the API dispensed. This time, stories and jobs will continue to be modeled as Item and comments will be extracted into its own model, Comment. One wrinkle is that an item can appear in any of the lists, with its own order. Since of item could appear in any of the Top, New, Jobs, Ask, or Show lists at the same time, there will be an intermediate model holding the order associated with the Item.

Creating the Models

Create the Item model, and then the TopItem model:

$ rails g model Item hn_id:bigint:index by:string hn_type:integer time:datetime text:text url:string host:string score:integer title:string descendants:integer 
$ rails g model TopItem item:references location:integer:index
$ rails db:migrate

Item will include a populate method that takes the JSON API response and plucks out the values it needs. Here is app/models/item.rb:

class Item < ApplicationRecord
  has_one :top_item
  enum hn_type: [:story, :job]

  def populate(json) 
    if json.nil?
      return
    end
    self.hn_id = json['id'] if json['id']
    self.hn_type = json['type'] if json['type']
    self.by = json['by'] if json['by']
    self.time = DateTime.strptime("#{json['time']}",'%s') if json['time']
    self.text = json['text'] if json['text']
    self.parent = json['parent'] if json['parent']
    if json['url']
      self.url = json['url'] 
      host = URI.parse( json['url'] ).host
      self.host = host.gsub("www.", "") unless host.nil?
    end 
    self.score = json['score'] if json['score']
    self.descendants = json['descendants'] if json['descendants']
    self.title = json['title'] if json['title']
  end
end

TopItem is simpler, and references the associated Item object. Here is app/models/top_item.rb:

class TopItem < ApplicationRecord
  belongs_to :item
end

Background API Processing

Loading the TopItems will move to a background job. This will allow more visitors to see the content as it’s loaded into their pages asynchronously.

$ rails g job LoadTopItems

The LoadTopItemsJob is going load the item ids and their order from the API, and then load each item individually. There isn’t anyway to get around the N+1 API calls, and placing all these API calls into a background job means the front end performance and interactivity feel fast. Leveraging Turbo streams, the content will load asynchronously. Here is app/jobs/load_top_items_job.rb:

class LoadTopItemsJob < ApplicationJob
  queue_as :default

  def perform
    top_stories_json = JSON.parse HTTP.get("https://hacker-news.firebaseio.com/v0/topstories.json?print=pretty").to_s

    top_stories_json.each_with_index do |hn_story_id, top_news_location|
      begin
        story_json = JSON.parse HTTP.get("https://hacker-news.firebaseio.com/v0/item/#{hn_story_id}.json?print=pretty").to_s
        if story_json.nil?
          return
        end
        item = Item.where(hn_id: hn_story_id).first_or_create
        item.populate(story_json)
        item.save

        top_item = TopItem.where(location: top_news_location).first_or_create
        top_item.item = item
        top_item.save

      rescue URI::InvalidURIError => error
        logger.error error
      end
    end
  end
end

You can call this job from rails console, or you can put it in db/seeds.rb:

LoadTopItemsJob.perform_now

This will populate the data, giving the app something to load from the database.

Refactoring the Controllers

Since there are models in the database, TopsController doesn’t need to go out to the API to fetch the items. It should now pull the top 30 items, which will be displayed on front page. Using the includes directive will load the Item associated with each TopItem. For the time being, there will be a link to manually refresh the top item data. This can go under the create method. It calls the LoadTopItemsJob.

ITEMS_PER_PAGE ||= 30 

class TopsController < ApplicationController 

  def show
    @stories = TopItem.order(:location)
                      .limit(ITEMS_PER_PAGE)
                      .includes(:item)
  end

  def create
    LoadTopItemsJob.perform_now
  end
end

The ItemsController should also load the item from the database, rather than the API directly. It now becomes an ActiveRecord call:

class ItemsController < ApplicationController 

  def show
    @item = Item.find_by_hn_id params[:id]
  end
end

Updating the Views

The first, biggest change is creating a partial for Item. It takes the previous JavaScript dictionary, and puts it in app/views/items/_item.rb, with some tweaks for using an ActiveRecord model. There are some special cases in case the Item is a job posting, not a story, which are designated via an enum.

<div class="border rounded-sm flex flex-col">
  <header class="border flex justify-between">
      <span class="rounded-full py-1 px-3 bg-red-600 text-white"><%= item.score %></span>
      <p class="py-1 px-3">
        <% "job" if item.job? %>
        <%= item.host unless item.host.nil? %>
      </p>
  </header>
  <article class="border flex flex-col">
    <p class="py-1 px-3 flex-grow">
      <% if item.url.nil? %>
        <%= link_to item.title, item_path(item.hn_id) %>
      <% else %>
        <%= link_to item.title, item.url, { target: '_blank', rel: 'noopener' } %>
      <% end %>
    </p>
    <p class="py-1 px-3">
      <em><%= item.time.strftime('%c') %></em>
    </p>
  </article>
  <% unless item.job? %>
    <footer class="border flex justify-between">
      <p class="py-1 px-3">
        <%= link_to pluralize(item.descendants, 'comment'), item_path(item.hn_id) %>
      </p>
      <p class="py-1 px-3">
        <a href="/user/<%= item.by %>"><%= item.by %></a>
      </p>
    </footer>
  <% end %>
</div>

This simplifies app/views/items/show.html.erb. It renders the Item partial:

<%= render @item %>

TopItem gets its own view too, app/views/top_items/_top_item.html.erb. The div wrapper will be used by Turbo streams later.

<div id="<%= dom_id top_item %>">
  <%= render top_item.item %>
</div>

The last refactor is app/views/tops/show.html.erb. It removes the lazy loading for each item. Since the data loading for Item happens in the background, it queries the database, and gets the most recent records. It displays the TopItem partial.

<div class=" w-3/4 mx-auto p-4">
  <header class="bg-red-600 py-1 px-3 text-white flex">
    <p class=" text-2xl flex-grow">Top Stories</p>
    <%= link_to "Refresh", tops_path, method: :create, class: "pt-1" %>
  </header>
  <div class="grid grid-cols-3 gap-4">
    <%= render @stories %>
  </div>
</div>

Now the app is refactored into using a vanilla Rails setup, with ActiveRecord, ActiveJob, and the whole Model-View-Controller stack.

Putting on Turbo Streams

The app is interested in updates to a TopItem, since a new item could be in its place, and each Item can change as comments and the score change. The app will set up a connection, and listen over web sockets to the changes by adding the turbo_stream_from directive. It will multiplex over ActionCable for each Item and TopItem, and replace the DOM that comes over the connection. The updates can be triggered after ActiveRecord actions by adding broadcasts to the ActiveRecord model.

First, add broadcasts in the models, app/models/items.rb and app/models/top_items.rb. Then in the Item partial, app/views/items/_item.html.erb, add turbo_stream_from by replacing it under the opening div tag:

<div class="border rounded-sm flex flex-col" id="<%= dom_id item %>">
  <%= turbo_stream_from item %> 

In app/views/top_items/_top_item.html.erb, add it under the opening div:

<div id="<%= dom_id top_item %>">
  <%= turbo_stream_from top_item %>

Go back to the web app, click the “Refresh” link, and watch the page update in real time.

HHNPWA vs. HNPWA

In the original HNPWA project, Stimulus was used to subscribe to updates for each item. Now that Turbo handles the subscriptions and updates, no extra JavaScript was required. On the backend side, in HNPWA, the updates that were sent to the front end needed to be explicitly sent over ActionCable. The Turbo Rails integration handles that seamlessly with a single ActiveRecord directive. That’s magic!

Subscribe for updates as this project takes shape. You can see all the code at Github: https://github.com/OnRailsBlog/hhnpwa

Other HOTWire Tutorials

• HOTWire HNPWA #1: Setting up for Top Stories
• Turbo Streaming Top Items – HOTWire HNPWA #2
• Paginating Top Items – HOTWire HNPWA #3
• Russian Doll Caching – Building HOTWire HNPWA #4
• Lazy Loading Lots of Comments HOTWire Tutorial #5