Improving Performance with Russian Doll Caching HOTWire Tutorial #6

Now that the HOTWire Hacker News Progressive Web App loads in the comments asynchronously, you should experience the slow loading on the comments page. Since each level of comments can load another level of comments, it will feel slow as everything loads into the page in chunks. You’ll also notice that the comments are always empty every time the article loads. But we can optimize this loading experience to make it feel more interactive. First, use a Russian Doll Caching strategy, and then, change the loading algorithm to cut down on the time it takes for a single comment to respond back to the front end.

Russian Doll Caching

The hierarchy of caching in the comments simplifies the initial page load. Loading the outermost Comment, from cache, includes any child comments in the HTML partial. For example, 400 comments should require only a few database queries on the initial page load, resulting in a huge performance win on the front end. It only requires adding the cache directive around each comment partial, app/views/comments/_comment.html.erb:

<%= cache comment do %>

<% unless comment.dead %>
  <div class="w-full">
    <% if comment.text and comment.by and comment.time %>
      <div class="border-l border-b pl-4 mb-4">
        <%= sanitize comment.text %>
        <footer class="border-t flex justify-between">
          <p class="py-1 px-3">
            <a href="/user/<%= comment.by %>"><strong><%= comment.by %></strong></a>
          </p>
          <p class="py-1 px-3">
            <%= comment.time.strftime('%c') %>
          </p>
        </footer>
      </div>
    <% end %>
    <div class="w-11/12 float-right">
      <%= render partial: 'comments/comments', locals: { item: @item, comments: comment.kids.order(:location) } %>
    </div>
  </div>
<% end %>

<% end %>

This comment partial also checks to make sure the details have been loaded before displaying them. This is an optimization for the next section, where a comment may not have been completely populated from the API.

Since the comments are now cached, let’s display them in the comment list partial, app/views/comments/_comments.html.erb:

<div class="flex flex-col">
  <% comments.each do |comment| %>
    <%= turbo_frame_tag dom_id(comment), src: item_comment_path(item.hn_id, comment.hn_id) do %>
      <%= render comment %>
    <% end %>
  <% end %>
</div>

Show the comments under an item in app/views/items/show.html.erb:

<header class="bg-red-600 py-1 px-3 mb-11 text-white flex">
  <p class="text-2xl flex-grow">
    <%= link_to "Top Stories", top_path %></p>
</header>
<%= render @item %>

<%= turbo_frame_tag "comments", src: item_comments_path(@item.hn_id) do %>
  <%= render partial: 'comments/comments', locals: { item: @item, comments: @item.kids.order(:location) } %>
<% end %>

These small change means that on a subsequent page refresh, everything loads immediately, and then the comments update in the background. However, what happens to the inner nested comments when they have an update, but the parent doesn’t? You need to bust the cache!

Busting the cache in the Russian Doll Caching scheme is when an “inner” item in the cache updates the immediate outer item with the hope it will force some kind of cache invalidation. Since the comments are indefinitely nested, this needs to happen all the way up to the outermost parent. We can use an ActiveRecord callback which will touch the parent Comment. This parent Comment will touch its parent, if one exists, which will only change the time stamp on the parent record. Here is the updated Comment model, app/models/comment.rb, with the after_save :update_parent callback:

class Comment < ApplicationRecord
  has_one :hn_parent, class_name: 'Comment', primary_key: 'parent_id', foreign_key: 'hn_id'
  has_many :kids, class_name: "Comment", primary_key: 'hn_id', foreign_key: 'parent_id'
  after_save :update_parent

  def populate(json) 
    if json.nil?
      return
    end
    self.hn_id = json['id'] if json['id']
    self.by = json['by'] if json['by']
    self.parent_id = json['parent'] if json['parent']
    self.text = json['text'] if json['text']
    self.dead = json['dead'] if json['dead']
    self.time = DateTime.strptime("#{json['time']}",'%s') if json['time']
  end

  def update_parent
    hn_parent.touch if hn_parent.present?
  end
end

Optimizing the API calls

Since the frontend structure of the page has changed, it might be good to revisit the comments_controller.rb to see if there are any changes necessary.

Each individual comment makes a request through Turbo for an update, so there isn’t a need to retrieve the comments children any more. This cuts down on many extra API calls, which means each individual comment is going to load quicker. Remember that the API call returns the descendants of a Comment, and now only this is stored. Rendering a Comment will have placeholder, lazy-loading frames for each descendant comment, which will negate the need for duplicate details API requests. The returned HTML will have the comment descendants in an empty list, in the correct order. Most of the changes to controllers/comments_controller.rb are cleaning up this descendants strategy. load_kids becomes save_kids, and will store the child comment hn_id and location.

class CommentsController < ApplicationController 

  def index
    @item = Item.find_by_hn_id params[:item_id]

    item_json = JSON.parse HTTP.get("https://hacker-news.firebaseio.com/v0/item/#{@item.hn_id}.json").to_s

    if item_json.nil?
      return
    end
    @item.populate(item_json)
    @item.save
    save_kids(@item.hn_id, item_json)
  end

  def show 
    @item = Item.find_by_hn_id params[:item_id]
    @comment = Comment.find_by_hn_id params[:id]
    item_json = JSON.parse HTTP.get("https://hacker-news.firebaseio.com/v0/item/#{@comment.hn_id}.json").to_s
    if item_json.nil?
      return
    end
    @comment.populate(item_json)
    @comment.save
    save_kids(@comment.hn_id, item_json)
  end

  private 

  def save_kids(parent_id, item_json)
    if item_json and item_json.has_key? 'kids'
      item_json['kids'].each_with_index do |kid_hn_id, kid_location|
        kid = Comment.where(hn_id: kid_hn_id).first_or_create
        kid.location = kid_location
        kid.parent_id = parent_id
        kid.save
      end
    end
  end
end

HHNPWA vs. HNPWA

The previous version used Russian Doll Caching very effectively, so this isn’t very different. HOTWire again helps load in the comments one by one, and this tutorial improves the full experience by loading comments from cache. It also optimizes the strategy for retrieving child comments so the experience feels faster, and there are fewer network calls to the Hacker News API.

Next Steps

Subscribe for updates as this project takes shape. You can see all the code at Github: https://github.com/OnRailsBlog/hhnpwa.

Blogging On Rails

Improving Performance with Russian Doll Caching HOTWire Tutorial #6

Russian Doll Caching

Optimizing the API calls

HHNPWA vs. HNPWA

Next Steps

Other HOTWire Tutorials

Looking to make your Rails app more interactive?

Leave a Reply Cancel reply