At my work, we use Marko as our templating engine, which supports progressive rendering out of the box. So maybe that is why I took it for granted when I decided I would try to implement the same thing for Explorable Places. There are a few pages there that are data-heavy and it would help to start rendering the page right away rather than wait for everything to be loaded and rendered.
I expected there to be a gem already for this, in some shape or form, but was pleasantly surprised to find out Rails supports this out of box, and has since Rails 3! Now, I got this working, but not without a few gotchas I thought I’d share.
Choose a templating language that supports streaming
This may seem obvious, especially since I mentioned it above, but it wasn’t until later I realized I was in a pickle. I chose Haml when I first started the project because I liked it and it was a sort of standard in Rails to begin with, with how well it works with Ruby. I never had streaming the rendering of the page in mind back then. As it turns out, Haml does not support streaming. In fact, most templates don’t. But ERB does, and so does Slim. In benchmarking, it seems that Slim is actually slightly faster than ERB, but not enough to make me learn it while getting this up and running, so ERB is what I went with.
Since I only want a few pages to stream for now, I only needed to convert the layouts, templates and partials that those pages touch. This takes significantly less time than converting the whole site. Haml2erb helped a lot with speeding this process up.
Avoid any middleware that alters the html output.
Since we are streaming in chunks, if you have any middleware that alters this, the chunking will be corrupted and invalid and your page won’t even load. You’ll just get a not-so-helpful error message from your browser. I’ve read that this includes New Relic’s gem, but in my case, there were a few others.
Bullet was easy, but Rack Mini Profiler was tougher since it loads automatically and I haven’t touched these gems in forever. The mini profiler won’t work even if you choose to log to file only. Both of these gems I had to load conditionally behind an environment variable so I could still use them easily from time to time in development.
This may be all you need to get things up and running, but in my case, and hopefully in your case too, there should be one more step.
Monkey patch to get gzipped chunks working
The current Rails streaming implementation works by rendering directly to a chunked body. This means it is chunked, and then gzipped by the
Rack::Deflater middleware that we use (and you should too!). This corrupts the encoding and again, your page won’t load save for a not-so-helpful error message. There is a good explanation of this on Stack Overflow with a link to the Http spec and the example of the monkey patch that I took from.
The patch expects you to use the
Rack::Chunked middleware in addition to the Deflater. It overwrites Rails’ implementation of streaming so that it only removes the
Content-Length header and makes sure not to add the
Transfer-Encoding header. It will rely on the chunked middleware for encoding. It also directly renders the template, rather than to the chunked body, again relying on the middleware to do the chunking. Here is what I used:
The result is that the rack middlewares handle this correctly, finally! Good luck and happy rendering!