Jekyll has been the engine behind this website – scandio.de – for over a year now. Ever since the site has grown, both content and feature wise which unfortunately happened in a not-developer-friendly way: Until recently, the generating time from the source took more than one minute on this site, making it a bit tedious to write blog posts or to work on the site in general.

This post dives into which culprits we identified and how we reduced the generation time from 64 to only 16 seconds.

It's compiling
© xkcd.com

What takes Jekyll so long?

To figure out on which object Jekyll spends that time, the --profile flag can be used on most Jekyll commands. For example, jekyll build --profile returns a list of the most time consuming templates and rendered objects.

Filename                                                            | Count |    Bytes |   Time
--------------------------------------------------------------------+-------+----------+-------
_layouts/post.html                                                  |   410 | 7116.58K | 38.884
_includes/header.html                                               |   523 | 1142.60K | 22.430
_layouts/blog.html                                                  |    42 | 1309.84K |  7.285
_includes/head.html                                                 |   523 | 2392.85K |  4.785
_layouts/page.html                                                  |    19 |  364.14K |  0.997
_layouts/simple.html                                                |    18 |  250.16K |  0.963
_includes/footer.html                                               |   523 | 1374.47K |  0.848
_layouts/front.html                                                 |    16 |  206.36K |  0.793
(...)

                    done in 64.748 seconds.

The culprit: Our translation handling

Already 38 seconds are being spent on the post.html template (which includes the header.html). What’s so special about these templates? By removing blocks from that template and comparing the generation benchmarks, the reason becomes obvious, very quickly: Our translation handling.

As you can see, this website offers a German and an English version. For content that is available in both languages we want to directly link to the translated version of that page. Since Jekyll doesn’t know about languages by default, here’s some background on our implementation:

  • Each post and page has a property lang which defines the language (de or en)
  • Additionally there is the property page-name which identifies translations (both language versions share the same page-name)

Our approach is custom to this page, and the translation mechanism is limited to two languages. So it is not comparable to the feature set of other i18n plugins but covers all our needs. Nonetheless the underlying concept can also be applied to other pages.

Two parts of our implementation were particularly expensive:

Linking to the previous and next post of the same language

The “Previous Post / Next Post” functionality relied on iterating the site.posts array within the post.html template. Jekyll already offers the post.next and post.previous properties, but those aren’t aware of any languages and would therefore mix posts in German and English. So we had to look for the previous and next post in the current language manually:

<!-- post.html -->
{% assign  document = site.posts | sort: "date" | reversed | where:"lang", page.lang %}
{% for links in document %}
  {% if links.title == page.title %}

    {% unless forloop.first %}
      {% assign prevurl = prev.url %}
      {% assign prevtitle = prev.title %}
    {% endunless %}

    {% unless forloop.last %}
      {% assign next = document[forloop.index] %}
      {% assign nexturl = next.url %}
      {% assign nexttitle = next.title %}
    {% endunless %}

  {% endif %}

  {% assign prev = links %}
{% endfor %}

{% if prevurl or nexturl %}
  {% if nexturl %}<a href="{{nexturl}}" class="next">{{nexttitle}}</a>{% endif %}
  {% if prevurl %}<a href="{{prevurl}}" class="previous">{{ prevtitle}}</a>{% endif %}
{% endif %}

Traversing the posts array for every single post object slowed Jekyll down to a great extent – and it is not particularly nice to have this logic in the template either. So we were wondering if we could make the information available as a dedicated property similar to page.next. In an attempt to achieve this, we wrote a Generator plugin to alter the post object to include this information before it is rendered.

# source/_plugins/postlanguage/postlanguage.rb
module Jekyll
  class LanguageNext < Generator

    safe true
    priority :high

    def generate(site)

      all_posts = site.posts.docs

      langs = ['de', 'en']
      langs.each do |lang|

        posts = all_posts.select {|post| post.data['lang'] == lang}
        posts.each_with_index do |post, index|

          if index == 0
            post.data['prev_lang'] = nil
          else
            post.data['prev_lang'] = posts[index - 1]
          end

          if index == posts.size - 1
            post.data['next_lang'] = nil
          else
            post.data['next_lang'] = posts[index + 1]
          end

        end
      end

    end

  end
end

(To make sure Jekyll picks up the plugin, the line plugins_dir: _plugins must be added to the _config.yml, if not yet present.)

So the code to render the links in the template can be reduced to these few lines:

<!-- post.html -->
{% if page.next_lang or page.prev_lang %}
    {% if page.next_lang %}<a href="{{page.next_lang.url}}" class="next">{{ page.next_lang.title }}</a>{% endif %}
    {% if page.prev_lang %}<a href="{{page.prev_lang.url}}" class="previous">{{ page.prev_lang.title }}</a>{% endif %}
{% endif %}

Improvement: 20 seconds

Linking translations with each other

The link to the translation of a page was implemented similarly: Loop over all pages and posts and look for an object with the same page-name but a different lang property.

<!-- header.html -->
{% assign posts = site.posts | where:"name", page.name | sort: 'path' %}
{% for post in posts %}
    {% if post.lang != page.lang %}
        <a href="{{ post.url }}" class="lang-button {{ post.lang }}"></a>
    {% endif %}
{% endfor %}

{% assign pages = site.pages | where:"page-name", page.page-name | sort: 'path' %}
{% for my_page in pages %}
    {% if my_page.lang != page.lang %}
        {% unless my_page.url contains '/page' %}
            <a href="{{ my_page.url }}" class="lang-button {{ my_page.lang }}"></a>
        {% endunless %}
    {% endif %}
{% endfor %}

Again, looping over all posts and pages in the template is very time consuming. Reusing the approach from above, made the translation of each document available as page.translation using a Generator plugin:

# source/_plugins/translation/translation.rb
module Jekyll
  class Translation < Generator

    safe true
    priority :high

    def generate(site)

      translations = {}

      site.pages.each do |page|
        name = page.data['page-name']
        lang = page.data['lang']
        if name
          if translations.key?(name)
            translations[name].each do |translation|
              if translation.data['lang'] != lang
                page.data['translation'] = translation
                translation.data['translation'] = page
              end
            end
          else
            translations[name] = []
          end
          translations[name].push(page)
        end
      end

      site.posts.docs.each do |post|
        name = post.data['name']
        lang = post.data['lang']
        if name
          if translations.key?(name)
            translations[name].each do |translation|
              if translation.data['lang'] != lang
                post.data['translation'] = translation
                translation.data['translation'] = post
              end
            end
          else
            translations[name] = []
          end
          translations[name].push(post)
        end
      end

    end
  end
end

Of course this solution can only cope with two languages. But it reduces the complexity to render the link to this:

<!-- header.html -->
{% if page.translation %}
	<a href="{{ page.translation.url }}" class="lang-button {{ page.translation.lang }}"></a>
{% endif %}

Improvement: 18 seconds

Current State

Now our current jekyll build looks more like this:

Filename                                                             | Count |    Bytes |  Time
---------------------------------------------------------------------+-------+----------+------
_layouts/post.html                                                   |   347 | 5778.48K | 3.055
_includes/header.html                                                |   454 |  542.65K | 1.498
_includes/head.html                                                  |   454 | 2069.25K | 0.639
_includes/footer.html                                                |   454 | 1191.76K | 0.527
_layouts/blog.html                                                   |    35 | 1005.89K | 0.393
sitemap.xml                                                          |     1 |   58.21K | 0.331
(...)
                    done in 16.137 seconds.

Along with some smaller adjustments, these changes reduced the generation time by almost 75% – and we learned a lot about the internals of Jekyll and how to implement Jekyll plugins, too. By using build options like --incremental and --limit_posts the time for most regeneration calls has been reduced to just 3 seconds. But most importantly: Working on this site has now become a lot more fun again. :-)