Boosting grunt performance

I guess you’ve heard about Grunt, the JavaScript task runner. It’s a great automation tool, but when you start using it and keep adding more stuff to it, at some point you’ll probably get to a point where the execution takes more time then you are willing to wait.

I’ve recently hit that point with a document portal I’m working on at Red Hat, so I went to look out for how to do some profiling of grunt and how to speed it up. While there are some articles and stackoverflows about the topic, neither of them contained all that I needed, so the purpose of this blog post is to give you a digest with all information in one place.

The grunt setup of the project is specific in the way that it uses no CSS preprocessor and no JavaScript minification (such as uglify), because the portal is not a single page app and grunt is used mainly for linting server-side JavaScript code. Another project-specific thing is my first optimization.

src path wildcards

The project is based on CMS Alfresco and uses its folder structure, which is rather deep. So I was heavily using wildcards in paths of the src files of jshint and jscs to save myself some typing (e.g. “**/group-manager.js” instead of “Data Dictionary/Scripts/com/redhat/pnt/group-manager.js”). The problem is that grunt had to traverse the folder structure of the whole project (including a huge “node_modules” folder) to find those files. Avoiding wildcards made the grunt to speed up from 36s to 5.6s. You’ll probably won’t hit this in many projects, but I just wanted to mention it.

time-grunt

The first step in any optimization should be to measure “what the hack takes so long”. In case of grunt the profiler tool is time-grunt. time-grunt creates a time report that shows how much time each task took and appends the report to the output of each grunt call. It can look like this:

Execution Time (2016-10-22 18:34:14 UTC+2)
loading tasks  717ms  ▇▇▇▇▇▇ 13%
jshint:all      1.7s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 30%
jasmine:src    465ms  ▇▇▇▇ 8%
jscs:all        2.7s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 48%
Total 5.6s

Now we see that in this case most time was spent in jscs and jshint.

grunt-newer

The first think we can do to speed up those tasks is not to run them on files that didn’t change since the last time. Luckily (as mentioned in Two tips to boost Grunt performance), there is a grunt plugin exactly for that, called grunt-newer. The installation is pretty easy and described in the aforementioned link:

npm install grunt-newer --save-dev
Once the plugin has been installed, it may be enabled inside your gruntfile.js with this line:
grunt.loadNpmTasks('grunt-newer');

To use grunt-newer, you just need to prefix respective tasks with ‘newer:’, e.g. “newer:jshint”. So let’s do this and look at what the time-grunt output will look like when we change only one file:

Execution Time (2016-10-23 08:39:14 UTC+2)
loading tasks  788ms  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 42%
jasmine:src    467ms  ▇▇▇▇▇▇▇▇▇▇▇▇ 25%
jshint:all     159ms  ▇▇▇▇ 9%
jscs:all       423ms  ▇▇▇▇▇▇▇▇▇▇▇ 23%
Total 1.9s

Wow. We got a great speed-up (from 5.6s to 1.9s). Now we can see that the slowest task is “loading tasks” and that is exactly what we’ll look into next.

Don’t load all tasks

The second tip is not to load all tasks every time. While the aforementioned blog post describes a quite complicated way to achieve that, there is a grunt plugin to do that for us automatically: jit-grunt. The installation is again trivial:

npm install jit-grunt --save-dev

Remove grunt.loadNpmTasks, then add the require('jit-grunt')(grunt) instead. Only it.

module.exports = function (grunt) {
  require('jit-grunt')(grunt);

  grunt.initConfig({
    ...
  });

  grunt.registerTask('default', [...]);
}Links

So I installed that and let’s have a look at what does it do with the time.

Execution Time (2016-10-23 10:31:26 UTC+2)
loading grunt-contrib-jasmine   63ms  ▇▇ 4%
jasmine:src                    521ms  ▇▇▇▇▇▇▇▇▇▇ 32%
loading grunt-newer             27ms  ▇ 2%
loading grunt-contrib-jshint   109ms  ▇▇ 7%
jshint:all                     129ms  ▇▇▇ 8%
loading grunt-jscs             365ms  ▇▇▇▇▇▇▇ 23%
jscs:all                       363ms  ▇▇▇▇▇▇▇ 23%
Total 1.6s

Now the speed-up is not that impressive (from 1.9s to 1.6s), but it’s important to mentioned that all tasks are run on JacaScript files in this project, there is for example no CSS preprocessor (such as sass). If we had a project with tasks that don’t run on JavaScript files, there would surely be a better speed boost.

grunt-parallel

Another boost I was considering was to run the tasks in parallel (using grunt-parallel), because I have the fortune of having three tasks that don’t depend on each other (jscs, jshint and jasmine tests). I tested it and the results were highly varying (between 1.1s and 1.6s), apparently the parallelism makes the performance less deterministic. grunt-parallel also causes the other tasks to be time-reported separately and thus to be missing form the overall report.

Execution Time (2016-10-24 20:01:46 UTC+2)
parallel:assets  1.4s  ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%
Total 1.4s

So grunt-parallel doesn’t predictably do much speed-up and adds some clutter to the output, so unless there are tasks that take several seconds I don’t recommend using it.

Another issues

Another issues I would like to address in the future include:

Tasks are loaded each time a watch is triggered, not just at launching grunt watch.
sass doesn’t work at all with grunt-newer, because sass imports cause that change in one file might require other files to be recompiled.
uglify doesn’t work well with grunt-newer. When one file is changed, it sill processes all files, because it doesn’t have them cached before it concatenates them with the rest. They could have

Conclusion

The best speed-up is obtained by running tasks only on changed files, with grunt-newer. Another good shot is to use jit-grunt to load grunt task just in time and avoid unnecessary loads. If grunt is sill slow after applying those, I recommend to use the profiler time-grunt to find out what is the next pain point.