Decrease PHPUnit Code Coverage Execution Time By 75%

Recently my team at work spent some time looking into the performance of our PHPUnit test suite. One of the things we wanted to improve was the rather lengthy code coverage execution time, as the suite recently crossed the 12 minute code coverage execution time threshold on our local machines. When it takes this long to generate a code coverage report, you just really don’t want to wait every time you have a feature to commit. As I’m sure you’re much more interested in how to decrease that coverage time, I’ll get right to it.

We did two things to decrease the execution time:

  • Updated to PHPUnit 7.4
  • Used a new feature available in PHPUnit 7.4 to offload the file whitelist to XDebug
    • This uses a feature that is only available in XDebug 2.6+

If you’re currently using PHPUnit7.4+ and XDebug 2.6+, or can upgrade to them, you can probably save a solid chunk of time.

Starting with PHPUnit 7.4, a couple great improvements were made related to code coverage execution time. First, they made a number of internal improvements to code coverage. Just updating from PHPUnit 7.3 to PHPUnit 7.4 decreased our execution time from ~12 minutes to ~8 minutes. Second, they added two very important options: --dump-xdebug-filter and --prepend.

The 2 above options introduced in PHPUnit 7.4 utilize a feature introduced in XDebug 2.6 called code coverage filtering. This new XDebug feature allows you to use a configuration file to directly tell XDebug which files it should or should not analyze during execution. Without this feature, we just ignore the results of the analyzation, which isn’t exactly efficient.

By using the 2 new options available in PHPUnit 7.4 to apply the file whitelist in XDebug (based on a phpunit.xml.dist), our testsuite execution time dropped from ~8 minutes to ~3-3.5 minutes. However, in our case, we had one last piece of low hanging fruit in that we don’t need to re-generate that XDebug filter every time, as it rarely changes. Without generating that file, our code coverage execution time consistently clocked in at the low 3 minute mark (executing the same tests without coverage takes a bit more than a minute).

Personally, I always like examples, so I’ll wrap this up with a couple basic usages. The first is an example of how we implemented it at work using Docker, Composer, and a bash file to wrap all the commands. The second is what that same execution might look if we weren’t using Docker or Composer and just executed a couple commands directly (without the bash script).

With Docker and Composer

  1. #!/usr/bin/env bash
  2. FILTER=/tmp/phpunit-xdebug-filter.php
  3. docker exec -it some-container bash -c "[ ! -f $FILTER ] && php ./vendor/bin/phpunit --dump-xdebug-filter=$FILTER"
  4. docker exec -it some-container php -d ./vendor/bin/phpunit --prepend=$FILTER --coverage-html=tests/html-coverage

Without Docker and Composer

  1. phpunit --dump-xdebug-filter=/tmp/phpunit-xdebug-filter.php
  2. phpunit --prepend=/tmp/phpunit-xdebug-filter.php --coverage-html=tests/html-coverage

I hope this brief overview of these excellent new PHPUnit and XDebug features are helpful to you, and I’ll see you next time.