Skip to main content
Version: v2

Testing

This page helps developers on a project built from Vortex pick the right test framework, run it, and find its results. Visual regression testing has a page of its own; see Visual regression.

Vortex ships 3 test frameworks, configured and wired into the continuous integration pipeline, so the first test you add runs in CI with no extra setup. Each covers a different layer of the site:

FrameworkCoversCommand
PHPUnitYour custom PHP code, through unit, kernel, functional, and functional JavaScript testsahoy test
BehatBehavior-Driven Development (BDD) scenarios, run end to end against the site, in a real browser when a scenario needs JavaScriptahoy test-bdd
JestThe JavaScript behaviors in your custom modules, tested in isolationahoy test-js

➡️ See PHPUnit > Writing tests, Behat > Writing tests, and Jest > Test file structure for where a new test goes.

Running​

Run the tests with the following Ahoy commands:

ahoy test                        # Run all PHPUnit test suites.
ahoy test-unit                   # Run PHPUnit unit tests.
ahoy test-kernel                 # Run PHPUnit kernel tests.
ahoy test-functional             # Run PHPUnit functional tests.
ahoy test-functional-javascript  # Run PHPUnit functional JavaScript tests.
ahoy test-bdd                    # Run Behat tests.
ahoy test-js                     # Run Jest tests.

Each framework page lists the Docker Compose equivalents and the options for running a single file, a tag, or a filter.

Run unit, kernel, and functional tests​

ahoy test runs all the PHPUnit test suites: unit, kernel, functional, and functional JavaScript.

Run BDD tests​

ahoy test-bdd runs the Behat tests against the provisioned site. Scenarios tagged @javascript drive Chromium in the browser container, and the rest run through an HTTP client.

Reports and coverage​

Each framework writes its reports under .logs/, and the continuous integration pipeline stores them as artifacts:

  • PHPUnit writes its JUnit report to .logs/test_results/phpunit/phpunit.xml and its coverage reports to .logs/coverage/phpunit/.
  • Behat writes its JUnit reports to .logs/test_results/behat/.
  • Jest writes its coverage reports to .logs/coverage/jest/.

Continuous integration​

Tests run in the test job of the continuous integration pipeline, which runs on 2 parallel containers by default. PHPUnit and Jest run on the first container, and Behat spreads its scenarios across every container (see Test parallelism).

Each framework has a variable that lets the build pass despite failing tests. Set it to 1, and the tool still runs and reports every failure in the job log:

  • VORTEX_CI_PHPUNIT_IGNORE_FAILURE
  • VORTEX_CI_BEHAT_IGNORE_FAILURE
  • VORTEX_CI_JEST_IGNORE_FAILURE

The PHPUnit coverage threshold check has no such variable, so it can still fail the build.

➡️ See Ignore tool failures.

Project conventions​

Your project's own test writing conventions live in its docs/testing.md file, which your project team keeps up to date. It covers the following topics:

  • PHPUnit: test class structure, base test classes, test data, data providers, and kernel tests for services
  • Behat: user story format, standard user types, test data, an example feature file, and the process for testing a content type

Vortex ships the file in the project's docs/ directory, and the Vortex CLI keeps it unless you decline its Preserve project documentation? prompt. Declining it removes the whole docs/ directory.