Ansible Role tests/ Directory: Legacy Travis CI Artifact vs Modern Testing

ansibletestingmoleculetravis-ciroles

Bottom line

The tests/ directory inside an Ansible role is a legacy artifact from 2015 created for Travis CI integration with Ansible Galaxy 2.0. It's not mentioned in the official Ansible role documentation, is not consumed by any modern testing tool by default. Is widely treated as optional boilerplate that can be deleted. The modern standard for testing Ansible roles is Molecule (for integration testing with provisioned infrastructure) and ansible-test (for collection sanity/unit tests). If you aren't using Travis CI or a custom runner that explicitly looks at tests/test.yml, the directory serves no purpose.

Key findings

  • Finding: The tests/ directory was introduced in December 2015 via Ansible PR #13489 ("Galaxy 2.0") specifically to support Travis CI. The generated tests/test.yml was designed to run ansible-playbook --syntax-check inside Travis builds. No other tool uses it by default.
  • Finding: The official Ansible documentation for role directory structure lists tasks/, handlers/, templates/, files/, vars/, defaults/, meta/, library/, module_utils/, and lookup_plugins/ - but doesn't list tests/. This is source-stated omission.
  • Finding: ansible-galaxy role init still scaffolds tests/test.yml and tests/inventory in 2025, but multiple independent guides (PyNet Labs, Spacelift, ComputingForGeeks) explicitly advise deleting it if you aren't using it.
  • Finding: Molecule is the community-accepted standard for role integration testing. It uses its own molecule/ directory, not tests/. Molecule supports Docker, Podman, VMs, and cloud instances, and can verify idempotence, run side-effect tests, and assert final state with verify.yml.

Background

Ansible roles follow a conventional directory structure to organize reusable automation. In 2015, Ansible Galaxy 2.0 added ansible-galaxy role init scaffolding that included a tests/ folder containing:

  • tests/test.yml - a minimal playbook applying the role to localhost
  • tests/inventory - an inventory file containing just localhost

This was bundled with a .travis.yml template that ran:

ansible-playbook tests/test.yml -i tests/inventory --syntax-check

The intent was to give role authors on Galaxy a free CI smoke-test. Travis CI would import the GitHub repo, find .travis.yml, and run the syntax check.

Current state

As of 2025–2026:

  • ansible-galaxy role init still creates tests/, but the directory is orphaned. Travis CI's free tier for open source is largely discontinued, and no other tool automatically consumes tests/.
  • The official Ansible "Role directory structure" docs don't describe tests/ as a standard role directory.
  • Molecule (maintained under the Ansible project umbrella) is the active testing framework. Molecule scenarios live under molecule/<scenario_name>/ and include converge.yml, verify.yml, molecule.yml, etc.
  • For collections, ansible-test is the primary tool for sanity, unit, and integration tests. It uses tests/integration/targets/ inside collections, not role-level tests/.

Technical or implementation details

Aspect tests/ directory Molecule
Location roles/<name>/tests/ roles/<name>/molecule/
Default contents test.yml, inventory molecule.yml, converge.yml, verify.yml, prepare.yml
What it does Runs a playbook against localhost Provisions infra, applies role, checks idempotence, verifies state, destroys infra
Drivers None (manual/ansible-playbook only) Docker, Podman, Delegated, Vagrant, cloud
Tooling support Travis CI (legacy) molecule test, molecule verify, CI matrixes
Assertion support None built-in verify.yml with assert, ansible.builtin.uri, etc.

The typical tests/test.yml is:

- name: Test role
  hosts: localhost
  remote_user: root
  roles:
    - myrole

This only proves the role can be parsed and applied to the local machine as root. It doesn't verify system state, test across OS versions, or enforce idempotence.

Evidence, comparisons, and related context

  • Molecule vs tests/: Molecule was designed to replace ad-hoc local testing. Red Hat's official blog (2020) and ComputingForGeeks (2026) both present Molecule as the standard. The Ansible forum discussion (May 2024–Jan 2026) shows active community consensus that Molecule is the right tool for role integration tests. ansible-test is preferred for collection sanity/unit tests.
  • Collections vs standalone roles: In collections, integration tests use tests/integration/targets/<target>/ with ansible-test. This is unrelated to the role tests/ directory. Ansible core dev mattclay noted in 2019 that collections were migrating to use tests/ (plural) at the collection root to match roles/ and plugins/ - but this refers to collection-level testing, not role-level tests/.
  • Galaxy 2.0 origin: Stack Overflow and GitHub PR #13489 confirm the tests/ directory was added for Galaxy 2.0 + Travis CI. The .travis.yml template was part of the same PR.

Limitations and critiques

  • The tests/ directory is not integrated with ansible-lint, ansible-test, or Molecule. It's invisible to modern CI unless you manually wire it.
  • tests/test.yml hardcodes remote_user: root and hosts: localhost, making it unsuitable for most real-world testing scenarios (e.G., SSH-based cloud VMs, unprivileged users).
  • There is no official documentation explaining what the tests/ directory is for inside a role. The Ansible forum thread from 2024 explicitly asks this question and receives no authoritative answer beyond "it's for Travis CI."
  • Keeping tests/ in a role can confuse newcomers who expect it to be automatically executed or validated by modern tooling.

Open questions

  • Will a future version of ansible-galaxy role init remove the tests/ directory or replace it with a Molecule scaffold?
  • Is there any active effort to make ansible-test consume role-level tests/ for integration testing, or has that path been permanently ceded to Molecule?

Practical takeaways

  • If you aren't using Travis CI: Delete the tests/ directory. It provides no value and isn't used by Molecule or ansible-test.
  • For proper role testing: Use Molecule. Initialize scenarios with molecule init scenario, write verify.yml to assert system state, and run molecule test in CI (GitHub Actions, GitLab CI, etc.).
  • For collections: Use ansible-test for sanity/unit tests and Molecule for integration tests that need multi-OS container/VM coverage.

Sources used