Pytest Integration

Use the pytest helpers to run one test body across registered implementations. This guide builds on Quickstart and assumes you know basic pytest tests and parametrization. Install the tools used in the synchronous examples with:

$ pip install alternative pytest pytest-benchmark

The code blocks below can be combined in a single test file. Register every candidate before applying a helper to a test function.

Equivalence Tests

Use alternative.Alternatives.pytest_parametrize_pairs() when each candidate implementation should match the reference:

from collections.abc import Iterable

import alternative
import pytest


@alternative.reference
def total_loop(values: Iterable[int]) -> int:
    """Sum integers with a straightforward reference loop."""
    result = 0
    for value in values:
        result += value
    return result


@total_loop.add(default=True)
def total_sum(values: Iterable[int]) -> int:
    """Apply the same contract with the built-in sum."""
    return sum(values)


@total_loop.pytest_parametrize_pairs()
def test_totals_are_equivalent(reference, implementation):
    """Every implementation returns the same total as the reference."""
    values = [1, 2, 3]
    assert implementation(values) == reference(values)

The helper supplies the callable arguments named reference and implementation; do not define fixtures for them. reference calls the reference implementation, and implementation takes each candidate in turn. Reference caching is disabled by default.

The reference is not paired with itself unless you set double_reference=True to check the test’s own assumptions. With no candidates, ordinary pairwise parametrization has no cases. Keep independent expected-result tests for the reference as well as equivalence tests.

Check Expected Results

Use alternative.Alternatives.pytest_parametrize() to run against every implementation, including the reference:

@total_loop.pytest_parametrize()
@pytest.mark.parametrize(("values", "expected"), [((), 0), ((1, -2, 3), 2)])
def test_expected_totals(implementation, values, expected):
    """Check all implementations against independent expected results."""
    assert implementation(values) == expected

This checks the reference itself, which pairwise agreement alone cannot do.

Benchmark All Implementations

Use alternative.Alternatives.pytest_parametrize() with pytest-benchmark to benchmark every implementation:

@total_loop.pytest_parametrize()
def test_total_benchmark(benchmark, implementation):
    """Benchmark each registered implementation."""
    assert benchmark(implementation, [1, 2, 3]) == 6

Pytest generates readable parameter names from the underlying function names.

Only the Default Implementation

Set only_default=True to limit parametrization to the reference and selected default implementation:

@total_loop.pytest_parametrize(only_default=True)
def test_selected_implementation_accepts_input(implementation):
    """The reference and default implementation both accept list input."""
    assert implementation([1, 2, 3]) == 6

This is useful for expensive test suites where exhaustive coverage happens in a smaller equivalence test.

If the reference is also the selected default, it appears only once, whether selected implicitly or with @alternative.reference(default=True). Pairwise only_default=True similarly compares the reference with itself once; double_reference=True does not duplicate that pair.

Testing Methods

Method alternatives can be tested with the same helpers by passing an explicit instance to the parametrized implementation:

class Counter:
    def __init__(self, value: int):
        """Store the integer used by the method examples."""
        self.value = value

    @alternative.reference
    def total(self) -> int:
        """Return the integer through a text round trip."""
        return int(str(self.value))

    @total.add(default=True)
    def total_direct(self) -> int:
        """Return the stored integer directly."""
        return self.value


@Counter.total.pytest_parametrize_pairs()
def test_method_totals_are_equivalent(reference, implementation):
    """Every method implementation returns the same total."""
    counter = Counter(3)
    assert implementation(counter) == reference(counter)

The pytest helpers parametrize callables. They do not change the selected default implementation for each test parameter; the library still keeps the active implementation stable once it has been used.

For class methods, access the helper through the class. The parameters bind to that class, including when accessed through a subclass:

class Factory:
    marker = "base"

    @alternative.reference
    @classmethod
    def build(cls, value: str) -> str:
        """Prefix the text with the accessed class marker."""
        return cls.marker + ":" + value

    @build.add
    @classmethod
    def build_extra(cls, value: str) -> str:
        """Format the same marker and text with an f-string."""
        return f"{cls.marker}:{value}"


@Factory.build.pytest_parametrize_pairs()
def test_factories_are_equivalent(reference, implementation):
    """Class method parameters already have their class receiver."""
    assert reference("a") == implementation("a") == "base:a"

Helpers accessed through an instance also bind instance methods to that instance. When a helper is used on a raw descriptor inside a class body, pass the receiver explicitly, as for the unbound instance-method example above.

Async Tests

The helpers preserve async def tests for async pytest plugins. For pytest-asyncio, install the plugin and mark each async test:

$ pip install pytest-asyncio
@alternative.reference
async def normalise_async(text: str) -> str:
    """Return normalized text from an async callable."""
    return text.strip().lower()


@normalise_async.add
async def normalise_async_lower(text: str) -> str:
    """Apply the same text contract with a direct str call."""
    return str.lower(text.strip())


@pytest.mark.asyncio
@normalise_async.pytest_parametrize_pairs()
async def test_async_names(reference, implementation):
    """Await both implementations before comparing their results."""
    assert await reference(" Ada ") == await implementation(" Ada ") == "ada"

@pytest.mark.asyncio can appear in either order relative to either helper. Leave reference caching disabled. Ordinary runtime calls also return awaitables for asynchronous implementations; the library does not run an event loop on the caller’s behalf.

Reference Caching

Use n_cache to reuse reference results across pairwise comparisons when the reference is expensive and its results are safe to share:

@total_loop.pytest_parametrize_pairs(n_cache=None)
def test_with_unbounded_reference_cache(reference, implementation):
    """The reference result may be reused across implementations."""
    assert implementation((1, 2, 3)) == reference((1, 2, 3))

The default, n_cache=0, disables caching. Use a positive integer for a bounded cache, or None for an unbounded cache, when the reference is expensive and arguments are hashable.

Caching coroutine, generator, or async-generator reference functions raises ValueError during test decoration. Keep n_cache=0 for these functions so each call creates a fresh awaitable or iterator. Callable instances with an asynchronous or generator __call__ follow the same rule. For synchronous factories, only enable caching when their results can safely be reused.

The cache belongs to the decorated test and can reuse results across input cases and candidate parameters. Arguments must be hashable, and the reference must be safe to call fewer times. Cached mutable results are shared objects: changing one can contaminate later comparisons. Use n_cache=0 for mutation, state, randomness, or single-use input. See Testing Observable Behavior for examples.

Collection Order

Applying a helper to a test function snapshots the implementation list and freezes registration. This normally happens while pytest imports the test module, before test execution. Import all candidates before these decorators run. Reading .implementations or .callable, calling the alternatives object, or calling .measure() also freezes registration. Further additions raise alternative.AddTooLateError.

That rule is deliberate. It prevents a test module from collecting a partial implementation list and then silently missing an implementation imported later.