Unit Tests

BBOT takes tests seriously. Every module must have a custom-written test that actually tests its functionality. Don't worry if you want to contribute but you aren't used to writing tests. If you open a draft PR, we will help write them :)

We use ruff for linting, and pytest for tests.

Running tests locally

We have GitHub Actions that automatically run tests whenever you open a Pull Request. However, you can also run the tests locally with pytest:

# lint with ruff
uv run ruff check

# format code with ruff
uv run ruff format

# run all tests with pytest (takes roughly 30 minutes)
uv run pytest

Running specific tests

If you only want to run a single test, you can select it with -k:

# run only the sslcert test
uv run pytest -k test_module_sslcert

You can also filter like this:

# run all the module tests except for sslcert
uv run pytest -k "test_module_ and not test_module_sslcert"

If you want to see the output of your module, you can enable --log-cli-level:

uv run pytest --log-cli-level=DEBUG

Example: Writing a Module Test

To write a test for your module, create a new python file in bbot/test/test_step_2/module_tests. Your filename must be test_module_<module_name>:

test_module_mymodule.py

from .base import ModuleTestBase


class TestMyModule(ModuleTestBase):
    targets = ["blacklanternsecurity.com"]
    config_overrides = {"modules": {"mymodule": {"api_key": "deadbeef"}}}

    async def setup_after_prep(self, module_test):
        # mock HTTP response
        module_test.blasthttp_mock.add_response(
            url="https://api.com/subdomains?apikey=deadbeef&domain=blacklanternsecurity.com",
            json={
                "subdomains": ["www.blacklanternsecurity.com", "dev.blacklanternsecurity.com"],
            },
        )
        # mock DNS
        await module_test.mock_dns(
            {
                "blacklanternsecurity.com": {"A": ["1.2.3.4"]},
                "www.blacklanternsecurity.com": {"A": ["1.2.3.4"]},
                "dev.blacklanternsecurity.com": {"A": ["1.2.3.4"]},
            }
        )

    def check(self, module_test, events):
        # here is where we check to make sure it worked
        dns_names = [e.data for e in events if e.type == "DNS_NAME"]
        # temporary log messages for debugging
        for e in dns_names:
            self.log.critical(e)
        assert "www.blacklanternsecurity.com" in dns_names, "failed to find subdomain #1"
        assert "dev.blacklanternsecurity.com" in dns_names, "failed to find subdomain #2"

Mocking HTTP responses

module_test.blasthttp_mock intercepts all outbound HTTP requests during tests. Requests to 127.0.0.1/localhost pass through to the real test HTTP server, but everything else is mocked.

    async def setup_after_prep(self, module_test):
        # JSON response
        module_test.blasthttp_mock.add_response(
            url="https://api.example.com/lookup?domain=blacklanternsecurity.com",
            json={"subdomains": ["www.blacklanternsecurity.com"]},
        )

        # plain text response
        module_test.blasthttp_mock.add_response(
            url="https://example.com/data.txt",
            text="some plain text response",
        )

        # error response
        module_test.blasthttp_mock.add_response(
            url="https://example.com/broken",
            status_code=500,
            text="Internal Server Error",
        )

        # match on specific method and headers
        module_test.blasthttp_mock.add_response(
            url="https://api.example.com/submit",
            method="POST",
            match_headers={"Authorization": "Bearer mytoken"},
            json={"status": "ok"},
        )

Mocking DNS

Use module_test.mock_dns() to control DNS resolution. Supports A, AAAA, CNAME, MX, TXT, and other record types:

async def setup_after_prep(self, module_test):
    await module_test.mock_dns(
        {
            "blacklanternsecurity.com": {"A": ["127.0.0.88"]},
            "www.blacklanternsecurity.com": {"CNAME": ["blacklanternsecurity.com"]},
            "mail.blacklanternsecurity.com": {"MX": ["mx.example.com"]},
        }
    )

Debugging a test

You can debug from within a test using standard Python logging via self.log:

def check(self, module_test, events):
    for e in events:
        self.log.critical(e.type)  # bright red
        self.log.warning(e.tags)  # orange
        self.log.info(e.data)  # blue
        self.log.debug(e.parent)  # grey (requires -d)

Test servers and ports

ModuleTestBase starts a local HTTP server for every test, and the test registers canned responses on it with module_test.set_expect_requests(). A web module's test points targets at that server, which is why so many tests contain a 127.0.0.1 URL: it's the fixture, not a real host.

Its port is no longer fixed. The suite runs under pytest -n, and bbot/test/worker.py offsets every base port by the worker index so the workers don't collide on one socket: the server is on 8888 under gw0, 8988 under gw1. So targets = ["http://127.0.0.1:8888"] names gw0's server rather than your own, and the test reaches another worker's server or nothing at all, depending on how pytest distributed that run.

Use HTTPSERVER_URL. It is http://127.0.0.1:<this worker's port>, built at import time from the same offset the fixture uses, so it always resolves to the server the test is actually talking to:

from bbot.test.worker import HTTPSERVER_URL


class TestMyModule(ModuleTestBase):
    targets = [HTTPSERVER_URL]

/tmp/.bbot_test has the same problem, since the first worker to finish deletes it. BBOT_TEST_DIR is the per-worker equivalent.

Constant	What it is
`HTTPSERVER_URL`	`http://127.0.0.1:<port>`, the usual value for `targets`
`HTTPSERVER_SSL_URL`	the HTTPS equivalent
`HTTPSERVER_PORT`, `HTTPSERVER_SSL_PORT`	the bare ports, for building a URL or a regex
`HTTPSERVER_HOSTPORT`, `HTTPSERVER_SSL_HOSTPORT`	`127.0.0.1:<port>`, no scheme
`LOCALHOST_URL`, `LOCALHOST_SSL_URL`, `LOCALHOST_HOSTPORT`	the `localhost` spellings, for when the Host header is what's under test
`HTTPSERVER_PORT_ALT`	a second port inside the same worker's block
`HTTPSERVER_ALLINTERFACES_PORT`	the `0.0.0.0` listener
`FASTAPI_URL`, `FASTAPI_PORT`	the FastAPI test app
`WEBSOCKET_PORT`	the websocket test server
`BBOT_TEST_DIR`, `BBOT_TEST_DIR_NAME`	the per-worker BBOT home, and its basename

Advanced test features

HTTP request handlers

For dynamic HTTP responses, use set_expect_requests_handler with a custom handler function:

import re
from werkzeug.wrappers import Response
from bbot.test.worker import HTTPSERVER_URL


class TestMyModule(ModuleTestBase):
    targets = [HTTPSERVER_URL]
    modules_overrides = ["http", "mymodule"]

    def request_handler(self, request):
        if request.path == "/api/data":
            return Response('{"results": ["found"]}', status=200)
        return Response("Not Found", status=404)

    async def setup_before_prep(self, module_test):
        module_test.set_expect_requests_handler(
            expect_args=re.compile("/.*"),
            request_handler=self.request_handler,
        )

OOB interaction mocking

For modules that use out-of-band interactions (interactsh):

async def setup_before_prep(self, module_test):
    self.interactsh_mock_instance = module_test.mock_interactsh("mymodule")

    from bbot.core.helpers.helper import ConfigAwareHelper

    module_test.monkeypatch.setattr(
        ConfigAwareHelper,
        "interactsh",
        lambda *a, **kw: self.interactsh_mock_instance,
    )

Other class attributes

class TestMyModule(ModuleTestBase):
    targets = [HTTPSERVER_URL]  # override default target
    modules_overrides = ["http", "mymodule"]  # control which modules are enabled
    module_name = "mymodule"  # if your class name doesn't match the module
    config_overrides = {"modules": {"mymodule": {"option": "value"}}}
    blacklist = ["bad.example.com"]  # set a blacklist
    seeds = ["seed.example.com"]  # set seeds separate from targets
    skip_distro_tests = True  # skip when running in CI distro tests (e.g. docker-dependent tests)

More examples

If you have questions about tests or need to write a more advanced test, come talk to us on GitHub or Discord.

It's also a good idea to look through our existing tests. BBOT has over 300 of them, so you might find one that's similar to what you're trying to do.