Halide 22.0.0 Halide compiler and libraries

Testing

Halide uses CTest as its primary test platform and runner.

# Organization

Halide's tests are organized beneath the top-level `test/` directory. These folders are described below:

Folder | Description
--- | ---
`autoschedulers/$AS` | Test for the `$AS` (e.g. `adams2019`) autoscheduler
`common` | Code that may be shared across multiple tests
`correctness` | Tests that check correctness of various compiler properties
`error` | Tests that expect an exception to be thrown (or `abort()` to be called)
`failing_with_issue` | Correctness tests that are associated with a particular issue on GitHub
`fuzz` | Fuzz tests. Read more at FuzzTesting.md
`generator` | Tests of Halide's AOT compilation infrastructure.
`integration` | Tests of Halide's CMake package for downstream use, including cross compilation.
`performance` | Tests that check that certain schedules indeed improve performance.
`runtime` | Unit tests for the Halide runtime library
`warning` | Tests that expected warnings are indeed issued.

The tests in each of these directories are given CTest labels corresponding to the directory name. Thus, one can use `ctest -L generator` to run only the `generator` tests. The `performance` tests configure CTest to not run them concurrently with other tests (including each other).

The vast majority of our tests are simple C++ executables that link to Halide, perform some checks, and print the special line `Success!` upon successful completion. There are three main exceptions to this:

First, the `warning` tests are expected to print a line that reads `Warning:` and do not look for `Success!`.

Second, some tests cannot run in all scenarios; for example, a test that measures CUDA performance requires a CUDA-capable GPU. In these cases, tests are expected to print `[SKIP]` and exit and not print `Success!` or `Warning:`.

Finally, the `error` tests are expected to throw an (uncaught) exception that is not a `Halide::InternalError` (i.e. from a failing `internal_assert`). The logic for translating uncaught exceptions into successful tests is in `test/common/expect_abort.cpp`.

# Debugging with LLDB

We provide helpers for pretty-printing Halide's IR types in LLDB. The `.lldbinit` file at the repository root will load automatically if you launch `lldb` from this directory and your `~/.lldbinit` file contains the line,

settings set target.load-cwd-lldbinit true

If you prefer to avoid such global configuration, you can directly load the helpers with the LLDB command,

command script import ./tools/lldbhalide.py

again assuming that the repository root is your current working directory.

To see the benefit of using these helpers, let us debug `correctness_bounds`:

$ lldb ./build/test/correctness/correctness_bounds
(lldb) breakpoint set --file bounds.cpp --line 18
Breakpoint 1: where = correctness_bounds`main + 864 at bounds.cpp:18:12, address = 0x0000000100002054
(lldb) run
Process 29325 launched: '/Users/areinking/dev/Halide/build/test/correctness/correctness_bounds' (arm64)
Defining function...
Process 29325 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x0000000100002054 correctness_bounds`main(argc=1, argv=0x000000016fdff160) at bounds.cpp:18:12
15 g(x, y) = min(x, y);
16 h(x, y) = clamp(x + y, 20, 100);
17
-> 18 Var xo("xo"), yo("yo"), xi("xi"), yi("yi");
19
20 Target target = get_jit_target_from_environment();
21 if (target.has_gpu_feature()) {
Target 0: (correctness_bounds) stopped.
(lldb)

Now we can try to inspect the Func `h`. Without the helpers, we see:

(lldb) v h
(Halide::Func) {
func = {
contents = {
strong = (ptr = 0x0000600002486a20)
weak = nullptr
idx = 0
}
}
pipeline_ = {
contents = (ptr = 0x0000000000000000)
}
}

But if we load the helpers and try again, we get a much more useful output:

(lldb) command script import ./tools/lldbhalide.py
(lldb) v h
... lots of output ...

The amount of output here is maybe a bit too much, but we gain the ability to more narrowly inspect data about the func:

(lldb) v h.func.init_def.values
...
(std::vector<Halide::Expr>) h.func.init_def.values = size=1 {
[0] = max(min(x + y, 100), 20)
}

These helpers are particularly useful when using graphical debuggers, such as the one found in CLion.