Improve the test UX

[0xTim]

The test UI is really great apart from one minor annoyance. When I rerun a test, it would be great to show a different icon other than the last state (like a spinning icon) whilst the test suite is compiling and running (before the test itself has run).

This makes it clearer the current state of the tests. Currently, for example, if I run a failing test which compiles and takes a while etc, when I glance at the test in the file, the explorer of the results panel, it shows as still failing.

Definitely a minor issue but a nice QoL improvement!

[award999]

@plemarquand do the vscode APIs provide a means to improve this?

[plemarquand]

I agree this is the ideal workflow. What we'd like to do is mark a test run as "started" immediately, before compilation begins, which would show a spinner in the test explorer indicating that progress is underway.

This was my original plan when I started working on integrating swift-testing with the test explorer, but I quickly ran in to a combination of limitations in swift-testing and VS Code that mean't it wasn't possible.

Ultimately it boils down to swift-testing's parameterized test feature and how it interacts with VS Codes testing API. With parameterized tests, we can't know what test cases are going to run until compilation is finished and execution begins. Immediately at launch swift-testing evaluates the arguments to each test and emits a list of test cases its going to run to the JSON event stream. The extension then captures these test cases and injects them in to the Test Explorer.

Here is the rub: with VS Code testing APIs as they stand today test cases cannot be added to a test run once that test run has started. This means we capture these test cases, put them in the test explorer, and only then can we start a test run that includes the suites/tests/and test cases we just generated.

In an ideal world VS Code would accept modifications to the list of tests once a run has started, but today (or at least at the time this feature was written) that list must be fixed at test run start.

[0xTim]

It looks like this works some of the time? Sometimes the tests go yellow indicating they're yet to be run. I'm guessing this is if SwiftPM decides it needs to do a build or now, but I'm pretty sure I've seen this when code has been added