When tests are all in the same suite it is annoying to have to read all
the common text after each name. Skip this to help the user.
Signed-off-by: Simon Glass <sjg@chromium.org>
Show the average duration of a test, so we can keep track of how it is
trending. Report the suite with the longest average test to encourage
people to improve it.
Add a function to update the stats based on the results from a single
suite and another to show the summary information.
Make this optional, since sandbox's SPL tests do not have a timer driver
and people may want to print results without times.
Signed-off-by: Simon Glass <sjg@chromium.org>
All tests should belong to a suite, but if there is a suite we don't
know about (e.g. not added to cmd_ut.c) then the totals will not add up.
Add a check for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than having the help in the longhelp, put it in the suite info
so 'ut info -s' can show it. This is tidier, particular due to the
removal of #ifdefs
This means that the help text is present in the image (although not
displayed with 'ut info -s') so the image-size increases. But with
UNIT_TEST enabled, we expect large images so this doesn't seem
important.
Signed-off-by: Simon Glass <sjg@chromium.org>
Put the suites in order by name, for easier code-maintenance. This also
helps find test results for a particular swuit in the 'ut all' output.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update this function to access a unit-test state, so that the caller can
collect results from running multiple suites.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is not needed anymore. If a test suite is not built, then it will
have no linker-list entries. So we can just check for that and know that
the suite is not present.
This allows removal of the #ifdefs and the need to keep them in sync
with the associated Makefile rules, which has actually failed, since the
help does not match what commands are actually present.
Signed-off-by: Simon Glass <sjg@chromium.org>
The 'info' test is not a real test. With the new suite array we can drop
this and the associated special-case code.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new suite-runner to run these tests instead.
It is not clear that these actually work, since they are not enabled on
sandbox for some reason.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new suite-runner to run these tests instead.
It is not clear that these actually work, since they are not enabled on
sandbox for some reason.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Linus Walleij <linus.walleij@linaro.org>
The current cmd_ut_sub[] array was fine when there were only a few test
suites. But is quite unwieldy now:
- it requires a separate do_ut_xxx for each suite, even though the code
for most is almost identical
- running more than one suite requires running multiple commands, and
there is no record of which suites passed or failed
- 'ut all' runs all suites but reports their results individually
- we need lots of #ifdefs in the array, mirroring those in the makefile
but maintained in a separate place
In fact the tests are all in the same linker list. The suites are
grouped, so it is possible to access the information without a command.
Introduce a 'suite' array, which holds the cmd_ut_...() function to
call, but can also support running a suite without that function. This
means that the array of struct cmd_tbl is transformed into an array of
'struct suite'.
This will allow removal of many of the functions, particularly those
without test-specific init.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Python test which runs 'ut all' and then checks that the expected
suites are present and all tests in each suite are run.
This can help to check that nothing is missing.
Update 'ut info' to ignore the 'all' suite when counting the number of
suites, since that is really just a combination of all the other suites.
Adjust the message for skipped tests so that appears even if no
particular test was selected. This helps the new 'test_suite' test see
what is going on.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is helpful to see a list of available suites. At present this is
handled by the longhelp for the 'ut' command, but this is not in a
format which can be easily parsed by python tests.
Add a -s option to show this. At present it is not possible to show the
number of tests in each suite, but future work will address this. For
now, show a ?
Signed-off-by: Simon Glass <sjg@chromium.org>
Having CONFIG_OF_EMBED=y && CONFIG_BLOBLIST=n leads to the link
error:
```
ld: /tmp/ccwtRVty.ltrans28.ltrans.o:(.data.rel+0x4b0): undefined \
reference to `do_ut_bloblist'
```
Fixes: 6ea5df39e8 ("test: Only enable bloblist test when supported")
Signed-off-by: Evgeny Bachinin <EABachinin@salutedevices.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
There is no particular need for the time tests to have their own test
command. Move them into the lib suite instead.
Update the test functions to match the normal unit-test signature.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
There is no particular need for the unicode tests to have their own test
suite. Move them into the lib suite instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
There is no particular need for the str tests to have their own test
suite. Move them into the lib suite instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
There is no particular need for bloblist to have its own test suite.
Move it into the common suite instead.
Add the missing help for 'common'.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
There is no particular need for compression to have its own test suite.
Move it into the lib suite instead.
Add the missing help for 'common' and update the docs.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com> # rpi_3, rpi_4, rpi_arm64, am64x_evm_a53, am64-sk
Complete this rename for all directories outside arch/ board/ drivers/
and include/
Use the new symbol to refer to any 'SPL' build, including TPL and VPL
Signed-off-by: Simon Glass <sjg@chromium.org>
As part of bringing the master branch back in to next, we need to allow
for all of these changes to exist here.
Reported-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Tom Rini <trini@konsulko.com>
When bringing in the series 'arm: dts: am62-beagleplay: Fix Beagleplay
Ethernet"' I failed to notice that b4 noticed it was based on next and
so took that as the base commit and merged that part of next to master.
This reverts commit c8ffd1356d, reversing
changes made to 2ee6f3a5f7.
Reported-by: Jonas Karlman <jonas@kwiboo.se>
Signed-off-by: Tom Rini <trini@konsulko.com>
Introduce a new subcommand to ut: ut hush.
For the moment, this command does nothing, future commits will add tests which
will be run on command call.
Note that CONFIG_HUSH_PARSER must be defined to compile this new subcommand.
Signed-off-by: Francis Laniel <francis.laniel@amarulasolutions.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
These make use of disk images which are not available on reak boards.
Add a new Kconfig to ensure these tests only run where they are valid.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new suite for 'cmd' tests, used for testing commands. These are
kept in the test/cmd directory.
For now it is empty, but it will be used for coreboot-command tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
