linux/tools/testing/selftests/alsa, branch v6.3

kselftest/alsa: Run PCM tests for multiple cards in parallel

2023-02-04T08:36:35Z

With each test taking 4 seconds the runtime of pcm-test can add up. Since generally each card in the system is physically independent and will be unaffected by what's going on with other cards we can mitigate this by testing each card in parallel. Make a list of cards as we enumerate the system and then start a thread for each, then join the threads to ensure they have all finished. The threads each run the same tests we currently run for each PCM on the card before exiting. The list of PCMs is kept global since it helps with global operations like working out our planned number of tests and identifying missing PCMs and it seemed neater to check for PCMs on the right card in the card thread than make every PCM loop iterate over cards as well. We don't run per-PCM tests in parallel since in embedded systems it can be the case that resources are shared between the PCMs and operations on one PCM on a card may constrain what can be done on another PCM on the same card leading to potentially unstable results. We use a mutex to ensure that the reporting of results is serialised and we don't have issues with anything like the current test number, we could do this in the kselftest framework but it seems like this might cause problems for other tests that are doing lower level testing and building in constrained environments such as nolibc so this seems more sensible. Note that the ordering of the tests can't be guaranteed as things stand, this does not seem like a major problem since the numbering of tests often changes as test programs are changed so results parsers are expected to rely on the test name rather than the test numbers. We also now prefix the machine generated test name when printing the description of the test since this is logged before streaming starts. On my two card desktop system this reduces the overall runtime by a third. Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20230203-alsa-pcm-test-card-thread-v1-1-59941640ebba@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - Add more coverage by default

2023-01-02T14:08:24Z

Add more coverage to our standard test cases: - 8kHz mono and stereo to verify that the most common mono format is clocked correctly. - 44.1kHz stereo to verify that this different clock base is generated accurately. - 48kHz 6 channel to verify that 6 channel is clocked correctly. - 96kHz stereo since that is a common audiophile rate. Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-7-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - Provide descriptions for the default tests

2023-01-02T14:08:17Z

Help people understand what the standard tests are trying to cover by providing descriptions which both serve as comments in the file and log messages in the program's output. Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-6-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - Support optional description for tests

2023-01-02T14:08:07Z

In order to help with the comprehensibility of tests it is useful for us to document what the test is attempting to cover. We could just do this through comments in the configuration files but in order to aid people looking at the output of the program in logs let's provide support for an optional 'description' directive which we log prior to running each of the tests. Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-5-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - skip tests when we fail to set params

2023-01-02T14:07:58Z

Since we don't know what the capabilities of an unknown card is any of our standard tests may fail due to not being supported by the system. Set a flag once we've configured the stream, just before we start data, to say that the system accepted our stream configuration. Since there shouldn't be a use case for tests that are specified for the individual system failing for those tests we also add a new test which fails if we are unable to configure the settings specified in the system specific configuration file. Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-4-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - Always run the default set of tests

2023-01-02T14:07:49Z

Rather than allowing the system specific tests to replace the default set of tests instead run them in addition to the standard set of tests. In order to avoid name collisions in the reported tests we add an additional test class element to the reported tests. Doing this means we always get a consistent baseline of coverage no matter what card we run on but retain the ability to specify specific coverage for a given system. Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-3-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - move more configuration to configuration files

2023-01-02T14:07:35Z

Obtain all test parameters from the configuration files. The defaults are defined in the pcm-test.conf file. The test count and parameters may be variable per specific hardware. Also, handle alt_formats field now (with the fixes in the format loop). It replaces the original "automatic" logic which is not so universal. The code may be further extended to skip various tests based on the configuration hints, if the exact PCM hardware parameters are not available for the given hardware. Signed-off-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-2-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: pcm - Drop recent coverage improvement changes

2023-01-02T14:07:03Z

In preparation to adopting a better, more comprehensive approach to adding the coverage that was just added using some changes from Jaroslav which were sent at the same time the recently added improvements were being applied drop what was applied. This reverts: 7d721baea138 "kselftest/alsa: Add more coverage of sample rates and channel counts" ee12040dd53a "kselftest/alsa: Provide more meaningful names for tests" ae95efd9754c "kselftest/alsa: Don't any configuration in the sample config" 8370d9b00c92 Revert "kselftest/alsa: Report failures to set the requested channels as skips" f944f8b539ea "kselftest/alsa: Report failures to set the requested sample rate as skips" 22eeb8f531c1 "kselftest/alsa: Refactor pcm-test to list the tests to run in a struct" Reviewed-by: Jaroslav Kysela Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221208-alsa-pcm-test-hacks-v4-1-5a152e65b1e1@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: Add more coverage of sample rates and channel counts

2022-12-01T19:02:14Z

Now that we can skip unsupported configurations add some more test cases using that, cover 8kHz, 44.1kHz and 96kHz plus 8kHz mono and 48kHz 6 channel. 44.1kHz is a different clock base to the existing 48kHz tests and may therefore show problems with the clock configuration if only 8kHz based rates are really available (or help diagnose if bad clocking is due to only 44.1kHz based rates being supported). 8kHz mono and 48Hz 6 channel are real world formats and should show if clocking does not account for channel count properly. Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221201170745.1111236-7-broonie@kernel.org Signed-off-by: Takashi Iwai

kselftest/alsa: Provide more meaningful names for tests

2022-12-01T19:02:13Z

Rather than just numbering the tests try to provide semi descriptive names for what the tests are trying to cover. This also has the advantage of meaning we can add more tests without having to keep the list of tests ordered by existing number which should make it easier to understand what we're testing and why. Signed-off-by: Mark Brown Link: https://lore.kernel.org/r/20221201170745.1111236-6-broonie@kernel.org Signed-off-by: Takashi Iwai