Hello, Codeforces.

I have wanted for a long time and now, finally, I sat down and did it.

Many of you are familiar with my Testlib project. This is a library that helps you develop problems if you are using C++. In the case of problems for Codeforces rounds, its use is mandatory. It seems to me that it is almost a de facto standard for preparing materials for completely different olympiads and competitions.

The history of С++-Testlib is almost lost in centuries, I published the first version in 2005.

Nowadays, making edits to Testlib has become tricky. The code grown to thousands of lines, also it has too long a history. This increases the likelihood of accidentally changing some aspect of behavior, which will jeopardize the holding of a Codeforces round or another important competition. In short, making a bug in this code or changing behavior anywhere is highly discouraged.

So I recently took the plunge and wrote some tests for Testlib. Here's what tests can do now:

• check that all cpp files in the repository are compiled (test test-000_compile-all-cpp)
• check the plausibility of the behavior of a simple sval.cpp validator on a series of files (test-001_run-sval test)
• check the plausibility of the behavior of the two checkers fcmp.cpp and wcmp.cpp on a series of files (test test-002_run-fcmp-wcmp)
• check that the behavior of the random number generator has not changed (test test-003_run-rnd)
• check the behavior of some functions and the behavior of InStream and opt in a series of unit tests (test test-004_use-test.h)

More good news. All this was automated through GitHub Actions. Tests are automatically run on any push or pull request. The launch is carried out on:

• 6 different operating systems (ubuntu-18.04, ubuntu-22.04, macos-11, macos-12, windows-server-2019, windows-server-2022),
• different versions of different C++ compilers (g ++, clang ++, msvc),
• different C ++ standards
• and both architectures (32 and 64 bits) are used.

In total tests run in 33 jobs! Look here https://github.com/MikeMirzayanov/testlib/actions/runs/2733832393

Of course, there are still few tests. Something else I will definitely add.

If you understand well how some functionality of Testlib works, then you can add a test for it. First of all, consider adding unit-tests to test-004_use-test.h (carefully study the design of this test).

Otherwise you might find it useful to use test-ref script in tests, which checks that a process has exited with the expected exit code, standard output, and standard error.

It is highly desirable to cover all new functionality with tests. If the functionality is such that it is not very difficult to write tests, then the presence of tests is required.

How to run tests locally?

You need to have installed bash and some tools. After just do cd tests && bash run.sh.

This will run all tests on all supported installed compilers on multiple C++ standards. Probably, just want to run tests on single g++ version using --std=c++11. You can use bash run.sh g++ v0 g++.

More examples:

• bash run.sh — to run all tests on all compilers on multiple standards
• bash run.sh g++ — to run all tests on g++ on multiple standards
• bash run.sh g++ 11 v10 — to run all tests on g++-10 with --std=c++11
• bash run.sh g++ 11 test-001_run-sval — to run test-001_run-sval on g++ with --std=c++11
• bash run.sh test-001_run-sval — to run test-001_run-sval on all compilers on multiple standards
• bash run.sh g++ 11 17 — to run all tests on g++ with --std=c++11 and with --std=c++17

Also you can use t.sh (or t.bat) to run shortcut of run.sh g++ 11 v0 $*. It is good idea to use it for local development to have quick response.