eku's blog

Program to test solutions to interactive problems
By eku, history, 8 years ago, In English

It's difficult to test solutions to interactive problems during or after a contest. For non-interactive problems, I usually redirect input from a file. For interactive problems I have to type out input manually, since input depends on output. If the interaction is long, this can take time.

A more automated way will require 2 more programs:

  1. A judge program with whom our solution will interact.
  2. A program which connects stdout of solution to stdin of judge and stdout of judge to stdin of solution. I call this program a 'croupier'.

The judge program will have to be written by the user each time since it's problem-specific. But the croupier is generic and can be reused.

My implementation of the croupier, apart from connecting inputs and outputs of the two programs, also prints the output from each program so that it's easier to debug.

You can find my implementation here: https://github.com/sharmaeklavya2/croupier

I wrote the croupier so that it could help me debug 750F - New Year and Finding Roots. The croupier helped me find out many mistakes in my solution, but it seems like there are more, because I can't get my solution accepted. Here is the judge program I used for it: https://gist.github.com/sharmaeklavya2/fb9751ba0d9d5df883d4d29288db8315

Update: There was a small bug in croupier because of which it sometimes printed output incorrectly. Thanks you awoo for reporting this. awoo also tested this program on Windows, which I couldn't do myself because I don't have Windows. You can find a detailed description of the bug here: https://github.com/sharmaeklavya2/croupier/commit/a2fcd82a52a83df71518dfec248271d752a55b66

  • Vote: I like it
  • +51
  • Vote: I do not like it

»
8 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Auto comment: topic has been updated by eku (previous revision, new revision, compare).

→ Reply
»
8 years ago, # |
  Vote: I like it +78 Vote: I do not like it

If you are running Linux, you may "implement" a croupier in just 2 lines of Shell:

mkfifo fifo
./solution < fifo | ./interactor > fifo
→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I had tried that, but that didn't show outputs from each program, which I need for debugging.

    One can work around that by printing to both stdout and stderr, though.

    I implemented croupier in python as an exercise, but the implementation got more complicated than I expected.

    → Reply
    • »
      »
      »
      6 months ago, # ^ |
        Vote: I like it 0 Vote: I do not like it 
      ./solution < fifo | tee >(sed 's/^/>> /' > out.txt) | ./interactor | tee >(sed 's/^/<< /' > in.txt) > fifo
(printf '\n'; cat out.txt) | paste - <(cat in.txt)
      → Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    can you post an example of interactor.cpp? thanks a lot

    → Reply
»
8 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It's important that your interactor can handle Idle Limit Exceededs. This happens when the participant read something when he should actually output something. If you can't deal with them, both interactor and tested program will hang forever. So a timer such as setitimer is necessary.

→ Reply
»
8 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Auto comment: topic has been updated by eku (previous revision, new revision, compare).

→ Reply
»
8 years ago, # |
  Vote: I like it +10 Vote: I do not like it

I agree that testing interactive problems is harder, but usually something like that will do. Instead of 

cin>>x;

do

int getInput() {
#ifdef LOCAL
return generateLocalInputOrAnythingYouWant();
#else
int x;
cin>>x;
return x;
#endif
}

x = getInput();
→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Can you make a croupier like this for c++/c too? Or at least tell me how to do that?

→ Reply