-is-this-fft-'s blog

By -is-this-fft-, history, 12 months ago, In English

Introduction

Run-twice problems (also called multipass or communication problems) are problems in which your program is executed twice. Here are some examples:

As you can see, there are many ways this format can be used, but the common themes are encoding/decoding, compression/decompression, constructing a bijection and similar ideas. The general principle is that you have to implement two functions and the second function has to somehow use the value returned by the first function. Run-twice is useful if the problem would be ruined if the second function had access to the input data of the first function.

It is possible to prepare run-twice problems on Polygon and to host them on Codeforces. I think there is a lot of unexplored design space for problems here. This is why I am showing how to prepare them here — to encourage problemsetters to set more run-twice problems. $$$~$$$

This tutorial is completely unofficial: everything I've written here is either figured out by experimentation or gathered from little bits I've found in blogs here. Also, I think it's fair to assume that unless Mike or someone else from Codeforces says so, all APIs mentioned here are "unstable" — they may be changed at any time and future versions may not be backwards compatible.

Even and Odd Combinations

Let's start by preparing problem B from Universal Cup Season 2 Stage 16, one of the easier problems from the set. I've slightly simplified the problem here (by removing multi-test) for ease of explanation.

Problem. Let $$$A_n$$$ be the set of even-length subsequences of the array $$$[1, 2, \ldots, n]$$$ and let $$$B_n$$$ be the set of odd-length subsequences of the same array. Construct a bijection between these two sets. To prove that you have constructed a bijection, your program will be run twice.

  • First run. The first line of the input consists of two integers $$$n$$$ and $$$k$$$ ($$$1 \le n \le 50$$$, $$$0 \le k \le n$$$). The second line of the input consists of $$$k$$$ integers $$$a_1, a_2, \ldots, a_k$$$ in ascending order. Print two lines of output; the first line must consist of two integers $$$n$$$ and $$$\ell$$$ ($$$0 \le \ell \le n$$$); the second line must consist of $$$\ell$$$ integers $$$b_1, b_2, \ldots, b_\ell$$$ in ascending order. $$$\ell$$$ must have different parity from $$$k$$$.
  • Second run. The input consists of the output from the first run. The output must match the input the first run.
Solution

Hopefully it's clear why run-twice is necessary here. If the problem simply asked to implement two functions, the participant could cheat by simply storing the input.

To start preparing a run-twice problem on Polygon, go to General Info -> Advanced and select "Run solutions twice". Now the workflow for running on a test case looks like this:

Blue arrows denote data controlled by you, the problemsetter — test inputs and data printed by your programs. Orange arrows denote data controlled by the participant. You cannot assume anything about them as the participant may choose to output anything they wish.

This setup is inadequate. The output of the first run is forwarded straight to the input of the second run. While this is what we eventually want, it does not work properly right now: we have no way of checking that $$$k$$$ has a different parity from $$$\ell$$$, or even that the output from the first run is in the required format.

The solution is to make the problem interactive. This is now the workflow:

  • The input is available to the interactor via inf.
  • The interactor can write data to the first run via cout, the submission can read this data from standard input.
  • The interactor can read data that the first run writes to standard input from the stream ouf.
  • If the interactor quits with a verdict other than _ok, this will be the final verdict of the solution.
  • Otherwise, data that the interactor writes to tout will be the input to the second run.
  • The checker can read the output of the second run from ouf, the original input from inf, and the output of the second run of the model solution from ans.

Making the problem interactive sounds a bit odd considering that the problem is not interactive from the participant's point of view at all. This is actually pretty common if you go in the unusual problem territory (though part of this is due to the limitations of Polygon's model). In fact, virtually every run-twice problem needs to be interactive (edit: if it is to be hosted on Codeforces). I challenge anyone to come up with a counterexample.

In our case, the interactor will be fairly simple. The interactor reads the input from inf, writes that to cout (where the submission can read it), then reads the first run's output from ouf. If this output is invalid in some way, the interactor quits with _wa, _pe or another failing verdict. Otherwise, it writes that output to tout; that will become the input for the second run.

Interactor

Finally, we need a checker to check that the output of the second run matches the original input. Since this is true for the model solution, we can use ncmp.cpp. However, if you don't trust the model solution, you can also write a simple checker that compares ouf (which is just the output of the second run) to inf.

Swap and Guess

Let's prepare another one. Classical problems about prisoners who cannot communicate but can agree on a strategy beforehand are also a great fit for run-twice problems. For example, let's prepare Anton's problem from this old post.

Problem. There are $$$2n$$$ cards on a table; each contains an unique integer from $$$1$$$ to $$$2n$$$. Your program will be run twice on each test case.

  • First run. The cards are face-up. You can pick a pair of cards and swap them.
  • Second run. The cards are face-down. You are given a number $$$x$$$ ($$$1 \le x \le 2n$$$) and must find the card with that number. To do so, you are allowed to turn at most $$$n$$$ cards over.

Unlike the last problem, the solution needs to know whether it is in the first or the second run. The easiest way is to have the first token in the input be first or second respectively.

Full input format
Solution

In this problem, the second phase is actually interactive. In the previous problem, the interactor was only invoked for the first run. To enable the interactor for the second run, go to General Info -> Advanced and enable "Interactive second invocation". Now the workflow looks like this:

Now we need to figure out exactly what the interactor should do and what the input files should contain. Of course, the input files should contain the permutation $$$p$$$ and the target card $$$x$$$ (although we will later discuss choosing $$$x$$$ adaptively). But keep in mind that there is no way for the interactor to tell whether it is in the first run or the second run, so we need to pass that information along somehow as well.

We'll say that the input files have the following format:

0 n x
p_1 p_2 ... p_2n

In the first run, the interactor will read that data from inf and pass the word first, $$$n$$$, and the permutation to the submission. The interactor will read the positions of the cards to swap from the submission. Then, it will write to tout data to be consumed by the second run of the interactor. It will be in this format:

1 n x
q_1 q_2 ... q_2n

Here, $$$q_1, q_2, \ldots, q_{2n}$$$ is the permutation after performing the swap. Importantly, we need to pass along $$$x$$$ from the first run of the interactor because the second run has no way of reading the original input file.

Some examples:

Input file 1st run submission input 1st run submission output 2nd run interactor input 2nd run submission input
0 3 6
1 4 2 5 6 3
first 3
1 4 2 5 6 3
2 3 1 3 6
1 2 4 5 6 3
second 3 6
0 5 1
4 2 5 1 3
first 5
4 2 5 1 3
3 3 1 5 1
4 2 5 1 3
second 5 1

Code for the interactor so far

In the second run, the interactor reads the input (which is not the original input, but the contents of tout from the last interaction) and passes the word first, $$$n$$$ and $$$x$$$ to the submission. Then, it must read at most $$$n$$$ guesses from the submission and reply to them. If one of the guesses turns out to be correct, the interactor exits successfully (which means the checker will now run), otherwise it exits with a "wrong answer" verdict (and no checker is run).

Codeforces requires that all problems, including interactive ones, have a checker. In this case, there is really nothing to check (the checker will not be run if the submission made too many incorrect guesses), so the checker will be almost a do-nothing program. What you can do, however, is pass some structured data via tout, which the checker can incorporate into the checker message. For example, let's add the number of guesses and the correct $$$x$$$ to the checker message.

Second half of the interactor
Checker
Adaptively choosing $$$x$$$

In some problems, it may be necessary to make the interactor adaptive. In this case, the permutation is obviously determined ahead of time, but we may want to choose $$$x$$$ based on how the solution chose to swap cards. For example, if the submission failed to make all cycles at most $$$n$$$ elements long, it makes sense to choose an $$$x$$$ on the longest cycle of the permutation.

Let's prepare an adaptive interactor that uses the following two strategies:

  • $$$x$$$ is picked uniformly at random from all possible values.
  • $$$x$$$ is picked uniformly at random from the longest cycle.

The input data will now obviously need to contain a parameter that indicates which strategy the interactor should use. But it also needs to contain a random seed. If you attempt to use rnd.next(int) in an interactor, it will always return the same values because registerInteraction does not set a seed value (and even if it did, it would always be the same one). Thus, this random seed should also be in the input data and passed along in tout to the second run of the interactor.

The input files will thus have the following format:

0 n strat seed
p_1 p_2 ... p_2n

and in the first run, the interactor will write to tout the input for the second run, with the following format:

1 n strat seed
q_1 q_2 ... q_2n

Here is the new interactor:

Interactor
  • Vote: I like it
  • +189
  • Vote: I do not like it

»
12 months ago, hide # |
 
Vote: I like it +9 Vote: I do not like it

Problems D, G and K from Universal Cup Season 2 Stage 5.

The statement link you provided is incorrect. It should be https://assets.ucup.ac/statements/statements-2-5.pdf instead of https://assets.ucup.ac/statements/statements-2-15.pdf.

»
12 months ago, hide # |
Rev. 2  
Vote: I like it +5 Vote: I do not like it

This is a good blog on the topic. I just want to share some alternative if you are making tasks like this for OI training (which is where they are more likely to appear, from my experience). I usually use the following model:

  • Participant include a library file, and they have to implement functions and can call prewritten functions.
  • The main function is implemented in the library file itself, it read the input, call participant's functions, and then write to the output for the checker / 2nd invocation.
  • To prevent participant from interacting with the input / output stream directly, I just use a custom io method that read the the data and the hash of that data using a secret hashing function.
  • In the past I have also used encoded input/output instead of a hash, but I changed to a hashing method recently since it allow participant to see the test case in plaintext after the contest.

This technically doesn't use any interactor, and it will only works for C++ or Python because of polygon limitation, but it is a closer simulation of what need to be done in OI contests.

»
12 months ago, hide # |
 
Vote: I like it +20 Vote: I do not like it

Hi, I'm the author of Universal Cup Season 2 Stage 16, and 50+ run-twice problems in total.
The comment I have is a bit too long, so I'll split it into a few separate points.

»
12 months ago, hide # |
 
Vote: I like it +20 Vote: I do not like it

It is true that an interactor enables run-twice problems with only a bit of extra support from the contest system.

  • On the 1st run, the interactor gets the 1st input, forms the 1st solution input, checks the 1st output, and prepares the 2nd input.
  • On the 2nd run, the interactor gets the 2nd input, forms the 2nd solution input, and checks the 2nd output.
  • If the 1st input is also needed on the 2nd run, it can be appended to the 2nd input on the 1st run, but shown only to the interactor.

So, the interactor has all available information, and it controls which parts to show to the solution.

What's left for the contest system is to:

  • run the solution+interactor pair twice instead of once,
  • store the intermediate output between runs, however, also
  • guard against storing other stuff between runs.

However, interactive problems, in their current state, bring their own set of issues. The basic perception is that there are two outcomes: OK and non-OK. The handling of protocol errors often makes WA, TL, RE, and the other outcomes indistinguishable. Note that interactive problems can be rethought a bit (Nyatl may have something to say on the matter), and some of the issues can go away, but this is the current state.

»
12 months ago, hide # |
Rev. 2  
Vote: I like it +20 Vote: I do not like it

For the reasons above, most of the run-twice problems I see are not interactive. What do they look like, then?

Basically, the process has an extra step or two.

  • 1st run: solution <input >output1
  • between runs: channel <input <output1 <hint >input2
  • 2nd run: solution <input2 >output2
  • after runs: check <input <hint >result

Here, the prepared files for each test are input and hint (when the latter is needed, we usually store it in place of a test answer; note that Polygon doesn't support this). The only new entity is channel (other suggested name is mutator): it gets output1, checks it for correctness, and prepares input2 for the 2nd run. In practice, channel uses the same Testlib interface as an interactor:

  • it gets input from the jury,
  • it gets output1 from the solution,
  • it gets hint from the jury,
  • it prints input2 as an interactor would print a message to the checker.

This requires a bit more support from the testing system, as it now has another step to perform, and the new jury program, channel, to store and run.

This approach also requires extra effort for the trickier problems. Note that the check call does not have access to intermediate data: it gets neither output1 nor input2. In the problems where this is required (for an example, see problem I from Universal Cup S02E16 problemset mentioned above), this data has to be transmitted somehow.

However, once this works, it saves the contestant from the issues brought by making all problems interactive: there is no step where the interactor and the solution run in parallel, so, there are no protocol errors which introduce ambiguous outcomes. In UCup S02E16, there are 11 problems: 1 interactive in both runs, 1 interactive in the 2nd run, and 9 non-interactive. This does make a difference.

  • »
    »
    12 months ago, hide # ^ |
     
    Vote: I like it +8 Vote: I do not like it

    If there is one feature that I would like to see from Codeforces, hints would definitely be it. I wrote this blog to show how run-twice can be done on CF, but to be honest I don't actually like (ab)using interactors this way. That would also enable e.g. approximation problems.

»
12 months ago, hide # |
 
Vote: I like it +20 Vote: I do not like it

OK, if the non-interactive approach is so good, why does Polygon not support it?

Well, Polygon does the thing that:

  • enables making run-twice problems somehow,
  • minimizes the changes needed for a testing system,
  • is as backwards compatible as possible.

The latter point is very important for a system used worldwide. The run-twice format is still highly experimental. Me and the other authors (snarknews, our SPbSU authors group, and more) were preparing run-twice problems for a few years to see what works, what doesn't, and what we want from the testing system and the contest preparation system. There are also natural extensions to the format, like running three or more times, switching the run number based on condition, and so on. The approaches we take to problem preparation won't necessarily be the best possible on our first try.

If the authors collectively decide that they now know enough to say what better support looks like, I'm sure Mike would listen.

  • »
    »
    12 months ago, hide # ^ |
     
    Vote: I like it +8 Vote: I do not like it

    The most flexible support for nonstandard problems I've seen is in the Estonian OI fork of CMS. You can upload a script/program that receives the input and submission executable (+command line arguments sometimes), do whatever you want with them. I can't remember if vanilla CMS supports something similar.

    This might not work so well for an "open" platform (might be too easy to abuse?) like Codeforces.

»
12 months ago, hide # |
 
Vote: I like it +10 Vote: I do not like it

One particular possible improvement, which could be useful for normal problems as well, is related to storing the hint. Currently, a normal problem checks the solutions as follows:

  • solution run: solution <input >output
  • after run: check <input <answer >result

What if, in each test, we want to store some extra information (hint) which we should hide from the solution? Currently, in non-Polygon problems, the authors just use answer as the place to store hint. Or we have to make the problem interactive when it actually isn't...

OK, but what if Polygon supported an "input preparation" phase:

  • before run: prep <input >sol-input
  • solution run: solution <sol-input >output
  • after run: check <input <answer >result

This prep would skip the secret part of input and only put in sol-input what the solution needs to know. If you look closely at the phases, you will see that this prep run is very similar to a channel run in non-interactive run-twice.

Another way to improve here would be making interactive problems better. This is also possible, but deserves a separate discussion.

»
6 months ago, hide # |
 
Vote: I like it +8 Vote: I do not like it

Now that we are seeing such problems in a testing round, I would like to ask — has anything changed in how these are prepared on Polygon?

  • »
    »
    6 months ago, hide # ^ |
     
    Vote: I like it 0 Vote: I do not like it

    Idk about the future, but I totally followed this guide to prep the testing round problems xD. Maybe KAN can link this as an official tutorial when you tick the run twice option in polygon? Really great blog btw.