Geothermal's blog

Remove Virtual Contestants from Contest Leaderboards
By Geothermal, history, 5 years ago, In English

As most of you are likely aware, cheating in virtual contests (typically by submitting prewritten or copied solutions in order to appear at the top of the leaderboard) is very common. Most recently, wannabecandidatemaster submitted solutions to all problems in this round in order to take first place in the standings, and bemandrei competed virtually in this round after competing in the round officially, and ended up in second place on the leaderboard.

In response to the rise of VC cheating (as well as some other issues with the VC system I'll describe below), I'd like to make a simple proposal: Remove all virtual participants from the contest leaderboards. The one exception is that you should see your own name on leaderboards for contests you VCed (that is, virtual contestants should be visible only to themselves). Similarly, virtual participants should not be counted in the solve statistics during future contestants' VCs.

Cheating on VCs is harmful for two reasons. First, it makes it impossible to review the scoreboard as it appeared at the end of the round; in particular, this makes it difficult to discern accurate rankings for out of contest participants. Though this may seem like a relatively minor issue, Div 2, 3, and 4 rounds regularly attract hundreds or even thousands of competitors too high rated to compete officially, so it has an exceptionally broad scope and affects a large number of users.

Second, and more importantly, VC cheaters damage the VC experience for future competitors by making the solve statistics inaccurate. Many competitors use the dashboard's view of how many people have solved each problem in order to make strategic decisions during contests. In the round when I first reached GM, I achieved a high rank by noticing a few early solves on D, correctly discerning that it was unusually easy, and solving it before B or C in order to achieve a better penalty time. This kind of strategy is very common, but the presence of VC cheaters makes it much less practical because in pretty much every contest, when there are a few early solves on the hardest problems, they're not because those problems are easy--they're the result of VC cheaters. The result is that VC cheaters make the VC experience less realistic, so removing them from the solve statistics and leaderboards is critical to achieving the intended goals of the VC feature.

One might claim that the solution is simply to identify cheaters and remove them from the standings after the fact. However, this is a worse approach than removing VCs from the leaderboards altogether for three reasons. First, this approach requires continual effort on the part of the site administrators to remove VC cheaters, whereas my proposed solution needs only a one-time update. Second, it is impossible to accurately detect cheaters, especially because many forms of cheating are less obvious--for example, VCers can look up the test data using a second browser in order to gain an advantage when debugging, and there is no way to accurately distinguish between this and simply guessing the error quickly.

Third and finally, VCs are fundamentally different from participating in a contest live, so even in the absence of cheating, they should not appear on the leaderboards. There are a number of reasons for this, but I see two of them as most important. First, VC solutions are judged on all tests, rather than just the pre-tests, so it is impossible for them to FST: if their solution fails a test not originally included in the pre-tests, they get a chance to fix their error and resubmit, giving them a huge advantage over in-contest competitors. Second, frequently, Codeforces users release blog posts discussing techniques that came up in recent rounds; competitors who read an article inspired by a certain round and then VCed that round would have an advantage over anyone who did that round live.

As a result, I claim that the ideal way to deal with VCs is to remove them from the leaderboards entirely. Please feel free to leave thoughts below; I'm happy to discuss this further with anyone interested or to answer questions about my proposal.

  • Vote: I like it
  • +1001
  • Vote: I do not like it

Comments Comments (31)
| Write comment?
»
5 years ago, # |
← Rev. 2   Vote: I like it +35 Vote: I do not like it

Sad to see that another newbie-4e1s3n8e7g is competing virtually in today's Educational Round, and secures the first position within less than 20 minutes. Below him follows another newbie at rank 2 :( standings

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +157 Vote: I do not like it

    The one you linked copied my solution to F, which I guess is what I get for making this post :P

    → Reply
»
5 years ago, # |
  Vote: I like it +95 Vote: I do not like it

I agree with all your points, and would like something similar to only seeing your own name. However, one of the most fun thing about vc's is doing it against a group of people, so I think instead of just seeing your own name you only see the people on your friends list's names.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    That's a good idea; I'd be open to that. The one issue is that this works in reverse in some cases--some people may not want friends to be able to see their VC scores since it adds an extra layer of pressure. That said, I don't have a major issue with either implementation.

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +25 Vote: I do not like it

      Why not have a separate virtual leaderboard?

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +2 Vote: I do not like it

        Are you suggesting something like AtCoder does?

        I want to be able to see what my rank would have been in the actual contest. Having a separate leaderboard would ruin that purpose.

        → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +14 Vote: I do not like it

      Why not just let someone decide if their VC scores should be private or public? It seems like the best of both worlds.

      → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +4 Vote: I do not like it

      "some people may not want friends to be able to see their VC scores": It is true that some people may want this but as it stands now, friends are able to see the VC scores. I think this issue is a separate issue as compared to the issue of cheaters in the leaderboard.

      → Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    This is the same idea in my mind, it will maintain the spirit of competitiveness and exclude cheaters from our self-tailored leaderboard.

    → Reply
»
5 years ago, # |
  Vote: I like it +39 Vote: I do not like it

I couldn't understand what some guys try to prove by submitting copied solutions to appear at the top of leaderboard.

→ Reply
»
5 years ago, # |
  Vote: I like it +203 Vote: I do not like it

"show unofficial"

Yep, doesn't fix the problem with "Problems" tab, but you can see the number of AC below the standings and they will be calculated correctly.

I'm against your proposal in general because Gym is much more important for me than CF rounds and in Gym Virtual Participation is the main option. Maybe it is good for official rounds only.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +55 Vote: I do not like it

    Good point about the gym--I'm totally open to implementing this for official rounds only.

    → Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it -8 Vote: I do not like it

    You could still devirtualize your participation in the end if this is the only issue.

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +3 Vote: I do not like it

      I'm not only talking about my own results, I want to see complete standings.

      "Show unofficial" solve all your issues, doesn't it?

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +12 Vote: I do not like it

        Show Unofficial doesn't provide a way to separate VCers and official participants taking part out of competition, or your own VC from other VCers. It's a decent workaround for now, but it's not a perfect solution.

        → Reply
        • »
          »
          »
          »
          »
          5 years ago, # ^ |
            Vote: I like it -21 Vote: I do not like it

          Let's be honest — you just want to keep bragging to your friends that you scored the first place in Div. 3 round that you mentioned in your post — https://mirror.codeforces.com/contest/1360/standings. And VC cheaters ruin that victory for you.

          But is it really an achievement for a red guy to win Div. 3 contest? It is not so "Show official" button solves the problem, essentially, correctly removing Div. 1 wanna-be-winners and all virtual contestants.

          → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        No it doesn't. It doesn't solve the issue about the number of participants who have "solved this problem" being inaccurate.

        → Reply
»
5 years ago, # |
  Vote: I like it +3 Vote: I do not like it

I highly approve your suggestion. There's clearly no reason to rank people that participate in a normal contest with people that participate in it after the editorial is out or after everyone submission's is available.

And even though it's useless to participate virtually and copy solutions(In terms of progression) we can never stop people from doing it.

→ Reply
»
5 years ago, # |
  Vote: I like it +9 Vote: I do not like it

bemandrei cheated on the recent Div 3 round and yet he still couldn't beat Geothermal!

Lol!

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it -39 Vote: I do not like it

    i wasn't cheating, i just tryed to make the 2 problems that left from the oficial contest, and i did them in a virtual one with the other problems made from the oficial contest

    → Reply
»
5 years ago, # |
  Vote: I like it +89 Vote: I do not like it

I'd prefer if this were an optional feature: add another checkbox next to "show unofficial". Also, add these checkboxes to the dashboard.

→ Reply
»
5 years ago, # |
  Vote: I like it +302 Vote: I do not like it

they all come close but they can't touch the king

→ Reply
»
5 years ago, # |
  Vote: I like it +8 Vote: I do not like it

Completely agree with the following solution that you presented:

Remove all virtual participants from the contest leaderboards. The one exception is that you should see your own name on leaderboards for contests you VCed (that is, virtual contestants should be visible only to themselves)

In this manner if I am doing Virtual Contest then I should see where I stand, and thats what matters. How other virtual contestants performed also matters but as you said may are cheating, so its better to have no information about other Virtual Contestants than have wrong information.

→ Reply
»
5 years ago, # |
  Vote: I like it +86 Vote: I do not like it

I'm surprised so many people are agreeing with this idea, when:

  1. As said above, you have the "show unofficial" checkbox; do you want to make statistics on a problem/contest? Make it on that. It's just as wrong to make statistics on people who VP on a contest as to make statistics including people that a contest hasn't been organized for, in my opinion.

  2. Solved count? I'm not sure how that works, but I don't think I have to VP anyways to "increase the solved count", if that's the idea. And, anyways, why would you care about the 3 people who increased the solved count from 100 to 103?

I'm not sure what this blog post is really about. Are you mad because you had a good place in a div2 contest and you don't see yourself as high in the leaderboards now after the cheaters submitted the solutions in the first 10 minutes? Well, I'm sorry to break it for you, but maybe you shouldn't consider yourself in the standings, as div2 contests are made for div2 participants, and you obviously aren't in that category.

Stop worrying about already solved issues.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I wouldn't word it that harshly, but tbh the blog does come off that way.

    → Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I also want an option to see the list of unofficial (not VC) participations and the ranks I achieved on them. It's sad that it's hard to keep track of Div. 2 or Div. 3 round I joined in contest time on which I performed well.

→ Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it +32 Vote: I do not like it

Please, do not remove virtual participants. Suppose, that we with members of our competitive programming class started virtual participation in old Educational Round and want to see results of each other during next 2 hours and results of all participants too, and each of us is interested in honest solving during regular training. We will not be able to see each other.

Instead of it button "downvote" for each member of virtual participation table can be added, and if participant will be downvoted, he will be removed from virtual participation. Maybe, button "upvote" too.

→ Reply
»
5 years ago, # |
  Vote: I like it +31 Vote: I do not like it

I would like to have VC working with pretests like usual competitions, since that would make them more realistic.

→ Reply
»
2 years ago, # |
  Vote: I like it +25 Vote: I do not like it

2 years later and I think this is still relevant.

→ Reply