NeoYL's blog

Do you like scoring based on number of submissions?
By NeoYL, history, 12 months ago, In English

Yes, for most of the online contests, number of wrong submissions contribute to penalty.

More wrong submissions, the lower placing you gonna get, despite solving the same number of problems in the same period of time.

But for some of the OI contests (e.g. IOI), it is fully based on scores only and completely ignoring the number of wrong submissions.

I know that Codeforces style (ICPC style) has this penalty system, but if you are given the choice, which would you choose?

  • Vote: I like it
  • +45
  • Vote: I do not like it

»
12 months ago, # |
  Vote: I like it +83 Vote: I do not like it

I'm good with the current one.

If a contest is fully based on scores and depends on the number of tests passed like IOI, it should encourage people to do more subtasks and brute-forcing (which is what IOI is supposed to), so it should take a hell of an amount of time to get a problem done. Moreover, unless there are partial results on floats or something, there should be at least 200+ people that are the same rank as each other since they have the same points. Calculating the ratings from there is not an easy thing. Time metric is a good way to rank participants. You need speed to be competitive after all.

A better idea is to make a contest based on points and time, but not depending on wrong submissions. Cool I guess, but I'm worried that it will enable people to submit as many WAs as they want since there is no penalty, then a long submission queue will happen sooner or later, and people will complain and downvote the contest announcement for no reason. Giving a penalty also motivates participants to care more about their codes and try to not get WA, which will help a lot in reducing bugs and stuff. A good bet for both sides.

→ Reply
»
12 months ago, # |
  Vote: I like it +2 Vote: I do not like it

Tbh I find getting a worse perf purely because of a random bug to be pretty annoying, but there would be way too many ties without penalties. The no-penalty system probably works well for OIs because there are a lot less participants.

→ Reply
  • »
    »
    12 months ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I mean, time can be a pretty big differentiating factor. I doubt that many ties are broken by submission penalty over time taken to solve.

    → Reply
    • »
      »
      »
      12 months ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      hm yeah, that makes sense. Then I personally like the no penalty system more :))

      → Reply
»
12 months ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

Personally, I'd choose the system fully based on scores.

→ Reply
»
12 months ago, # |
← Rev. 2   Vote: I like it +5 Vote: I do not like it

Lyde did a good job of summarizing why IOI-style scoring would be hard to apply to a contest with a large number of participants.

However, I'm also not really a fan of the traditional ICPC-style penalty system. $$$20$$$ minutes is usually a large amount of penalty, and if you end up getting a lot of wrong submissions on a problem no one is making wrong submissions in (for example, due to a random typo in your code), it will hurt a lot. Then again, it depends on your style in doing contests – personally, I'm not a huge fan of meticulously checking or re-reading my code before submitting just to avoid the large time penalty.

I personally feel like the current system in Codeforces is already good enough. Because the decay rate of points for larger problems is faster (and linear to the maximum points on the problem), while the penalty stays the same ($$$-50$$$ points), a WA on a problem scoring more is usually only worth several minutes of 'time penalty'. In other words, there's more leeway in the harder problems, which may have more implementation detail to keep track of than the easier problems.

→ Reply
»
12 months ago, # |
  Vote: I like it 0 Vote: I do not like it

Ranking on the basis of timing would be better.

→ Reply