A lot of people love to talk about cheating, so let's gather all of our opinions into one set of definitely-not-misleading numbers! For the sake of this post, we will define cheating as "anything that violates the Codeforces rules with the intention of affecting the standings," and we will define being caught as "either being banned or having your submissions in a contest skipped." These may not be the most proper definitions, but I think it'll yield the most useful results.
In any given contest, on average what percentage of users (who make at least one submission) cheated during that contest? (This includes users who are caught.)
In any given contest, on average what percentage of cheaters are eventually caught?
What percentage of active users (who have made at least one in-contest submission in the last six months) have cheated at least once?
Do you think online unproctored competitive programming is doomed?
- No, online unproctored competitive programming will remain alive
- No, but only if something is done in the near future
- Yes, I think in the long term, online unproctored competitive programming will be unviable
- Yes, I think we are doomed in the very near future (within a year)
What do you think the appropriate punishment for cheating should be?
- Permanent IP ban on first violation
- Permanent account ban on first violation, permanent IP ban after multiple violations
- Temporary ban on first violation, permanent account ban after multiple violations
- Warning after first violation, temporary ban after multiple violations
- Warning after multiple violations
- No punishment (please don't choose this)
What measures should be taken to combat cheating? (You may choose multiple options.)
- Email/SMS verification on account creation
- Small monetary fee on account creation
- Granting well-known figures (setters, coordinators, etc) the ability to ban users freely
- Proctored contests
- Banning/skipping more users based on statistical evidence (which may increase false positive rate)
- Please choose this option if you voted for any of the above, so we can see each result as a percentage
At what levels do you think cheating is a significant issue? That is, what rating are most cheaters able to obtain? (You may choose multiple options.)
- Gray/green
- Cyan/blue
- Purple/yellow
- Red/nutella
- Please choose this option if you voted for any of the above, so we can see each result as a percentage
At median, how much rating do you think a cheater gains, compared to their actual competitive programming level?
Which of the following forms of cheating do you think occur at a widespread level? (You may choose multiple options.)
- Using generative AI to assist with solutions
- Receiving solutions from YouTube or similar streaming platforms
- Receiving solutions from private Telegram chats or similar
- Discussion with much stronger users (unpaid)
- Discussion with much stronger users (paid)
- Discussion with users of a similar level
- Please choose this option if you voted for any of the above, so we can see each result as a percentage
At median, how much rating do you think a legitimate gray/green contestant loses due to cheating?
Feel free to suggest more questions (or more answer choices) in the comments; I'll add anything I see.
I feel like there are more cheaters than people realize; because it is not possible to count cheaters that don't get caught
There are at least two more well-known forms of cheating which, in my opinion, are kinda widespread (not as widespread as using AI nowadays, but more frequent than simple discussions with other users): teaming in individual contests and rating boosting. Maybe the former is similar to "discussion with users of a similar level", but teaming is much more than discussing the problems.
A couple of months ago, I would answer "No, online unproctored competitive programming will remain alive" on the 4th question. Right now, I'm far less optimistic. I always thought that rating was just a number and most dedicated competitive programmers would not be affected that much by anything related from cheating, except for the rating points. However, I realized that cheating can severely affect even people who don't care about their placement — and that is because scumbags who instantly throw the problem into LLM after loading it create a substantial amount of useless work for the testing system. Even if they all are removed, at least some damage to the testing process — which either creates a huge queue, or makes Mike add trillion more invokers and increases the cost of running CF — is irreversible. And, unfortunately, not all cheaters are removed, so cheating does even more damage, especially to people caring for their placement.
I don't see any way to combat this issue other than requiring some sort of verification. However, I don't really understand which form of verification it should be. Email verification is very easy to bypass; phone verification is a bit more difficult, but still can't protect from everyone. There are some forms of photo verifications (making the user put a photo in a profile, then asking them to make another photo of themselves in a certain way that would verify that this is a real person using the account), but they can't really work if you want to keep at least some level of privacy for the users. Any other ideas?
Honestly, part of me thinks that the best solution is just to forego some degree of privacy, and force users to verify by government ID in order to participate in rated contests. Then, if somebody is a repeat offender, they can be permanently banned from contests.
This seems extreme, but also, at any onsite competition you're required to give your real identity anyways, so it isn't actually that unreasonable imo. Besides, I think most strong contestants have already revealed their identities publicly, and for those who haven't, they can still keep their identity private to the general population, just reveal themself to the Codeforces admins.
I don't really like this solution, but I can't think of a better one.
Well, Roblox has already implemented ID verification for voice chats.
This is probably a terrible comparison but I mean... if roblox is an extremely popular game with billions of players and even they require ID verification through a 3rd party, why not codeforces...?
No they didn't. I have voice chat access and I just used SMS to verify. I purchased the verification for 10 cents (privacy is a human right). ID is only for 17+ experiences.
Well,seems this is the only solution.But I think there is a problem.Some users cannot reveal their real ID because of dictators' monitor.If big brother is watching you,you can't discuss on CF freely.
Phone number verification is a good approach to avoid multiple accounts, but I think there should also be a portal where users can report IDs that they suspect belong to the same person or are involved in cheating.
If the report is correct, the user who reported it should get some kind of points similar to "contribution" point .
LeetCode has already implemented this feature. While it doesn’t completely eliminate cheating, it has significantly reduced it.
There are several issues with a reporting system. The first is obviously false reports. Well an admin can then review those reports, but it still takes human resources and time that is probably better spent elsewhere.
Well then I thought: why not limit to reporting to div. 1 participants? The issue with this is: it's not hard to reach div. 1 using llms lol. Well how about limiting to red+ only? I doubt red+ participants are really interested in reporting cheaters anyhow, especially when they're not directly affected.
I personally don't get why privacy is such a huge concern. The amount of times I had to present my ID to get any services (I'm talking non-government organisations, obviously) is almost infinite, so I just don't see why doing it one more time is an issue. Same with the phone number — even games/game related third-party services require those on a regular basis.
It might seem not a concern for you, but imagine being a little kid who just found out about codeforces, they might not even have a legal ID yet, not considering the fact that whether they're willing to give up their ID or not (most of them won't anyways, it is too much of a hassle).
And the Democratic Party of the US wants people to be able to VOTE FOR GOVERNMENT OFFICIALS and the PRESIDENT without an ID.
isn't it crazy? and this past election they couldn't pull it off properly so 10m voters "didn't show up to the polls"
What if you are too young to have an ID?
My comment on your second paragraph: IMO if cheating gets very easy and prevalent, the ratings will lose their meaning and hence the incentive for cheating diminishes: for an extreme example, companies will now treat Codeforces ranking as unreliable and ignore it during recrutation (I often see on discussions about cheating this being given as a primary force for the cheaters). Maybe even the ratings will be entirely removed from the page. So now anyone cheating gains literally nothing for their everyday life. In a sense, "stop caring about the rating!" naturally emerges, as now rating means nothing or doesn't exist. So in a sense, I'm rather confident that Codeforces contests will survive the cheating epidemic.
On the other hand, I'm much more worried about the onsite, "proctored" competitions, like national OIs, Meta Hacker Cup, etc.! Why? Typically, those have an online qualification round, since, understandably, you are only able to host so many onsite participants. So somehow you have to select the best contestants, typically first organizing some online qualifications. I can easily imagine a situation, in which a majority of onsite rounds contestants are cheaters in such a setup (and perform very poorly, while legit contestants didn't even make it to onsite rounds), especially if AI gets to red+ levels (which I doubt would happen anytime soon). The same principle applies to any onsite contest (especially with prizes) with an open registration, so national olympiads in most subjects, best student project awards, etc.
The only (major) competition that is proctored at every level that I can think of is ICPC: universities are incentivized to organize internal selections to pick the best teams, and then nationals, regionals and finals are sort of guaranteed to have the strongest possible legit contestants. So IMO Codeforces and ICPC are safe.
Another suggestion to combat cheaters: new accounts are not allowed to access the website during contest-time unless they pay a one-time fee (smth like 10$).
Optional: this fee could be removed for educational institutions. imo this exemption should not be based on having an edu email address. Instead, it should require official contact from the country's olympiad organization, and only apply to middle/high school students.
Nah, money is not a deterrent. Many cheaters have $10 and many legit users from poorer countries would rather not spend it on this website.
The idea is paying for access during contest time. Legit users can still practice problems for free.
Many cheaters have 10$I disagreeI am aware that it adds a barrier for honest users from developing countries, but it helps reduce abuse from bot accounts and puts pressure on cheaters by increasing the risk of losing access they've paid for. I don't think an average cheater would pay 10$ for a second or third time.
I limited the exemption to middle and high school students because I see access to contests as part of a basic educational right especially as more countries include computer science in their pre-university curricula -- this actually needs more explanation but I have no energy to write I hope you get my point. Maybe you fully disagree which I respect.
For university students or professionals, contests are an extracurricular or hobby activity, so it’s reasonable to expect some payment for premium features like contest time access.
Well overall I don't think this kind of measure with tweaks happening since even basic features are taking ages to be implemented in this platform.
lol my teacher made us write a argumentative essay in class today and I used this blog for evidence
xd what did you write about? i'd be interested in reading it lmao
I am for opt-in ID verification. Rating changes would be calculated with respect to the verified users only. The sum of rating changes would only be 0 for verified users (this how rating currently works correct?). Unverified users taking part in contests would receive their rating changes based on how they did with respect to the verified users. Becoming verified would require a rating reset of course. Users should have some sort of badge on their profile if they are verified.
I think this could change would drastically reduce the number of cheaters over all, not just among verified users, because cheaters likely want to show off their rating for some purpose, but showing off an unverified rating would hopefully become meaningless. Bans for verified users are actually permanent. No just making a new verified account.
Participating as a well meaning user who for whatever reason does not wish to ID verify, would be basically the same was participation today except their (unverified) rating wouldn't be impacted if a ton of (unverified) people disproportionately cheated above them.
Perfect solution! Regarding reseting the rating, I think we could implement something similar to new accounts: their rating goes back to 0, but their actual hidden rating is their last rating, so they would have to take 2 verified contests for both ratings to match. I was going to suggest more verified contests, but div. 1 people would have to wait some months to get their rating back. Also we could make an exception for people with an account before this change, to not harm anyone.
I agree that this is the best way to proceed — I am all for this type of system.
That being said, why should becoming verified require a rating reset? Presumably, verification has nothing to do whether or not you can cheat — it only means that if you cheat, it will be tied to your real identity, and you cannot make another verified account afterwards. Maybe a more thorough check for cheating is applied to verified users, and this can be done retroactively if a person later verifies their account, but I'm not sure what the rating reset does, since there's no reason to believe that the past performances are any less legitimate.
Overall, great suggestion though. I hope this gets implemented for real.
gg ez wp thanks for playing 2ez4me outplayed and outskilled sit down get rekt ez clap diff gap better luck next time no sweat barely tried tutorial complete maybe practice more uninstall recommended skill issue like taking candy from a baby too easy not even close ez game ez life carried my team destroyed absolutely demolished child's play that was quick next victim please you made that too easy back to training mode calculated git gud trash light work didn't even break a sweat free win speedrun complete welcome to my highlight reel keyboard diff mouse diff headset diff chair diff that's going in my montage yawn... boring too predictable read like a book
maybe you're right lol :)
that being said, are they doing essentially the same thing (wrt avoiding getting caught) on all 100 accounts?
if so, the sol is just to retroactively analyze all past submissions at account verification, then the 100th one gets caught like the first 99
otherwise, they've essentially found a way to bypass cheat detection anyways, so nothing would really help
no, can just try different pattern on every account until one pattern works
at the end of the day, everything comes down to the cheat detection system
if somebody has found a pattern of code which can trick the cheat detection system, then no verification system will make a difference anyways
the only way to solve this issue is just with a better method of detecting cheaters, but that seems hard...
(well, maybe IP ban would also work :) then you can't spam accounts to test different patterns)
I think the only real solution (though not a perfect one) is to verify users based on in-person performances that are close to their current rating. Obviously, this means most users won’t be verified, so it might make sense to introduce different classes of verification.
It’s not hard to imagine someone cheating to an lgm level without any clear signs of cheating, which is why I think this kind of system is necessary.
maybe... but in your proposed system, what functional differences do the different classes of verification have?
if a contest's rating changes are calculated based solely on a couple hundred people who have done on-sites, this will be highly variable/inaccurate, especially at low levels where very few of them have probably done on-sites
besides I'm not sold on the integrity of all on-sites anyways... I'm sure cheating is commonplace at ICPC in some regions
I guess I was mostly considering GM+ ratings. If you're at that level, you should be able to do fairly well at in-person contests and I think it would be a bit sad if it was easy for users to cheat to that level.
Also at lower levels cheating isn't as big of a problem because their rank doesn't change as much (going from rank 6000 to 6300 is much better than going from 500 to 800.)
I think you can make rating changes fairly accurate by relying on:
Priors for problem ratings based on problem setters and testers
Some sort of weighted system considering users of second class of verification (ID but no in-person contests).
"In any given contest, on average what percentage of cheaters are eventually caught?"
A lot are caught, but hardly anything happens :( (as fair as I know).
"Small monetary fee on account creation"
Not everyone is old enough to own credit card.