Hello, Codeforces!
I carefully read most of the ideas that are written in the comments and posts. I have no way to answer everything. Otherwise, I would only do what I answered. Excuse me. There are many good ideas, some easier to implement, some more difficult.
The following idea seems vivid to me. Idea is to limit the maximum rating increase for rounds where there is already an upper rating limit at registration. For example, if the upper limit is $$$u$$$, then your rating after participating in the round cannot be higher than $$$u+100$$$ even with a very successful performance. For example, after winning in Div.3 round the rating can't be more than $$$1699$$$.
It is easy to implement, it does not entail a significant redistribution of ratings. It seems I like it. What do you think about it?
Actually, probably a good idea is to make a bottom cap for rating changes: in any case of any (even the worst) performance, the rating shall not fall by more than $$$100$$$. Еverything happens during the rounds — there may be technical difficulties, force majeure, or just catastrophic bad luck. What are your thoughts about it?
— Mike
Hiruem Your Idea is hit
I'm surprised lol, glad Mike liked my comment! At least now there'll be more deserving experts and CMs!
P.S. — Also I think the bottom cap idea is also good, some participants get discouraged if they receive huge negative delta.
I'm glad that Mike noticed your comment! (It isn't ironic). And I also want to say that this idea was suggested a year ago by my classmates — tem_shett and MikMirzoyanov — This is a link to the post. But, unfortunately, tem_shett was disliked and he even deleted the post (here is another classmate's post about it). And I am glad that their idea may be implemented on Codeforces.
Bottom cap should be 200, top cap is ok at 100. And also this should all apply to hidden ratings, not displayed ratings (hidden rating can't be more than u + 100).
imo both top and bottom should be 150. Rationale: I think lower cap=upper cap is necessary, otherwise the rating distribution will probably slowly shift lower (if more decrease than increase is allowed). Also, having to quit in-contest should not mean needing 2+ good contests to cover up.
+100 is a little too restrictive. I think a majority have had a contest with delta > +100. +150 seems better, while being a rounded cap. It also keeps div-1 eligibility achievable within 4 contests for new users, instead of 6.
No, the lower cap applies no matter what rating you are -- the upper cap only applies when you are at the top rating. Therefore lower cap always has to be bigger than upper cap since more people hit the lower cap. If lower cap and upper cap are the same then inflation will happen.
Aah yeah that's right. However, even 50 points extra might be too much for lower cap as the majority of users are under 1900. Maybe 125 upper and -150 lower then.
Also worth considering: Upper cap only for 'trusted users' of Division (>=2). If someone has hit 2100 before, don't upper cap them (or keep a more flexible upper cap) so that they can regain from that one bad contest (or when they had to go AFK in between)
No, this is bad, because then alts will be not restricted anymore. Probably just something like bound = max(u + 100, user's max rating) is good.
Maybe somebody can take a bunch of past rounds and analyze them with different values of upper and lower caps to see the actual effect?
Suppose during contest time an accident has occurred with an contestant. Is it ok to give him/her that much penalty for an accident?
It has been ok for a decade and should keep being ok.
Getting injustice for a long period of time does't make that justice
I don't think that it is about justice or injustice, 'cause some shit happens all the time and monitoring them will be definitely difficult as well as analyzing whether 'accident' was real or not.
Right, sure, it should be applied to hidden ratings.
Why $$$200$$$? What's wrong with $$$100$$$? I like some symmetry in the case of $$$100$$$. Also, I think in most cases a decrease of $$$>200$$$ is a result of planned rating destroy for some prank. Therefore, such the cap (200) will not particularly work in life.
Hello Mike. Can you please clarify if we will be sorted into div 1/2 based on our hidden ratings or displayed ratings? (i.e. hidden 1900, displayed 1200, so will I be in div 1 or div 2)? If it is the former, then it will come to a point where some people had already exceeded the upper cap for div 2 (2200/2000) and yet can't participate in div 1 because of low displayed rating, and thus they cannot possibly increase their rating.
They will compete in div2 and then the displayed rating goes up even if hidden rating stays the same.
Understandable, but do you think that it's fair that 2200-strong people have to keep maximum performance in 3 continuous div 2 for no reason except waiting for their displayed ratings catch up? (Also, wouldn't that generates some considerable degree of inflation in div 2, as 2200-strong people who will receive loads of top places does not receive any rating changes, and will dump said ratings to weaker participants?).
Actually it will cause rating deflation because the points that were supposed to go to these contestants will just vanish into thin air.
Anybody can hit the lower cap. However only top contestants can hit the upper cap. Therefore it will cause rating inflation if the lower cap is set too low (= set to same value as upper cap). 200 is good enough in my opinion.
MikeMirzayanov If there is an upper_bound of +100 then the other rating changes should be normalised with respect to 100 because If two candidates at the same rating,one perform much better then the other,If they will get rating changes of +80 and +100(which could be much higher,but due to the upper_bound get restricted to +100).This would be unfair for the candidate.
I won't advise for using bottom cap because if there are technical difficulties in a contest than everyone is affected by it also bad luck is just a part of life and everyone should learn to deal with it, for a hardworking person negative rating are inspiration to perform better.
He meant technical difficulties for any person. Such as, electricity was cut for this person, internet not connecting,etc...
[Deleted]
Lets say a person submitted problem A and it got accepted. Then suddenly, electricity is cut out. How can he submit other solutions or even read the rest of the problem statements(if we consider his laptop was not charged or the person is participating from a desktop)
[Deleted]
I was literally saying an example.
that would be counted as having bad luck which as i have already mentioned is just a part of life and people should learn to deal with this.Also if someone drop ratings because of events like this than it should be easy for them to pickup their ratings in the next 1-2 contests.
If this can be implemented, it will be really good. I am sure everyone has a bad day during contests, so there should be a limited decrease, so that it does not ruins anyone’s hardwork significantly. Also anyone can have a good contest once in a blue moon, so having a limited increase helps.
Just a quick clarification -
There won't be any such limit for div1s right?
Yes, no such limits (simple upper cap) for contests without explicit upper rating bound (Div1 and Div1+Div2).
The top performers might complain. If a cap of +100 is introduced a guy getting rank of top 10 and others in top 200. Relative to their rating may both end up getting +100.
The change seems good to me otherwise. (besides the above issue I mentioned)
What if the delta(rating change) is normalised between lower & upper bound instead of capping the extreme delta values. This might prevent the issue you mentioned to some extent but it may make the rating value stiff for many!
Yes, these changes will definitely be for the better. Codeforces rating system is becoming silimar to that of AtCoder where after 15-20 contest your rating kind of converges if you are not improving.
If the rating system is based on the mean rating being invariant, how will caps change this? For example, if one person creates lots of accounts to get huge rating changes on purpose, will this shift the center of distribution?
The rating system is not based on anything anymore, it is just a bunch of klutches, fixes and workarounds smashed one on top another.
I wouldn't say it is bad, it seems that it is really hard to come up with meaningful system which is easy to recalculate.
I think it's a good idea. It will avoid people from creating fake accounts...
I wish you could have done that. Separate Discuss Section Or Editorial for each problem. Asking editorialists to prepare a separate editorial for each problem is quite easy, and that would make the discussion a lot better at Codeforces.
Unnecessary, And probably no one would even follow that...
Has something like this ever happened with you that you want to see all discussion related to problem C, but you found spoilers for D and E that you wanted to try later, like I did a binary search in D and used DP for E?
You are right... But separate thread for each problem might cause too much chaos. Also most of the editorials have drop downs for each problem solution, So its better to look at the editorial first (which is probably more detailed then the discussion) Still I agree with the spoiler part...
What about the rule of sum of rating changes of participants in a contest?
So the maximum rating change will be +100 or it will be capped by the upper limit of the round +100? Because it is mentioned that the maximum decrease will be -100 for symmetry, but if the limit is u+100 some people can gain 200 if the rating is low enough but can lose at most 100? How is this symmetry?
If you so want to do something with rating system, why not to change it completely instead of raping old one to death?
Surely you could've phrased that better?
+++ this
If you have a better rating system, why don't you share it with us!
We are working on this. See this subtree of comments for full discussion.
Please make CF like ~2 years ago. Div.2 upper bound 1900 no Div.3 and no Div.4 and ....
[Deleted]
Do you really find div3/4 contests helpful or do you just wanna see yourself placed high? Personally, I'm not impressed with div3/4 problem qualities at all. Also, there are like billion resources for beginners already. So I really see no point in those contests.
Div2C == Div1A, but Div3A is same as Div2A and sometimes harder. this is why new divisions dont make sense, 1 real div3 where div3C == div2A, would be good for all ranges of beginners.
I don't understand why Red people cry about div-3,4 how is it affecting your group anyways there is a reason that those rounds are rated for only low rated people. You should instead ask for more div-1 or more quality in div-1 rounds. You think that it takes away the concentration from div-1 rounds but as Mike told the number of div-1 rounds has stayed same for many years.
What about making sub tasks contest instead of div3&4. In that way beginners could get points in +2 tasks. Like there's ICPC style in edu rounds but no IOI
Agree, after div3 one can get much more rating, than after div2(I've experienced it for myself)
Sounds good. But I have a little concern about the asymmetry of the rating change. I suggest to write a simulator (if you haven't done it yet) and evaluate your changes before deploying them. Also, the whole system starts looking like a set of hacks, magick constants and so on. May be it's time to think about more robust and simpler system?
Suggestion seems good but my only problem is that it would not be ideal to see rank 1 and rank 50 to be at the same rating after the contest. That does not seem completely fair to me.
Solution -
Instead of making so many changes to the rating system, why not just increase the difference of rating gaps between different divisions?
For example, 2100 seems a very loose bound for Master since due to rating inflation, its become very easy to become a master. Why not just restore the master rating to 2200 (like it was 3 years back). Same holds for becoming a Candidate Master now.
If performance in div 2 contests provides poor discrimination between people who "should be" 2000 and people who "should be" 2200 it seems reasonable for rank 1 and rank 50 to both end up at 2000. The main goal of ratings should be to represent revealed information from performances as accurately as possible.
its become very easy to become a masterRecent div 2 rounds were harder than ever, so maybe because of quarantine people are giving more time to cp and performing well which might be the reason for more masters.Really? In recent educational round, people became yellow by solving only problems which are solved by 1000+ participants. If every Div2E was as high quality as Div1C then there would be no problem in having purples participate in Div2. But average Div2E has lower quality than average Div1C, especially in educational rounds. One shouldn't be master by solving problems too similar to known ones.
Yes you are right problem E of Div2 only rounds are on an average easier than Div1C problems and an upper bound will help in resolving this matter.But recent Div2 rounds has become difficult for an expert or maybe you can say it has become difficult to continuously perform well for an expert.
As someone who became yellow after the last educational round, I feel I can put my two cents in.
First of all, both D and E had less than 1000 official solves. It makes no sense to count the solves of unofficial (rated yellow or higher) participants just because we're not talking about them here.
Secondly, you kinda forget that C was rather tough as well. I mean, around 2750 contestants got it AC'ed out of... almost 8000. That's an accuracy rate of around 34%, which, for a problem of this level and type (not, say, one where everyone is tempted to squeeze a brute-force O(n^2) and thousands of people submit an asymptotically incorrect solution) is very, very rare... That's not to mention that most of the people who did, eventually, get AC, didn't get it right from the first try.
Thirdly, D and E are not "easy" problems at all and do require some thinking. Oh, and did I mention the accuracy rate of D is less than 30%? Plus, the profile of those two problems is quite different — which is backed up by the fact that although both of these problems received under 1k solves, only around 500 solved them both (and yes, there are also many people who solved ABDE).
And now just think that you need to solve all three of those problems sufficiently fast and without many penalty attempts. And if, for some reason, you got stuck on, say, C, that would leave you in the best case with a small positive delta. So I believe that if you managed to do that, you definitely deserve a yellow rating.
Your rating does not change if you do not submit anything during contest. Have you considered removing this rule?
I don't know if it is currently the case, but it is easy to abuse this rule:
Spend some (1-5) minutes at the start of the contest to get an overview of the problems.
Based on this overview decide whether to participate or not.
Perhaps it is not a big problem, but I suspect it is by the amount of registered users that do not participate in the average contest.
Imho abusing such thing makes no sense.
Well, for example, sometimes I don't know if I can participate or not, so I register so that there won't be a situation where I want to participate, but I can't because I'm not registered... Therefore, if the rating was deducted for registration without participation, it would be very unpleasant(((
Maybe a system can be implemented where if someone enter's a contest than he will be considered for final standings and thus the participant's rating will change.
You are literally repeating the OP on a comment refusing that idea.
I have idea to close out of competition participants at div3/div4.There are a lot of people who participate out of contest. It can help to make testing system faster. What you think about that idea?
I dont like this suggestion, I dont see many people complaining about the long queue in div. 3/4 rounds. Maybe you feel like that this is an issue because your friends that participate in it are all out of competition? Plus this would increase the number of alt accounts
Also it can help to make pretests a little bit larger. Or you think that it is not needful?
I'm sure that getting rid of out of competition participants would not make a significant difference in order to enable stronger pretests. What could be done first is making Div. 1 pretests stronger (or making pretest = systest), because div. 1 has way less participants (around 10 times lower or even less).
Look at the last div3: 19030 official contestants and 3109 unofficial contestants. Cutting out the unofficial contestants won't do much for the queue.
Regarding this upper limit for lower divisions, I would like to apply this for the color change, for example: one could only become orange through a Div. 1 contests. That way, purples would look forward for Div. 1 contests, because that is the only way they can become oranges, instead of looking forward for Div. 2 contests, which is a "easier" way of doing that. I feel like some purples avoid Div. 1s (it would be nice to see the number of purple participants in Div. 1s vs Div 2s, they should be the same, but I feel like they arent).
Well, actually quite the opposite is true...
Personally I didn't experience any negative attitude towards div. 1 rounds in the sense that they are "difficult" for purple competitors. If you solve moderately fast (generally) AB — the tasks which are AC'ed by most (but not all!) participants, you have a good chance to achieve a positive delta. The only possible case of achieving a negative delta as a purple competitor in a div. 1 contest would be to end up near the end of the scoreboard — which is, mildly speaking, difficult to achieve unless you have really bad luck (yes, once I solved 0 problems, but it was partially my fault since I gave up after the 1st hour)
On the other hand, in a div. 2 contest you must end up near the top of the scoreboard to achieve a positive delta, and in the top-200 for a decent increase. Several contests in a row. And that would mean solving all of the problems that > 300 competitors solve rather fast. In my opinion, if you deserve the rank of candidate master, the first option is even easier than the second one.
I feel like your argument is too much based on your personal experience, but that is fair, mine is based on my impressions as well. Thats why I would like to see some statistics regarding that, such as the number of CMs that participated in Div1s vs div2 only rounds
The statistics sound like a good point. However I wonder whether they would be representative, since, for example, I miss many rounds due to personal reasons (not because I don't want to participate in the particular contest).
Speaking about personal experience, well... that's partly true. Mostly though I was talking about the rating system. In a div. 1 round, a CM will be rated lower than the vast majority of the participants (well, unless it's something like 2090). So even a mediocre performance is enough and the only thing he has to avoid is end up in the lower 20% or something. In a div. 2 only round, though, the CMs should end up in the top 1% of the ratings in order to get a sufficient increase, just because there are tooooooons of greens, cyans, blues and lower-rated purples. Of course almost all of them a priori won't pose a competitive risk, but apart from similar-rated CMs, there are also some blues that display stellar (or just good, but within the top 1%) performance and also unrated smurfs. Soooo...
Wondering whether the statistics would be representative makes no sense to me. If you miss many rounds for personal reasons, you would roughly miss in equal proportion div 1s and div 2 only rounds. Also we are not even only taking into account a percentage of the participants which is common in real life for gathering statistics and extrapolating things, in which case it makes sense to say that something is not representative. But in this case we are measuring all the participants, how could you say that this is not representative? Maybe you dont know the meaning of representative?
Sorry if I was unclear. I'll try to rephrase myself — I meant that if you know how many CMs participated in div. 1 only vs. div. 2 only rounds, that doesn't give you an accurate picture of how many competitors actually chose to participate in either of the two options.
And no, div. 1 and div. 2 rounds are not set with the same frequency — just from this Tuesday and up to Sunday there are 3 div. 2 only rounds in a row (excluding the ones before), and only 1 div. 1 round next Thursday and then who knows when will the next one take place...
(I'm not complaining, just saying that div. 1 rounds are harder to make and so appear with different frequencies).
I really hope I sounded clearer this time
Then what would be the point for a purple participate in div2 contests? You are sending mixed signals..
The point would be becoming a high purple instead of a low purple
Rank 1 and rank 20 have same rating change... It seems unfair and participants will waiting for the end of contest instead of solving div2F. How about give back the rating more than upper bound in the next contest?
I'm for capping it at +-150. 200 is too much, 100 means that e.g. in the last round, I shouldn't have tried to solve D-F at all.
Your last round was Division 1. The upper bound for rating change is $$$u+100$$$, where $$$u$$$ is the upper bound of registration for the contest. Division 1 doesn't have any upper bound.
I missed that in the OP. Then the problem appears with combined rounds, which always gave bigger rating changes due to the greater number of participants. 100 in that case seems fine, I'd still go with 150 for the maximum abs. change.
codeforces is becoming like codechefWhy not use Atcoder rating formula for Codeforces ratings?
Why shouldn't someone solved 2900 rated problem have just more than 2300 rating after div2?
I mean if they get the same performance in div1 , they will be more than 2300. But why they should't be more than 2200 after div2?
I don't know pros and cons to be kept in mind while dealing with rating systems. So ignore if it is bad idea. What about you can have maximum change within your colour ? Like if you are cyan you can go least to the starting of cyan and most to the end of cyan in one round and only cross over in next round.
Or we can say that each colour have their own rating system that deals with that range problems more effectively rather than making the higher rated people suffer for the problems CF face due to lower rated users.
I think there is opportunity to find a better solution by more carefully considering the problem(s) that such a change is meant to address. Judging by the comments that I've read, it seems that there are two main concerns that users have:
I find concern #2 to be an uncompelling reason to adjust the rating calculation method whether or not its hypothesis is true. If It is easier to perform at X-rating-level by competing against opponents at the somewhat lower Y-rating-level, that means that the distribution of user ratings is not in equilibrium and users near skill-level-X should statistically tend to become more separated in ratings from those near skill-level-Y over time, no matter what ad-hoc adjustment is made to the rating calculation system. The only ways to avoid this type of inflation are:
(a) Introduce a matching deflation for the less-skilled users or
(b) To change the scale factor of $$$\frac{\log{10}}{400}$$$ used by the rating calculation.
Option (b) may cause artifacts and temporary weirdness for users far away from the contentious Div1/Div2 line, in addition to being unappealing due to changing the scale away from a standard value that many users expect. Option (a) can be achieved in several ways, among which the more tasteless may discourage users who see their graphs go down over time or generate negative reactions, but is the one that appeals more to me, if concern #2 is to be addressed at all.
Concern #1 has real merit. The proposed patch of preventing users from increasing in rating to more than a certain level is probably the simplest way to try to manage this concern, but it would be better (if practical) to achieve the same goal by a means more directly related to the problem: Give head-to-head comparisons between users both rated close to the cap for a round less weight than those involving a user far from the rating cap. Obviously this has more implementation overhead than capping the achieved rating, since the weighted place achieved by a user for rating calculations would then depend on that user's rating, but I think this is well within reason, very possibly requiring fewer than ten lines of code. As far as the weight function itself goes, I would be drawn to something like
There should be a bottom cap!!
As if someone performs their best then they deserve their rating increase more than 100 also(if possible).
But if some mishap happens during a contests because of any possible reason then their rating should not decrease by more than 100..
That would just lead to inflation.
YES this will be like Atcoder!! Also it would incentive people to do more contests.
I think such a system of putting caps will lead to rating inflation on the platform. Such a system can be applied to newer participants, but then we have the newer rules for starting from newbie which are sufficient I think. Increasing the number of rounds required to be called a trusted participant is better I think for lower-rated participants like us
Four years ago I analyzed the data of Codeforces, and did my best to invent a rating system designed specifically for competitive programming. It's similar to Elo rating system like CF rating system (in a sense that all these systems are based on the logistic distribution), but also contains various modifications required for competitive programming. I already know how to handle the problems you suggested (too high rating by participating in a lower division, avoid extreme rating falls, beginners' ratings gradually decrease, etc.) It also contains some parameters, and I know how to set those parameters to fit your philosophy (you like stable ratings or wildly shaking ratings, etc).
Please contact me if you think AtCoder system is good — I'm willing to cooperate!
Thanks, I'll contact you to discuss it!
Looks like we independently went after some of the same problems: https://arxiv.org/abs/2101.00400 (on Codeforces history, the distribution looks like this)
Do you plan to publish yours soon? I'm curious.
I think instead of a bottom cap, every user can be given 3(maybe) chances a year to cancel the rate change in a contest.
Users can choose to use the chance if they run into unpredicted emergencies, or if their performance is well under expected. By setting an annual limit, this mechanism will not be abused.
Knowing that the rating won't fall by more than 100 may reduce the pressure felt in contest. I guess pressure handling is an important thing in contests. Maybe we will miss that now. Also if the rating is not supposed to increase more than 100, will miss the excitement of getting 200+ or something like that.
This system looks good only for upsets, if anyone accidentally has a poor performance, this system will save him/her. But life is good with ups and downs , rather than a plain curve.
Read this The cap is on the maximum rating and not on maximum delta
Oh, I thought it was on the maximum delta as it said "rating won't be higher than u+100" . If its on maximum rating, seems good then.
I think lower limit of 100 is okay. It helps not to be discouraged. But I guess upper limit of 100 is pretty short. It's a good positive delta but not great. It may discourage someone from solving more problems(as Xellos says) or hacking. Great rating change gives more encouragement, more confidence. It may also a memorable moment for someone. I think 150-200 positive delta is decent.
The limit is not on the maximum delta but rather on the max rating after a round so for example after a div 2 round no one will have a rating of more than 2200 but someone rated 1800 before contest can have +300 delta.
Thank you for pointing out my mistake.
Maybe it's a good idea. But, there is one thing. Some participants solve 1900-2000 rated problems during the contest. Would that be fair to only increase the rating to 1700 in that case?? Now, if someone can solve 1343E - Weights Distributing during contest, he surely deserves to be rated more than 1700.
And this kind of thing would be worse for div2 participants. Like someone solved a 2500 rated problem, then find out he can be at most 2200 in that round.
Is it really a problem when a contestant is overrated after one contest? Wouldn't it be fixed on later ones?
the cap should be like if a person eligible in div 3 participates in div3, he cannot cross 1699 rating and so would be good
The bottom cap idea is really really good.
Bottom cap for an LGM with bad performance is just wrong. And it's not that much different between -100 and -150, so I don't think it will make a difference.
"for rounds where there is already an upper rating limit at registration"
I don't think that Div. 1 rounds apply?
Shish, you are absolutely right. But still, sometimes I think I deserved -120 (and maybe even more).
MikeMirzayanov, do you have any plan to increase the lower bound for master to 2200? Even with rating caps, it's too easy to become master in a not-so-balanced div2 round, especially since so-called standardard point distribution of Div2 treats Div2D and Div2E almost similarly, whereas Div1B and Div1C usually have better difficulty gap and point gap. Speed matters more in Div2 than Div1, as it's tough to take risks of solving harder problem skipping easier with such point distribution.
Please consider creating a new group for 2100-2199, who won't be rated in Div2 contests, (maybe color them violet with orange initial).
I agree with that lots of people became master easily, but adding another color?? Then it will be like this :
2100 — 2199 => International Candidate Master
2200 — 2299 => Master
2300 — 2399 => International Master
Is it interesting? I don't think so.
Then we can remove "International Master", or we can shift IM, GM and IGM by +100, but I don't think that's necessary as percentage of GM+ is already low enough.
3-5 years ago, it used to be like this:
1900-2199 => CM
2200-2299 => Master
2300-2399 => IM
We can simply go back to that, just don't let 2100+ be rated in Div2.
Don't you think it'll just shift top Div2 participants even further (closer to GM)?
Why? Div2 will be rated for same group of people: <2100. Rating system will be same, just color of 2100-2199 will change.
And why would you want a new color? I mean, nothing would really change, just the fact that you'll differentiate newcoming masters. I'm actually waiting for a rating revolution to come soon on codeforces, so we should all stay tuned.
good idea , once i got -111 due to technical problems