nicola's blog

Problem Recommender System For Codeforces Users
By nicola, 10 years ago, In English

Hi Codeforces!

I am really pleased to introduce My research project called PROPER

  • This is an web application that recommend you some problems that have adequate difficulty to practice. After you solve the recommended problems you may give PROPER your feedback that the problem is Hard/Adequate/Easy. This feedback will be used to make better recommender system.

  • System automatically collect status data from CodeforcesAPI, and calculate adequate problems to user personally, Problem is recommended based on your status and other coder's status interactively.

  • We use Matrix Factorization algorithm which is one of the famous recommender system model.

the PROPER address is: http://recommender.cf

I've evaluated this system in statistical way and it was quite good.

So I'd like to survey about this system for actual users.

I hope it works well for users and this system helps you practice better.

Thank you for reading!!

  • Vote: I like it
  • +106
  • Vote: I do not like it

Comments Comments (35)
| Write comment?
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Interesting idea, but I'm not sure it's a good way to practice. Intuitively, if I solved a lot of problems on a specific subject (or of specific difficulty), then I'm already good at it and it doesn't make sense to offer me even more similar problems.

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    You may think it is recommending simmilar problems that the coder has already solved.

    If you thought so, it is not.

    We thought adequate problems for a coder means the hardest problems that the coder can solve.

    So our algorithm is following below steps.

    1. See what was the hardest problems in contests that you have attended. If you solved a,b,c,d problems in a contest, the hardest problem is d ( we assumed problems in a contest are provided in order by difficulty. )

    2. Based on these, we applied recommendation model.

    Sorry for brief explanation. Thanks to say it is interesting.

    → Reply
»
10 years ago, # |
  Vote: I like it +60 Vote: I do not like it

It recommends me my own problem, lol

→ Reply
  • »
    »
    10 years ago, # ^ |
    ← Rev. 2   Vote: I like it +4 Vote: I do not like it

    Does it recommend problems that you are author?

    I didn't exclude them when I write code. sorry.

    I think it is not so common case. So I will fix later.

    Thanks to let me know

    → Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

nice idea, i hope it will be available for people who compete in div.2 too.

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    Thanks for your interest. I hope so, but this is a kind of testing with small sample(nearly only 5000 users). If many users are feeling it is not so good to practice, I should find the reason and improve something.

    I will apply this for every users when this system achieve good result.

    → Reply
    • »
      »
      »
      10 years ago, # ^ |
      ← Rev. 2   Vote: I like it 0 Vote: I do not like it

      I don't see how Div1 only users would account for ~5000 users when people registering for a Div1 round is usually around 1500. Also ain't you committing a crucial statistical error here, because Div1 users would surely rate a question easy as compared to someone who is from Div2 and would rate it moderate/hard. Correct me if I am missing my mark!

      → Reply
      • »
        »
        »
        »
        10 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Well.. It was nearly 5000 users when I crawled contest standing data that contest name contains "Div. 1"

        See following link. http://mirror.codeforces.com/ratings/page/45

        There are nearly 9000 users have rating of purple color. And we can see almost of them didn't participated in contest of "Div. 1"

        I think it is the reason of 5000 users.

        And "Easy, Adequate, Hard" information are not reflecting this system now in any way.

        Just collecting now.

        → Reply
»
10 years ago, # |
  Vote: I like it +2 Vote: I do not like it

It recommends me super easy problems.

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Yes it does.. I was reasoning but couldn't get any idea. I think you deserve to get B~C problems though this system is offering A~B problems. Thanks to notify.

    → Reply
»
10 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

It asked me for my handle and when I entered "RodionGork" it show me no problems but instead asked to sign in. Funny :)

UPD Nevertheless idea is cool and interesting, though its implementation could be tricky and far from bringing really useful results :)

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    You may see Readme page http://recommender.cf/readme

    It said that 'It works for only users who have attended Div 1 contest.'

    → Reply
    • »
      »
      »
      10 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Ah, sorry for being that inattentive.

      Though it may tell me "You did not participate in Div1 contests" instead of offering to log in :)

      → Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Funny :)

    That is because you didn't attend any div. 1 contest. there's no data.

    Actually I have exactly same problem with you. :)

    → Reply
»
10 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

Could you add links to russian versions of problems?

→ Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

feels like about right difficulty ;)

good job creating such a convenient system!!

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    I was relived of hearing it works well for you.

    → Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Firstly, I want to thank you, nicola, for taking out time to build this. It sure does look interesting. Would love to use it.

Though, I do have a query: what happens when I solve a problem which is recommended by PROPER? For example, I solved 185A - Plant as a recommended problem. I signed out, and signed in again. And it is still displayed in the list of problems to be solved when I sign in.

Shouldn't it be removed? Or, an alternate problem be added in place of it? Or, am I missing something here?

Also, "If you solve the problem and sign in, you will meet a feedback message downside of address line(Easy/Adequate/Hard)." — where does this feedback message come?

→ Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Screenshot

Why recommender doesn't recommend problems for me?

→ Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Hi

the system doesn't recommend any problem when I first use the application.

why ?

→ Reply
»
10 years ago, # |
  Vote: I like it +11 Vote: I do not like it

It doesn't work if your name has a "." in it.

→ Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It recommends some problems that I already solved. And some other problems are too easy :(

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    It recommends some problems that I already solved. Yes, for me too. It is not good.

    → Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It didn't show me any problems when I wrote my handle, so I tried with my previous handle (b4 New Year) and it worked. I guess that is something to be fixed. Overall, nice work.

→ Reply
»
10 years ago, # |
  Vote: I like it +5 Vote: I do not like it

Wow, I am insulted; It recommends me 15 "A" problems :(

→ Reply
  • »
    »
    10 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    It looks like you've never solved a B problem on a div1 round; so, it's normal.

    → Reply
  • »
    »
    10 years ago, # ^ |
    ← Rev. 3   Vote: I like it 0 Vote: I do not like it

    Me too, then I recognized that I've only solved A and C in div1 contest, never got B solved, I guess he hasn't thought of that :D

    → Reply
»
10 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

It refers me problems of Div1 A / Div2 C that I have already solved. There is no consideration that I solved that Div1A while competing in it's respective Div2 contest or in practice.

For example 477A - Dreamoon and Sums of Div1 is same as 476C - Dreamoon and Sums of Div2 and I solved it in Div2 practice and not in Div1.

btw Nice initiative :)

→ Reply
»
10 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

It does not recommend me any problems whereas I'm quite sure I have not solved all the problems here on Codeforces.

→ Reply
»
10 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I loved it but I need to send the auth code every time i need to sign in.It neither remains signed in.

→ Reply
»
10 years ago, # |
  Vote: I like it +4 Vote: I do not like it

Why it is down? :(

→ Reply