crap_the_coder's blog

UpCode — Upload all your accepted solutions to GitHub, hassle-free
By crap_the_coder, 2 years ago, In English

Do you have many accepted solutions in CodeForces/CodeChef/Atcoder, but you can't share a convenient link to your GitHub to flex your submissions? Maybe it would take too long to upload all the files because you 1000+ accepted submissions?

Introducing UpCode, the program that uploads all your accepted solutions to your GitHub. Just enter your username and a GitHub API key and it will automatically upload all your accepted submissions from CodeForces/CodeChef/Atcoder to a GitHub repository.

For example, here's my repository that was generated by UpCode: https://github.com/crapthecoder/Competitive-Programming

Disclaimer

To use UpCode, perform the following steps:

  • Make sure python is installed, the project will not work otherwise.
  • Generate a GitHub API key from here. Make sure the repo section is checked.

  • Download the project from GitHub and extract it.
  • Navigate to the UpCode-master folder, replace the path with cmd, and press enter.

  • Run pip install -r requirements.txt
  • Run python3 main.py. If it doesn't work, try python main.py
  • Enter credentials as prompted

If all goes well, then you should see some lines like CodeForces/GNU C++20 (64)/1234E | Hello World/123456789.cpp CREATED. After that, you should be able to see the updates in your GitHub repository.

NOTE:

  • If you are running the CodeForces scraper, a Chrome browser will open. DO NOT close it or interact with it. You can safely ignore it and do something else.

  • DO NOT run multiple instances of the program for the same repository. Also, do not turn off or sleep the PC when the code is executing.

  • It is recommended to not run the code during a CodeForces contest, as a popup notification could possibly stop the code from scraping properly. Please wait until the contest is over.

  • If there are some errors, please ignore them as long as the entire program doesn't stop. The code will go through the failed submissions again to make sure they get added.

  • If the code stops executing, please try downloading the latest update again, as there have been many major updates/bug fixes thanks to the community. If there are still any issues, let me know in the comments and I will make sure to help you out!

  • CodeForces API is sometimes inconsistent and might not always be available, in which case you will be notified by the program. In that case, please wait for a while and try again.

  • Due to the Atcoder API only updating every week, there won't be any changes to the Atcoder folder in the repository if you run UpCode multiple times before it updates.

Special thanks to:

  • advik123 for being my mentor in this project.
  • ghosty for agreeing to test if he was mentioned in the blog. So here you go nerd. (Also for helping me make this blog)
  • aryan12 for existing and sharing his wisdom to the world.
  • parth_kabra for finding like 20 bugs and testing my project.
  • evenvalue for not bullying me (he bullied me into saying this)

Thank you to everyone for pointing out bugs, or suggesting some new feature. The CodeForces scraper was very hard to create, and still not as efficient as I would like it to be. But it should work, though it might take some time. If there are any bugs, please do let me know, as I want to make sure this project works for everyone :D

TL;DR:

Go to this link, follow the instructions, and bam. Your accepted submissions from CodeForces/CodeChef/Atcoder will get uploaded to GitHub.

  • Vote: I like it
  • +107
  • Vote: I do not like it

Comments Comments (45)
| Write comment?
»
2 years ago, # |
  Vote: I like it +6 Vote: I do not like it

Sriv bhaiya OTZ

→ Reply
»
2 years ago, # |
  Vote: I like it +6 Vote: I do not like it

orz

→ Reply
»
2 years ago, # |
  Vote: I like it +6 Vote: I do not like it

orz sir

→ Reply
»
2 years ago, # |
← Rev. 2   Vote: I like it +9 Vote: I do not like it

Looks pretty cool. Also I am sure, existence of aryan12 was very useful.

→ Reply
»
2 years ago, # |
  Vote: I like it +3 Vote: I do not like it

I also made a scrapper which will download your solutions. It works on Atcoder,Codeforces & Codechef. Link : https://github.com/aryan57/cp-crawler

→ Reply
  • »
    »
    2 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    Hey! I would like to know if your CodeForces scraper works reliably. In my code, after making around 50 requests, CodeForces starts returning HTML that does not contain the solution. Does CF do the same to your code?

    EDIT: Found the cause of the issue:

    If anyone has any information on how to properly make requests such that both CF and the scraper are happy, please let me know!

    → Reply
»
2 years ago, # |
  Vote: I like it +11 Vote: I do not like it

Pretty nice tool. Can you add support for Atcoder too.

→ Reply
»
2 years ago, # |
← Rev. 2   Vote: I like it +6 Vote: I do not like it

Thank you for this tool!

Is it possible for you to add the problem link in a text file in each problem's folder? And maybe name the problem folders with the problems' names and codes both? If not for other judges, at least for codeforces?

→ Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    Thanks for the great suggestion! I will add that soon.

    → Reply
    • »
      »
      »
      2 years ago, # ^ |
      ← Rev. 2   Vote: I like it 0 Vote: I do not like it

      Maybe you could just insert a comment at the top of the code file with the problem link, and mention that the comment was auto-generated by UpCode. Not sure how feasible this would be though.

      You could start with just C++, maybe.

      → Reply
      • »
        »
        »
        »
        2 years ago, # ^ |
        ← Rev. 2   Vote: I like it 0 Vote: I do not like it

        I have made it so that a text file will be created, but in the process I found another issue. But a temporary fix has increased the time the program takes. If you are fine with the code taking a while, you can go ahead and use it now.

        → Reply
»
2 years ago, # |
  Vote: I like it +8 Vote: I do not like it

Very nice tool indeed!

I wrote a similar scraper for cses, if you were interested in it. It's currently on my github.

→ Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Amazing tool. When i was trying to download the files i faced an issue, can you tell how do i resolve it ?

    Spoiler
    → Reply
    • »
      »
      »
      2 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Glad you've found it useful!

      I've changed something and now it should work. However I don't have this issue and can't tell whether it's fixed or not.

      If it still doesn't work, check out this question on stackoverflow.

      → Reply
»
2 years ago, # |
  Vote: I like it +1 Vote: I do not like it

I am getting this error for codeforces solutions.

→ Reply
  • »
    »
    2 years ago, # ^ |
    ← Rev. 2   Vote: I like it +6 Vote: I do not like it

    I am sorry, that's because CodeForces lets the code request atmost 80 times before not letting it make any more requests for a while. There was another issue.

    But no worries, as the code makes sure to go through the failed solutions once again to get it uploaded.

    I am currently working on a way to make it so that this error will not appear any more.

    → Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    The issue should be (partially) fixed now. I hope to make more improvements in the future, but the updated code should hopefully work better for you.

    → Reply
»
2 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Is it me or all codes in your repository are empty?

→ Reply
»
2 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Very useful, but I stopped saving solutions as files instead just having "A.cpp" and clear it after every contest many red coders do the same thing I noticed

→ Reply
»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It is the best thing...

well I know u from TLE

→ Reply
»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

can you make the extension upload non-uploaded solutions only? and btw thank you !

→ Reply
  • »
    »
    2 years ago, # ^ |
    ← Rev. 2   Vote: I like it +1 Vote: I do not like it

    The issue is that to know which one to avoid, I have to scrape it first ;_;
    For example, to avoid some CodeChef solution, I first need to know its link.

    → Reply
»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Hello,

I am facing an issue where not all the problems are uploaded (less than 50%) and when trying to scrape on Code Chef I get ERROR:root:'NoneType' object is not subscriptable FOR None

→ Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I am very sorry, I somehow missed your comment. The issue has been fixed now though, so you can use it now!

    → Reply
»
2 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Bro does it automatically upload the code to github as soon as the code is accepted or we have to run the main.py whenever we want to upload the codes? BTW This Scraper is pretty awesome. Thanks.

→ Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    You need to run main.py everytime you want to upload the codes. It doesn't do anything automatically.

    → Reply
»
11 months ago, # |
  Vote: I like it 0 Vote: I do not like it

crap_the_coder I am getting this error while running for codeforces API. Image

→ Reply
  • »
    »
    11 months ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Ah I am sorry, the project has pretty much been archived right now, so a lot of errors will pop up. The websites keep changing a lot and I can't maintain it properly right now. I might do so in the future though.

    → Reply
»
11 months ago, # |
  Vote: I like it 0 Vote: I do not like it

Thanks

→ Reply
»
8 months ago, # |
  Vote: I like it 0 Vote: I do not like it

Has it stopped working for Codechef? Btw working great for CF orz

→ Reply
»
3 weeks ago, # |
  Vote: I like it 0 Vote: I do not like it

It doesn't work for codeforces , I keep getting this error: ValueError: There is no such driver by url https://chromedriver.storage.googleapis.com/LATEST_RELEASE_129.0.6668 Also, I think codeforces doesn't allow you to see submissions unless you are logged in

→ Reply
  • »
    »
    3 weeks ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Same Issue, Please let me know if you find a fix or a workaround!

    → Reply
    • »
      »
      »
      2 weeks ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      I fixed the issue but still codeforces is blocking me. I dont think it is possible anymore to use a web scraper because codeforces implimented a captcha for each request.

      → Reply