Hi beautiful people of Codeforces! Hope you'd all be doing fine and keeping safe.↵
↵
I'd like to start by mentioning that it's been a while since I graduated from university ever since which I haven't completely been able to indulge myself in competitive coding in the same way I did back in my university. But I sure haven't forgotten those days.↵
I had some time off from work recently and decided to spend my time giving back to the community from where I have learned so much. ↵
↵
In the thought of so, I've built this tool which I like to call [`Harwest`](https://github.com/nileshsah/harwest-tool) (a play on the word Harvest since the package name wasn't available anymore :'D)↵
It's one of those tools which I would have loved to have during the time when I used to practice rigorously on this platform.↵
↵
**Harwest** allows you to manage all the submissions that you've made on Codeforces, or on other OJs (WIP) into one single Git repository which you can then push to your personal Github / BitBucket / Gitlab or any other SCM platform of your choice.↵
↵
Harwest completely automates this entire process with minimal effort from your side of answering just a few basic questions, and even goes to the limit of pushing the Git repository to the remote (Github/BitBucket..) for you!↵
↵
Here's a sample repository built solely using Harwest: https://github.com/nileshsah/harwest-sample↵
↵
↵
↵
The project directory structure is as follows:↵
↵
~~~~~↵
```↵
nellex@HQ:~/accepted$ tree↵
.↵
├── codeforces↵
│ ├── <contest-id>↵
│ │ └── <problem-level>↵
│ └── 592↵
│ └── A.cpp↵
├── README.md↵
├── submissions.json↵
└── .git↵
```↵
~~~~~↵
↵
↵
↵
Harwest creates a local git repository for you, scrapes all your submissions directly from Codeforces, maintains them in a nicely organized way, and commits them to the repository with the actual date of your submission!↵
↵
Now what the last statement enables us to do is to maintain a complete contribution graph on GitHub (or others) accurate to the time when you actually made the submission and not when you made the commit. Yes! It means complete tracking of your submissions on a day by day basis. Keep track of the number of days you've been lazy for, the number of problems you solved on a given day, and maintain a complete **streak** of daily practice! Heck, let this streak be your new motivation to practice continuously to become a better version of yourself every single day↵
↵
↵
↵
↵
I've detailed the entire process of configuring Harwest on your local computer with an example in [https://github.com/nileshsah/harwest-tool#installation](https://github.com/nileshsah/harwest-tool#installation) Let me know if something needs more clarification.↵
↵
It's built using Python so you would require `PIP` with `Python` version 3.5 or above to install the package which you might have to set up in case you haven't used it before. It's a great package manager which I'd recommend learning a bit about nonetheless since it'll come in handy in the future.↵
↵
Along with PIP, you'd also need `git` on your computer which would come pre-installed if you're using Linux or MacOS though might would have to work your way up in case you're using windows. I'd also strongly encourage you to learn about Git since it's something that is heavily used in the software industry and early exposure to it would sure make your life a lot easier in the future. There are numerous documents on the internet which can help you get started on Git, one of them is [https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute-beginners-86fa1d32ff71/](https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute-beginners-86fa1d32ff71/) ↵
Though For operating Harwest, you'd require minimal to no experience with Git.↵
↵
The first time Harwest runs, it'll scrape all your submissions starting from the first page till the last which might be a bit time-consuming if you had made a lot of submissions. If you don't want to parse the results till the very end, you can stop processing by hitting \<Ctrl\> + C. The next runs of Harwest will no more try to scrape the results from the old pages and instead will push your new changes directly. You can then make use of the `harwest codeforces --start-page X` feature to restart scraping from the page where you left off.↵
↵
I haven't got a chance to extensively test Harwest much and hence the term [Beta]. It seems to be working fine on MacOS and Ubuntu but I guess the auto push feature is causing a bit of an issue on Windows especially if you're using _GitHub for Windows integration_ Do keep me posted if you face any other issues and I'd be happy to fix it at once. ↵
↵
As for a workaround for the Windows user, if you want to leverage the automated push feature then specify the remote URL for your repository when asked as [https://username:password@github.com/username/repository.git](https://username:password@github.com/username/repository.git) to bypass the login step during the push. Alternatively, you can always leave it empty and do a push from the generated repository by yourself.↵
↵
Nonetheless, I had a fun time building this tool and I really hope it would be helpful for you. Contributions to the project are more than welcome!↵
↵
## TL;DR↵
↵
Automate the management of your submissions on GitHub with <https://github.com/nileshsah/harwest-tool> to get something like <https://github.com/nileshsah/harwest-sample>↵
↵
---↵
↵
**Update <2020-12-27>: Support for AtCoder is now available!**↵
↵
Harwest now supports **AtCoder**! Massive thanks to [user:Creations,2020-12-27] for leading the effort of finding the ideal way to integrate with the platform. Not to forget the crucial contributions from [user:I_love_Hoang_Yen,2020-12-27], [user:YouKn0wWho,2020-12-27], and others for fixing some major performance issues and making Harwest more stable. Much appreciated.↵
↵
The sample submissions repository <https://github.com/nileshsah/harwest-sample> has been updated to reflect what new changes this update brings. In order to switch to this release, please upgrade Harwest by running:↵
↵
~~~~~↵
pip install --upgrade harwest↵
~~~~~↵
↵
Ensure that you're on version `harwest-0.3.1`, The new release is backward compatible and will not break any of your existing harvested submissions. After switching to the new version, you can then harvest submissions from AtCoder by running↵
↵
~~~~~↵
harwest atcoder↵
~~~~~↵
↵
In addition to the new integration with AtCoder, we have introduced `--full-scan` capability on platforms to avoid the pain of dealing with re-starting harwest from the page where it stopped processing due to an error. It can be triggered as:↵
↵
~~~~~↵
harwest codeforces --full-scan↵
OR↵
harwest atcoder --full-scan↵
~~~~~↵
↵
Please do let me know if you face any difficulties in the process, would be happy to help at once.↵
↵
Happy Harwesting! :)↵
↵
↵
↵
I'd like to start by mentioning that it's been a while since I graduated from university ever since which I haven't completely been able to indulge myself in competitive coding in the same way I did back in my university. But I sure haven't forgotten those days.↵
I had some time off from work recently and decided to spend my time giving back to the community from where I have learned so much. ↵
↵
In the thought of so, I've built this tool which I like to call [`Harwest`](https://github.com/nileshsah/harwest-tool) (a play on the word Harvest since the package name wasn't available anymore :'D)↵
It's one of those tools which I would have loved to have during the time when I used to practice rigorously on this platform.↵
↵
**Harwest** allows you to manage all the submissions that you've made on Codeforces, or on other OJs (WIP) into one single Git repository which you can then push to your personal Github / BitBucket / Gitlab or any other SCM platform of your choice.↵
↵
Harwest completely automates this entire process with minimal effort from your side of answering just a few basic questions, and even goes to the limit of pushing the Git repository to the remote (Github/BitBucket..) for you!↵
↵
Here's a sample repository built solely using Harwest: https://github.com/nileshsah/harwest-sample↵
↵
↵
↵
The project directory structure is as follows:↵
↵
~~~~~↵
```↵
nellex@HQ:~/accepted$ tree↵
.↵
├── codeforces↵
│ ├── <contest-id>↵
│ │ └── <problem-level>↵
│ └── 592↵
│ └── A.cpp↵
├── README.md↵
├── submissions.json↵
└── .git↵
```↵
~~~~~↵
↵
↵
↵
Harwest creates a local git repository for you, scrapes all your submissions directly from Codeforces, maintains them in a nicely organized way, and commits them to the repository with the actual date of your submission!↵
↵
Now what the last statement enables us to do is to maintain a complete contribution graph on GitHub (or others) accurate to the time when you actually made the submission and not when you made the commit. Yes! It means complete tracking of your submissions on a day by day basis. Keep track of the number of days you've been lazy for, the number of problems you solved on a given day, and maintain a complete **streak** of daily practice! Heck, let this streak be your new motivation to practice continuously to become a better version of yourself every single day↵
↵
↵
↵
↵
I've detailed the entire process of configuring Harwest on your local computer with an example in [https://github.com/nileshsah/harwest-tool#installation](https://github.com/nileshsah/harwest-tool#installation) Let me know if something needs more clarification.↵
↵
It's built using Python so you would require `PIP` with `Python` version 3.5 or above to install the package which you might have to set up in case you haven't used it before. It's a great package manager which I'd recommend learning a bit about nonetheless since it'll come in handy in the future.↵
↵
Along with PIP, you'd also need `git` on your computer which would come pre-installed if you're using Linux or MacOS though might would have to work your way up in case you're using windows. I'd also strongly encourage you to learn about Git since it's something that is heavily used in the software industry and early exposure to it would sure make your life a lot easier in the future. There are numerous documents on the internet which can help you get started on Git, one of them is [https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute-beginners-86fa1d32ff71/](https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute-beginners-86fa1d32ff71/) ↵
Though For operating Harwest, you'd require minimal to no experience with Git.↵
↵
The first time Harwest runs, it'll scrape all your submissions starting from the first page till the last which might be a bit time-consuming if you had made a lot of submissions. If you don't want to parse the results till the very end, you can stop processing by hitting \<Ctrl\> + C. The next runs of Harwest will no more try to scrape the results from the old pages and instead will push your new changes directly. You can then make use of the `harwest codeforces --start-page X` feature to restart scraping from the page where you left off.↵
↵
I haven't got a chance to extensively test Harwest much and hence the term [Beta]. It seems to be working fine on MacOS and Ubuntu but I guess the auto push feature is causing a bit of an issue on Windows especially if you're using _GitHub for Windows integration_ Do keep me posted if you face any other issues and I'd be happy to fix it at once. ↵
↵
As for a workaround for the Windows user, if you want to leverage the automated push feature then specify the remote URL for your repository when asked as [https://username:password@github.com/username/repository.git](https://username:password@github.com/username/repository.git) to bypass the login step during the push. Alternatively, you can always leave it empty and do a push from the generated repository by yourself.↵
↵
Nonetheless, I had a fun time building this tool and I really hope it would be helpful for you. Contributions to the project are more than welcome!↵
↵
## TL;DR↵
↵
Automate the management of your submissions on GitHub with <https://github.com/nileshsah/harwest-tool> to get something like <https://github.com/nileshsah/harwest-sample>↵
↵
---↵
↵
**Update <2020-12-27>: Support for AtCoder is now available!**↵
↵
Harwest now supports **AtCoder**! Massive thanks to [user:Creations,2020-12-27] for leading the effort of finding the ideal way to integrate with the platform. Not to forget the crucial contributions from [user:I_love_Hoang_Yen,2020-12-27], [user:YouKn0wWho,2020-12-27], and others for fixing some major performance issues and making Harwest more stable. Much appreciated.↵
↵
The sample submissions repository <https://github.com/nileshsah/harwest-sample> has been updated to reflect what new changes this update brings. In order to switch to this release, please upgrade Harwest by running:↵
↵
~~~~~↵
pip install --upgrade harwest↵
~~~~~↵
↵
Ensure that you're on version `harwest-0.3.1`, The new release is backward compatible and will not break any of your existing harvested submissions. After switching to the new version, you can then harvest submissions from AtCoder by running↵
↵
~~~~~↵
harwest atcoder↵
~~~~~↵
↵
In addition to the new integration with AtCoder, we have introduced `--full-scan` capability on platforms to avoid the pain of dealing with re-starting harwest from the page where it stopped processing due to an error. It can be triggered as:↵
↵
~~~~~↵
harwest codeforces --full-scan↵
OR↵
harwest atcoder --full-scan↵
~~~~~↵
↵
Please do let me know if you face any difficulties in the process, would be happy to help at once.↵
↵
Happy Harwesting! :)↵
↵
↵
