nellex's blog

[Beta] Harwest — Git wrap your submissions this Christmas!
By nellex, history, 4 years ago, In English

Hi beautiful people of Codeforces! Hope you'd all be doing fine and keeping safe.

I'd like to start by mentioning that it's been a while since I graduated from university ever since which I haven't completely been able to indulge myself in competitive coding in the same way I did back in my university. But I sure haven't forgotten those days. I had some time off from work recently and decided to spend my time giving back to the community from where I have learned so much.

In the thought of so, I've built this tool which I like to call Harwest (a play on the word Harvest since the package name wasn't available anymore :'D) It's one of those tools which I would have loved to have during the time when I used to practice rigorously on this platform.

Harwest allows you to manage all the submissions that you've made on Codeforces, or on other OJs (WIP) into one single Git repository which you can then push to your personal Github / BitBucket / Gitlab or any other SCM platform of your choice.

Harwest completely automates this entire process with minimal effort from your side of answering just a few basic questions, and even goes to the limit of pushing the Git repository to the remote (Github/BitBucket..) for you!

Here's a sample repository built solely using Harwest: https://github.com/nileshsah/harwest-sample

The project directory structure is as follows:

```
nellex@HQ:~/accepted$ tree
.
├── codeforces
│   ├── <contest-id>
│   │   └── <problem-level>
│   └── 592
│       └── A.cpp
├── README.md
├── submissions.json
└── .git
```

Harwest creates a local git repository for you, scrapes all your submissions directly from Codeforces, maintains them in a nicely organized way, and commits them to the repository with the actual date of your submission!

Now what the last statement enables us to do is to maintain a complete contribution graph on GitHub (or others) accurate to the time when you actually made the submission and not when you made the commit. Yes! It means complete tracking of your submissions on a day by day basis. Keep track of the number of days you've been lazy for, the number of problems you solved on a given day, and maintain a complete streak of daily practice! Heck, let this streak be your new motivation to practice continuously to become a better version of yourself every single day

I've detailed the entire process of configuring Harwest on your local computer with an example in https://github.com/nileshsah/harwest-tool#installation Let me know if something needs more clarification.

It's built using Python so you would require PIP with Python version 3.5 or above to install the package which you might have to set up in case you haven't used it before. It's a great package manager which I'd recommend learning a bit about nonetheless since it'll come in handy in the future.

Along with PIP, you'd also need git on your computer which would come pre-installed if you're using Linux or MacOS though might would have to work your way up in case you're using windows. I'd also strongly encourage you to learn about Git since it's something that is heavily used in the software industry and early exposure to it would sure make your life a lot easier in the future. There are numerous documents on the internet which can help you get started on Git, one of them is https://www.freecodecamp.org/news/an-introduction-to-git-for-absolute-beginners-86fa1d32ff71/ Though For operating Harwest, you'd require minimal to no experience with Git.

The first time Harwest runs, it'll scrape all your submissions starting from the first page till the last which might be a bit time-consuming if you had made a lot of submissions. If you don't want to parse the results till the very end, you can stop processing by hitting <Ctrl> + C. The next runs of Harwest will no more try to scrape the results from the old pages and instead will push your new changes directly. You can then make use of the harwest codeforces --start-page X feature to restart scraping from the page where you left off.

I haven't got a chance to extensively test Harwest much and hence the term [Beta]. It seems to be working fine on MacOS and Ubuntu but I guess the auto push feature is causing a bit of an issue on Windows especially if you're using GitHub for Windows integration Do keep me posted if you face any other issues and I'd be happy to fix it at once.

As for a workaround for the Windows user, if you want to leverage the automated push feature then specify the remote URL for your repository when asked as https://username:password@github.com/username/repository.git to bypass the login step during the push. Alternatively, you can always leave it empty and do a push from the generated repository by yourself.

Nonetheless, I had a fun time building this tool and I really hope it would be helpful for you. Contributions to the project are more than welcome!

TL;DR

Automate the management of your submissions on GitHub with https://github.com/nileshsah/harwest-tool to get something like https://github.com/nileshsah/harwest-sample

Update <2020-12-27>: Support for AtCoder is now available!

Harwest now supports AtCoder! Massive thanks to s59_60r for leading the effort of finding the ideal way to integrate with the platform. Not to forget the crucial contributions from I_love_Hoang_Yen, YouKn0wWho, and others for fixing some major performance issues and making Harwest more stable. Much appreciated.

The sample submissions repository https://github.com/nileshsah/harwest-sample has been updated to reflect what new changes this update brings. In order to switch to this release, please upgrade Harwest by running:

pip install --upgrade harwest

Ensure that you're on version harwest-0.3.1, The new release is backward compatible and will not break any of your existing harvested submissions. After switching to the new version, you can then harvest submissions from AtCoder by running

harwest atcoder

In addition to the new integration with AtCoder, we have introduced --full-scan capability on platforms to avoid the pain of dealing with re-starting harwest from the page where it stopped processing due to an error. It can be triggered as:

harwest codeforces --full-scan
OR
harwest atcoder --full-scan

Please do let me know if you face any difficulties in the process, would be happy to help at once.

Happy Harwesting! :)

  • Vote: I like it
  • +270
  • Vote: I do not like it

Comments Comments (57)
| Write comment?
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Auto comment: topic has been updated by nellex (previous revision, new revision, compare).

→ Reply
»
4 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

I installed Harwest but when I try to run the command.

It says harwest: command not found.

I’m using Ubuntu.

→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 3   Vote: I like it +8 Vote: I do not like it

    Hey sam571128! Thanks for trying. I'm guessing it might be so because the directory under which pip installed the package may not be present in your PATH. Could you try running the command pip3 uninstall harwest

    ```
nellex@HQ:~$ pip3 uninstall harwest
Found existing installation: harwest 0.2.0
Uninstalling harwest-0.2.0:
  Would remove:
    /home/nellex/.local/bin/harwest
    /home/nellex/.local/lib/python3.5/site-packages/harwest-0.2.0.dist-info/*
    /home/nellex/.local/lib/python3.5/site-packages/harwest/*
Proceed (y/n)? n
```

    > /home/nellex/.local/bin/harwest

    The path /home/nellex/.local/bin is something you'd like to register under $PATH as export PATH=/home/nellex/.local/bin:$PATH or you can execute the command directly as /home/nellex/.local/bin/harwest

    Let me know if that works.

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      Sorry for replying so late. But yea, it worked.

      → Reply
»
4 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

This is such a good idea! unfortunately I'm getting an error when running harwest codeforces (and putting in my handle):

equest.py", line 1352, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)>
→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 6   Vote: I like it 0 Vote: I do not like it

    Thanks a ton AkshajK I've updated the code to skip SSL verification. Can you try installing the new version by running pip3 install harwest==0.2.9 Let me know if the problem persists.

    → Reply
»
4 years ago, # |
← Rev. 3   Vote: I like it +19 Vote: I do not like it

Would like to extend my heartiest thanks to YouKn0wWho for helping me out in fixing some of the major encoding issues in Harwest on the windows platform. In case you were a windows user and have already tried Harwest then you might notice ^M characters in the generated submission files. If so, then I'd recommend deleting the locally created repository and starting over with harwest --init after updating to the latest version by running pip3 install harwest==0.2.9 Apologies for the inconvenience.

As for YouKn0wWho, the amount of hard work you've put in this year itself is simply remarkable! https://github.com/ShahjalalShohag/ProblemSolving

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I have installed it and pushed my submissions to github. Now if I make more submissions, how do I push them again so that it only takes the submissions which are not pushed yet?

→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 3   Vote: I like it 0 Vote: I do not like it

    Hey anshu_2761! Thanks for giving Harwest a try! Harwest keeps track of your past submissions and fetches only the ones which are new. Whenever you'd like to sync the submissions in your GitHub repository with that on Codeforces, just run harwest codeforces and it'll do the job for you. Will also push automatically if you have set it up with auto-push else you can manually cd into the local submissions directory and issue a git push origin master yourself.

    I believe this was your question right? Also, I'd recommend running a pip3 install harwest==0.2.9 again since I pushed in some new set of fixes which would be good to pull.

    → Reply
»
4 years ago, # |
  Vote: I like it +5 Vote: I do not like it

Nice tool!

However, the process was stopping a lot before finishing all pages, and I had to use the commnad harwest codeforces --start-page x multiple times, then I wrote this python snippet to do the job.

import subprocess
for i in range(300):
  subprocess.call(['harwest', 'codeforces', '--start-page', str(i)])
→ Reply
  • »
    »
    4 years ago, # ^ |
    ← Rev. 2   Vote: I like it +13 Vote: I do not like it

    Thanks Mohammad_Yasser! Nice hack I'd say! I would like to apologize that I didn't stress-test the tool enough to check its breaking limits. I_love_Hoang_Yen just helped me out in introducing a performance update (PR#3) which should address your issue a bit. Updating to the latest version of Harwest should help. Could you try executing the commands:

    ```
pip uninstall harwest
pip install harwest
```

    If it doesn't fetch the latest version of 0.2.10 then maybe you can do a force install using pip install harwest==0.2.10

    Let me know if that helps and thanks for trying Harwest out :)

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      Thank you!

      It worked fine. The update is a significant improvement.

      → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      It didn't work out for me. Like it does until page 9 and then when I check my repository, it has done only until some submission of page 8. I start again but it stops at page 9 again. If I start at page 10 instead, it does until some pages(14 or 15) and then again same problem occurs and then again after that. Can u help me out here? :( I would appreciate it

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Sorry to hear that nmnsharma007 and apologies for getting back to you late. I'm taking a look at what could be the possible issue behind it. Meanwhile, maybe you can use the hack which Mohammad_Yasser proposed of running the script: https://ideone.com/01Yc9C

        ^ It should just be a one-time overhead when you're setting up Harwest for the first time. But I'm on it to identify what could be going wrong for you. Hope that works.

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
          ← Rev. 2   Vote: I like it 0 Vote: I do not like it

          No problem at all. By the way,where should I put the script?Thanks for the help :) UPD: It worked. Thanks

          → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It is just me or this tool doesn't work with gym submissions?

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    Hey timreizin! Thanks for trying Harwest!

    You'd be correct that the first iteration of this tool does not support gym submissions or any other submission which is not public :/ The approach was taken to avoid having an extra overhead of requesting and maintaining the user's codeforces password. Though it sure is something that we're looking at including in the next iterations of the tool. In fact, it has already been reported here: https://github.com/nileshsah/harwest-tool/issues/7

    Will prioritize it in case more people would like to see this feature. Hope that's alright.

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      Thanks for fast reply. Thank you for this tool.

      → Reply
»
4 years ago, # |
  Vote: I like it +10 Vote: I do not like it

Thanks for this amazing tool!

Please add support for other platforms as well :)

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    Glad you liked it hellodummy! :) We're already working on an integration with AtCoder (all credits to s59_60r).

    Which other platforms would you like to see on Harwest?

    → Reply
»
4 years ago, # |
  Vote: I like it -10 Vote: I do not like it

https://www.stopstalk.com/dashboard something similar to yours.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +21 Vote: I do not like it

    Thanks for bringing this up daddy_puff! StopStalk is undoubtedly a wonderful tool to track your activity across different OJs. Huge respect for its creator.

    Harwest though serves a bit of a different space. It enriches the belongingness of your submissions by allowing you to host them on a platform of your choice rather than being present on a remote system.

    Harwest fosters the idea of taking immense pride in putting up your solutions on a platform like GitHub since it reflects on the amount of hard work you had put in to succeed early on in your career. As the years go by, you'll realize that all these other tools will fade away but it will only be your Git profile that sticks with you for a while. And I promise you, every time you look back, these contributions will bring back nothing but the good memories of the not so long past: https://github.com/ngthanhtrung23/CompetitiveProgramming

    → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Thanks for the amazing tool please also consider including codechef, atcoder

Bug(maybe): After scanning all pages it showing I sovled 319 but my actual solves are 371

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Thanks sainad! We'll be launching the support for AtCoder in a matter of few days and will then proceed to add integration for CodeChef. Contributions are always welcomed!

    The discrepancy that you mentioned might arise from the fact that some of your solutions might not be publicly available, especially gym submissions which Harwest at the moment does not scan. In order to support these private submissions, we'll have to enable login flow on Harwest which we might consider in the future if we have more bandwidth. Hope it is alright for now.

    → Reply
»
4 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

Auto comment: topic has been updated by nellex (previous revision, new revision, compare).

Support for AtCoder has been added! Please refer to the update section of the blog for further details.

→ Reply
»
4 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

GitHub
It seems that I can never get all my submissions (I should have around 600). I've tried harwest codeforces -f multiple times but all of them yield different results, yet none producing all my submissions. Looking at the commmit graph, it seems that harwest ignores a certain period of time, which has happened in all my previous attempts (the actual time is different though). Bug?

Edit: When I run something similar to this, I manage to get all my submissions.

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Hi skittles1412! Thanks for bringing this up. It sure is quite strange as to why you had to face this problem. I'm mostly guessing that it might be so because some of the requests to fetch the submissions are failing from the server-side. Ideally running a full-scan should have picked up these missed solutions. Hmm, I'll try to reproduce the issue at my end and check if I can locate it. Meanwhile, thanks for sharing a temporary fix.

    → Reply
»
4 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

I believe the atcoder harwest for java submissions does not work

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    Hi bleh0.5! Could you let me know what issue are you facing? Harwest should support Java submissions from AtCoder

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
      ← Rev. 3   Vote: I like it 0 Vote: I do not like it

      Error message:

      Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.496.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\users\bleh0\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\scripts\harwest.exe\__main__.py", line 7, in <module>
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\harwest.py", line 115, in main
    args.func(args)
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\harwest.py", line 74, in atcoder
    process_platform(args, "AtCoder", AtcoderWorkflow)
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\harwest.py", line 90, in process_platform
    workflow(configs).run(start_page_index=args.start_page, full_scan=full_scan)
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\lib\abstractworkflow.py", line 96, in run
    response.append(self.__add_submission(submission))
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\lib\abstractworkflow.py", line 29, in __add_submission
    solution_file_path = self.__get_solution_path(submission)
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\lib\abstractworkflow.py", line 60, in __get_solution_path
    lang_ext = config.get_language_extension(submission_lang)
  File "C:\Users\bleh0\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\harwest\lib\utils\config.py", line 49, in get_language_extension
    raise ValueError(
ValueError: ("Please provide correct file extension for the language 'Java8 (OpenJDK 1.8.0)' in", 'C:\\Users\\bleh0\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python39\\site-packages\\harwest\\lib\\resources\\language.json', 'file')

      I just assumed it had to do with being unable to do java because I recently switched languages

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +8 Vote: I do not like it

        Thanks for sharing the issue bleh0.5. The necessary file extension has been updated as part of the issue https://github.com/nileshsah/harwest-tool/issues/14

        Please consider upgrading your Harwest version to 0.3.2 to solve the issue at your end. You can do so by running pip3 install --upgrade harwest Please ensure it says — Successfully installed harwest-0.3.2

        Do let us know if it still doesn't work for you.

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Ok it works, thanks for the amazing tool!

          → Reply
»
4 years ago, # |
  Vote: I like it +5 Vote: I do not like it

nellex thank you so much for this awesome tool. Loved it.

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Traceback (most recent call last): File "c:\users\shubham\appdata\local\programs\python\python38\lib\runpy.py", line 193, in _run_module_as_main return run_code(code, main_globals, None, File "c:\users\shubham\appdata\local\programs\python\python38\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\Users\SHUBHAM\AppData\Local\Programs\Python\Python38\Scripts\harwest.exe__main.py", line 7, in File "c:\users\shubham\appdata\local\programs\python\python38\lib\site-packages\harwest\harwest.py", line 110, in main print("Hey there!", "\U0001F44B", File "c:\users\shubham\appdata\local\programs\python\python38\lib\encodings\cp1252.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_table)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f44b' in position 0: character maps to 

__  __                              __
 / / / /___ _______      _____  _____/ /_
/ /_/ / __ `/ ___/ | /| / / _ \/ ___/ __/

/ __ / /_/ / / | |/ |/ / ( ) /_ /_/ /_/__,_/_/ |__/|__/___/____/__/

==========================================

→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    This is a rather interesting case shubham_409, can you run the following commands on your terminal before running Harwest and let me know if it fixes the issue?

    set PYTHONIOENCODING=utf-8
set PYTHONLEGACYWINDOWSSTDIO=utf-8
    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      It worked like a charm using powershell but not by git bash. Thanks for making it. It would be great if it could support other ojs also.

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +3 Vote: I do not like it

        Glad that you liked it shubham_409. We're looking at on-boarding CodeChef next. Will let you know as soon as it's available :)

        → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Can you please tell what's the issue?

Spoiler
→ Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    Hey raja_n, thanks for reporting the issue. It's something that we're aware of hence the paragraph in blog about the workaround:


    As for a workaround for the Windows user, if you want to leverage the automated push feature then specify the remote URL for your repository when asked as https://username:password@github.com/username/repository.git to bypass the login step during the push. Alternatively, you can always leave it empty and do a push from the generated repository by yourself.

    It was reported on https://github.com/nileshsah/harwest-tool/issues/10 and it turns out that it might get fixed if you update your git version. Can you possibly try that and let me know if that fixes the issue for you as well? Thanks!

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      I have updated git version but now it's showing below error:

      Error
      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +1 Vote: I do not like it

        Hey raja_n, seems like there are already some files in your CP-Practice repository: https://github.com/Rajan-226/CP-Practice I'd recommend deleting this repository on GitHub and creating a new empty one without any README/Licence files. It should work then.

        Another alternative is to cd to your local directory of CP Practice and issue a git pull command to fetch the remote changes onto your local.

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Thanks for replying fast. Yes, it works.

          Are you planning to add any other platform like codechef, leetcode?

          → Reply
          • »
            »
            »
            »
            »
            »
            4 years ago, # ^ |
              Vote: I like it +1 Vote: I do not like it

            Adding other platforms would be great!

            → Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I was expecting that it will become green in github at the time when I submitted a correct solution in codeforces. Am I wrong somewhere ?

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

C:\python\Scripts>harwest Traceback (most recent call last): File "c:\python\lib\site-packages\git__init__.py", line 83, in refresh() File "c:\python\lib\site-packages\git__init__.py", line 73, in refresh if not Git.refresh(path=path): File "c:\python\lib\site-packages\git\cmd.py", line 278, in refresh raise ImportError(err) ImportError: Bad git executable. The git executable must be specified in one of the following ways: — be included in your $PATH — be set via $GIT_PYTHON_GIT_EXECUTABLE — explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the $GIT_PYTHON_REFRESH environment variable. Use one of the following values: — quiet|q|silence|s|none|n|0: for no warning or exception — warn|w|warning|1: for a printed warning — error|e|raise|r|2: for a raised exception

Example: export GIT_PYTHON_REFRESH=quiet

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\python\lib\runpy.py", line 193, in run_module_as_main return run_code(code, main_globals, None, File "c:\python\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\python\Scripts\harwest.exe__main.py", line 4, in File "c:\python\lib\site-packages\harwest\harwest.py", line 7, in from harwest.lib.codeforces.workflow import CodeforcesWorkflow File "c:\python\lib\site-packages\harwest\lib\codeforces\workflow.py", line 1, in from harwest.lib.abstractworkflow import AbstractWorkflow File "c:\python\lib\site-packages\harwest\lib\abstractworkflow.py", line 5, in from harwest.lib.utils.repository import Repository File "c:\python\lib\site-packages\harwest\lib\utils\repository.py", line 5, in from git import Repo, GitCommandError File "c:\python\lib\site-packages\git__init_.py", line 85, in raise ImportError('Failed to initialize: {0}'.format(exc)) ImportError: Failed to initialize: Bad git executable. The git executable must be specified in one of the following ways: — be included in your $PATH — be set via $GIT_PYTHON_GIT_EXECUTABLE — explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the $GIT_PYTHON_REFRESH environment variable. Use one of the following values: — quiet|q|silence|s|none|n|0: for no warning or exception — warn|w|warning|1: for a printed warning — error|e|raise|r|2: for a raised exception

Example: export GIT_PYTHON_REFRESH=quiet

Harwest is not working.What should i do??

→ Reply
»
4 years ago, # |
  Vote: I like it +3 Vote: I do not like it

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

orz

You're awesome great sir. This is absolutely wonderful.

Thanks a bunch! :)

→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Hi nellex

Currently, the recent GitHub projects use main as the master branch instead of the master. So when I use the feature to automatically push code to GitHub using harwest, it shows an error as my project doesn't have master as my branch.

cmdline: git push origin master stderr: 'error: src refspec master does not match any'

This is the error you get

My suggestion is to take a branch as an argument during the init process in which you want to push the code and push code to that branch automatically.

→ Reply
  • »
    »
    2 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I too am having the same error, but idk how I can -
    "take a branch as an argument during the init process in which you want to push the code and push code to that branch automatically."

    → Reply
»
3 years ago, # |
  Vote: I like it +3 Vote: I do not like it

I found this error massage anyone can help me?

Error Massage:

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\runpy.py", line 194, in _run_module_as_main return run_code(code, main_globals, None, File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\runpy.py", line 87, in run_code exec(code, run_globals) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\Scripts\harwest.exe__main.py", line 7, in File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\harwest\harwest.py", line 115, in main args.func(args) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\harwest\harwest.py", line 70, in codeforces process_platform(args, "Codeforces", CodeforcesWorkflow) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\harwest\harwest.py", line 90, in process_platform workflow(configs).run(start_page_index=args.start_page, full_scan=full_scan) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\harwest\lib\abstractworkflow.py", line 105, in run self.repository.push() File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\harwest\lib\utils\repository.py", line 52, in push self.git.push(*args) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\git\cmd.py", line 542, in return lambda *args, **kwargs: self._call_process(name, *args, **kwargs) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\git\cmd.py", line 1005, in _call_process return self.execute(call, **exec_kwargs) File "C:\Users\USER\AppData\Local\Programs\Python\Python38-32\lib\site-packages\git\cmd.py", line 822, in execute raise GitCommandError(command, status, stderr_value, stdout_value) git.exc.GitCommandError: Cmd('git') failed due to: exit code(1) cmdline: git push origin master stderr: 'To https://github.com/yousuf7902/Codeforces_Solved-Auto-Generate.git ! [rejected] master -> master (fetch first) error: failed to push some refs to 'https://github.com/yousuf7902/Codeforces_Solved-Auto-Generate.git' hint: Updates were rejected because the remote contains work that you do hint: not have locally. This is usually caused by another repository pushing hint: to the same ref. You may want to first integrate the remote changes hint: (e.g., 'git pull ...') before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details.'

→ Reply
»
2 years ago, # |
  Vote: I like it +3 Vote: I do not like it

nellex I am getting this error. I had scrolled through the blog comments and found that there was a suggestion to update harwest which I have done. 

Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Python310\Scripts\harwest.exe\__main__.py", line 7, in <module>
  File "C:\Python310\lib\site-packages\harwest\harwest.py", line 115, in main
    args.func(args)
  File "C:\Python310\lib\site-packages\harwest\harwest.py", line 70, in codeforces
    process_platform(args, "Codeforces", CodeforcesWorkflow)
  File "C:\Python310\lib\site-packages\harwest\harwest.py", line 90, in process_platform
    workflow(configs).run(start_page_index=args.start_page, full_scan=full_scan)
  File "C:\Python310\lib\site-packages\harwest\lib\abstractworkflow.py", line 96, in run
    response.append(self.__add_submission(submission))
  File "C:\Python310\lib\site-packages\harwest\lib\abstractworkflow.py", line 29, in __add_submission
    solution_file_path = self.__get_solution_path(submission)
  File "C:\Python310\lib\site-packages\harwest\lib\abstractworkflow.py", line 60, in __get_solution_path
    lang_ext = config.get_language_extension(submission_lang)
  File "C:\Python310\lib\site-packages\harwest\lib\utils\config.py", line 49, in get_language_extension
    raise ValueError(
ValueError: ("Please provide correct file extension for the language 'GNU C++20 (64)' in", 'C:\\Python310\\lib\\site-packages\\harwest\\lib\\resources\\language.json', 'file')
→ Reply
  • »
    »
    3 weeks ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    My error got resolved by doing the following

    export PATH=/home/outbreakkp/.local/bin:$PATH nano /home/outbreakkp/.local/lib/python3.10/site-packages/harwest/lib/resources/language.json then added the following in the structure "C++17 (GCC 7-32)": ".cpp",

    → Reply