johnathan79717's blog

A Codeforces Contest Parser and Sample Tests Generator
By johnathan79717, 11 years ago, In English

Many thanks to brunoja. He is the main contributor now. Please try it out and give us feedback. You're also welcome to fork the project or join us as a collaborator.

https://github.com/johnathan79717/codeforces-parser

Codeforces Parser v1.4.1

Summary

Codeforces is a website for competitive programming. It holds contests, so-called Codeforces Rounds, about every week.

This is a python program that parses the sample tests from the contest problem pages. For each problem, it generates the sample input/output files and a shell script to run sample tests.

You can also find this article here, http://mirror.codeforces.com/blog/entry/10416

Example:

./parse.py contest_number (e.g. ./parse.py 464)

Where 464 is the contest number, not the round number! Check the URL of the contest on your browser, that is the number you are supposed to use.

Effect:

What will happen, for example, if ./parse.py 464 is executed?
  1. Directories 464/A, 464/B, 464/C, 464/D and so on are created depending on the contest number of problems.
  2. For each problem, main.cc is copied and renamed to the problem letter to the corresponding directory. You can put the path of your usual template in parse.py:20.
  3. Problem page is downloaded from Codeforces website, and parsed. Sample input/output files are generated, e.g. input1, output1, input2, output2 and so on. You can create your own test cases after that, just keep the same naming format as others test cases.
  4. A script test.sh is generated. You can use it to compile and run the sample tests after you finish coding. Just run ./test.sh in the problem directory.
What will happen if ./test.sh is executed?
  1. Compilation: g++ -g -std=c++0x -Wall main.cc. You can change the compile options in parse.py:21.
  2. Run each sample tests on your program (a.out), and check the output by diff. If it's correct, print Accepted, or else print the sample test that went wrong.

Collaborators and Versions:

List of CodeForces Collaborators:

If you have any suggestions and/or bugs drop a message!

Versions Changes:
  • 1.4.1: Minor fixes, such as typos, bugs and special characters handling.
  • 1.4: Changed how the parser gets the problems. During the competitions the page is slightly different. Fixed some invalid character on input and output causing the script to crash. Forcing a new line on the input/output if there is none. Fixed some line number information in this README file.
  • 1.3: Some minor fixes and code organizing. Also fixed some typos. Removed the sample from default input and output files.
  • 1.2: Fixed some typos and constants. Fetching contest info, printing contest name and problem names. The contest may now have more or less than 5 problems, it will auto detect. The script will now generate the template with the problem letter. Fixed test cases fetching. The script was stopping for escaped html characters, such as '&lt'. Fixed script to work with python 3.
  • 1.1: Cleaner generation of the test script, now it auto detects the test cases, making you able to create your own cases. Echo color output, for accepted we get a green message, otherwise it is red. Added the time measurement for running the test cases. For the runtime error case, it now outputs the input case. Created some constants, such as compile options. These user modifiable constants should be easily spotted at the first lines of the python script.
  • 1.0: Initial Version.
Todo, Bugs & Troubleshootings:
  • In OS X it is necessary to install the gnu-time to measure time.
  • This parser currently works only on Unix OSes. If you want to add Windows/Other support let us know.
  • Vote: I like it
  • +63
  • Vote: I do not like it

Comments Comments (23)
| Write comment?
»
11 years ago, # |
← Rev. 2   Vote: I like it +3 Vote: I do not like it

Hi! Thanks for this, really helpful :). In the test.sh script there is a line before running your code: rm sample_*. That is deleting the sample cases before running them, is it correct?

Also, you may want to change -std=c++11 to -std=c++0x, it works for older gcc versions (like mine, gcc 4.6).

→ Reply
  • »
    »
    11 years ago, # ^ |
    ← Rev. 3   Vote: I like it 0 Vote: I do not like it

    Thanks, that's a stupid mistake. I've removed it. Also changed to -std=c++0x. Thanks a lot for the suggestion :)

    → Reply
    • »
      »
      »
      11 years ago, # ^ |
      ← Rev. 4   Vote: I like it +3 Vote: I do not like it

      it is better to replace on "rm -R my_output{0}\n"

      → Reply
      • »
        »
        »
        »
        11 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Your right! That's what I had in mind when I wrote that line I guess :P Thanks a lot.

        → Reply
      • »
        »
        »
        »
        11 years ago, # ^ |
        ← Rev. 2   Vote: I like it +3 Vote: I do not like it

        You should not remove the a.out! Also add a -R to the rm command, otherwise it will stop the script (the first time you run it, there isn't an output file yet..).

        → Reply
        • »
          »
          »
          »
          »
          11 years ago, # ^ |
            Vote: I like it +3 Vote: I do not like it

          Oh, of course, you are right)

          → Reply
        • »
          »
          »
          »
          »
          11 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Thanks again... I really should have tested thoroughly every time before commit and push. Nice to have you guys helping.

          → Reply
»
11 years ago, # |
  Vote: I like it +3 Vote: I do not like it

from where we can download this?

and can it run under windows or it needs Linux?

and thanks for your work, it looks great :D

→ Reply
  • »
    »
    11 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    The project is at https://github.com/johnathan79717/codeforces-parser

    You can run git clone https://github.com/johnathan79717/codeforces-parser.git or there's a button at the bottom right corner of the page that allows you to download a zip file.

    We haven't try it on windows. You're welcome to try it out and make a windows version.

    → Reply
»
11 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Hi, I can not use it for proxy authentication? Can you add some support for that? As you are using urlopen for downloading the problem statement, Urlopen does not allow it to have a proxy with user authentication. Do you have some workaround for this?

→ Reply
  • »
    »
    11 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Hi, could you elaborate in what situation do you need proxy authentication? Thanks. This may work, but I haven't tried it yet.

    → Reply
    • »
      »
      »
      11 years ago, # ^ |
        Vote: I like it +3 Vote: I do not like it

      Hi, I usually use internet on the linux pcs installed in our institute labs, So I need to use proxy authentication for that purpose. For now I have kind of patched the things by using FancyURLopener class in the urrlib. I will try to find a good solution for it. I saw your link, it was quite similar to what I was doing, Thank you

      → Reply
»
11 years ago, # |
  Vote: I like it +5 Vote: I do not like it

I think it would be fantastic if Codeforces provides an API to obtain the test cases to not have to parse the entire HTML.

→ Reply
»
11 years ago, # |
  Vote: I like it +16 Vote: I do not like it

What is the license of your code? I have my own tester and I would like to use your parser. :)

→ Reply
»
10 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Hi! I just merged the branch I was using with some fixes and I thought it might be good to announce it here :). The current version is the 1.4.1. Thanks!

→ Reply
»
6 years ago, # |
← Rev. 3   Vote: I like it +8 Vote: I do not like it

Thanks a lot johnathan79717 and brunoja.

I'm using it with vim to improve workflow.

$ sudo mv parse.py /usr/bin/parse

And following functions in .vimrc (vim configuration file)

"My functions for gvim && competitive"

nnoremap <C-z> :cd %:p:h <bar> call Parse()<CR>

"self check test cases"
autocmd filetype cpp nnoremap <C-a> :cd %:p:h <bar> call TestCase()<CR>

"To check the solution, while keeping cpp file in buffer, press Ctrl+x"
autocmd filetype cpp nnoremap <C-x> :cd %:p:h <bar> !xterm -e "./test.sh; read"<CR><CR>

"To parse the contest, press Ctrl+z"
function! Parse()
    let new_contest = input('Enter codeforces contest code: ')
    exec '!xterm -e "parse ' . l:new_contest. ';read"'
endfunction

"For addition test cases that you may want to add, press Ctrl+a"

function! TestCase()
	 let index = 1
	 while !empty(glob("input".index))
		  let index += 1
	 endwhile
	 echo "Input case: " . index . " "
	 let in = input('Input case: ')
	 echo " "
	 let out = input('Output case: ')
	 echo " "
	 let confirm = input ('Are you sure of writing this test case (y/n) : ')
	 if confirm == 'y'
		  silent exec '!echo "' . in . '" > input' . index
		  silent exec '!echo "' . out . '" > output' . index
	 endif
endfunction

And little changes in parser.py, if you like.

language_params = {
        'c++14' : {
            'TEMPLATE'    : '/home/convict/temp.cpp',
            'DEBUG_FLAGS' : '-DDEBUG -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC',
            'COMPILE_CMD' : 'g++ -g -std=gnu++14 -Wall -Wall -Wextra -pedantic -Wshadow -Wformat=2 -Wfloat-equal -Wconversion -Wlogical-op -Wshift-overflow=2 -Wduplicated-cond -Wcast-qual -Wcast-align  -D_FORTIFY_SOURCE=2 -fsanitize=address -fsanitize=undefined -fno-sanitize-recover -fstack-protector  -O2 $DBG',
            'RUN_CMD'     : './a.out'
            },
        }
→ Reply
»
4 years ago, # |
  Vote: I like it 0 Vote: I do not like it

please tell how it will detect the division no.

→ Reply