A modern CLI tool that brings the entire problem-setting workflow to your terminal
TL;DR: I built Polyman — a command-line tool that lets you create, test, and verify competitive programming problems entirely on your local machine before touching Polygon. It supports test generation, validation, multiple solutions, standard checkers, and full Polygon integration.
Installation: npm install -g polyman-cli
The Problem with Problem Setting
If you've ever tried to create a problem for Codeforces (or other competitive programming platforms), you know the pain:
- Upload a solution to Polygon → Wait → Compile error → Fix → Repeat
- Upload tests one by one → Realize they're invalid → Delete → Upload again
- Test your checker → It fails → Download, fix locally, re-upload
- Want to test multiple solutions? Good luck switching between them all on Polygon
- Need to regenerate 100 tests? Hope you enjoy clicking buttons
Sound familiar?
I got tired of this workflow. So I built Polyman — a tool that lets you do everything locally, with your favorite text editor, your terminal, and full automation.
What is Polyman?
Polyman is a CLI tool (think testlib.h, but for the entire problem-setting workflow) that helps you:
- Create problems with a single command — complete folder structure, templates, everything
- Generate tests programmatically using C++ generators (with
testlib.h) - Validate inputs automatically against your constraints
- Run multiple solutions (correct, wrong answer, TLE, etc.) and verify their verdicts
- Use standard checkers (or write custom ones) to verify outputs
- Sync with Polygon — pull problems, work locally, push changes back
- Verify everything with a single command before uploading
All from your terminal. No browser tabs. No waiting for Polygon to compile. No clicking through endless menus.
Quick Example: Creating Your First Problem
Let's create a simple problem — "Sum of Two Numbers":
# Create problem
polyman new sum-problem
cd sum-problem
# Download testlib.h
polyman download-testlib
Now you have this structure:
sum-problem/
├── Config.json # Problem configuration
├── solutions/ # Your solutions
├── generators/ # Test generators
├── validator/ # Input validator
├── checker/ # Output checker
├── testsets/ # Generated tests
└── statements/ # Problem statements
Write your solution (solutions/Solution.cpp):
#include <iostream>
using namespace std;
int main() {
int a, b;
cin >> a >> b;
cout << a + b << endl;
return 0;
}
Write your validator (validator/Validator.cpp):
#include "testlib.h"
int main(int argc, char* argv[]) {
registerValidation(argc, argv);
int a = inf.readInt(1, 1000, "a");
inf.readSpace();
int b = inf.readInt(1, 1000, "b");
inf.readEoln();
inf.readEof();
return 0;
}
Write your generator (generators/Generator.cpp):
#include "testlib.h"
int main(int argc, char* argv[]) {
registerGen(argc, argv, 1);
int testNum = atoi(argv[1]);
int maxValue = min(100 * testNum, 1000);
int a = rnd.next(1, maxValue);
int b = rnd.next(1, maxValue);
cout << a << " " << b << endl;
return 0;
}
Configure your problem (Config.json):
{
"name": "sum-problem",
"timeLimit": 1000,
"memoryLimit": 256,
"inputFile": "stdin",
"outputFile": "stdout",
"solutions": [
{
"name": "main",
"source": "./solutions/Solution.cpp",
"tag": "MA"
}
],
"checker": {
"name": "ncmp",
"isStandard": true
},
"testsets": [
{
"name": "tests",
"generatorScript": {
"commands": [
{
"type": "generator-range",
"generator": "gen",
"range": [1, 20]
}
]
}
}
]
}
Now verify everything:
polyman verify
That's it. One command. It will:
- Generate all 20 tests
- Validate all inputs
- Run your solution on all tests
- Verify all outputs with the checker
- Tell you if anything is wrong
If everything passes, your problem is ready to upload to Polygon!
Real Power: Working with Multiple Solutions
Want to test that your problem actually rejects wrong solutions? Add them to Config.json:
"solutions": [
{
"name": "main",
"source": "./solutions/correct.cpp",
"tag": "MA"
},
{
"name": "wrong",
"source": "./solutions/wrong.cpp",
"tag": "WA"
},
{
"name": "slow",
"source": "./solutions/slow.cpp",
"tag": "TL"
}
]
Run polyman verify and it will:
- Verify main solution gets AC on all tests
- Verify wrong solution gets WA on at least one test
- Verify slow solution gets TLE on at least one test
If any solution doesn't behave as expected, it tells you immediately. No more uploading 5 different solutions to Polygon and manually checking each one.
Standard Checkers: No Need to Reinvent the Wheel
Polyman includes all the standard testlib.h checkers:
polyman list-checkers
Shows you:
wcmp — Compare tokens (whitespace-insensitive)ncmp — Compare sequences of numbersfcmp — Compare floating-point numberslcmp — Compare lines exactlyyesno — Compare yes/no answers- And 20+ more...
Just set "isStandard": true in your config. No need to download or compile them manually.
Polygon Integration: The Best of Both Worlds
Already have problems on Polygon? No problem. Polyman syncs with Polygon seamlessly:
# Register your API key (one-time)
polyman remote register
# Pull a problem from Polygon
polyman remote pull 123456 ./my-problem
cd my-problem
# Work on it locally...
# Make changes, test everything
# Push changes back to Polygon
polyman remote push .
# Commit your changes
polyman remote commit . -m "Added new test cases"
# Build and download package
polyman remote package . --full
Work locally with your tools, then push to Polygon when ready. No more choosing between local development and Polygon's features.
Advanced Features
Organized Test Groups
"testsets": [{
"name": "tests",
"groupsEnabled": true,
"groups": [
{"name": "samples"},
{"name": "small"},
{"name": "large"}
],
"generatorScript": {
"commands": [
{
"type": "manual",
"manualFile": "./manual/sample1.txt",
"group": "samples"
},
{
"type": "generator-range",
"generator": "gen-small",
"range": [1, 30],
"group": "small"
},
{
"type": "generator-range",
"generator": "gen-large",
"range": [1, 70],
"group": "large"
}
]
}
}]
Selective Generation/Validation
polyman generate --testset tests --group samples # Only samples
polyman validate --testset tests --index 5 # Only test 5
polyman run main --testset tests --group large # Only large tests
Multiple Languages
Polyman supports C++ (g++, clang, MSVC), Java, and Python solutions. Mix and match as needed.
Custom Checkers
Write your own checker with testlib.h, test it with checker_tests.json, and Polyman handles the rest.
Installation
Requirements:
- Node.js v14+
- C++ compiler (g++, clang, or MSVC)
- Java/Python (optional, for those solution types)
Install:
npm install -g polyman-cli
Verify:
polyman --version
That's it. You're ready to create problems.
Documentation & Learning Resources
The tutorial walks you through creating your first problem from scratch. The guide covers everything — validators, generators, checkers, testsets, remote operations, best practices, troubleshooting, and more.
Why I Built This
I love creating problems, but I hated the workflow. Every time I had to:
- Switch between my editor and Polygon
- Wait for compilations
- Manually test edge cases
- Upload tests one by one
I thought: "There has to be a better way."
So I built Polyman. It's the tool I wish I had when I started problem setting. It lets you work the way developers actually work — locally, with automation, with proper testing, with version control.
Try It Out!
If you're a problem setter (or want to become one), give Polyman a try:
npm install -g polyman-cli
polyman new my-first-problem
cd my-first-problem
polyman download-testlib
# ... create your problem ...
polyman verify
Follow the tutorial if you're new, or dive into the complete guide if you want to see everything it can do.
Found a bug? Have a feature request? Open an issue on GitHub. PRs are welcome!
Star the repo if you find it useful: github.com/HamzaHassanain/polyman ⭐
Want to Contribute?
Polyman is open source and I'd love your help making it better! Whether you're a beginner or experienced developer, there are plenty of ways to contribute.
Open Issues & Feature Requests
Here are some exciting features we're working on that you can help with:
- Auto-Deriving Config Names from File Paths (#12) — Simplify configuration by automatically inferring names from file paths
- Smart Compilation Cache with Dependency Tracking (#11) — Speed up workflow by caching compilations intelligently
- Diff Viewer for Local vs Remote Changes (#10) — See what changed before pushing to Polygon
- Shell Auto-Completion Support (#9) — Add tab completion for bash/zsh/fish
- Comprehensive Unit Test Coverage (#8) — Help us reach better test coverage
- Parallel Execution for Independent Operations (#7) — Make test generation and validation faster
- Java Class Naming Conflict Resolution (#6) — Handle Java/Polygon naming conflicts better
- Time Scaling for Java & Python (#5) — Automatically adjust time limits for different languages
All issues are tagged with good first issue, help wanted, enhancement, or documentation to help you find where you can contribute. Check out the full list of issues on GitHub!
How to Get Started
- Fork the repository
- Pick an issue (or suggest a new feature!)
- Create a branch and make your changes
- Submit a pull request
Questions?
Drop them in the comments! I'd love to hear:
- What features would make your problem-setting workflow easier?
- What problems have you faced when creating problems?
- What would you want to see in future versions?
UPDATE
Notes Regarding Windows Users
When using this project on Windows, please be aware of the following considerations:
Line Endings:
Windows uses different line endings (CRLF) compared to Unix-based systems (LF). Ensure your text editor is configured to handle line endings appropriately to avoid issues in scripts and configuration files.
UNIX
{
"tests": [
{
"input": "5 10\n1 2 3 4 5\n",
"expectedVerdict": "VALID"
},
{
"input": "0 0\n\n",
"expectedVerdict": "INVALID"
}
]
}
WINDOWS
{
"tests": [
{
"input": "5 10\r\n1 2 3 4 5\r\n",
"expectedVerdict": "VALID"
},
{
"input": "0 0\r\n\r\n",
"expectedVerdict": "INVALID"
}
]
}
Notes About The TUTORIAL.md File
We always use Unix-style line endings (LF) in the TUTORIAL.md file to maintain consistency across different operating systems. If you are using Windows, please ensure your text editor can handle LF line endings correctly to avoid any formatting issues.
PLEASE REPORT ANY ISSUES YOU ENCOUNTER WHILE USING THIS PROJECT ON WINDOWS. YOUR FEEDBACK IS VALUABLE TO US!
File Paths:
Windows uses backslashes (\) for file paths, while Unix-based systems use forward slashes (/). Be cautious when specifying file paths in scripts or configuration files to ensure compatibility.
{
"filePath": "C:\\Users\\Username\\Documents\\project\\file.txt"
}
Solutions, Validators, Generators, and Checkers That Exceed The Time Limit:
I had a hard time trying to ensure the proper cleanup of the processes terminated duo to time limit exceeded on Windows, Most of the time, you will find that the processes are not being killed properly, leading to resource leaks and other unexpected behavior.
That is most of the time you will recive that error saying something like:
EBUSY: resource busy or locked, open 'C:\Users\hamza\sum-problem\solutions-outputs\tlee\tests\output_test1.txt'
This error mostly happens while wrting to files that are being used by another process (the one that was not killed properly).
For such cases, open your Task Manager and manually kill the processes that are still running, search for the name of the executable you were trying to run (that is the name of your solution, validator, generator, or checker executable).
Compilation Errors ALSO MAY HAPPEN DUE TO THE SAME REASON
If you face compilation errors that you are sure are not related to your code, try to manually kill the processes as mentioned above, and then try to recompile.
Notes Regarding Polygon
I found no way to disable the automatic update of the checker, so it will give you a warning (when useing a standard checker). You may need to consider doing that manually if you want to avoid that warning.
Notes Regarding The Validators new line.
Please keep in mind that on Windows, the new line character is represented by a carriage return followed by a line feed (\r\n), while on Unix-based systems, it is represented by just a line feed (\n). Also keep in mind that some code editors will use \n even on Windows.
So when you write a manual test case for your validator, make sure to use the correct new line character based on the operating system you are using.
Best option is to use the windows Text Editor (the built-in one) to write your test cases, as it will use the correct new line characters for your OS.
