I implemented codeforces from scratch

По техническим причинам тестирование временно остановлено. Разбираемся, прогнозов пока нет. Приносим извинения за сбой. ×

→ Обратите внимание

До соревнования
EPIC Institute of Technology Round Summer 2024 (Div. 1 + Div. 2)
3 дня
Зарегистрироваться »

*есть доп. регистрация

→ Лидеры (рейтинг)

№	Пользователь	Рейтинг
1	tourist	3845
2	jiangly	3707
3	Benq	3630
4	orzdevinwang	3573
5	Geothermal	3569
5	cnnfls_csy	3569
7	jqdai0815	3532
8	ecnerwala	3501
9	gyh20	3447
10	Rebelz	3409

Страны | Города | Организации

Всё →

→ Лидеры (вклад)

№	Пользователь	Вклад
1	maomao90	171
2	awoo	163
2	adamant	163
4	nor	151
4	maroonrk	151
4	-is-this-fft-	151
7	TheScrasse	148
8	atcoder_official	146
9	Petr	145
10	pajenegod	144

Всё →

→ Найти пользователя

→ Прямой эфир

Детальнее →

Блог пользователя nubskr

I implemented codeforces from scratch

Автор nubskr, история, 3 недели назад, По-английски

Hi codeforces, I was curious about how this platform worked, so I implemented my own version from scratch,It has all the essentials that an online judge has and I also added various functions such as collaborative coding and group voice chat with up to 10 users

The users can scrap problems from codeforces or they can just create their own problems

This is how it looks: image for reference

You can check it out here

and feel free to ask anything,It all was fueled by curiosity :)

Update: I'm planning to implement main test case Parsing in the next update.

+181

nubskr
3 недели назад
39

Комментарии (39)

Написать комментарий?

ZombieUser

3 недели назад, # |

-10

So Cool!! One of the best projects I have ever seen!!

→ Ответить

AvaraKedavra

3 недели назад, # |

+38

If you are using free tier on AWS for hosting your website then,keep in mind that Amazon won't notify you when you cross the freely usable limit on EC2 or other instances.
Once happened with me and I got a bill of $1000 (which is a lot of money in my country).

You have built a nice website btw. Very impressive.

→ Ответить

adityachk2002

3 недели назад, # ^ |

They have not hosted it. It's on localhost

→ Ответить

AvaraKedavra

3 недели назад, # ^ |

He has mentioned EC2 instance somewhere. I read it on his github page.

→ Ответить

123gjweq2

3 недели назад, # ^ |

bro is hosting it from his pc?

→ Ответить

pathetique

3 недели назад, # |

-75

looks dumb

→ Ответить

fonmagnus

3 недели назад, # |

+14

How do you get the full testcases data for the problem? For afaik, there is no convenient way (yet) to get them

→ Ответить

nubskr

3 недели назад, # ^ |

← Rev. 2 →

I don't, the scraper only scrapes the data on the problem page, you can check out the scraper here , the main test cases need to be added manually.

I'm planning to add main test cases parsing in future though (atleast the ones which are accessible)

→ Ответить

uff_too_hard

3 недели назад, # |

+17

What if someone submits a malicious code to corrupt the backend?

→ Ответить

nubskr

3 недели назад, # ^ |

Apologies for the late reply,was caught up in something, every compilation request triggers the creation of a docker container which runs alpine linux (which also happens to be one of the most lightweight distro out there with the docker container requiring no more than 8 MB), other than that each container has nothing but the compiler and the code to be executed in it , it compiles, sends out the output and get's destroyed, it all happens in a period of like 2 seconds, so even if someone sends in some code which intends to modify the backend files, they can't, because the container does not have access to it.

Even if they try to do the cpp equivalent of "sudo rm -rf /*", it won't matter, because each container is created fresh on demand and has nothing to with last thing it compiled, You should read more about docker

tldr: The container are a isolated runtime environment which only has access to the request code, they can't even access the internet, one can cause any mayhem they want in there, it won't matter.

→ Ответить

ExplodingFreeze

3 недели назад, # ^ |

While Docker containers are definitely far more secure than they used to be a few years ago, I still wouldn't consider it a replacement for a proper sandbox when executing untrusted code.

→ Ответить

nubskr

3 недели назад, # ^ |

← Rev. 2 →

Upon more research, it seems that kernel level exploits might still be an issue as it's shared with the main backend, at the same time my risk profile isn't as high, I think that keeping the systems updated should protect against most well known vulnerabilities, the kernel exploits seem more like a black swan scenario.

I'll still consider adding complete virtualization though at some point in the future.

→ Ответить

uff_too_hard

3 недели назад, # ^ |

Can you provide more resources to learn about docker

→ Ответить

nubskr

3 недели назад, # ^ |

You can check this out

→ Ответить

yariss

3 недели назад, # |

how do you handle security aspects

→ Ответить

nubskr

3 недели назад, # ^ |

hey, you can check my above reply to the comment if you're wondering about malicious code injection attacks, here are some other things:

As the architecture in the readme,each compilation requeset to the backend has has a rate limiter attached, each IP address is limited to 100 submissions per 15 minutes,and like I said in the previous reply, the submission codes never touch the actual backend, but a docker container.

And each submission has a time limit for 2 seconds, if it doesn't end its execution within that time, the container terminate and a TLE status is sent, so that it doesn't keep running indefinitely.

Basically I'm running this script inside the docker container to evaluate submissions.

Feel free to ask anything :)

→ Ответить

-is-this-fft-

3 недели назад, # ^ |

+22

#include <fstream>
#include <cstdlib>

int main () {
  ofstream fout ("verdict.txt");
  fout << "Accepted" << '\n';
  fout.close();

  system("chmod 444 verdict.txt");
}

:))

→ Ответить

nubskr

3 недели назад, # ^ |

← Rev. 3 →

:(( , after a second thought, someone can also modify the expected output to match their output, I'll patch this soon

→ Ответить

animeshz916

3 недели назад, # ^ |

He's talking about remote code execution, that if you haven't isolated the runtime access, the code will be able to run arbitrary function in the os, maybe exploiting your servers for free data storage foe instance.

→ Ответить

nubskr

3 недели назад, # ^ |

No he's not,Have you read any of the replies I wrote to above comments?

→ Ответить

nubskr

12 дней назад, # ^ |

Hey there, I recently shipped a new version of the submissions system here, also can you please try to hack this one as well, I tried to cover all the sneaky ways and implement proper sandboxing.

Some highlights:

All the codes are executed by another non root user who can't touch either verdict or expected output
I made sure that the code injections can't get to root (check the code for how I did it ;))
Since the user can't get to root, they can't see the verdict and expected_output, they can do anything they want with the output though and I tried to handle some sneaky aspects.

:))

→ Ответить

-is-this-fft-

12 дней назад, # ^ |

← Rev. 2 →

I think now you need someone more experienced to break into this system. :)

Only one thing comes to mind: compiling probably isn't really that safe. At the very least you should timeout it (since you can write pretty much arbitrarily complex preprocessor macros), but I'm not sure, maybe it is possible to do worse things with it (especially since you are compiling as root).

If you want to look into sandboxing even more, here is a repo on how IOI does it. There's also a bunch of papers linked in the README about other considerations they took into account.

→ Ответить

uff_too_hard

12 дней назад, # ^ |

What does this thing do?

→ Ответить

nubskr

11 дней назад, # ^ |

Hey there, it gets a random string from urandom of length 69

The script then sets this as the password for root so someone can't get into root userspace

→ Ответить

-is-this-fft-

3 недели назад, # ^ |

By the way, if you are using diff to compare outputs, at least use diff --ignore-trailing-space --strip-trailing-cr.

(That is, if you want to keep things simple and not introduce custom checkers.)

→ Ответить

SK1PY

3 недели назад, # ^ |

Also --brief in order not to make big files for no reason.

But it's best not to use diff anyway for the quadratic complexity.

→ Ответить

imgoingtowininfocup

3 недели назад, # |

-58

nobody asked

→ Ответить

Math_is_Good

3 недели назад, # |

which tool you used to create that high level system design?

→ Ответить

pagol

3 недели назад, # ^ |

You can do the same with LucidChart!

→ Ответить

nubskr

3 недели назад, # ^ |

I used draw.io

→ Ответить

nubskr

3 недели назад, # |

Auto comment: topic has been updated by nubskr (previous revision, new revision, compare).

→ Ответить

nobleknight

3 недели назад, # |

← Rev. 2 →

Nice one. I was also trying to build something like this. During that I heard about Linux Containers. They are lightweight and faster to boot than docker, but harder to implement. You can refer https://youtu.be/SD4KgwdjmdI?si=DVDdtuBRFFh5z0YX.

Correct me if I am wrong. Also please share resources from which you have taken reference.

→ Ответить

nubskr

2 недели назад, # ^ |

← Rev. 2 →

Hey, I checked it out and it seems like works in a similar way like mine but better

For starters, it also uses docker as a base and then inside the container runs a custom script to code executions, both mine and the one that is used there are similar in various ways(the latter one being a lot more mature) but mine also determines the verdict inside the container which was a stupid idea and easily exploitable as someone mentioned above.

One fix that I'm planning to add is to make the container just spit out the program output and then compare that with the expected output outside the container, that way the container won't need to have access to the verdict and expected output file, making it essentially bulletproof against those attacks

→ Ответить