HireForge: AI-Run Coding Assessments

1. 1. Describe the role. Start from a template or paste a job description. The must-have skills become the criteria candidates are scored on.
2. 2. Generate the codebase. An agent writes the project, its tests and the planted issues in a background job. Each refinement adds a new numbered version instead of overwriting the last.
3. 3. Publish. Checks run first: the tests run, the planted issues really do make them fail, and nothing meant for the company leaks into the candidate's brief. The published version is then locked, so every candidate takes the same test.
4. 4. Take it. The candidate opens a private link and gets a timed workspace: an editor, a terminal, the test runner and an AI assistant.
5. 5. Review. The reviewer reads the diff and everything the candidate did, and writes a scorecard. The company can re-weight the criteria, shortlist and compare candidates, and ask questions about the results.

Candidate code never runs on the host. Each session gets its own container under gVisor, on its own network with the internet off. If a company needs packages installed, a proxy for that session lets through the package registries and the hosts the company names, and nothing else.

The AI assistant runs in a separate container and never holds an API key. Anything an agent can read, a candidate can get by asking it, so the keys stay in a gateway on the host that works out which session a request belongs to and applies that session's budget.

The model judges; code does the arithmetic. The diff comes from real git, the tests are actually run, and totals, rankings and most red flags are worked out by code, so a model can't miscount or be talked into a better score. The reviewer gets read-only tools and no shell.

• • The scorecard gives Engineering, Use of AI and Overall scores, each backed by quotes from what the candidate did.
• • A code view shows where every added line came from: typed by the candidate or written by the assistant.
• • A replay walks through the key moments of the session.
• • "Ask the results" answers questions about the candidates, and code checks every citation in the answer before it is shown.

A FastAPI backend with PostgreSQL, and a job queue in Postgres that a separate worker takes generation and review jobs from. The frontend is React and TypeScript, with CodeMirror for the editor and xterm.js for the terminal. Every model call goes through one small client of my own for Anthropic and DeepSeek, and a test makes sure nothing else calls a provider.

spec→generate→publish→sandbox→review→results

• • About 1,250 Python tests and 485 frontend tests, all of which run offline against a fake model, so CI never needs a key.
• • The data formats are defined once in TypeScript, emitted as JSON Schema and mirrored in Python. CI fails if any of them drift apart.
• • The test script lives outside the candidate's repository and is checked by hash, so it can't be swapped for one that always passes.
• • Every git command on the host goes through one wrapper, so a hook planted in a candidate's repository never runs.
• • Asking for another company's data gets a 404, not a 403, so ids can't be probed. A test tries it on every route.