PR Reviewer
A self-hosted GitHub App that reviews pull requests with AI and posts real reviews with inline comments. Code, not the model, decides the verdict; re-reviews read only what changed; and every AI call is logged with its cost.
- Role
- Solo engineer
- Timeline
- Jul 2026 – present
- Stack
- Next.jsReactTypeScriptMongoDBGitHub AppsOctokitZodSWRVitestPM2GitHub Actions
The problem
AI review bots comment on everything, approve PRs they only half read, and hide what each review cost.
What it does
- Reviews a PR when you request a review from a connected account, and posts APPROVE, REQUEST_CHANGES or COMMENT with inline comments.
- Code decides the verdict: if part of the diff was skipped (token budget, timeout, failed chunk), the review is downgraded to COMMENT and lists what it skipped.
- A scope guard drops findings the diff cannot support, but always keeps security findings.
- Re-reviews compare against the last reviewed commit and read replies to earlier findings.
- Review profiles, per-repo models and guidelines, ignore globs and per-author overrides.
- A dashboard with reviews, verdicts, an audit of every AI call (prompt, response, tokens, latency, cost) and CSV export.
How it is built
- Next.js 16 and MongoDB. Reviews run after the webhook responds, and large PRs are split into token-budgeted chunks processed in parallel.
- Supports multiple LLM providers behind one interface, with output checked by a schema and one repair attempt.
- Webhooks are verified with HMAC and de-duplicated. Tokens and API keys are encrypted at rest with AES-256-GCM.
- About 200 tests. CI runs lint, test, typecheck and build. Deploys roll out to a VPS with a health check and automatic rollback.
My role
Designed and built it end to end, including the deploy pipeline.