Repository navigation
Modernize project and evaluate profiles locally - #16
Conversation
The external evaluator API is being shut down. The app now calculates the grade in its own endpoint, GET /api/evaluation?user=<username>, which reads public profile data from the GitHub GraphQL and REST APIs. - Add the evaluation logic in server/ and a Vercel function in api/ - Serve the same endpoint from the Vite dev server - Detect the default identicon from the avatar PNG size - Find technologies in skillicons, devicon, shields.io and README text - Count pinned repositories, which the old API always returned as 0 - Add unit tests with Vitest and run them in CI - Document the GITHUB_TOKEN setup for local use and for Vercel Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Reviewer's GuideData structuresDefine the shared evaluation result, criterion keys, and username validation that the server and front end rely on.
APIExpose the evaluation endpoint as a Vercel function and define its request validation, response, and caching behavior.
Uses of the APISwitch the front-end evaluation client to the local endpoint and shared result contract, and update the grade criteria type and photo label.
ImplementationImplement GitHub profile and avatar retrieval, profile evaluation, identicon detection, and README technology detection.
Add Vite middleware so local development serves the same evaluation handler as the deployed endpoint.
ConfigurationAdd the test command and configure CI to run it, while updating TypeScript, Vercel, and Vite project configuration for the server endpoint.
DocumentationDocument the local evaluation criteria, endpoint architecture, token setup, scripts, project structure, and deployment steps.
TestsAdd unit tests for avatar detection, profile evaluation, request handling, and technology extraction.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
HeadersInit exists only in the DOM type library, so the type check of the server project failed on Vercel. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Hey - I've found 9 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="api/evaluation.ts" line_range="4" />
<code_context>
+import { handleEvaluationRequest } from "../server/handler.js";
+
+// Vercel serverless function: GET /api/evaluation?user=<username>
+export function GET(request: Request): Promise<Response> {
+ return handleEvaluationRequest(request, process.env.GITHUB_TOKEN);
+}
</code_context>
<issue_to_address>
**Production evaluation endpoint is unavailable**
When the app is deployed to Vercel using this `api/` file, vercel does not register the named `GET` export as a function handler, so requests never reach `handleEvaluationRequest` and profile evaluations fail in production.
Export the handler in the format Vercel expects for this function, or configure the deployment to use the named `GET` export.
</issue_to_address>
### Comment 2
<location path="package.json" line_range="39" />
<code_context>
"typescript": "^5.9.3",
- "vite": "^8.3.8"
+ "vite": "^8.3.8",
+ "vitest": "^5.1.0"
}
}
</code_context>
<issue_to_address>
**CI stops before tests and build**
When CI runs without a committed lockfile, `npm ci` exits because the repository has no `package-lock.json`, so the workflow never reaches the test or build steps.
Commit a `package-lock.json` that matches the dependency manifest.
Also at `.github/workflows/ci.yml:17`.
</issue_to_address>
### Comment 3
<location path="server/handler.ts" line_range="31" />
<code_context>
+ }
+
+ try {
+ const result = await evaluateProfile(new GitHubClient(token), username);
+ if (!result) return json(404, { error: "User not found" });
+ return json(200, result, {
</code_context>
<issue_to_address>
**Unauthenticated requests exhaust GitHub quota**
When an unauthenticated caller repeatedly requests uncached or nonexistent usernames, `handleEvaluationRequest` calls `evaluateProfile` for cache misses, spending the shared GitHub token’s quota; not-found responses return before receiving a cache header, so repeated misses exhaust the quota and evaluations fail for everyone.
Rate-limit uncached requests and cache not-found responses.
</issue_to_address>
### Comment 4
<location path="server/github.ts" line_range="46" />
<code_context>
+ /** Returns `null` when the user does not exist. */
+ async getProfile(login: string): Promise<GitHubProfile | null> {
+ const response = await this.fetchImpl(`${GITHUB_API}/graphql`, {
+ method: "POST",
+ headers: this.headers(),
+ body: JSON.stringify({ query: PROFILE_QUERY, variables: { login } }),
</code_context>
<issue_to_address>
**Stalled upstream calls exhaust requests**
When gitHub or the avatar host accepts a request but does not complete it promptly, `GitHubClient` awaits `fetchImpl` without a timeout or abort signal for the GraphQL, README, and avatar requests. If an upstream connection stalls, `handleEvaluationRequest` remains pending until the platform terminates the invocation, and the caller receives a failed evaluation.
Apply a bounded timeout to each upstream request and return a controlled error when it expires.
Also at `server/github.ts:67`, `server/github.ts:84`.
</issue_to_address>
### Comment 5
<location path="server/github.ts" line_range="76" />
<code_context>
+// Vercel serverless function: GET /api/evaluation?user=<username>
+export function GET(request: Request): Promise<Response> {
+ return handleEvaluationRequest(request, process.env.GITHUB_TOKEN);
+}
</code_context>
<issue_to_address>
**Large READMEs exhaust function resources**
When a public profile README is unusually large, `getProfileReadme` buffers the entire raw README, then `evaluateProfile` passes it to `findTechnologies`, which scans it repeatedly. A caller can target a profile with a very large README to consume excessive memory and CPU in the serverless function, causing evaluations to time out or fail.
Enforce a maximum README response size before buffering or parsing its contents.
</issue_to_address>
### Comment 6
<location path="server/handler.ts" line_range="31" />
<code_context>
+ }
+
+ try {
+ const result = await evaluateProfile(new GitHubClient(token), username);
+ if (!result) return json(404, { error: "User not found" });
+ return json(200, result, {
</code_context>
<issue_to_address>
**Development loads duplicate GitHub calls**
When a developer loads an evaluation page with the development server running, react StrictMode runs the evaluation effect twice in development; aborting the first browser fetch does not cancel `handleEvaluationRequest` or its `evaluateProfile` call, so both requests make the GitHub profile, README, and avatar requests and consume the configured token’s quota.
Coalesce concurrent evaluations for the same username in the development server, or otherwise prevent the StrictMode cleanup request from starting duplicate GitHub work.
</issue_to_address>
### Comment 7
<location path="server/handler.ts" line_range="34" />
<code_context>
+ const result = await evaluateProfile(new GitHubClient(token), username);
+ if (!result) return json(404, { error: "User not found" });
+ return json(200, result, {
+ "Cache-Control": "public, s-maxage=600, stale-while-revalidate=3600",
+ });
+ } catch (error) {
</code_context>
<issue_to_address>
**Profile grades stay stale too long**
When a profile changes after its cached result has expired and a request arrives during the stale-while-revalidate window, vercel can serve the cached response for up to an additional hour while revalidating it, so a changed profile continues to show its old grade and `evaluatedAt` after the documented 10-minute cache period.
Remove the extra stale window or shorten it to the maximum staleness the app intends to expose.
</issue_to_address>
### Comment 8
<location path="server/technologies.ts" line_range="99" />
<code_context>
+ for (const match of readme.matchAll(/skillicons\.dev\/icons\?i=([\w,+#.-]+)/gi)) {
+ ids.push(...match[1].split(","));
+ }
+ for (const match of readme.matchAll(/devicons?\/(?:devicon\/[^"'\s)]*?)?icons\/([\w+#.-]+)\//gi)) {
+ ids.push(match[1]);
+ }
</code_context>
<issue_to_address>
**Devicon technologies are omitted**
When a README uses a Devicon image URL with a version suffix such as `@latest`, the URL pattern in `findTechnologies` does not match that Devicon path, so it removes the URL without extracting its icon ID and awards too few technology points.
Update the Devicon URL pattern to match version-suffixed paths such as `devicon@latest/icons/...`.
</issue_to_address>
### Comment 9
<location path="server/evaluate.test.ts" line_range="5" />
<code_context>
+import { evaluateProfile } from "./evaluate.js";
+import type { GitHubClient, GitHubProfile } from "./github.js";
+
+const PHOTO = new Uint8Array([0xff, 0xd8, 0xff]);
+
+function fakeClient(profile: GitHubProfile | null, readme: string | null): GitHubClient {
</code_context>
<issue_to_address>
**Default avatars still earn photo points**
When the evaluator is changed to grant the photo point without checking whether the avatar is a default identicon, the “gives 10 points for each passed criterion” test still passes if `evaluateProfile` unconditionally awards the photo point: its fake client returns this JPEG for the avatar, which the current identicon check already treats as non-identicon. The test therefore does not catch default avatars earning points.
Use a valid default-identicon PNG fixture in an `evaluateProfile` test and assert that `hasPhoto` is false.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 9 findings to address first, and if the evaluation logic is wrong, the endpoint can return an incorrect grade that may remain in Vercel’s shared cache after a revert until it expires or is refreshed. The result is bounded and repairable; the endpoint only reads public GitHub data and does not persist changes or trigger external actions.
Blocking findings: api/evaluation.ts:4, package.json:39, server/handler.ts:31, server/github.ts:46, server/github.ts:76, and 4 more
| import { handleEvaluationRequest } from "../server/handler.js"; | ||
|
|
||
| // Vercel serverless function: GET /api/evaluation?user=<username> | ||
| export function GET(request: Request): Promise<Response> { |
There was a problem hiding this comment.
🔴 Critical · Production evaluation endpoint is unavailable
When the app is deployed to Vercel using this api/ file, vercel does not register the named GET export as a function handler, so requests never reach handleEvaluationRequest and profile evaluations fail in production.
Export the handler in the format Vercel expects for this function, or configure the deployment to use the named GET export.
Prompt for AI agents
In `api/evaluation.ts` at line 4:
**Production evaluation endpoint is unavailable**
When the app is deployed to Vercel using this `api/` file, vercel does not register the named `GET` export as a function handler, so requests never reach `handleEvaluationRequest` and profile evaluations fail in production.
Export the handler in the format Vercel expects for this function, or configure the deployment to use the named `GET` export.| "typescript": "^5.9.3", | ||
| "vite": "^8.3.8" | ||
| "vite": "^8.3.8", | ||
| "vitest": "^5.1.0" |
There was a problem hiding this comment.
🔴 Critical · CI stops before tests and build
When CI runs without a committed lockfile, npm ci exits because the repository has no package-lock.json, so the workflow never reaches the test or build steps.
Commit a package-lock.json that matches the dependency manifest.
Also at .github/workflows/ci.yml:17.
Prompt for AI agents
In `package.json` at line 39:
**CI stops before tests and build**
When CI runs without a committed lockfile, `npm ci` exits because the repository has no `package-lock.json`, so the workflow never reaches the test or build steps.
Commit a `package-lock.json` that matches the dependency manifest.
Also at `.github/workflows/ci.yml:17`.| } | ||
|
|
||
| try { | ||
| const result = await evaluateProfile(new GitHubClient(token), username); |
There was a problem hiding this comment.
🟠 High · Unauthenticated requests exhaust GitHub quota
When an unauthenticated caller repeatedly requests uncached or nonexistent usernames, handleEvaluationRequest calls evaluateProfile for cache misses, spending the shared GitHub token’s quota; not-found responses return before receiving a cache header, so repeated misses exhaust the quota and evaluations fail for everyone.
Rate-limit uncached requests and cache not-found responses.
Prompt for AI agents
In `server/handler.ts` at line 31:
**Unauthenticated requests exhaust GitHub quota**
When an unauthenticated caller repeatedly requests uncached or nonexistent usernames, `handleEvaluationRequest` calls `evaluateProfile` for cache misses, spending the shared GitHub token’s quota; not-found responses return before receiving a cache header, so repeated misses exhaust the quota and evaluations fail for everyone.
Rate-limit uncached requests and cache not-found responses.| /** Returns `null` when the user does not exist. */ | ||
| async getProfile(login: string): Promise<GitHubProfile | null> { | ||
| const response = await this.fetchImpl(`${GITHUB_API}/graphql`, { | ||
| method: "POST", |
There was a problem hiding this comment.
🟡 Medium · Stalled upstream calls exhaust requests
When gitHub or the avatar host accepts a request but does not complete it promptly, GitHubClient awaits fetchImpl without a timeout or abort signal for the GraphQL, README, and avatar requests. If an upstream connection stalls, handleEvaluationRequest remains pending until the platform terminates the invocation, and the caller receives a failed evaluation.
Apply a bounded timeout to each upstream request and return a controlled error when it expires.
Also at server/github.ts:67, server/github.ts:84.
Prompt for AI agents
In `server/github.ts` at line 46:
**Stalled upstream calls exhaust requests**
When gitHub or the avatar host accepts a request but does not complete it promptly, `GitHubClient` awaits `fetchImpl` without a timeout or abort signal for the GraphQL, README, and avatar requests. If an upstream connection stalls, `handleEvaluationRequest` remains pending until the platform terminates the invocation, and the caller receives a failed evaluation.
Apply a bounded timeout to each upstream request and return a controlled error when it expires.
Also at `server/github.ts:67`, `server/github.ts:84`.| throw new Error(`GitHub REST API responded with ${response.status}`); | ||
| } | ||
| return response.text(); | ||
| } |
There was a problem hiding this comment.
🟡 Medium · Large READMEs exhaust function resources
When a public profile README is unusually large, getProfileReadme buffers the entire raw README, then evaluateProfile passes it to findTechnologies, which scans it repeatedly. A caller can target a profile with a very large README to consume excessive memory and CPU in the serverless function, causing evaluations to time out or fail.
Enforce a maximum README response size before buffering or parsing its contents.
Prompt for AI agents
In `server/github.ts` at line 76:
**Large READMEs exhaust function resources**
When a public profile README is unusually large, `getProfileReadme` buffers the entire raw README, then `evaluateProfile` passes it to `findTechnologies`, which scans it repeatedly. A caller can target a profile with a very large README to consume excessive memory and CPU in the serverless function, causing evaluations to time out or fail.
Enforce a maximum README response size before buffering or parsing its contents.| } | ||
|
|
||
| try { | ||
| const result = await evaluateProfile(new GitHubClient(token), username); |
There was a problem hiding this comment.
🟡 Medium · Development loads duplicate GitHub calls
When a developer loads an evaluation page with the development server running, react StrictMode runs the evaluation effect twice in development; aborting the first browser fetch does not cancel handleEvaluationRequest or its evaluateProfile call, so both requests make the GitHub profile, README, and avatar requests and consume the configured token’s quota.
Coalesce concurrent evaluations for the same username in the development server, or otherwise prevent the StrictMode cleanup request from starting duplicate GitHub work.
Prompt for AI agents
In `server/handler.ts` at line 31:
**Development loads duplicate GitHub calls**
When a developer loads an evaluation page with the development server running, react StrictMode runs the evaluation effect twice in development; aborting the first browser fetch does not cancel `handleEvaluationRequest` or its `evaluateProfile` call, so both requests make the GitHub profile, README, and avatar requests and consume the configured token’s quota.
Coalesce concurrent evaluations for the same username in the development server, or otherwise prevent the StrictMode cleanup request from starting duplicate GitHub work.| const result = await evaluateProfile(new GitHubClient(token), username); | ||
| if (!result) return json(404, { error: "User not found" }); | ||
| return json(200, result, { | ||
| "Cache-Control": "public, s-maxage=600, stale-while-revalidate=3600", |
There was a problem hiding this comment.
🟡 Medium · Profile grades stay stale too long
When a profile changes after its cached result has expired and a request arrives during the stale-while-revalidate window, vercel can serve the cached response for up to an additional hour while revalidating it, so a changed profile continues to show its old grade and evaluatedAt after the documented 10-minute cache period.
Remove the extra stale window or shorten it to the maximum staleness the app intends to expose.
Prompt for AI agents
In `server/handler.ts` at line 34:
**Profile grades stay stale too long**
When a profile changes after its cached result has expired and a request arrives during the stale-while-revalidate window, vercel can serve the cached response for up to an additional hour while revalidating it, so a changed profile continues to show its old grade and `evaluatedAt` after the documented 10-minute cache period.
Remove the extra stale window or shorten it to the maximum staleness the app intends to expose.| for (const match of readme.matchAll(/skillicons\.dev\/icons\?i=([\w,+#.-]+)/gi)) { | ||
| ids.push(...match[1].split(",")); | ||
| } | ||
| for (const match of readme.matchAll(/devicons?\/(?:devicon\/[^"'\s)]*?)?icons\/([\w+#.-]+)\//gi)) { |
There was a problem hiding this comment.
🟠 High · Devicon technologies are omitted
When a README uses a Devicon image URL with a version suffix such as @latest, the URL pattern in findTechnologies does not match that Devicon path, so it removes the URL without extracting its icon ID and awards too few technology points.
Update the Devicon URL pattern to match version-suffixed paths such as devicon@latest/icons/....
Prompt for AI agents
In `server/technologies.ts` at line 99:
**Devicon technologies are omitted**
When a README uses a Devicon image URL with a version suffix such as `@latest`, the URL pattern in `findTechnologies` does not match that Devicon path, so it removes the URL without extracting its icon ID and awards too few technology points.
Update the Devicon URL pattern to match version-suffixed paths such as `devicon@latest/icons/...`.| import { evaluateProfile } from "./evaluate.js"; | ||
| import type { GitHubClient, GitHubProfile } from "./github.js"; | ||
|
|
||
| const PHOTO = new Uint8Array([0xff, 0xd8, 0xff]); |
There was a problem hiding this comment.
🟡 Medium · Default avatars still earn photo points
When the evaluator is changed to grant the photo point without checking whether the avatar is a default identicon, the “gives 10 points for each passed criterion” test still passes if evaluateProfile unconditionally awards the photo point: its fake client returns this JPEG for the avatar, which the current identicon check already treats as non-identicon. The test therefore does not catch default avatars earning points.
Use a valid default-identicon PNG fixture in an evaluateProfile test and assert that hasPhoto is false.
Prompt for AI agents
In `server/evaluate.test.ts` at line 5:
**Default avatars still earn photo points**
When the evaluator is changed to grant the photo point without checking whether the avatar is a default identicon, the “gives 10 points for each passed criterion” test still passes if `evaluateProfile` unconditionally awards the photo point: its fake client returns this JPEG for the avatar, which the current identicon check already treats as non-identicon. The test therefore does not catch default avatars earning points.
Use a valid default-identicon PNG fixture in an `evaluateProfile` test and assert that `hasPhoto` is false.
Summary
The external evaluator API is being shut down. This PR moves the evaluation into this repo. It follows #15, which upgraded the stack.
GET /api/evaluation?user=<username>. It reads public data from the GitHub GraphQL and REST APIs and calculates the 10 criteria.api/and also innpm run dev, through a Vite middleware.server/and the shared types inshared/.Changes in the criteria
Required setup
Add a
GITHUB_TOKENenvironment variable in the Vercel project before you merge. A fine-grained token with public repository access only is enough. Without it, the endpoint returns 500 and the site shows the error page.Pending before merge
package-lock.json. The install could not run because the local registry token expired. CI fails until the lockfile exists, on this branch and onmain.GITHUB_TOKENis set.Test results
The 14 unit tests pass. They ran with Bun, because Vitest is not installed yet. The endpoint was tested against real GitHub users, a user that does not exist (404), and an invalid username (400).
The Vercel preview build passes. On the preview,
/evaluation/:usernameloads, and/api/evaluationruns and returns500 Server is not configured, becauseGITHUB_TOKENis not set yet.🤖 Generated with Claude Code
Summary by Sourcery
Move GitHub profile evaluation into the project and expose it through a locally and remotely hosted API.
New Features:
Bug Fixes:
Enhancements:
CI:
Deployment:
Documentation:
Tests: