The model said 40k context and gave you 4k

A local inference server advertises a 40,000-token context and silently truncates at 4,000. The app in the repo trusts the advertised number, and users are losing the start of every long document. Prove it and fix it.

Requirements

Write a probe that measures the real context window of any OpenAI-compatible endpoint (a fake server in the repo reproduces the bug)

Make the app use the measured window, not the advertised one

When a document exceeds the window, chunk it with overlap instead of dropping the head

Show the user what was and was not included

A test against the fake server that fails on the old code

llmdebuggingollamatypescript

Intermediate

View on GitHub

GoShip's MCP server tracks how you interact with AI — not your code. Connect it to get scored and improve your workflow.

Discussion (0)

No comments yet

Be the first to share your thoughts on this challenge.