If it were, I can in theory see situations where improving content cleanliness is worth blowing away the KV cache.
But I absolutely can't see how feeding the entire context into a more expensive model multiple times per task, just to propose context edits that might indirectly help, could ever be worthwhile.
I added a 2nd while loop so that it takes user input. And vendored my tiny llm lib (so it's 150 lines now, and dependency free :)
---
As for context-sculpting, the economics are different when not touching the context gives you the >98% discount everyone's doing now. (Although it might be worth fiddling with the suffix... not sure yet!)
e.g. this issue: "ToolSearch saves ~15K tokens per request in prompt size, but at the cost of breaking prefix-based caching for models like DeepSeek that rely on stable prefixes. For heavy users of DeepSeek through OpenRouter, the savings from smaller prompts are dwarfed by the increased cost from cache misses."
i definitely considered something like this for the local-first harness i made ... i just don't think most people have the RAM to be able to run two good models yet. maybe i'm wrong though. but i also think a single "agent" can compartmentalize itself into subdivisions better than we imagine (i.e., much much better than any single human can). i ended up creating a broker, though, so at least the tool calls don't eat up as much context. and the auto-reset thing is definitely legit.
But I absolutely can't see how feeding the entire context into a more expensive model multiple times per task, just to propose context edits that might indirectly help, could ever be worthwhile.
https://minimal-agent.com/
I followed this tutorial earlier today and I'm having a lot of fun with it.
https://gist.github.com/a-n-d-a-i/cb5e929b4c87b8d185760d0264...
I added a 2nd while loop so that it takes user input. And vendored my tiny llm lib (so it's 150 lines now, and dependency free :)
---
As for context-sculpting, the economics are different when not touching the context gives you the >98% discount everyone's doing now. (Although it might be worth fiddling with the suffix... not sure yet!)
e.g. this issue: "ToolSearch saves ~15K tokens per request in prompt size, but at the cost of breaking prefix-based caching for models like DeepSeek that rely on stable prefixes. For heavy users of DeepSeek through OpenRouter, the savings from smaller prompts are dwarfed by the increased cost from cache misses."
https://github.com/QwenLM/qwen-code/discussions/4065