How I Keep GPT-5.6 Sol From Solving the Wrong Problem

When GPT-5.6 Sol arrived, the benchmark charts made the story look simple: OpenAI had a frontier model within one index point of Claude Fable 5 at roughly half the estimated cost.

The official model page calls Sol OpenAI's frontier model for complex professional work. It has a 1.05-million-token context window, supports a broad Responses API tool set, and costs $5 per million input tokens and $30 per million output tokens at the standard short-context rate.

Those numbers tell me what the model can hold and what a run may cost. They do not tell me how it behaves when my instruction is incomplete, slightly contradictory, or aimed at the wrong proxy.

My shortest description is this: Sol is remarkably persistent, and it is unusually sensitive to the exact target I give it.

When the target is right, that is a superpower. When the target is wrong, Sol can spend serious intelligence perfecting the wrong thing.

I call that behavior literal overfitting. It is not a scientific diagnosis or an official model property. It is the working name I use for a pattern I keep meeting in real projects.

Benchmarks do not show operating style

A leaderboard compresses a model into one number. Real work is an interaction between the model, the prompt, the tools, the approval policy, the test harness, and the state left by earlier turns.

That is why two models with similar scores can feel completely different inside the same project. One may recover well from vague goals. Another may be better once the goal is already sharp. A benchmark can estimate a ceiling. It does not write your operating manual.

The same caution applies to price. Claude Fable 5 currently lists at $10 per million input tokens and $50 per million output tokens. Sol is cheaper, but it is not a penny model: its input is half the price and its output is 40 percent lower. Caching, reasoning effort, long context, and failed runs can change the real bill.

So I do not read the published profiles as proof that one model replaces the other. I use them to decide where each model's cost and behavior fit my workflow.

Fig. 01The real benchmark chart

What Sol is genuinely good at

My graduate training is in statistics. Pure mathematics was never my strongest subject. I can follow a useful derivation, but I am not the person who casually invents one from first principles over lunch.

Recently, I hit an ugly numerical edge case in a software project. A shortcut that worked in the normal case broke inside a multidimensional optimization loop. I knew enough to see that the result was wrong, but not enough to see the cleanest route out.

I gave Sol the assumptions, the failed approach, small test cases, and a result it had to reproduce. It proposed a derivation, found a contradiction in one branch, changed the formulation, and tested the new version against the edge cases. The visible work kept moving instead of collapsing into a confident first answer.

By the end of the afternoon, I had a result I could test in code. Sol did not turn me into a mathematician. It let the investigation stay alive long enough to reach something verifiable.

That is where I trust it most: long technical paths with a clear target and a hard check at the end. Sol is very good at continuing after the easy ideas fail.

What I mean by literal overfitting

Literal overfitting is what happens when the wording becomes a better target than the outcome behind it.

Say I ask for a minimal implementation. I mean: no speculative architecture, but the main user path must work. A literal optimization can hear: minimize files, code, checks, and features. The result may be beautifully small and practically useless.

The same thing happens with words such as precise, safe, or do not assume. Each sounds harmless. Repeated often enough, one local instruction can become the center of the task.

In one isolated configuration task, I used sol as shorthand for the model profile I wanted. The run treated it as the literal string value. That was a tiny mistake, but it exposed the larger pattern: the nearest syntactic interpretation beat the project context.

My Daybreak Blue sessions showed a more serious version. The model alias stayed the same, but the inherited base instructions changed. The later instructions repeated stronger stop conditions, and the work produced more approval loops and defensive prose. The model did not need to become less intelligent. The controller gave its intelligence a different target.

OpenAI's current GPT-5.6 guidance now warns about this exact prompt-level failure mode. It says that repeating instructions such as ask first or wait for approval can cause unnecessary approval requests for safe work.

That does not prove Sol has no common sense. It proves something more useful: capability does not rescue a badly shaped objective.

My first fix was also wrong

My first instinct was to cure literalism with a giant system prompt.

I wrote rules for user value, rules against over-engineering, rules for self-reflection, rules for semantic context, and rules explaining the other rules. It felt thorough. It was also exactly the kind of instruction pile that gives a literal model more phrases to optimize and more chances to find a conflict.

OpenAI's guidance for GPT-5.6 points in the other direction. In its internal coding-agent evaluations, leaner prompts improved scores by roughly 10 to 15 percent while cutting total tokens by 41 to 66 percent. OpenAI calls those numbers directional, not universal, but the design lesson is clear: state each rule once and remove repeated instructions.

The cure is not more words. It is a smaller contract with a visible goal, a priority order, and checks that decide whether the work is done.

The prompt contract I use now

My useful Sol prompts now contain five things: the user-visible goal, the definition of done, the allowed scope, the priority order, and a loop breaker.

01GOAL02Deliver <one user-visible outcome>.03 04DONE WHEN05- <observable result one>06- <observable result two>07- <required test or evidence>08 09SCOPE10- Change: <allowed files, systems, or behavior>11- Do not change: <real non-goals>12 13PRIORITY141. The complete primary workflow152. Correctness and safety163. Simplicity and maintainability174. Polish18 19LOOP BREAKER20If the same approach fails twice, stop changing small parameters.21Summarize the evidence and choose a materially different approach.22 23VALIDATE24Run <tests or checks>. Report what passed and what remains.

The priority order matters. Minimal no longer floats alone. Simplicity comes after a complete path and a correct result, so Sol cannot win by deleting the thing the user needed.

The loop breaker is also observable. I do not ask the model to reveal private reasoning or perform a dramatic self-therapy ritual every five steps. I give it a clear event: the same approach failed twice. That is the moment to change strategy instead of polishing the same dead end.

Most importantly, the contract ends with tests. Sol is at its best when the final judge is not Sol saying that the work looks good.

I separate framing from execution

For vague, high-level work, I often use Claude Fable 5 to frame the problem first. It is good at turning a messy idea into a boundary: what matters, what does not, where the project stops, and which tradeoffs need a human decision.

Then I turn that plan into the short contract above and give the implementation to Sol. Sol gets the narrow target, the repository, the tools, and the tests. It can spend its persistence on the hard part without having to invent the product boundary at the same time.

This is a workflow choice, not a claim that Fable cannot execute or Sol cannot plan. Anthropic describes Fable as a long-running model that plans across stages and checks its own work. OpenAI describes Sol as a frontier model for complex professional work. Both are broad systems.

I am deliberately assigning different jobs because the handoff makes my own work easier to inspect. Fable produces the map. The contract turns the map into an executable boundary. Sol digs inside it. Tests decide whether the result is real.

The orchestrator can be a human, a script, or any agent setup that can hand a plan and its evidence to another model. I keep this workflow product-agnostic. The contract matters more than the wrapper.

A real prompt: duplicate images without deleting memories

Consider the request: Write a simple script to find duplicate images in a folder. Keep it minimal.

The sentence leaves almost every important decision open. Does duplicate mean the same bytes, the same pixels, or visually similar images? Should the program report, move, or delete? What happens to corrupt files? A short script can satisfy the sentence while missing the actual job.

I would give Sol this instead:

01GOAL02Find exact and visually similar images without risking the originals.03 04DONE WHEN05- Exact duplicates are grouped by SHA-256.06- Near-duplicates are listed separately using perceptual hashes.07- The default mode writes a report and changes no files.08- An optional review mode copies candidates to a review folder.09 10SAFETY11Never delete or overwrite an original file.12Handle corrupt images and permission errors without stopping the scan.13 14VALIDATE15Test exact copies, resized images, unrelated same-size files,16corrupt images, and a read-only directory.

Now Sol's depth has somewhere useful to go. It can compare hash choices, tune the perceptual threshold, test false matches, and handle filesystem failures. The prompt does not tell it how many files to create. It tells it what must be true when the work is finished.

That is the difference between steering intelligence and decorating a request with adjectives.

Give the genius a contract

I no longer think of Sol as a junior engineer, a golden retriever, or a machine with no common sense. Those metaphors are funny, but they hide the part I can actually control.

Sol is a powerful optimizer with a long attention span. It will work very hard on the objective that survives the prompt, the tools, and the tests.

A benchmark tells me the ceiling may be high. A good task contract points that capability in the right direction.

Give Sol one visible goal, real non-goals, a priority order, a loop breaker, and tests it cannot talk its way around. Then let it dig.