Porting a Production Workload Between OpenAI and Claude

Last updated: August 2026

A client asked us to move a document classification service from one frontier vendor to the other. The estimate they had been handed was two days, and the API surface really is about two days. The other three weeks went into things no migration guide mentions: an invoice well above the rate card, a cache hit rate that collapsed on day one, and a dashboard that stayed green while some traffic got answered by a different model.

This is not a which-is-better piece. We wrote that one: ChatGPT vs Claude vs Gemini. This is the port itself, in the order the problems arrive, with both directions in the tables. Every number was read off vendor documentation on 11 August 2026, and one is scheduled to change three weeks later.

The Easy Half, and Why Nobody Should Bill You For It

Endpoint shape, message roles, system prompt placement, streaming events, tool schemas. All mechanical: one engineer, one day, plus an afternoon of test fixes.

Failure One: Token Counts Do Not Transfer, Even Inside One Lineup

Two vendors quoting the same dollar per million tokens are not quoting the same price, because they are not counting the same tokens. It holds inside one vendor's own family too. From Anthropic's pricing page on 11 August 2026: Claude 4.7 and later models use a newer tokenizer that produces approximately 30 percent more tokens for the same text, with the exact increase depending on content and workload shape. Sonnet 4.6 and earlier use the previous one. Moving up the lineup at an identical rate is a bigger bill for identical documents, and nothing on the rate card says so.

Do not estimate Claude tokens with tiktoken. It is OpenAI's tokenizer, and across our corpora it undercounts Claude by roughly 15 to 20 percent on prose and further on code. Anthropic's own instruction is to recount against the model you intend to deploy rather than reuse a number measured on an earlier one. POST /v1/messages/count_tokens is free and counts under whichever model ID you pass.

# Re-price a migration on your own corpus, not on the rate card.
# The only authoritative token count is the one the target vendor returns.
# pip install anthropic tiktoken
import glob
import tiktoken
from anthropic import Anthropic

client = Anthropic()
enc = tiktoken.get_encoding("o200k_base")   # OpenAI side, counted locally

# USD per 1M input tokens, read off the vendor pricing pages on 2026-08-11.
PRICE_IN = {
    "gpt-5.6-terra":   2.00,
    "claude-sonnet-5": 2.00,   # introductory; 3.00 from 2026-09-01
    "claude-opus-5":   5.00,
}

SYSTEM = open("prompts/classifier_system.txt", encoding="utf-8").read()
totals = {"gpt-5.6-terra": 0, "claude-sonnet-5": 0, "claude-opus-5": 0}

for path in sorted(glob.glob("corpus/*.txt"))[:500]:
    doc = open(path, encoding="utf-8").read()
    totals["gpt-5.6-terra"] += len(enc.encode(SYSTEM + doc))
    for model in ("claude-sonnet-5", "claude-opus-5"):
        r = client.messages.count_tokens(
            model=model,                    # counts under THAT model's tokenizer
            system=SYSTEM,
            messages=[{"role": "user", "content": doc}],
        )
        totals[model] += r.input_tokens

base = totals["gpt-5.6-terra"]
for model, tokens in totals.items():
    cost = tokens / 1e6 * PRICE_IN[model]
    print(f"{model:16} {tokens:>10,} tok  {tokens / base:5.2f}x  ${cost:8.2f} / 500 docs")

# count_tokens is free and rate limited separately (2,000 RPM on Start,
# 4,000 on Build, 8,000 on Scale as of 2026-08-11). Then replay the same
# 500 documents through both APIs for real and diff the usage blocks.

Run it before the business case. On one recent job the corpus repriced at 1.28x, turning a projected 20 percent saving into break-even. Headline prices, read 11 August 2026:

Watch the Sonnet 5 row. An introductory rate expiring weeks after your research is the number that ends up in a board deck, quoted for a year.

Failure Two: Caching Semantics, Not Caching Existence

Both cache prompt prefixes, and not the same way, so a layout that is optimal on one is a cost regression on the other. The fix is reordering the prefix, not changing models.

The minimum prefix row catches people because it varies by model, not by vendor: 512 tokens on Opus 5, 1,024 on Sonnet 5 and Opus 4.8, 2,048 on Opus 4.7, 4,096 on Opus 4.6, 4.5 and Haiku 4.5. A 1,500 token system prompt caches on one and silently caches nothing on the next. If both usage cache counters read zero, you are paying full price.

• Prefix order is a hierarchy: tools, then system, then messages. Editing one tool description invalidates everything below it, which is how a docstring edit wipes a hit rate.
• Effort is part of the prompt. Changing output_config.effort between requests re-renders the prompt and always invalidates the cached message blocks, so varying effort inside one cached conversation costs more than it saves. One exception: Anthropic documents high as behaving exactly like omitting the field, so passing the model's own default invalidates nothing.

Failure Three: A Refusal Is an HTTP 200

Claude Fable 5 and Claude Opus 5 carry safety classifiers that can decline a request. You get a normal 200, empty content, stop_reason set to refusal, and stop_details naming the category: cyber, bio, frontier_llm, reasoning_extraction or general_harms. Branch on stop_reason and nothing else. Both category and explanation come back null when a refusal maps to no named category, and that null is a permanent valid state rather than a placeholder. The explanation text is documented as unstable, so display it, never parse it. A refusal before any output is not billed, a mid-stream one bills what streamed, and both burn rate limit budget without tripping a status-code SLO.

Retrying elsewhere is opt in. Set fallbacks to "default" with the server-side-fallback-2026-07-01 beta header, or name up to three models. The answering model shows up in the top level model field and as a fallback_message entry in usage.iterations. The parameter is rejected inside batches and unavailable on Bedrock, Google Cloud and Microsoft Foundry, where SDK middleware covers it. Two gaps to size for: a category with no recommended fallback still refuses, and if the fallback model is itself rate limited or overloaded the retry never runs and the original refusal comes back. Fallbacks degrade to refusals under exactly the load that makes you want them.

# A refusal is HTTP 200. If you only catch exceptions, your success rate stays
# green while a share of traffic is answered by a different model.
from anthropic import Anthropic

client = Anthropic()
PRIMARY = "claude-opus-5"

def classify(document: str) -> dict:
    r = client.beta.messages.create(
        model=PRIMARY,
        max_tokens=1024,                     # required on /v1/messages
        fallbacks="default",                 # opt in; omit it and a refusal stands
        betas=["server-side-fallback-2026-07-01"],
        output_config={"effort": "medium"},  # pin it; the API default is "high"
        system="Classify the document. Reply with exactly one label.",
        messages=[{"role": "user", "content": document}],
    )

    served_by_fallback = any(
        it.type == "fallback_message" for it in (r.usage.iterations or [])
    )

    if r.stop_reason == "refusal":
        # content is empty. Not billed if nothing streamed, but it still
        # consumed rate limit budget. Count it as a failure, not a success.
        return {
            "ok": False,
            "reason": "refusal",
            "category": r.stop_details.category if r.stop_details else None,
            "model": r.model,
        }

    return {
        "ok": True,
        "label": r.content[0].text.strip(),
        "model": r.model,             # NOT always PRIMARY once fallbacks is on
        "fallback": served_by_fallback,
    }

# Emit r.model and served_by_fallback as metric dimensions from day one.
# Without them your eval numbers average over an unknown model mix.

Turn fallback on, but log the answering model from request one. Otherwise your eval set measures a mixture of two models and you lose a week on a routing change you mistook for a regression.

Failure Four: Reasoning Defaults Vary by Model and by Surface

Almost nobody sets this parameter on the first pass, which is the problem: both vendors pick a value for you, and they do not pick the same one. On Anthropic the parameter is output_config.effort, values low through max. The API default is high on Opus 4.7, 4.8, Opus 5, Sonnet 5, Sonnet 4.6 and Fable 5, and passing high is documented as identical to omitting it. Thinking tokens bill as output and count toward max_tokens, which is why the docs suggest 64k at xhigh or max.

On OpenAI the values run from none to max, each model supporting a subset, and the guidance page states GPT-5.6 defaults to medium if you omit it. Because the model reasons by default, developers report that a Chat Completions request carrying function tools fails with a 400 telling them to use /v1/responses or set reasoning_effort to none, on requests that never set it. That error string comes from the OpenAI forum and third-party issue trackers, not a docs page, but it matches OpenAI's own steer toward the Responses API for reasoning and tool calling.

So your notebook calls chat.completions without tools and works, your service calls it with tools and 400s, both on the same model ID. Pin effort explicitly in every environment, and sweep it on your evals rather than reusing the old model's level.

Failure Five: Rate Limits Follow the Account, Not the Contract

Throughput is priced off your payment history with that specific vendor. Your contract does not move it and the volume you were already running elsewhere does not either. A pipeline that ran fine on a three-year-old account gets 429s on a fresh one at identical volume, on cutover day.

OpenAI publishes per-model numbers on the model pages, not the rate limit guide. These rows were read for one model on 11 August 2026 without cross-checking model by model, so pull the numbers for the ID you deploy. New Anthropic organizations, and any with thin usage history, can land in the Evaluation tier below the published Start numbers.

Anthropic also documents acceleration limits: a sharp jump in usage produces 429s even inside your stated limit. A big-bang cutover is exactly that pattern, so ramp over days. The one asymmetry in your favour is that cache_read_input_tokens do not count toward ITPM on current Claude models, only uncached input and cache writes do, so the prefix work from failure two buys throughput as well as margin.

Tool Use and Structured Output: The Fields the Code Touches

// The same tool on both providers. Small differences, all load bearing.

// OpenAI, POST /v1/responses
{
  "model": "gpt-5.6-terra",
  "tools": [{
    "type": "function",
    "name": "lookup_invoice",
    "description": "Fetch an invoice by number.",
    "parameters": {
      "type": "object",
      "properties": {"invoice_no": {"type": "string"}},
      "required": ["invoice_no"],
      "additionalProperties": false
    },
    "strict": true
  }],
  "reasoning": {"effort": "medium"},
  "text": {"format": {"type": "json_schema", "name": "result", "schema": { /* elided */ }}}
}
// Model emits a function_call item carrying call_id.
// You reply with a function_call_output item keyed by the SAME call_id.

// Anthropic, POST /v1/messages
{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "tools": [{
    "name": "lookup_invoice",
    "description": "Fetch an invoice by number.",
    "input_schema": {
      "type": "object",
      "properties": {"invoice_no": {"type": "string"}},
      "required": ["invoice_no"],
      "additionalProperties": false
    },
    "strict": true
  }],
  "output_config": {
    "effort": "medium",
    "format": {"type": "json_schema", "schema": { /* elided */ }}
  }
}
// strict on Anthropic needs BOTH required and additionalProperties: false.
// Ship it with only required and the schema is rejected.
//
// Model emits a tool_use content block carrying id.
// You reply with a tool_result content block, inside a USER message, whose
// tool_use_id matches that id. Wrong role, no error: the model just answers
// as if the tool never ran.

Two things bite. Anthropic expects tool_result inside a user message, and a wrong role produces no error, just an answer that ignores the tool. And tools inject a system prompt costing 286 tokens on Opus 5 at tool_choice: auto, 675 on Opus 4.7, per request: invisible in code, visible on the bill.

The Compatibility Shim Will Pass Your Tests and Wreck Your Budget

Anthropic ships an OpenAI SDK compatibility layer: point base_url at https://api.anthropic.com/v1/, swap the key and model name, and your client runs. Anthropic says plainly it is for testing and comparing capabilities, not a production-ready solution. Read what it drops against the failures above.

• reasoning_effort is ignored. The cost and latency dial from failure four is gone. A thinking block can be smuggled through extra_body; output_config.effort has no documented equivalent.
• Prompt caching is not supported, and usage.prompt_tokens_details comes back empty, so failure two is unavailable and invisible.
• response_format and tool strict are ignored. Schema conformance stops being guaranteed while your parser keeps working, most of the time.
• message.refusal is always empty. The telemetry from failure three never reaches you.
• System and developer messages are hoisted into one leading system message, so a staged mid-conversation instruction jumps to the top.

seed, logit_bias, the penalties and user go the same way, and none of it raises an error, so your tests pass. Use the shim for evaluation, then write the native client.

If the Workload Is Offline, Batch Changes the Arithmetic

Before arguing about a 20 percent rate difference, check whether the workload needs to be synchronous. Both vendors halve the price of asynchronous work, which dwarfs the cross-vendor gap.

Batch and cache discounts stack on Anthropic. For a nightly enrichment job, moving to batch on your existing provider usually beats the whole migration. Fix the workload shape first.

The Sequence That Works

• Shadow for a week, do not switch. Fan real traffic out to both providers, discard the second response, log usage, latency, stop_reason and the answering model.
• Re-price from the shadow logs, including cache writes, cache reads, reasoning tokens and tool-use overhead, at dated prices.
• Rebuild the prefix for the target's cache rules before comparing. Measuring an optimized prompt against an unoptimized one is how teams talk themselves out of a sound port.
• Pin everything the target allows: model ID with its date suffix, effort, max_tokens, every beta header you rely on.
• Cut over one route at a time, cheapest first, old client behind a flag. Keep the flag for a full billing cycle, not a week: get a month-end invoice under the new provider before you delete anything.
• Ship two new alerts: share of responses from a non-primary model, and cache read ratio.

When I Would Not Do the Port at All

If the only driver is the rate card, do not port. A 20 percent headline gap disappears into a tokenizer that counts your documents differently, a cache prefix you have not rebuilt, effort defaults you have not swept, and a few weeks of engineering. On the last three we modelled, the saving landed inside the noise. The ports worth doing answer to something else.

• Data residency. Anthropic applies a 1.1x multiplier across input, output and cache pricing when you pin inference_geo to us on Claude 4.6 and later. Compliance becomes a line item you can price.
• Refusal behaviour that fits your domain. Security teams collide with cyber and life sciences teams with bio on benign work. Check the category first: reasoning_extraction means your prompt asks the model to print its own chain of thought, an hour of work either way.
• A latency profile you need. Anthropic's fast mode on Opus 5 and Opus 4.8 runs at $10 in and $50 out per million as of August 2026, exactly double standard, and it is first-party API only. If your compliance story requires Bedrock or Foundry, this reason evaporates.
• A capability you depend on: guaranteed schema conformance, a 1M token context at standard pricing, a server-side tool, or a governance feature your security review demands.

None of that argues for staying put. The teams that get hurt are the ones treating a provider swap as a config change. Budget three weeks and keep the old client warm through one invoice.