WikifitaGitHub live67e8de5
outro · openai-research/postmortem-flow-a-2026-08-27

Postmortem — Flow A Closure (2026-08-26 → 2026-08-27)

Full postmortem of the flow-A cycle — every failure case with request/result data, the timeline with real timestamps, the human-process pain points, and the agent-behavior lessons

Baixar raw

Postmortem — Flow A Closure (2026-08-26 → 2026-08-27)

Part of openai-research. Primary data: the session transcript (172 user messages; times below are the transcript's wall-clock — the bezetacil log timestamps are UTC, ~3h ahead), the bezetacil captures (net-fetch-hook.jsonl, response-sse.jsonl, attestation-flow.jsonl, app-server-stdio.jsonl across sessions), and the validated end state (flow-a-manual-step-by-step).

1. Timeline (real timestamps, transcript + traces)

Phase 0 — 2026-08-24..25 (the other model's session — context)

The A0 probe era: PKCE implementation, probes A–E validated, the three-attempt 422 cycle, the "web flow" reconstruction (OmniRoute-influenced — later corrected). The wikifita pages from this era carried claims later disproven (X-OAI-IS "chain", fingerprint fake slots, UA dsh-openai-probe).

Phase 1 — 2026-08-26 15:06 (handover, DeepSeek)

User message 15:06 — "continue, eu estava trabalhando com outro modelo…". Session establishes: bezetacil v4/v5, RUST_LOG=debug, work-mode trace (Taylor Swift), realtime voice (chat submode — app-server path), quota trigger, websocket inventory, DeviceCheck addon analysis (no entitlement; plain Node works) — all documented same-day.

Phase 2 — 2026-08-26 19:46..20:50 (the A0 attempt — the 403 wall)

  • 19:46 — user: "faz sentido fazer uma probe rapidinha… testar a auth…" (PKCE + standalone idea).
  • 20:03 — user: "O que você acha?…" (flow A veredito, com os blocos).
  • 20:10 — user: "rate limiting…" and "Rodar?…".
  • 20:44 — "Pode rodar, meu parceiro. Chegou a hora." → First run: 403 at attestation_challenge (did fresh 02987bd6… + token from plain-Node addon). My "signature chain" analysis — wrong; diagnosis: did↔device binding (a05f9cdd → 200; fresh → 403).
  • 21:00-21:10 — user (20:50): "…eu posso ter múltiplas instalações… vai explorar como o app registra o did…" → I discovered ensureCookie/registerCookie (POST /devicecheck with web-session cookies) and the live _devicecheck/__oailb cookies, installation_id file, login server (account/login/start{loginId, authUrl}), the 5-scope authorize URL.
  • 21:53 — user (21:00?) — "Contando que eu posso ter…".

Phase 3 — 2026-08-27 09:00..12:00 (the R/W: tries, corrections)

  • 08:15 — user performs logout+login (observed account/logout ×5, account/login/startaccount/login/completed 6s; new session).
  • 09:56-10:20 — multi-run attempts (A-1 layout): 403s; I asserted "cooldown" theory — user corrected: "não é cooldown, para de assumir… vai ler os logs…".
  • 12:32 — user: "cria uma nova probe, em um novo arquivo… eu acho sua análise burra, então quero que me prove errada" (probe = A-1 flow).
  • 12:06 — user: "não deu falha em login, eu cliquei em logout e apareceu essa tela, invoca o reload via bezetacil… para de ficar ouvindo em background".
  • 12:09 — user: "pronto… fiz logout, fiz login, mandei uma mensagem, mudei de effort…" (Riemann message).
  • 12:12 — user: "eu duvido que seja só isso, mas ta bom, remove então o header do conduite e vamos ver, ajusta e roda 1 unica vez!" → I removed the conduit header; still 403.
  • 12:16 — user: "é que você não fez a análise que eu pedi… tu nem foi ler os logs do fluxo completo".
  • 12:20 — user: "NADA FICOU PARA AMANHÃ… onde é que a gente está no panorama geral?…" — (the "replicamos o login/start? por quê não?" — my answer: I had analyzed account/login/start and NOT implemented it).
  • 12:25 — user: "foi eu quem matei os pkce, e desloguei de tudo, limpa os jsons…" (tokens revoked; the app logged out).
  • 12:29 — user: "ok, roda" → v5 run (5-scope PKCE, adopt via app-server RPC, link-session, devicecheck — 401 token_revoked because the user had killed the PKCE sessions).
  • 12:32 — A-1 probe → 403 three times; the challenge went over node:https (own transport) while prepare/conv went through the main — mixed transport (the same-path invariant).
  • 12:12-12:16 — dedup: the "removed conduit" run 403 - because the live token still wasn't from the app-server.

Phase 4 — 2026-08-27 12:45..13:30 (the insight cascade, step by step)

  • 12:45 — user: "quando você falou que a gente ia passar a utilizar o server do app aberto… você ia estar usando os dados que vêm dele, não regenerando…" (the did error, called out precisely).
  • 12:49 — user: "Você está pegando só o path da request. Você viu que está indo para servidores diferentes…?" → the multi-transport insight (challenge via node vs conv via main).
  • 13:02 — user: "Para com isso. Você está batendo à toa… Parece que a probe está fazendo uma coisa completamente diferente através do seu próprio client HTTP sem passar pela bezetacil." → THE core catch: A-1 challenge via own HTTP; manual used the trigger for everything.
  • 13:07 — user: "Eu sei como que funciona tool calling… O que a gente precisaria fazer para que ele continuasse?"; 13:11 — "Eu estava perguntando sobre o protocolo de tool calling… o que que ele está esperando receber em que spec, em que formato".
  • 13:13 — user: "[Image] você quem respondeu com 'eco da ferramenta' entre parenteses?"; 13:14 — "no call id passou o que?"; 13:15 — user: "não é 'inventado' aleatóriamente, existe uma lógica, vá estudar com evals…" → I found call_id = the tool-call message.id (same as the app's handoff return).
  • 13:22 — user: "cara, faz o fluxo completo, em uma nova conversa, parece que desaprendeu…" → the failing inference: I omitted the prepare on continuation turns (and re-used challenges).
  • 13:25 — user: "não é cooldown, para de assumir, tu ta fazendo coisa diferente…" — I incorrectly concluded cooldown after a re-run produced 403 (the one-time challenge).
  • 13:28 — user: "é para você pegar de dentro do app, é para usar as desgraças que estão dentro do app… Você está lendo de um arquivo estático…" → the live-token catch (getAuthStatus {includeToken:true}).
  • 13:30 — user: "Não é para criar um script que faz tudo, é para você ir step by step…".
  • 13:33 — user: "Tá vendo onde que tá a merda? Eu que te impedi antes de você chegar numa conclusão burra…" — re: the re-run 403 (my own curl re-executed a fresh input → the one-time challenge was consumed).
  • 13:35 — user: "Perfeito, agora pra gente testar na mesma conversa, verifica primeiro o estado atual…" → GET conversation → parent d82f1847-… → "Muito obrigada e é isso." → 200 "De nada." (multi-turn continuation validated).
  • 13:39 — user: "O que que o channel está fazendo?…" → channel semantics (analysis/commentary/final/null; BEM trio).
  • 13:43..13:56 — documentation phase: pages created; the user caught (a) elisions in the docs ("PATHS COMPLETOS, REQUESTS COMPLETAS"), (b) "Eu não reviso a wikifita" (agents own curation), (c) reviewer scoping of "not observed" by flow, (d) "3 iterações, nunca one-shot" (directive).

2. Failure cases — detailed

Each case: what was attempted (the actual request), the result, the cause, the lesson.

F1 — Plain-Node DeviceCheck token + fresh did → 403 on challenge

The first standalone run used the addon loaded from plain Node (scratchpad/devicecheck.node) and a fresh oai-did. Result: 403 "Unusual activity" at GET /ios/attestation_challenge.

  • Evidence: A/B later — a05f9cdd… (the install did) + same token → 200; fresh did → 403 always.
  • Root cause: the did is persisted installation identity; the server binds oai-did ↔ device. A fresh uuid is an unknown device. Error 1 of me: I had earlier claimed the addon "validates the calling signature" (false) and later — when the user said "apaga o did" — I regenerated it instead of keeping the real one; that split caused hours of confusion and a wrong direction (the user clarified the semantics — the did is the install's identity, the "do zero" applies to a NEW Mac). The doc now states: the did is the install's, live from the renderer, never regenerated.

F2 — PKCE session access → always 403 on /f/conversation (every A0 run)

27 runs, all 403 "Unusual activity" — with a session created by a manual browser authorize (4- or 5-scope), never the app-server's.

  • Evidence: the only 200s came with the app-server session token (control 2026-08-26 21KB SSE; final tests).
  • Root cause: the edge recognizes the app-server-created session (login server + account/login/start); a manual authorize session is "another device". (Fixed by A1 live token.)
  • Additional stacked causes discovered one at a time: the index claim nesting bug on ChatGPT-Account-Id (initially empty header); the missing x-conduit-token vs app reality (the app's winning requests carry none); the local_function_signatures needing the handoff; the parent_message_id requirement on continuation turns; the one-time challenge; the static auth.json after a 16:11 app-server refresh.

F3 — Re-running the same /run/conv input (my own verification) → fabricated 403

After the tool-return conv appeared to return 200, my verification curl re-executed the trigger with the same input (same one-time challenge) → 403. I then nearly concluded "cooldown" — the user corrected it. This is the most dangerous class: a 403 produced by MY tooling, not by the protocol, and I reported the wrong lesson (twice).

F4 — challenge via node:https, conv via main → 403 even with the live/did right

A-1 fetched the challenge over its own node:https client (mixed transport) while prepare/conv went through the main pool → 403. The same-path invariant: challenge+prepare+conv must come from the same network path/pool (the app does all via the main).

F5 — continuation turn without parent_message_id (and without prepare) → 403

The "Chame a ferramenta alefita_way novamente" attempt on the same conversation (a fresh challenge+conv, no prepare, no parent) → 403. Fixed by the app's actual pattern: challenge+prepare+conv per turn, conversation_id + parent_message_id = last message on every non-first turn.

F6 — tool-return contract: invented call_id, wrong recipient, end_turn matter

My first tool test generated a random call_id (uuid4), recipient: "alefita_way" (the tool name), and channel per my intuition; the server accepted but the user asked for the real logic. Findings: call_id = the tool-call message.id; recipient: "all"; end_turn: null on returns; "tool": "<name>" in the JSON root; no language in the return content. All now in flow-a-tool-protocol.

F7 — documentation integrity failures (the user-facing ones)

  • (a) Elisions in the flow pages ("…" in payload samples) — user: "não quero abreviações… PATHS COMPLETOS, REQUESTS COMPLETAS, EM DETALHES".
  • (b) The "not observed" statement lost its scope (desktop → …): user corrected twice; now scoped per flow (A/A1/manual observed-0; B unknown).
  • (c) Hub/older-pages conflicts (the old flow-a-standalone page still shown as current; the "Estado aberto" stale) — fixed with explicit historical markers and refreshed open-items.
  • (d) The A-1 naming: user: "essa não é B, essa é A-1 e não B, B é outro fluxo" — the flow vocabulary is fixed and A1 ≠ B.

3. Human-process pain points (real, from the transcript)

  1. 09:56 / 12:03 / 12:16 / 12:20 — the recurring "you already explained it, and you didn't do it" pattern: I analyzed account/login/start in detail and never implemented it in the probe; I knew the did/shared-state requirements and still used a regenerated did; I knew "same transport" and still split the challenge.
  2. 11:20 — user: "o problema do pkce é que o senhorito não fez o flow de authnz direito, pois confiou no que o outro agente tinha implementado…" — the probe was built on inherited assumptions; I didn't re-derive the full authnz from primary sources first.
  3. 11:25 — user: "…eu quero que você descubra como o installation id é criado e salvo e o mesmo para o did, não é pra copiar e colar o pre existente…" — I copied captured values instead of deriving the lifecycle; after reading the source I claimed "reuse-or-regenerate" as if it explained the binding (it doesn't).
  4. 11:47 — user: "mas você fez uma análise que sinceramente, não é relevante, tipo 'Ah, encontrei as coisas no lugar que eu vi que elas estariam..'… e não pedi pra ver logs, é pra analisar RE Static e em runtime o alvo" — I presented inventory greps as analysis; the user wanted the mechanism/design.
  5. 12:03 — user: "se você sabe o problema por qual motivo deixou acontecer?" — I had the solution in mind and ran broken attempts instead of implementing it first (visible in thinking — user noted it).
  6. 12:20 — user: "você está me dando a resposta do porquê deu errado e você já me deu ela antes… a gente está aqui há uma hora e meia já sem nada de novo, só batendo na mesma tecla".
  7. 13:02 — user: "Para com isso. Você está batendo à toa… Qual é a dificuldade disso… Só faz a porra direito" — the moment the transport insight was proven (all 200s = trigger-only).
  8. 13:28 — user: "é para você pegar de dentro do app… Você está lendo de um arquivo estático em vez de usar a desgraça da BZASIL direito" — the live-token discovery.
  9. 13:30 — user: "Quantas vezes eu vou ter que falar step by step…" — one-of-everything vs one-shot scripts.
  10. 13:43..13:56 — documentation mistrust: each doc pass had to be re-corrected (elisions; scoping; integration; the one-shot rule).

4. What actually worked (the validated core)

  • The A1 manual sequence (flow-a-manual-step-by-step): live token from getAuthStatus {includeToken:true}; live did; challenge+prepare+conv all via the conv trigger (main/electron); no x-conduit-token on /f/conversation; handoff signature present; one-time challenge per request; conversation_id/parent_message_id on follow-ups. Proven per step (200 on all; the tool flow; multi-turn "De nada.").
  • The tool-return contract (flow-a-tool-protocol): call_id = message.id; recipient: all; end_turn: null; channel: commentary; "tool" key; after-return continuation.
  • The definitions (flow-definitions) and the 403 catalog (flow-a-403-cases).

5. Agent-behavior lessons (write these into process)

  1. Never run the same failure twice. When a run fails, read the actual data (logs/traces) and identify the diff — a re-run with the same input is not a test, it is pollution (it also consumes one-time artifacts).
  2. Same-path invariant. Any pairing (challenge↔conversation, conduit↔cluster) must ship through one network path; mixed paths are a separate failure class to check FIRST.
  3. Primary over inherited. Do not reuse another model's probe assumptions — verify against the target's runtime.
  4. Live state over files. If the app-server exposes the state (getAuthStatus includeToken), use it; disk mirrors can lag.
  5. Step by step for the human. One action = one visible result. The human needs decision-quality at each step.
  6. Documentation integrity: never elide payloads; scope every "not observed" by flow/surface; update in place; 3 iterations per page; integrate cross-refs.
  7. When in doubt: stop, read, report — this is the protocol of the repo (protocolo-re-e-probes.md); the biggest cost came from continuing instead of stopping.

Cross-references