What Landed
Joseph Kim, OpenAI's Head of Financial Services, spoke on Latent Space. He said roughly 95% of ChatGPT for Financial Services, built with Morgan Stanley and Evercore, runs on the same public plugin primitives available to outside builders. The part that decides whether it wins is licensed data, firm templates, and integrations. In OpenAI's testing, some third-party financial MCP connectors from major vendors had error rates above 40%, so OpenAI chose to license and bundle the data itself. Kim also said public finance evals are saturated or broken. When flat GPT-6 "Astra" finance scores were investigated, the causes were wrong answer keys and poorly specified prompts.
Why It Matters
A frontier lab measured MCP connectors and found that being available is not the same as being reliable in production. That gives an advisor a concrete case for connector benchmarking before a buyer builds an architecture on them. Kim's safety model mirrors existing org controls: employee-style read/write permissions, with the analyst→associate→VP→MD chain reviewing agent output. Regulated buyers can adopt that governance frame without inventing a new one. Broken public evals mean vendor finance benchmarks carry little weight, so internal task-level evals become a real deliverable. Kim also said Codex adoption outside engineering jumped once it shipped as a desktop app, which suggests packaging held back adoption more than model capability did. Treat the 2027 move into associate and MD work as a vendor roadmap claim, not evidence.
Worth Raising With Customers
- Benchmark MCP connectors before committing. Measure error rates on your own queries. If the data is core, budget for licensed data you control end to end, even if that means paying for licenses twice.
- Map agent permissions onto your existing control structure. Use role-based read/write gating plus your current review hierarchy. Auditors already understand that framing.
- Build internal, task-level evals for your own workflows. Public finance benchmarks are compromised, by the vendor's own admission, so don't use them for model or vendor selection.