A friendly cartoon robot wearing a leather horse harness, reins trailing off to the side

The SaaS era broke precedent : for the first time, enterprises stored their data on vendors’ clouds, rather than servers in their buildings. Will this 20 year trend persist in the world of AI? The question is evocative & the debate of the moment.

Satya Nadella wrote over the weekend :

“You essentially pay for intelligence twice, once with money, & again with something even more valuable : the proprietary knowledge you must reveal to make that intelligence useful.”1

A week before, Alex Karp on CNBC’s Squawk Box said :

“[frontier labs] are stealing the weights & alpha of my business.”2

Karp & Nadella are both partners & competitors to the frontier labs. Their alignment on the same warning, in the same week, reflects a broader fear about data loss taking hold across enterprise software.

On July 13, a security researcher reverse-engineered xAI’s Grok Build binary & found that a session with zero AI calls had uploaded the developer’s codebase to xAI’s cloud.3 xAI has since disabled the behavior.

AI models need data to learn ; this isn’t new.

Google Analytics does this on the web. A user visits a candle shop’s landing page, clicks on the gift section, finds a coupon & checks out. The recommendation system improves for the next visitor.

Like the shopper at the candle shop, each time someone queries an AI, information is produced. This data is called a trajectory.

More data is always better for AI & AI labs pay for it. Startups produce trajectories by paying experts to use AI, capturing the data. Others train new AIs to synthetically create trajectories, mimicking users. Collectively, these companies generate roughly $10b in revenue & are some of the fastest growing startups ever.4

Unlike in SaaS, where this data lived in databases accessible only to the customer, trajectories can be fed back into a model to improve AI. The customer’s data can become part of a vendor’s intellectual property.

Could internal data commingle with trajectories? Or trade secrets? How do you answer customer support tickets? What is your brand identity? What are your employees paid?

All of it flows through a harness.

The harness is the software through which the user works with the AI, like Claude Cowork or Cursor.

Who Holds the Reins? Schematic of the AI harness showing the inference loop between enterprise, harness, and model, and the training loop from session traces to the next model

The harnesses that win are the ones that maximize user productivity by being intelligent about how they marshal the AI.

CIOs & CEOs will demand zero data retention. Data fully deleted, not fully anonymized. The technologies around anonymization aren’t yet strong enough to guarantee it.

The trend needs to evolve & make the same guarantees software did : vendors have no access to enterprise data & don’t use it for their own purposes.

The last 20 years were the argument that vendors could be trusted with data. The next 20 will demand the same guarantees.

  1. Satya Nadella, “The Reverse Information Paradox”, July 12, 2026. ↩︎

  2. “Palantir’s Karp bashes token-based AI model as ‘completely wrong’”, CNBC Squawk Box, July 1, 2026. ↩︎

  3. @hrkrshnn on X, July 13, 2026. ↩︎

  4. @deedydas on X, “Every single startup selling AI Training Data (July 2026)”, July 12, 2026. ↩︎