Commerce AI Training Data: Outcomes Beat Chats

13 minute read

The easiest AI training data for a commerce agent is often the least useful. Agent teams collect conversations because conversations are easy to see. A user asks for a product, the agent reasons through options, calls tools and gives an answer. The transcript looks like the natural training record.

For commerce, it is only the front half of the story.

The useful question is not merely what the agent said. It is what state the agent observed, which action it selected, what outcome it predicted, what the commerce systems executed and what later happened. Did the item ship on time? Did the buyer keep it? Was the payment reversed? Did the merchant honor the quote? Did a supposedly compatible part cause a return?

A conversation can teach language and preference expression. A verified trajectory can teach consequences.

That makes the long-term commerce learning object:

state + action + predicted outcome + verified actual outcome

Building it is harder than storing chat logs. It is also much more defensible as infrastructure.

Why chat transcripts are weak AI training data

Suppose a buyer tells an agent:

Find a quiet dishwasher under EUR 650 that fits a 45 cm cabinet.
I need delivery and installation in Berlin this week.

The transcript may show the agent comparing three options and recommending product B. It does not establish:

• which catalog version the agent queried;
• which variants passed the 45 cm constraint;
• whether inventory and installation were checked live;
• which price and delivery estimate were shown;
• whether a mandate authorized checkout;
• what the merchant accepted;
• whether fulfillment met the promised date;
• whether the buyer returned the product;
• why the buyer rejected products A and C.

The transcript may even contain confident prose that conflicts with the tool result. Treat it as one observation stream, not the system of record.

Operational telemetry has a similar limit. OpenTelemetry defines traces as the path of a request through an application and logs as records of events. Its signals documentation gives developers a common way to observe distributed systems. A commerce trajectory can use that trace context, but it also needs business identities and outcome state that may arrive days or months later.

Model a trajectory as linked records

A useful commerce trajectory connects several typed records without forcing them into one giant JSON document.

Intent
  -> observed state
  -> candidates
  -> plan and prediction
  -> authority and policy decision
  -> tool calls and results
  -> offer and transaction
  -> payment and fulfillment events
  -> return, dispute or retention outcome

Use stable identifiers and immutable event references:

type CommerceTrajectory = {
  trajectoryId: string
  traceId: string
  buyerContextRef: string
  intentRef: string
  stateSnapshotRef: string
  candidateSetRef: string
  selectedActionRef: string
  predictionRef?: string
  mandateRef?: string
  decisionRef?: string
  transactionRef?: string
  receiptRefs: string[]
  outcomeRefs: string[]
  schemaVersion: string
  createdAt: string
}

The references can point to records held under different access controls. A merchant may retain order details, a payment provider may own settlement state and the agent platform may retain only digests or purpose-limited features. The trajectory is a join structure, not permission to centralize every payload.

The W3C PROV-O recommendation provides a useful conceptual base for interoperable provenance through entities, activities and agents. Commerce systems will need domain-specific objects, but provenance vocabulary helps preserve who produced a claim, which activity used it and what record resulted.

Capture state at decision time

An outcome cannot be interpreted without the state the decision used.

For product selection, record or reference:

• normalized hard constraints and soft preferences;
• candidate identifiers and catalog versions;
• price, inventory and delivery observations with timestamps;
• compatibility evidence;
• merchant capability status;
• trust and authority state;
• policy version;
• market and currency;
• model and tool versions used for the proposal.

Do not copy a mutable product row and call it historical state. Store an immutable snapshot, a versioned reference or a digest plus recoverable source evidence.

{
  "snapshotId": "state:01K2C91",
  "observedAt": "2026-08-11T10:14:22Z",
  "market": "DE",
  "catalogVersion": "catalog:2026-08-11T10:00Z",
  "candidates": [
    {
      "variantId": "variant:dw-451-silver",
      "priceMinor": 62900,
      "currency": "EUR",
      "inventory": 3,
      "deliveryDate": "2026-08-14",
      "installationAvailable": false,
      "sourceDigests": ["sha256:aa81...", "sha256:07fc..."]
    }
  ]
}

The example is intentionally explicit about installationAvailable: false. If the agent still recommends the product, evaluators can identify whether intent parsing, ranking or explanation failed. A conversation alone may hide that error behind fluent justification.

Store the prediction before seeing the result

Outcome learning becomes much less useful if the prediction is reconstructed after the outcome arrives. Record the model's forecast at decision time.

type OutcomePrediction = {
  predictionId: string
  actionId: string
  modelVersion: string
  predictedAt: string
  estimates: {
    transactionSuccess: number
    onTimeFulfillment: number
    returnWithin30Days: number
    expectedBuyerUtility: number
  }
  featureSnapshotDigest: string
  uncertainty?: Record<string, number>
}

These values are illustrative. Production teams need calibrated definitions, time horizons and ground-truth rules. "Transaction success" might mean checkout accepted, payment settled or order fulfilled. Choose one and encode it in the schema.

Version the model, features and calibration method. Otherwise, an accuracy chart mixes predictions that meant different things.

Predictions never authorize an action. A model may forecast a high probability of successful payment while a deterministic mandate denies the amount. The denial remains final.

Verify outcomes against authoritative systems

An agent saying "the order arrived" is not a verified fulfillment event. Outcome verification needs source-specific rules.

type VerifiedOutcome = {
  outcomeId: string
  transactionId: string
  outcomeType:
    | 'payment-settled'
    | 'payment-reversed'
    | 'order-fulfilled'
    | 'delivery-late'
    | 'item-returned'
    | 'dispute-opened'
  status: 'observed' | 'corroborated' | 'contested'
  source: {
    system: string
    recordId: string
    assurance: 'merchant-record' | 'provider-record' | 'bilateral'
  }
  observedAt: string
  effectiveAt: string
  evidenceDigest: string
}

Verification does not turn every external assertion into truth. A signed merchant event proves what the merchant signed. A provider settlement record carries stronger evidence for payment status than a model transcript, but it may still be corrected or reversed. A delivery scan does not prove the buyer received the right item.

Use explicit assurance and dispute states. Link corrections instead of overwriting the earlier observation. This is where action receipts help: they bind identity, authority, transaction, decision and execution evidence. They do not by themselves prove the later real-world outcome.

AP2 follows the same general separation. Its specification links Checkout and Payment Mandates with corresponding receipts so parties can reconstruct what was authorized and observed during a dispute. It also states that retention and retrieval details are outside the current specification. See the AP2 dispute-evidence section. A commerce outcome layer still has to connect that transaction evidence to fulfillment, return and dispute events.

Keep action receipts and outcomes separate

Collapsing a receipt and an outcome into one record creates two problems.

First, outcomes arrive later. A payment can be accepted, then fail or reverse. An order can be fulfilled, then returned. Rewriting the receipt destroys the history of what the system knew at execution time.

Second, authority and success are different dimensions. An unauthorized action can produce a commercially successful result. An authorized action can fail operationally. The dataset should preserve both.

Action Receipt
  transaction: txn-418
  authority: allowed
  execution: submitted
  observed_at: T0

Outcome Event
  transaction: txn-418
  payment: settled
  effective_at: T1

Outcome Event
  transaction: txn-418
  order: returned
  effective_at: T2

This append-only shape supports evaluation without pretending the final status was known from the beginning.

Learn from trajectories without inventing causality

A trajectory records what happened after an action. It does not prove the action caused the outcome.

Product B may have a lower return rate because it was offered to different customers. A merchant may have better fulfillment outcomes because it serves easier regions. An agent version may appear to improve conversion during a seasonal promotion.

Use outcome data for several levels of analysis:

1. Descriptive: what outcomes followed which decisions?
2. Predictive: can state and action forecast an outcome on held-out data?
3. Counterfactual: what might have happened under another action?
4. Causal: does an intervention change the outcome?

The last two require stronger assumptions or experiments. Do not label an observational association as agent uplift.

Preserve selection probabilities or ranking scores where appropriate. Randomized exploration, when ethically and commercially acceptable, can help estimate alternatives. Hard safety, authority and legal constraints remain outside exploration. No experiment should test whether exceeding a payment mandate improves conversion.

Build labels that survive business reality

Naive labels produce naive agents.

purchase_completed = true may reward an agent for an order that was canceled ten minutes later. refund = true may treat a goodwill refund, fraud reversal and returned defective product as the same outcome. no_return_after_30_days may be censored because the return window is longer.

Define label contracts:

label: retained_purchase_45d
entity: order_line
positive_when:
  - fulfillment_status == delivered
  - no_return_record_by == delivery_time + 45d
exclusions:
  - merchant_return_window_days > 45
  - outcome_evidence_status == contested
version: 1

Track delayed labels, corrections and coverage. A model trained only on transactions with complete outcomes may inherit systematic bias if difficult merchants or regions have poorer instrumentation.

Negative outcomes need detail. Returns caused by size, compatibility, damage, delivery delay or buyer preference teach different lessons. Do not force the model to infer the reason from customer-service prose when a structured return code exists.

Privacy and data rights are part of the schema

The richest trajectory is not automatically the lawful or appropriate one to retain.

A useful data contract should attach:

• data controller and contributing party;
• purpose of collection;
• allowed uses, including whether model training is permitted;
• retention and deletion policy;
• tenant and merchant boundaries;
• personal-data categories;
• aggregation or de-identification requirements;
• geographic restrictions;
• evidence and consent references.

OpenAI's merchant-feed terms, for example, define how submitted merchant content may be used within OpenAI services and add requirements when feed content contains personal data. That is a reminder that a technical feed or receipt is not a general training license. See the OpenAI Merchant Feed Terms.

Cross-merchant learning is especially sensitive. A platform must not reveal one merchant's prices, customer behavior or operational weaknesses to another. Use explicit consent, isolation, aggregation, minimum cohort sizes and purpose-bound feature generation. Some data should never leave the merchant boundary.

Avoid storing full prompts by default. Extract the minimum structured features needed for the approved purpose, retain evidence references where necessary and apply separate access controls to raw conversations.

Instrument the trajectory across services

Use one trace or transaction identifier across agent runtime, resolver, policy, gateway, merchant connector, payment and outcome services. OpenTelemetry's semantic conventions provide common names for technical operations. Commerce-specific conventions should add stable business identifiers without putting sensitive values into high-cardinality metrics.

trace_id: technical request path
trajectory_id: learning episode
transaction_id: governed commercial action
order_id: merchant execution object
receipt_id: signed action evidence
outcome_id: later verified observation

These identifiers may have one-to-many relationships. One trajectory can contain several tool calls and transactions. One order can produce multiple outcome events. Do not assume a single distributed trace remains open until a return arrives weeks later. Link later traces through durable business identifiers.

Failure cases worth designing now

Backfill and correction tests matter as much as live ingestion. Outcome systems spend much of their life reconciling late and contradictory events.

Implementation checklist

• Treat conversations as one input stream, not the transaction record.
• Define stable intent, candidate, action, transaction and outcome identities.
• Capture immutable state references at decision time.
• Record predictions before outcomes with model and feature versions.
• Separate authority decisions, action receipts and later outcomes.
• Verify outcomes against named authoritative sources.
• Represent assurance, corrections and contested states explicitly.
• Use append-only events rather than mutable final-status fields.
• Define label contracts, horizons, exclusions and coverage.
• Avoid causal claims from observational trajectories alone.
• Propagate trace context while linking long-lived events through business IDs.
• Keep raw prompts under stricter access and retention than structured features.
• Attach purpose, consent, retention and training rights to data products.
• Isolate merchant data and govern any cross-merchant aggregation.

The Intelliger target and the current product

Intelliger's agentic-commerce blueprint proposes an Outcome Ledger, Commerce World Model, Decision Engine and privacy-governed Commerce Data Network. The target learning object is the linked state, action, predicted outcome and verified actual outcome described in this article. Those capabilities would improve simulation, routing, fraud detection, merchant recommendations and model behavior.

They are roadmap architecture, not deployed product claims. OATI's current developer preview can create signed transaction objects, evaluate deterministic Commerce constraints, generate receipts and run reference Commerce flows. The durable evidence worker, production reconciliation, Outcome Ledger, World Model and cross-merchant learning network are incomplete or target-state components. Independent cryptographic and protocol review remains open.

That boundary also explains the product thesis. A prompt archive gets larger with usage, but much of it is repetitive, sensitive and detached from business reality. A governed trajectory connects the decision to evidence about what followed. If Intelliger earns the right to build that dataset through real merchant integrations, explicit permissions and verified outcomes, it can learn something a conversation alone never reveals: whether the agent's choice worked.

Those trajectories become trustworthy only when signed action receipts establish the transaction record. The agentic commerce platform thesis explains how verified outcomes could support routing and decision systems without turning chat logs into a data moat by assumption.