SAP Commerce plus Mirakl demos beautifully. Offers flow in, orders split across shops, refunds post back on cue. Then you run it for a quarter and learn the real shape of the work: the integration is not the happy path. It is the failure paths. This is a working guide to building them.
The connector's sync model is simple to describe: scheduled jobs on the SAP Commerce cronjob framework pull data from the marketplace APIs on a timer - offers, orders, refunds, shop data - with a smaller set of synchronous calls in the buy flow. The demo shows the data moving. It does not show the three properties that will define your next two years of operations:
- At-least-once, not exactly-once. Retries and overlapping windows mean duplicates will arrive. If your consumers are not idempotent, a duplicate becomes a double side effect.
- Per-item failure, not per-run failure. A job that processes 4,999 of 5,000 items and swallows one exception reports success. The one item is a customer.
- A polling floor. Data is as fresh as your cron interval on a good day, and as stale as your last silent failure on a bad one.
Everything below follows from taking those three seriously.
Retry that does not amplify the failure
The naive loop - catch, sleep, try again, forever - is not resilience. Against a rate-limited API it is an attack on your own quota. Mirakl publishes per-endpoint rate limits and answers excess with HTTP 429 and a Retry-After header. The header is not advice. It is the contract.
- Exponential backoff with jitter. Backoff spreads retries over time; jitter stops your API nodes from retrying in lockstep and arriving as a synchronized wave.
- Honor
Retry-Afterexactly. If the platform says 30 seconds, 29 is just another 429. Read the header, schedule the retry, do not guess. - Cap and park. After N attempts, write the item - full payload, last error, timestamp - to a dead-letter store and move on. The stream must never block on its slowest item.
- Separate retry budgets per endpoint. A catalog import burning write quota must not starve order sync. Different flows, different budgets, different alarms.
The amplification failure is worth naming, because it hits on your best day, not your worst. A big catalog change lands. The import saturates the write limit. A retry loop that ignores Retry-After turns one throttled hour into an evening of them, and order sync queues behind the mess. Rate limits are an architecture constraint. Budget them like capacity, not like errors.
Idempotency on the money paths
Orders, refunds, credit adjustments - anything that moves money must be safe to process twice, because at-least-once delivery guarantees that one day you will.
A refund is the canonical case. Two systems each hold an opinion about the amount - the marketplace computed one number, your commerce or ERP side computed another - and when they disagree, the posting is rejected. Without idempotency and parking, that rejection either blocks the whole stream or vanishes into a log. With them, it parks with its payload and shows up in the morning report as what it actually is: a pricing disagreement wearing a refund's clothes.
The mechanics are not exotic:
- Give every operation a natural idempotency key. Marketplace order id + line + operation type covers most flows. If you cannot name an operation's key, you do not understand the operation yet.
- Record the key before the side effect. A processed-operations ledger, written in the same transaction as the posting - in SAP Commerce terms, an item type with a unique business key does the job. Check the ledger before you post, not after.
- Make re-runs boring. Replaying yesterday's window must produce zero new side effects and say so explicitly. If replay is scary, the design is wrong.
At-least-once delivery plus idempotent consumers equals effectively-once processing. That combination is the whole trick, and it is buildable in a sprint.
Reconciliation is a component, not a spreadsheet
Every integration diverges. The only question is whether you find out from your own system or from finance at month-end.
- Map the state machines. Marketplace order states on one axis, commerce and ERP states on the other. The mapping table is the specification; any pair outside it is divergence by definition.
- Give in-flight items a grace window. An order mid-transition is not divergent, it is in transit. Alert on aged divergence only - disagreement that has outlived its grace period.
- Publish a daily report. Counts by divergence type, and a named owner for each type. A divergence without an owner is a divergence you have decided to keep.
Then treat the divergence rate as a health metric with a target. Zero is not the target - the target is known, bounded in age, and owned. A marketplace integration that cannot state its current divergence count is not integrated. It is co-located.
Alert on absence, not just exceptions
The failures that hurt are the silent ones. A scheduler that stopped. A job that runs, matches nothing, and reports success in one millisecond. A trigger someone disabled during an incident and forgot to re-enable. None of those throw.
- Expected-run monitors. "This job completes every N minutes" is an alertable contract. Page when a window passes without a completion event, not only when an exception appears.
- Volume floors. Zero refunds processed on a weekday is not calm. It is a signal. Alert on suspicious quiet, not just on noise.
- Staleness metrics. The age of the newest synced offer and order is a number. Graph it. Alarm on it. It is the single most honest gauge of integration health you can put on a screen.
"Nothing threw" is the most dangerous sentence in integration operations.
Telling it to business
All of this costs real budget - in practice a quarter or more of integration effort goes into paths that only matter when something fails. Here is the translation that gets it funded.
Do not sell "reliability". Sell the incident you are deleting. A silent refund failure is a customer waiting on their money, a discrepancy in the books, a support escalation, and an engineer reconstructing three days of history under pressure. Each of those costs more than the retry policy that would have prevented it, and the comparison is checkable.
Then sell the number. Reconciliation converts unknown unknowns into a daily count with named owners. The honest pitch to a CFO is one sentence: we are buying a dashboard that tells us exactly how wrong the two systems are, every morning, so that the answer to "are the books right" is never "we believe so".
"The integration works" is not a state. It is a measurement with a timestamp. Fund the measurement.
The demo sells the connector. Production runs your failure paths. At marketplace scale the failure paths are not the edge of the product - they are the product. Build them like it.
If it can't handle real load with real data and real users across multiple markets, it's not architecture. It's a slide deck.