Custom API integration services: the six stages, with week numbers

Most integration service pages list the systems they can connect and stop there. That list is the easy part. Two systems can exchange records in an afternoon.

What takes six weeks is everything after the connection works: the historical backfill, the reconciliation that proves both sides agree, the cutover that does not lose a day of orders, and the question of who gets the alert at 2am when the nightly file does not arrive.

Here is the whole sequence, with what happens in each stage and roughly when.

Stage 1: Systems audit#

We read both systems before proposing anything. That means real access, not a screen share.

What comes out of it is an inventory: every object we will touch, the fields that actually carry data as opposed to the fields that exist, the volume, and the integration surface each system offers. A documented REST API, a nightly CSV on an SFTP server and a database replica are three very different projects.

The audit is also where we find the thing that decides the architecture: whether there is a stable external ID on both sides. If there is not, the first deliverable is agreeing one, because every upsert for the life of this integration keys on it.

Stage 2: Mapping and contract#

Field mapping is the part everyone expects. The contract is the part that prevents the arguments.

The mapping workbook is a row per field: source, destination, transformation, and what happens when the value is missing or malformed. It gets signed off by someone who owns the data, not by whoever is managing the project.

The contract covers the questions a mapping cannot answer:

• Which system is authoritative when both have a value and they disagree?
• Is this one way or two way? Two way sync is roughly triple the work of one way, because now you need conflict resolution and loop prevention.
• What is the sync frequency, and what is the acceptable staleness?
• What counts as a failure worth waking someone for?

Getting this wrong is the single most common reason an integration gets rebuilt in year two.

Stage 3: Build in staging#

The service gets built against a staging environment on both sides, never against production.

What goes in regardless of the systems involved: idempotent writes keyed on that external ID so a replay cannot duplicate anything, exponential backoff on retries, a dead letter queue for records that fail repeatedly, searchable logs, scoped credentials, and explicit handling for rate limits and batch sizes.

For file based sources, which is more of the real world than vendors admit, that also means schema validation on arrival, row level error reporting, automatic reprocessing, and alerting when the file simply does not show up. A pipeline that only alerts on errors is blind to the most common failure, which is silence.

Stage 4: Backfill and reconciliation#

This is where the weeks go, and where most published process descriptions stop.

Backfill means loading history, not just new records from today forward. A CRM without the previous two years of orders is a CRM your sales team will not open.

Reconciliation means proving it worked. Not spot checking. A side by side report, object by object, that a human signs before cutover: row counts, sums where sums make sense, and an explicit list of everything that did not migrate with the reason attached.

Here is the shape of a real sync report, so the abstraction is concrete:

Run 02:00 UTC
  14,208 rows read
   1,046 created
  13,158 updated
       4 quarantined  -> see exception report

Four quarantined rows is a healthy number. Zero usually means the validation is not doing anything.

Stage 5: Cutover and monitoring#

Cutover is a decision with a rollback plan, not an event.

The pattern that works: run both systems in parallel long enough to compare outputs, cut over during the lowest volume window you have, then run a delta pass to catch anything that changed during the freeze.

Then somebody watches it. For the first weeks that is us.

Stage 6: Support window and handover#

The integration is not finished when it runs. It is finished when someone on your side can operate it.

That means a runbook: how to reprocess a failed batch, what each quarantine reason means and who resolves it, where the credentials live and how to rotate them, what to do when the vendor deprecates the endpoint. The code goes to your repository and the infrastructure sits in your accounts.

An integration only we can maintain is not a deliverable, it is a dependency.

What this costs and why#

Integration work starts at $7,500 and runs 3 to 8 weeks, fixed after a paid scoping week. The range is wide because the audit in stage 1 is what collapses it: two documented REST APIs with clean IDs sit at the bottom, a file based source with no stable key and nine years of history sits at the top.

The same six stages apply whether the far end is an ERP, a warehouse system, a billing platform or a system whose only outbound interface is a CSV drop. The CSV case is not a degraded version of this process, it is the same process with a different stage 3.

If the work is less about connecting systems and more about replacing a person who reads and retypes between them, the question becomes what you can actually test before you build.

Frequently asked questions#

How long does an API integration take?#

Three to eight weeks for a single integration, with the systems audit and the reconciliation step taking more of that than the build. Two way sync sits at the top of the range because conflict resolution and loop prevention roughly triple the logic.

What if our system has no API?#

Then it almost certainly has an export. A scheduled CSV to an SFTP server or an S3 bucket is a completely workable integration surface, with schema validation, keyed upserts and a quarantine report. It is slower and it is not worse.

Who owns the integration when the project ends?#

You do. Code in your repository, infrastructure in your accounts, and a runbook covering reprocessing, credentials and adding a field. The support window is for questions, not for gatekeeping.

Can we add a field later without calling you?#

That is what the runbook section on adding a field is for. Most mapping changes are configuration. A new object or a change to the authoritative system is a real change and should be scoped.