The client's biggest supplier sent stock updates as a fixed-width text file over FTP, twice a day, in a format that predated most of our team's careers. Renegotiating that was not on the table.
The constraint we had to accept
We could not change the feed, the schedule, or get documentation better than a two-page email from someone who had since left the supplier's business. Every design decision had to assume the feed would stay exactly this bad indefinitely, because betting on it improving had already cost the client a year of delay before we arrived.
What the feed actually did to us
- Column positions drifted occasionally without notice, so a field that was inventory count on Tuesday became a warehouse code on Wednesday.
- Encoding was inconsistent within the same file, apparently generated by more than one system at the supplier's end.
- Silence was not distinguishable from "no changes" — a missing file could mean nothing moved or could mean the job upstream had failed.
The translation layer we built
A dedicated ingestion service that validates every incoming file against a shape it expects — column count, type of each field, plausible ranges for quantities and prices — before any of it touches the client's live stock data. Anything that fails validation goes to a quarantine queue with the specific row and reason, rather than either crashing the whole import or silently accepting bad data.
When you cannot fix the source, the job becomes making its unreliability visible and contained rather than pretending it does not exist.
Detecting silence
We added a simple check: if no file has arrived by the expected time, alert, rather than waiting for someone to notice stock levels looked stale. This caught three supplier-side failures in the first two months that would previously have gone unnoticed until a customer ordered something that was not actually in stock.
What we would tell another team facing this
Do not build for the feed you were promised. Build for the feed you were sent last Tuesday, because that is the one that will keep arriving, warts included, for as long as the relationship lasts.