Unified Marketplace Integration Platform
The Problem
Extending internal systems (ERP, POS, inventory) to support marketplace channels required building a multi-platform integration layer connecting with Amazon, Noon, HungerStation, Jahez, AliExpress, and Trendyol. Each platform operates with different API standards, authentication models, rate limits, and data contracts. Even within a single platform, multiple integration models exist — for example, Amazon Marketplace, FBA, and invoice/report flows — each with distinct workflows, data structures, and operational constraints. The solution focused on designing scalable, event-driven pipelines that handle high volumes of orders and transactions, support near real-time inventory and pricing synchronization, enable reliable bidirectional data flows, and maintain consistency between internal and external systems across all channels.
Architecture
Key Design Decisions
- Amazon's mandatory AWS infrastructure (SQS, Lambda, S3) required a cross-cloud bridge. Event-driven consumption via Lambda → Azure Service Bus kept the Azure pipeline consistent across all three Amazon models (Marketplace, FBA, reports) without polling or tight coupling to AWS internals.
- Service Bus as the integration backbone across all channels — decoupling external producers from internal consumers, enabling per-channel throughput control, message replay on failure, and protection against upstream rate limit spikes.
- Transformation and normalization layer before any internal write — each marketplace payload maps to a canonical internal schema with idempotency checks, preventing duplicate processing under retries, redelivery, or throttle-driven backoff.
How It Works
Amazon KSA runs three independent integration models: Marketplace and FBA orders are ingested via Lambda functions consuming from SQS, forwarded to Azure Service Bus, processed by Azure Functions (.NET 8), and persisted to Azure SQL. Invoice and report flows use a separate pipeline with S3 as the data source. Noon, HungerStation, Jahez, AliExpress, and Trendyol each run dedicated Azure-native bidirectional pipelines — inbound order ingestion and outbound inventory/pricing feeds — built on the same Service Bus and Azure Functions pattern. All pipelines include rate limit handling, retry with backoff, and structured logging with correlation IDs.
Impact
