TL;DR
- Legacy system integration is the #1 technical challenge in enterprise AI: and it is almost always underestimated
- You do not need to replace legacy systems: build an AI layer on top using API, ETL, event-driven, or replication patterns
- Data quality problems in legacy systems require 60–80% of total project time to resolve: budget for it
- Choose your integration pattern based on latency requirements: real-time needs API or events; analytical needs ETL
- Enterprise integration platforms (MuleSoft, Azure Integration Services, Boomi) significantly reduce build time vs custom connectors
- Start with a data audit before committing to an AI use case: data quality determines whether the AI is possible
Why Legacy Integration Is the Hardest Part of Enterprise AI
Enterprise AI projects routinely fail not because of bad models or wrong algorithms, but because of the data plumbing underneath. Most large enterprises run on systems built in the 1990s or early 2000s: mainframes, SAP R/3, Oracle EBS, bespoke on-premise applications. These were designed for transactional processing, not data sharing.
These systems hold the data that makes enterprise AI valuable: decades of transaction records, customer histories, operational data, quality records, financial ledgers. But accessing that data for AI is rarely simple. Common challenges include:
No Programmatic Access
Legacy systems designed for human interaction, not API consumption. Data export is often manual or batch-only.
Proprietary Data Formats
Data stored in formats specific to the vendor (IDOC, IDoc XML, COBOL copybooks) requiring specialist translation.
Data Quality Issues
Decades of manual entry, system migrations, and schema changes produce inconsistent, duplicate, and incomplete data.
Operational Risk
Any direct integration with a core legacy system carries risk of performance degradation or data corruption in business-critical systems.
Knowledge Loss
The engineers who built or understand the legacy system may have left. Documentation is often sparse or non-existent.
Siloed Architecture
ERP, CRM, WMS, and finance systems contain separate data models with no shared identifiers: joining them is non-trivial.
Common Legacy Architecture Types
Different legacy systems require different integration approaches. Understanding the architecture type is the first step to planning the integration correctly.
Mainframe Systems (IBM z/OS)
Still in use at most large banks, insurers, and government departments. Mainframes process millions of transactions per day with extraordinary reliability. Data is stored in VSAM files or Db2 for z/OS. Integration options include:
- IBM MQ (message queuing) for event-driven data extraction without impacting mainframe performance
- JDBC/ODBC connections to Db2 for z/OS for direct database queries (read-only, non-peak hours)
- z/OS Connect EE for REST API exposure of CICS and IMS programs
- Offline data replication via tape or file transfer to cloud data warehouse
SAP ERP (R/3, S/4HANA, ECC)
SAP is the backbone of operations for most large UK and European enterprises. SAP exposes data through several mechanisms:
- OData APIs: Modern, REST-like APIs available in SAP S/4HANA for reading and writing SAP data: the recommended approach for new integrations
- RFC/BAPI: Remote Function Calls and Business Application Programming Interfaces: the classic SAP integration mechanism, complex but widely supported
- IDOCs: SAP's native document format for bulk data exchange: best for large batch transfers
- SAP Integration Suite (formerly SAP Cloud Platform Integration): SAP's own iPaaS, optimised for SAP-to-SAP and SAP-to-cloud integrations
- Direct Db2/HANA access: Read-only access to the underlying database: fast but bypasses SAP's authorisation model (use with extreme caution)
On-Premise Relational Databases (Oracle, SQL Server, PostgreSQL)
The most common legacy data store in mid-market enterprises. Integration is generally more straightforward than mainframe or SAP, but data quality and access control remain challenges. Standard approaches include:
- Change Data Capture (CDC) using Debezium
- read replicas for AI workloads without impacting production
- ETL pipelines to a cloud data warehouse
Integration Patterns: Choosing the Right Approach
| Pattern | How It Works | Latency | Pros | Cons | Best For |
|---|---|---|---|---|---|
| API Layer | Build REST/GraphQL APIs in front of legacy system; AI calls APIs | Real-time (ms) | Clean abstraction; preserves legacy system integrity; enables future migration | Build cost; API performance depends on legacy system performance | AI systems that need real-time data or must write back to legacy system |
| ETL Pipeline | Extract data from legacy on schedule, transform, load into data warehouse; AI trains/runs on warehouse | Batch (hours) | No impact on legacy system; enables rich data quality processing; good for large volumes | Stale data; complex pipeline maintenance; storage cost | ML model training, historical analytics, reporting AI, non-real-time predictions |
| Event-Driven (CDC) | Capture database change events in real time via CDC (Debezium); stream to Kafka; AI consumes events | Near real-time (seconds) | Fresh data; decoupled; scalable; AI reacts to business events immediately | Complex to set up; requires Kafka or equivalent; CDC may not be available on all legacy databases | Fraud detection, real-time recommendations, anomaly detection, operational AI |
| Database Replication | Create a read replica of the legacy database; AI reads directly from replica | Near real-time to batch | Simple to set up; no impact on production performance; AI has full data access | Replication lag; storage cost; schema changes in primary break replica; security concerns | Analytical AI, model training, reporting where near-real-time is sufficient |
| Middleware / iPaaS | Enterprise Integration Platform (MuleSoft, Boomi, Azure Integration Services) mediates between AI and all legacy systems | Configurable | Pre-built connectors for 1,000+ systems; centralised monitoring; reusable integrations | High licensing cost; adds latency; another system to manage | Complex multi-system integrations; organisations with many legacy systems to connect |
Data Quality Requirements for AI
AI models are only as good as the data they are trained on and the data they receive at inference time. Legacy systems often have significant data quality problems that must be resolved before AI can work reliably. Use this checklist to assess and remediate data quality before committing to a use case.
Data Quality Checklist for AI Integration
Completeness
Missing values assessed per field: is the missing rate <5% for all fields required by the AI model? Missing value imputation strategy defined for fields where missing values are unavoidableAccuracy
Sample validation performed: manual review of 200+ records against known-correct source confirms >95% accuracy Known data entry errors (e.g. free-text fields with inconsistent formatting) identified and normalisation rules definedConsistency
Entity matching across systems verified, customer IDs in CRM match customer IDs in ERP for >98% of records Categorical values standardised, e.g. "UK", "United Kingdom", "GB" normalised to a single valueTimeliness
Data freshness assessed: how old is the most recent record? Is this acceptable for the AI use case? Data update frequency confirmed: if AI runs hourly, data pipeline delivers updates at least as frequentlyUniqueness
Duplicate records identified and deduplication strategy applied before AI training Primary key integrity verified: no duplicate PKs in training datasetRelevance
Historical data still reflects current business process, data from before major system migrations or business model changes excluded or flagged Feature relevance verified, features used by AI model have been validated by domain experts as genuinely predictiveIntegration Tools and Platforms
| Tool / Platform | Type | Strengths | Best Use Case |
|---|---|---|---|
| MuleSoft Anypoint | iPaaS / API Management | Largest connector library; excellent SAP, Salesforce, and legacy connectors; enterprise-grade governance | Large enterprises with many systems; SAP-centric architectures |
| Azure Integration Services | Cloud iPaaS | Deep Azure/Microsoft stack integration; Logic Apps, Service Bus, API Management; pay-as-you-go | Azure-native AI deployments; Microsoft ecosystem shops |
| Apache Kafka | Event Streaming | Extremely high throughput; durable event log; powers CDC pipelines; de facto standard for event-driven AI | Real-time AI inference; fraud detection; operational AI that must react to events |
| Debezium | Change Data Capture | Open source CDC for PostgreSQL, MySQL, SQL Server, Oracle, MongoDB; streams database changes to Kafka | Extracting real-time change events from on-premise databases without impacting production |
| dbt (Data Build Tool) | Data Transformation | SQL-based transformation pipeline; version controlled; excellent for preparing AI training datasets from warehouse data | Building curated AI feature stores from raw legacy data in cloud warehouse |
| AWS Glue / Azure Data Factory | Cloud ETL | Managed ETL services; serverless; pre-built transformations; integrates with cloud AI/ML services | Batch ETL from legacy systems to cloud data warehouse for AI training |
| Boomi / Informatica | iPaaS / Data Integration | Strong mid-market offering; Informatica excels at data quality and MDM alongside ETL | Mid-market enterprises; projects requiring data quality management alongside integration |
Case Study: Manufacturing Company Integrates AI with Legacy ERP
A UK-based precision components manufacturer with £180M annual revenue wanted to implement AI-driven demand forecasting and predictive maintenance. Their IT landscape was typical of an established manufacturer:
- SAP ERP (ECC 6.0) running on-premise
- a 12-year-old Wonderware SCADA system for production monitoring
- a separate Oracle database for quality management
The Integration Challenge
The AI system needed three data inputs:
- historical sales orders and inventory from SAP
- machine sensor data from SCADA
- quality inspection records from Oracle
None of these systems talked to each other. The SCADA system had no API and exported data only via CSV on a manual basis. The SAP system used RFC-based interfaces. The Oracle database was accessible but contained 14 years of inconsistently formatted quality data.
The Integration Architecture
SAP SAP Integration Suite configured to extract sales orders, purchase orders, and stock levels via BAPI calls on a 4-hour batch cycle. Data landed into an Azure Data Lake Storage account. Azure Data Factory performed data quality checks and loaded clean records into Azure Synapse Analytics. SCADA An OPC-UA data bridge was installed alongside the existing SCADA system: reading sensor data (temperature, vibration, cycle time) in real time without modifying the SCADA system. Data was streamed to Azure Event Hubs and then to a time-series database (Azure Data Explorer) for the predictive maintenance model. ORACLE Debezium CDC connector deployed against the Oracle database, streaming quality record changes to Kafka. A custom normalisation pipeline standardised 14 years of free-text quality codes into 23 structured failure categories. Normalised data loaded into Synapse Analytics alongside the SAP data. AI LAYER Two AI models deployed: (1) Demand forecasting model trained on Synapse data, producing 13-week forward forecasts written back to SAP via BAPIs; (2) Predictive maintenance model reading from Azure Data Explorer, triggering maintenance alerts when anomaly thresholds exceeded, surfaced in a Power BI dashboard alongside SAP maintenance order creation.Results
31% reduction in inventory holding cost through improved demand forecasting accuracy 67% of machine failures predicted and prevented before they caused production downtime £1.4M annualised savings in Year 1 from combined demand and maintenance improvements 6 mo. from project start to both AI systems in production: including data quality remediationThe Key Lesson: Integration Architecture First, Model Second
In the manufacturing case study above, the model development itself took 6 weeks. The integration architecture design, data quality remediation, and pipeline build took 18 weeks. This ratio is typical. For any enterprise AI project involving legacy systems, plan your integration architecture and data quality programme first. Do this before any model development begins.
The critical code below illustrates a simple pattern for exposing SAP BAPI data via a REST API wrapper. This is the first step in building an API layer on top of legacy systems:
# FastAPI wrapper exposing SAP BAPI data as a REST endpoint
# Sits between AI inference service and on-premise SAP system
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import pyrfc # SAP RFC Python connector
import logging
app = FastAPI(title="SAP Integration Layer")
# SAP connection config (loaded from secure vault in production)
SAP_CONFIG = {
"ashost": "sap-prod.internal.company.com",
"sysnr": "00",
"client": "100",
"user": "AI_SERVICE_USER",
"passwd": "***", # Use Azure Key Vault / AWS Secrets Manager
"lang": "EN"
}
class InventoryRequest(BaseModel):
material_number: str
plant: str
@app.get("/api/v1/inventory/{material_number}")
async def get_inventory(material_number: str, plant: str = "1000"):
"""Fetch current stock level from SAP via BAPI_MATERIAL_STOCK_REQ_LIST"""
try:
with pyrfc.Connection(**SAP_CONFIG) as conn:
result = conn.call(
"BAPI_MATERIAL_STOCK_REQ_LIST",
MATERIAL=material_number,
PLANT=plant
)
# Extract unrestricted stock from result
stock = next(
(r for r in result.get("STOCKREQLIST", [])
if r["REC_TYPE"] == "BS"), # BS = unrestricted stock
None
)
return {
"material": material_number,
"plant": plant,
"unrestricted_qty": float(stock["QTY_UNRESTR"]) if stock else 0.0,
"unit": stock["UNIT"] if stock else "EA"
}
except pyrfc.RFCError as e:
logging.error(f"SAP RFC error: {e}")
raise HTTPException(status_code=502, detail="SAP connection error")
except Exception as e:
logging.error(f"Unexpected error: {e}")
raise HTTPException(status_code=500, detail="Internal error")Connect Your Legacy Systems to Enterprise AI
SpiderHunts Technologies specialises in connecting AI to the legacy systems where your data actually lives: SAP, mainframes, on-premise databases, and bespoke ERP systems. We design the integration architecture, fix the data quality, and build production-grade AI on top: without replacing your existing infrastructure.
Discuss Your Legacy Integration Project