Migrating a 15-Year-Old SAP ERP to Snowflake Without Breaking the Supply Chain

The Challenge

This third-party logistics company operates 6 distribution centers across the eastern United States, handling warehousing, fulfillment, and last-mile delivery for roughly 200 enterprise clients. Their entire operation runs on SAP ECC 6.0 - and it's been customized to the point of no return.

Over 15 years, consultants and in-house developers had added 230 custom ABAP programs, 18 custom Z-tables, and a pricing engine built on top of SAP's condition technique that had been modified so many times that nobody could explain all the edge cases. The SAP Basis team estimated an S/4HANA migration would take 18-24 months and cost north of $4M. That wasn't happening anytime soon.

But leadership needed analytics that SAP couldn't deliver. Specifically, they wanted cross-client shipment optimization - the ability to look at all 200 clients' shipment data together and find consolidation opportunities. They also needed real-time visibility into warehouse utilization across all 6 DCs. Their ops team was making staffing decisions based on yesterday's numbers because SAP reports ran overnight in batch.

Running analytical queries directly against SAP was out of the question. The last time a business analyst tried running a report against the LIPS (delivery item) and LIKP (delivery header) tables during business hours, it locked a production table for 45 minutes. Three distribution centers couldn't process outbound shipments during that window. The VP of Operations was not subtle about expressing his feelings on the matter.

They needed a way to get data out of SAP safely, transform it into something analysts could actually use, and visualize it - without touching the 230 custom ABAP programs or risking another production outage.

What We Built

Extraction via Talend with SAP RFC/BAPI connectors: This was the critical design decision. We didn't read from SAP tables directly - that's how you get those 45-minute locks. Instead, we used Talend's SAP RFC connector to call standard BAPIs (BAPI_DELIVERY_GETLIST, BAPI_MATERIAL_GETLIST, etc.) and custom RFC function modules that the SAP team exposed for us. RFC calls go through SAP's application layer, which respects SAP's own locking and buffering mechanisms. The SAP system didn't even notice the additional load.

Incremental extraction framework: We built a change-data-capture pattern using SAP's change pointers (BDCP/BDCPS tables) and timestamps on transaction tables. For master data (materials, customers, vendors), we used change pointers to pull only records modified since the last extraction. For transactional data (deliveries, shipments, warehouse orders), we filtered on the ERDAT/ERZET (creation date/time) and AEDAT/AEZET (change date/time) fields. Full extracts ran weekly as a safety net; incrementals ran every 30 minutes.

Medallion architecture in Snowflake:

• Bronze layer: Raw SAP structures, column names preserved (VBELN, MATNR, WERKS - the SAP names nobody outside the SAP team understands). We kept these because the SAP team needed to validate the data using their existing knowledge of SAP field names.
• Silver layer: Denormalized, business-friendly tables. SAP's heavily normalized structure (a delivery header in LIKP, items in LIPS, status in VBUK, partner functions in VBPA) got flattened into a single deliveries table with human-readable column names. The 18 custom Z-tables were joined with standard SAP tables here.
• Gold layer: Aggregated KPIs. Warehouse utilization by DC and zone, carrier on-time performance, client shipment volumes with 7/30/90-day trends, and the cross-client consolidation opportunities that leadership wanted.

Tableau dashboards: The gold layer powers 12 Tableau dashboards. The flagship is the DC Operations dashboard showing real-time (well, 30-minute delayed) warehouse utilization heat maps, inbound/outbound shipment queues, and carrier performance scorecards. Operations managers at each DC have this on a wall-mounted screen.

What we didn't do: We didn't migrate SAP's business logic. The 230 custom ABAP programs continue to run in SAP exactly as they did before. We didn't touch the pricing engine. We didn't modify any SAP configuration. We simply read from SAP safely and built an analytics layer on top.

Results

Tech Stack

What We Learned

• RFC/BAPI extraction is non-negotiable for production SAP systems. Direct table reads (using SAP's Open SQL or direct database connections) bypass SAP's application server and can cause table locks, especially on heavily customized systems. RFC calls go through SAP's application layer, respect its buffering, and are how SAP itself recommends external systems access data. The extra setup time is worth it.
• SAP field names are a barrier to adoption. In the bronze layer, we kept SAP's original column names (VBELN, MATNR, POSNR). The SAP team loved it because they could validate data using the names they already knew. But the business analysts refused to use it. The silver layer with human-readable names (delivery_number, material_code, item_number) was where adoption actually happened. If you skip the silver layer, the analytics team won't use the data.
• Change pointers aren't available for all SAP objects. We planned to use change pointers for everything, but some custom Z-tables didn't have change pointer configuration. For those, we fell back to timestamp-based incremental extraction. Two extraction patterns means more code to maintain, but it's better than running full extracts on large tables every 30 minutes.
• The 22-week timeline was driven by SAP complexity, not Snowflake. Snowflake and Tableau were set up in weeks 1-3. The remaining 19 weeks were spent mapping SAP's data model (which is spectacularly normalized), building and testing the RFC extraction jobs, handling SAP-specific edge cases (archived data, deleted records, status codes that mean different things in different transaction types), and getting sign-off from the SAP team on data accuracy. If you're planning a SAP extraction project, plan for the SAP side to take 3-4x longer than the analytics side.