Integrations never seem to happen the way we plan them. I think the main reasons are,
- we start with limited information about the actual processes, applications, and the people involved.
- We assume that every team/entities works in a
Standardway - We think everyone in the team has the same interest
Whether it was the Courier App integration I did four years back or the SAP integration I completed last month, it was the same.
Rather than pointing fingers, this case study explores the reality of 'Internal IT integrations'.
The Objective, a.k.a Requirement.
Eliminate the ~15-minute manual data entry process for every Business Partner request. By automating the flow from our Workflow App to SAP.
The Premise
- The App: An internal workflow platform I built two years ago—now significantly more complex.
- Dev Team: A "lean" squad (meaning: me, another contract developers).
- SAP Team: An internal Functional team, and an offshore Integration Team
- QA Team: __ (None).
- The Lead (Me): Handling a "slight" overflow of responsibilities,
- Architecting new features
- Babysitting existing modules
- Answering user queries & tickets
- Managing stakeholder expectations
AI Coding: NO
The Beginning
One of my primary goals upon joining was to automate the Business Partner (BP) onboarding process.The mission: reduce the Master Data team’s workload and streamline the entire resource chain. Finally, the time had come.
This wasn't our first attempt at automation. Earlier in 2024, we built a module for BP unblocking, but the approach was... questionable.
The Approach: A traditional "DB-first" strategy proposed by the team managing Master Data. They wanted my app to connect directly to the HANA DB. I pushed for SAP APIs instead, but the answer was a "No" from the management.
Left with no other option, I committed—and immediately hit my first bitter reality. Connecting Laravel to SAP HANA is anything but straightforward! After weeks of searching through every blog and technical forum I could find, I finally implemented an ODBC connector for SAP HANADB! Our Infra team wasn't comfortable with Linux, so I handled the setup myself.
The Leassons (problems):
- The Chase 1: I spent weeks chasing the DB and SAP teams just to find the right tables.
- The Chase 2: Every access issue required another round of meetings with the Basis team.
- Big One: The DB password expires every 90 days! Once, while I was on vacation, the module stopped working. Stakeholders called, users were worried, and I had to log in from home just to find a password expiry issue. We still have to reset it manually to this day.
The (Only) Advantage: This ODBC connector eventually became a lifesaver during the actual integration, as the team ultimately came up with only one API :-) :-(
The Beginning (of this integration)
I could feel this integration heading my way even while I was flooded with other projects. It was hard to tell whose priority it actually was, as multiple stakeholders were claiming it as their own KPI. As the Solutions Architect and Lead Software Engineer on the ground, I knew one thing for sure: it was time for me to get ready for another "firefight." The previous (db) integration had finally stabilized, but my plate was already full with several other projects.
Technical Deep-Dive: SAP & HANA Findings
To build a reliable bridge, I had to look beyond the standard API documentation and understand how data sits within the HANA Database. Our mapping logic had to account for core tables to ensure the Customer-Vendor Integration (CVI) remained intact:
- General Identity (BUT000): Managing general header data.
- Role-Specific Masters (KNA1 / LFA1): Handling the specific attributes for Customers and Suppliers.
- Financial & Banking (BANKA / T001): Get bank master and company code validations.
- The "Invisible" Schema: Beyond the main headers, I mapped numerous secondary tables to handle, like:
- Payment Terms (T052): Enforcing conditional logic for credit.
- Reconcile Accounts (SKB1): Ensuring the G/L accounts were correctly linked for accounting.
- Identification (BUT0ID): Capturing government-issued ID documents and licenses.
- Data Encoding: I discovered that special characters (e.g., Finnish or Arabic names) required specific UTF-8 handling to prevent "broken" strings in the SAP GUI.
Engineering Challenges & Solutions
1. Solving the Idempotency Problem
Early in the testing phase, we encountered "Duplicate BP creation."
-
The Issue: Our background job supervisor was configured with
retries=3. If the SAP API was slow to respond, the supervisor triggered a second and third attempt, creating three identical records in SAP. -
The Solution: I implemented a Unique Transaction ID . This is to ensure that even if a job retried, the system recognize the "In-Progress" status and to block duplicate POST requests. As an immediate step, the retry was removed.
2. The "One API" Challenge: Leveraging HANA DB
In many enterprise setups, you have a suite of microservices. Here, I had exactly one: a BPCreate BAPI. There were no sub-APIs for master data, default lists, or validation checks.
- The Solution: To solve the lack of supporting APIs, I utilized a HANA DB ODBC connector I had configured for a previous project. By connecting directly to the HANA layer, I was able to fetch validation rules and default lists (like Bank Keys or Tax Categories) in real-time, bypassing the need for dozens of non-existent APIs.
3. Navigating the Documentation Vacuum
There was no Swagger UI, no technical manual, and no Excel schema. The "documentation" was a single Postman collection that worked intermittently.
- The Solution: I took a "Reverse-Engineering" approach. I built a comprehensive master sheet of every field, data type, and mandatory attribute. Every time a new error surfaced, I documented the accepted values and constraints, essentially building the company’s first functional API documentation for this integration.
4. Decoding the "Internal Server Error"
Initially, the API response was a generic: An internal server error occurred: MPL ID.... This made debugging impossible for the end-user.
- The Solution: I worked closely with the SAP team to push for custom error handling within the standard API. Today, 90% of validation errors are identified and returned with clear messages (e.g., "Invalid Postal Code" or "Missing Industry Key"), allowing users to fix their own data entry issues without technical support.
5. Environment Drift & Authorizations
We faced unforeseen authorization failures on Production (PRD) that never appeared in the Sandbox or Dev environments.
- The Fix: We performed a granular audit of the Integration User permissions. We ensured that every segment—Credit, Sales, and Purchase—had the specific SAP Authorization Objects required for a "Commit," ensuring the automated user had the same rights as a manual power-user.
6. Operating Without a QA Safety Net
With no dedicated QA team, the burden of stability was on the development lead. To mitigate this, I built "Hardened" Validations within Simplifi. The system now "pre-validates" data against SAP logic before the request is even submitted, catching 95% of errors at the source.
Realizations from the Trenches
The "Hidden" Timeline
Estimation in SAP projects is rarely linear. We realized that for every hour of coding, there were two hours of "chasing" dependencies—waiting for Functional Consultants (FCs) or ABAPers to verify field logic. This led to shifting timelines and many late nights, including working through the Eid holidays to ensure the Production environment remained stable.
The Need for Governance
As the project scaled from 10 to 400+ users, we realized that spreadsheets were no longer enough. We introduced a formal Project Management/Roadmap tool to track the 14+ concurrent workstreams. This provided the visibility needed to show management exactly where the bottlenecks were—usually in external dependencies rather than internal code.
The Outcome
Despite the technical and environmental challenges, the Phase 3 rollout was a success.
-
99% Reduction in Manual Effort: A 15-minute task is now a 10-second background sync.
-
Data Integrity: By replacing manual text fields with SAP-fetched dropdowns, the error rate has dropped significantly.
-
Scalability: The system is now prepared to handle the next phase of automation, including Bulk Credit Assessments and Treasury extensions.
Conclusion
Building an integration of this scale isn't just about writing code; it’s about system resilience. It requires the patience to debug legacy payloads and the leadership to drive a project forward when requirements change mid-stream. For any developer tackling SAP: Focus on the data integrity first; the speed will follow.