The challenge
The client, operating on an SAP S/4HANA environment, faced a significant data governance gap due to the historical lack of native connectivity between Collibra and SAP systems. While the client possessed Silwood Safyr (now called ERP/CRM Integrator for SAP) – a third-party connector designed to bridge this gap – the solution was complex to configure and underutilized.
The primary pain point was data discoverability. SAP data structures utilize cryptic technical naming conventions (e.g., table names like MARA) which carry no meaning for business users. Consequently, non-technical stakeholders could not effectively search for or understand the data residing within their SAP landscape, making it difficult for business users to find the data they needed and use it in their daily work.
The solution
To resolve these issues, a technical implementation project was executed, focusing on the configuration of the Silwood Safyr connector, the development of a semantic and logical layer as well as the automation of ingestion workflows.
- Connector configuration & optimization: The team successfully configured the Silwood Safyr connector to ingest metadata from SAP S/4HANA into Collibra. This required multi-stakeholder coordination involving Collibra, Safyr, and SAP technical teams to enable necessary “ABAP functions” and establish connectivity.
- Scoped metadata ingestion: To prevent data swamps, the team avoided an “all-in” ingestion approach. Instead, Murdio designed a scalable ingestion strategy based on clearly defined Subject Areas, allowing the client to precisely control what data was brought into Collibra. This approach ensured that only relevant tables and branches were ingested, while also creating a framework that could be easily expanded over time as new domains were onboarded.
- Automated SAP Ingestion Pipeline: To eliminate manual maintenance, the team implemented an automated workflow using Windows Task Scheduler and custom ERP Integrator scripts. Instead of manually executing imports via the UI, these scripts are triggered automatically at defined intervals (e.g., weekly or monthly) to refresh SAP metadata in Collibra. The solution includes a log monitoring protocol, allowing the technical team to verify success status and rapidly intervene manually only if specific connectivity errors or data volume exceptions occur.
- Semantic & logical layer development: A custom workflow was built to translate technical SAP metadata into a logical & semantic layer. This process mapped physical assets (e.g., technical table names) to business-friendly assets. For example, a technical table was linked via a relation to a business term like “Material Information,” making the data comprehensible.
- Governance & lifecycle management: The solution included defining the Community and Domain structure within Collibra to establish data ownership. Additionally, technical documentation and best practices were created to manage connector updates and Subject Area modifications, ensuring the solution’s maintainability.
The results
The project delivered tangible improvements in data accessibility and governance efficiency:
- Democratized SAP data discoverability: The most significant outcome was the enablement of “data discoverability” for non-technical users. Business stakeholders can now search for common business terms (e.g., “material data”) in Collibra and instantly view the corresponding technical SAP objects and their relationships.
- Transparent data lineage: The implementation bridged the gap between IT and Business by providing a clear lineage between the semantic layer (business context) and the physical layer (SAP S/4HANA tables).
Scalable & Automated Framework: The client was left with a documented, operational framework. The combination of scoped Subject Areas and automated scheduling allows the metadata environment to stay current with minimal manual oversight as the SAP landscape evolves
