Data Engineer Associate - Finance and Supply Chain

PURPOSE OF THE POSITION:
 

This position supports the design, development and operationalization support of data ingestion, transformation and loading processes across multiple data sources within Finance Data Warehouse (DW)and Data Lake platforms. The position requires strong understanding of business and analytical requirements. The Data Engineer works under guidance and mentorship of the senior data engineer and other team members to develop ETL/ELT pipelines and data governance processes that ensure data quality, security and scalability of data solutions.

MAJOR RESPONSIBILITIES:
 

Design, Develop and Deploy ETL/ELT Pipelines

• Work closely with Finance Business Users/SMEs, Business and Data Analysts to gather and translate the business requirements to clarify requirements and supports creation of technical specifications.
• Develop, Test and Deploy ETL/ELT pipelines for data ingestion, ingestion and loading using established data framework design and CI/CD practices.
• Implement automation for data workflows, job orchestration and dependency management either batch or real time data workflows.
• Participate in sprint ceremonies, release planning, development effort assessment and documentation creation and reviews.

Platform Operations and Support

• Provide Level 1/2 support for Finance DW pipelines which entails performing root cause analysis and code fix/enhancementacross data pipelines, storage layers and downstream integration/analytics requirements.
• Assist in monitoring production ETL/ELT jobs, execution of runbook and incidents monitoring.

Data Quality and Data Governance

• Supports the Data Quality Analyst to define and implement business rules, data validation, data integration rules and data quality/anomaly detection.
• Development of data validation, cleansing to ensure data accuracy and consistency.
• Ensure compliance with data governance standards and maintain metadata, data lineage and documentation.

REQUIREMENTS:

• Understanding of Microsoft Fabric components, specifically components of data lake and data warehouse (Lakehouse, Warehouse, Dataflows, Gen2, Pipeline, Notebooks).
• 1-2 years of overall experience in ETL/ETL development and including hands-on development and support of and data warehouse/data lake implementation.
• Familiarity of data warehouse concepts and data modelling, including dimensional modelling, medallion architecture (Bronze-Silver-Gold) and end to end (ETL/ELT) pipeline design.
• Basic knowledge of Python, Apache Spark, SQL and exposure to Cloud Native data integration tools.
• Demonstrated ability to collaborate effectively with cross-functional teams, including data scientists, analysts, and software engineers to deliver end-to-end data solutions and adaptability to agile environment.
• Exposure in implementing data quality, validation, and governance practices to ensure data integrity and compliance with organizational and regulatory and security standards.
• Familiarity with ERP system, budget and planning software and Finance Datawarehouse implementation is an advantage.
• Understanding of data governance and lineage. Experience with CI/CD practices using Git integration for Fabric.
• Working knowledge of PowerBI datasets and semantic modelling to enable BI and self-service.
• Experience with large data in a multinational organization. Strong willingness to learn and develop technical skills.
• Demonstrate capability to execute defined tasks and raises issues proactively.
• Has training, education or experience using AI