Data Engineer, Azure

Data Engineer, Azure
London/Hybrid
up to £87k + bonus + excellent benefits

Azure, Fabric, Python

Data Engineer with extensive cloud experience is sought by thriving financial services organisation in the City of London. As part of the central data platform team, you will be responsible for developing the Data Engineering standards to be followed by federated data teams across the business.

Within these standards our client is looking to create a Data Engineering development environment that has the flexibility to ensure engineers work as efficiently as possible while operating within the security constraints outlined by the business.

Responsibilities

• Work with Product Owners to establish product requirements.
• Design and develop observable and secure data pipelines.
• Source, integrate and clean data using medallion architecture.
• Ensure data completeness, integrity and validity.
• Create Lakehouses and Warehouses.
• Optimise data pipelines, processing workflows and Fabric warehouses for performance.
• Maintain Data Catalogue and ensure data duplication does not occur.
• Ensure data security is implemented and enforced.
• Set and own standards in the Data Engineering space.
• Work with the DevOps team to establish CI/CD processes.
• Work with other members of the central platform team to monitor the Microsoft Fabric feature roadmap and integrate new features into the established eco-system.
• BCP/DR strategy.
• Work with other members of the central platform team to define an efficient project process to deliver new data products.

Skills

• Minimum 5 years working in a cloud environment using data engineering tools on a variety of complex products.
• Python
• Microsoft Fabric
• Power BI
• Semantic Models
• Azure Data Factory
• Azure Synapse
• SQL Server
• ETL/ELT
• Database Design
• Data Warehousing
• Datalake/Lakehouse

For a full consultation on this exciting new opportunity, get in touch with Arc IT today!