Data Engineer - Azure

Venn Group are working with a local authority client in London to recruit an Azure Data Engineer to expand their Internet of Things and Smart Cities capabilities.

• IR35 Status: Inside
• Rate: £550 umbrella per day
• Working pattern: Hybrid - London

As a Data Engineer, you will play a pivotal role in shaping how data is harnessed to drive decision-making and enhance services across the council. You will work within a dedicated product team, designing and developing high-quality data solutions that ensure data integrity, accessibility, and scalability.

You will be responsible for building and running data products and services that generate high-quality insights, supporting the council's mission to use data effectively in service delivery. This is an exciting opportunity to influence how a high-profile local authority utilises data to benefit its residents and stakeholders.

Responsibilities:

• Develop, optimise, and maintain scalable data pipelines and architectures
• Ensure high data quality and availability, both internally and externally shared datasets
• Build and run data products to provide high-quality insights
• Work with modern data engineering tools and cloud-based technologies, such as Azure Data Factory, Synapse, and Snowflake
• Implement dimensional data models and work with SQL and NoSQL databases, including time-series databases like InfluxDB
• Utilise Python for data pipeline development and visualisation
• Deploy applications using Docker and contribute to CI/CD pipelines

Desirable skills:

• Extensive experience in developing scalable, reusable data pipelines on cloud services
• Strong cloud data engineering experience, particularly in Azure Data Factory, Synapse, and Snowflake
• Proficiency in Python, SQL, and NoSQL databases, with a focus on time-series databases (e.g., InfluxDB)
• Knowledge of dimensional modelling and deploying applications using Docker
• Familiarity with CI/CD operating models and best practices in data engineering