SC Data Engineer - CGEMJP

Role Title: Data Engineer
Duration: contract to run until 31/03/2025
Location: Remote but potentially one day a month in Newcastle Upon Tyne
Rate: up to £623.70 p/d Umbrella inside IR35
Clearance required: Active SC

Job Description:

• 3-5 years experience
• Listen to the needs of technical and business stakeholders, and interpret them
• Effectively manage stakeholder expectations
• Manage active and reactive communication
• Support or host difficult discussions within the team or with diverse senior stakeholders
• Understand and help teams to apply a range of techniques for data profiling
• Source system analysis from a complex single source
• Bring multiple data sources together in a conformed model for analysis
• Establish enterprise-scale data integration procedures across the data development life cycle, and ensure that teams adhere to them
• Manage resources to ensure that data services work effectively at an enterprise level
• Identify areas of innovation in data tools and techniques, and recognise appropriate timing for adoption
• Establish standards, keep them up to date and ensure adherence to them
• Keep abreast of best practice in industry and across government
• Understand the concepts and principles of data modelling and can produce relevant data models
• Work across government and industry, recognising opportunities for the reuse and alignment of data models in different organisations
• Design the method to categorise data models within an organisation
• Design an appropriate metadata repository and present changes to existing metadata repositories
• Understand a range of tools for storing and working with metadata
• Provide oversight and advice to more inexperienced members of the team
• Ensure that the most appropriate actions are taken to resolve problems as they occur
• Co-ordinate teams to resolve problems and to implement solutions and preventative measures
• Use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations
• Collaborate with others to review specifications where appropriate
• Show a thorough understanding of the technical concepts required for the role, and can explain how these fit into the wider technical landscape
• Review requirements and specifications, and define test conditions
• Identify issues and risks associated with work
• Analyse and report test activities and results

Key Technical Skills

* AWS: S3, Lambda, EMR, SMS, SQS, and additional services related to data infrastructure. Terraform. Databricks. Kafka. Data Lake, Warehouse, Lakehouse architecture and design. Python/Pyspark. Data platforms and notebooks: Jupyter, Databricks, Azure. Gitlab: repository and CI/CD. Java (Spring Boot) experience is a plus

All profiles will be reviewed against the required skills and experience. Due to the high number of applications we will only be able to respond to successful applicants in the first instance. We thank you for your interest and the time taken to apply!