Senior Data Engineer

Main Responsibilities:

• Build and manage data integration processes, ETL workflows, and data pipelines to handle large data streams.
• Create data models and structured data solutions to maintain high standards of data quality, consistency, and reliability.
• Ensure optimal performance of data pipelines for streamlined data retrieval and processing.
• Integrate raw information from various sources, demonstrating your ability to work with diverse data sets.
• Work closely with other data professionals to form robust data governance, validation, and development practices.
• Stay updated with the latest technological advancements in data engineering to ensure maximum efficiency and software quality.
• Engage actively in a self-organizing Scrum team, exhibiting your agile mindset.

Main Skills Required:

• 5 years of experience as a Data Engineer
• Big Data Ecosystem, including Cloud technologies
• Expert-level Python skills, along with intermediate proficiency in PySpark, Hadoop, Hive, Airflow, and SQL.
• Experience with AWS technologies will be an added advantage.
• Hands-on experience in optimizing the performance of Spark processes.