Senior Data Engineer (AWS)

An exciting opportunity to be part of a technology and analytics driven consulting company whose core capabilities span the disciplines of Data Science, Data Engineering and Software Engineering.

Your responsibilities will include:

· Analyse and organise raw data

· Design data engineering solutions to meet business requirements

· Build scalable data pipelines that clean, transform and aggregate data from different sources using appropriate tools and technologies

· Collaborate with data scientists to prepare data sets for analytical modelling

· Identify ways to enhance data quality and reliability

· Manage technical delivery of projects

· Mentor junior data engineers

Qualifications and experience needed:

· 8+ years of relational SQL database

· Solid understanding of Apache Spark as a distributed data processing framework

· Experience in designing, implementing and performance tuning solutions in Apache Spark

· Experience in complex data warehouse and data lake design

· Experience in building, optimising and manging data pipelines for large and complex datasets

· Experience using AWS cloud data platform technologies:

o Amazon S3

o Amazon Redshift

o AWS Glue

o Amazon Athena

o Amazon Elastic MapReduce (EMR)

o AWS Step Functions

o AWS Lambda

to implement data pipelines and analytical solutions

· Experience in Python would be beneficial

· Experience with streaming technologies such as Spark streaming or Kafka would be beneficial

· A relevant data engineering certification would be beneficial

  • Graduate or Postgraduate degree in Computer Science, Information Technology, Statistics, Engineering or related technical field.

Apply Now

Other Jobs Available

Past Close Positions

Apply Now

Other Jobs Available

Past Close Positions

Scroll to Top