Senior Data Integration Engineer

We are actively seeking an experienced Senior Data Integration Engineer to join our team. In this role, you will play a crucial part in designing, developing, testing, deploying, maintaining, and enhancing data integration pipelines for our cutting-edge projects. If you possess a wealth of experience in data integration and pipeline development and thrive in a dynamic, collaborative environment, we welcome you to be a key player in shaping the future of our data-driven initiatives.
Responsibilities
Design, develop, test, deploy, maintain, and improve data integration pipelines
Utilize Python and common libraries for efficient data integration and pipeline development
Employ advanced SQL for analytical purposes and to optimize complex queries
Collaborate with cross-functional teams to ensure the success of data integration projects
Implement and maintain data integration and pipeline developments in an AWS environment
Optimize and debug user-defined functions, views, and indexes for efficient data integration
Utilize GitLab for source control and Jenkins for build and continuous integration
Contribute to the enhancement and maintenance of data integration pipelines
Provide insights and recommendations for improving data integration processes
Collaborate with the wider engineering team to drive standards of excellence across all data integration projects
Requirements
At least 3+ years of hands-on experience in data integration and pipeline development
Strong expertise in Amazon Web Services (AWS) Cloud for data integration with Apache Spark, Glue, Kinesis, and S3
Proficiency in Python for data integration, with real-life experience in Python development
Solid understanding of advanced SQL for analytical purposes and complex query optimization
Experience with Databricks in an AWS environment for effective data integration
Knowledge of source control systems such as GitLab and continuous integration tools like Jenkins
Analytical experience with databases, including writing complex queries, query optimization, and debugging
Strong communication and collaboration skills, essential for effective teamwork
Fluent English language skills at an Upper-Intermediate level
Nice to have
Familiarity with other data integration tools and technologies