Senior Data Engineer - Azure Databricks

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. We help organisations solve complex business challenges by combining deep domain understanding with modern data and AI capabilities.
Our teams work across strategy, analytics, engineering, and product delivery to create scalable, high-value solutions that improve decision-making, efficiency, and growth.
We are looking for a skilled Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and platforms using Azure and Databricks. The ideal candidate will work closely with analytics, data science, and business teams to deliver reliable, high-performance data solutions that enable data-driven decision-making.
Key Responsibilities
Design, develop, and optimize end-to-end data pipelines using Azure Databricks
Build and maintain scalable ETL/ELT frameworks for structured and unstructured data
Ingest data from multiple sources including APIs, databases, files, and streaming systems
Work with Azure Data Lake Storage (ADLS Gen2) for efficient data storage and retrieval
Develop data transformation logic using PySpark and Spark SQL
Ensure data quality, reliability, and performance across pipelines
Implement data modeling concepts to support analytics and reporting use cases
Collaborate with Data Scientists, BI teams, and business stakeholders to understand data requirements
Monitor, troubleshoot, and optimize Databricks jobs for performance and cost efficiency
Apply best practices around security, governance, and compliance within Azure
Participate in code reviews, documentation, and continuous improvement initiatives
5+ years of hands-on experience as a Data Engineer
Strong experience with Azure Databricks
Proficiency in PySpark, Spark SQL, and Python
Experience working with Azure Data Lake Storage (ADLS Gen2)
Solid understanding of data warehousing and data modeling concepts
Experience building batch and/or streaming data pipelines
Familiarity with Azure Data Factory or similar orchestration tools
Strong SQL skills and experience with relational databases
Experience using Git or similar version control systems
Strong analytical, problem-solving, and communication skills
Experience with Delta Lake
Knowledge of streaming technologies (Kafka, Azure Event Hubs, Spark Structured Streaming)
Exposure to CI/CD pipelines using Azure DevOps
Experience working in Agile/Scrum environments
Understanding of cloud cost optimization strategies