Sr. Data Engineer

We're seeking a Senior Data Engineer to join our Delivery Team. In this role, you'll design and implement modern data architectures that enable our clients to make data-driven decisions.

Requirements

• 5+ years of experience in data architecture, data engineering, or analytics solution design.
• Hands-on experience with data lake and warehouse technologies (e.g., Databricks, Snowflake, Redshift, Synapse).
• Deep understanding of data modeling, data integration, and ETL/ELT design.
• Proficiency in SQL and one or more programming languages (Pyspark/Python, Scala), particularly for complex data transformations and optimization within Spark
• Solid understanding of data governance, security, and privacy best practices.
• Proven experience in designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
• Deep practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
• Experience in building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
• Extensive experience in successfully applying and enforcing the Medallion architecture (Bronze, Silver, Gold layers) within a Databricks environment
• Experience in designing and implementing CI/CD pipelines (using tools like Azure DevOps, GitHub Actions, GitLab CI) specifically tailored for Databricks workflows, notebooks, and cluster configurations, enabling automated deployment and testing
• Experience in planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
• Strong working knowledge of at least one major cloud provider (AWS, Azure) regarding data storage, networking, and security concepts relevant to Databricks deployment.
• Proven ability to engage with clients, present technical solutions, and communicate complex ideas clearly.
• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.
• Excellent problem-solving, communication, and collaboration skills.