Job Overview

Roles & Responsibility:

• As an Azure Data Engineer, you will be responsible for designing, building, and maintaining data pipelines to support business and analytics needs. You will work closely with data analysts and business stakeholders to understand their needs and develop solutions to meet those needs.
• 4+ Years of experience in Data Engineering with cloud platforms such as Azure and/or AWS.
• 4+ years of hands-on experience in Azure technologies such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage, or equivalent AWS services such as AWS Glue, Amazon EMR / AWS Glue Spark, Amazon Redshift, and Amazon S3.
• Design, develop, and manage scalable data pipelines that integrate information from various sources into a centralized data lake or data warehouse using Azure Data Factory or AWS Glue, storing data in Azure Data Lake Storage or Amazon S3.
• Extract, transform, and load (ETL/ELT) data from a variety of sources including on-premises databases, cloud-based data stores, and APIs, leveraging tools such as Azure Data Factory, Azure Databricks, AWS Glue, or AWS Lambda.
• Optimize and tune data pipelines for performance, scalability, and cost efficiency across cloud platforms using best practices and automation tools.
• Ensure data quality, integrity, and security throughout the data lifecycle by implementing data governance policies, monitoring frameworks, and compliance standards.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and design scalable data models, pipelines, and analytical datasets to support business insights and decision-making.
• Monitor and troubleshoot data pipelines, identifying and resolving issues in a timely manner to minimize impact on business operations.

Desired Candidate Profile:

• Any Degree
• Proven experience as a data engineer, with a focus on designing and building data solutions on Azure/AWS.
• Strong proficiency in SQL.
• Experience with data modeling, schema design, and performance optimization techniques.
• Familiarity with data integration patterns, ETL/ELT processes, and data warehousing concepts.
• Excellent problem-solving and analytical skills, with the ability to troubleshoot complex data issues and drive resolution.
• Effective communication and collaboration skills, with the ability to work effectively in a remote and cross-functional team environment.

Experience with modern data lakehouse architectures using platforms such as Databricks, Delta Lake, or equivalent technologies is a plus.

MOTIVATIONAL/CULTURAL FIT

As a data engineer, must be driven by the challenge of turning raw data into valuable insights. With expertise in optimizing data pipelines and a passion for innovation, must be committed to powering informed decisions, and driving success for the organization.

Competencies