DATA ENGINEER

Job Overview

Responsible for designing and establishing the solution architecture for analytical platforms and/or data-enabled solutions.Has broad understanding of the entire data and analytics eco-system of tools and technologies, with hands-on experience across all three core data domains of data science, data engineering, and data visualization. Responsible for fleshing out the details of the end-to-end component architecture being implemented by the team. Contributes to the team's development velocity, if necessary. Understands how an organization's data platform integrates within the overall technical architecture. Understands the business purpose the role fulfills, and the roadmap for the lifecycle of an organization's systems. Sets the direction and establishes the approach for integrating information applications and programs.

Roles & Responsibilities

As an Azure Data Engineer, you will be responsible for designing, building, and maintaining data pipelines to support business and analytics needs. You will work closely with data analysts and business stakeholders to understand their needs and develop solutions to meet those needs.

• 4+ Years of Experience in Azure with Data Engineering.
• 4+ Hands on experience in Azure technologies such as Azure Data Factory, Azure Databricks and Azure Synapse Analytics, Azure Data Lake
• Design, develop, manage data flows that integrate information from various sources into a common data lake using Microsoft Azure data Factory.
• Extract, transform and load data from variety of sources, including on-premises databases, cloud-based data stores and API calls.
• 4+ years of working with SQL, implement data processing and transformation logic using SQL, handling complex windowing functions and joins.
• Optimize and tune data pipelines for performance, scalability, and cost-efficiency, leveraging best practices and automation tools.
• Ensure data quality, integrity, and security throughout the data lifecycle, implementing data governance policies and compliance standards.
• Collaborate with data analysts and business stakeholders to understand their data needs and develop data models and solutions to meet those needs.
• Monitor and troubleshoot data pipelines, identifying and resolving issues in a timely manner to minimize impact on business operations.

Competencies And Experience Required/Desired

• Any Degree
• Proven experience as a data engineer, with a focus on designing and building data solutions on Azure.
• Knowledge of Azure data services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL Datawarehouse, and Azure Data Lake Storage.
• Strong proficiency in SQL.
• Experience with data modeling, schema design, and performance optimization techniques.
• Familiarity with data integration patterns, ETL/ELT processes, and data warehousing concepts.
• Excellent problem-solving and analytical skills, with the ability to troubleshoot complex data issues and drive resolution.
• Effective communication and collaboration skills, with the ability to work effectively in a remote and cross-functional team environment.
• To summarize: Experience working on ETL, Microsoft Azure SQL Data Warehouse, SQL Scripting, with exposure to other services on the Azure landscape, API Data Extraction

Competencies