Senior Azure Data Engineer

Back

Senior Azure Data Engineer

@ CGI

Position Description:

• Join this exciting new project as a Senior Azure Data Engineer!
• With your expertise in developing and deploying application on MS Azure DataBricks, you will work with a high-performing team to implement a Cloud Modernization Project.
• You will support and interact with the client daily in an Agile way of working.
• You will have the opportunity to show and grow with technology in a customer-facing role.
• We are looking for someone who is self-motivated to drive solutions and proactively create (in sufficient detail) the documentation required to support and describe technical solutions.
• You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL experience.
• You will work with cloud architects to establish pipelines and develop spark code using DataBricks platform.
• You should have sufficient knowledge and maturity to effectively deal with technical issues and help to support the broader team.

Your future duties and responsibilities:

• The senior data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
• Will streamline code, rationalize datasets and tables to arrive at single source of truth
• Will build detailed architecture diagrams for all workflows and processes
• Migrate existing clients from the existing On-Prem solution to the Azure Solution

---

Qualifications:

Required qualifications to be successful in this role:

• 5 years of hands on experience with cloud automation and scripting in an Azure environment.
• Experience in stream processing tools (Spark structured streaming)
• Experience in server less and cloud technologies like Azure functions
• Experience in data modeling and in developing data warehouse and data lake solutions.
• Experience in monitoring and logging services in Azure
• Experience in development of apache Spark code using PySpark/Scala
• Experience in using Azure DataBricks Platform
• Experience in using Azure Data Factory to call Azure DataBricks notebook activities
• Experience in using Azure Storage (Blob/Data Lake Store), DataBricks Delta
• Experience in using Git based repositories
• Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
• Exposure to Azure DevOps

Years of Experience/Skill Set:

• 10 years of overall experience in Information Technology
• 7 years of experience in ETL
• 5 years of SQL experience
• 3 years if Azure DataBricks experience
• 5 years of experience with Spark (PySpark/Spark Scala)
• 5 years of experience with Modern Language (Python/Scala)
• 2 years of experience in Agile

Educational Requirements: Bachelor's Degree in Computer Science or Related Field

Skills:

• ETL
• Python
• Scala
• Spark SQL
• SQL
• Azure DevOps

---

How to Apply:

Apply online at https://www.cgi.com/en/careers

Visit Site to Apply