Senior Azure Data Engineer

Back

Senior Azure Data Engineer

@ CGI

Position Description:

• Join this exciting new project as a Senior Azure Data Engineer!
• With your required expertise in developing and deploying application on MS Azure Databricks, you will work with a strong team to implement a Cloud Modernization Project.
• You will support and interact with the client daily in an Agile way of working.
• You will have the opportunity to show/grow with technology in a customer-facing role.
• You should be self-motivated to drive solutions and proactively build (in sufficient detail) the documentation required to support and describe technical solutions.
• You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL.
• You will work with cloud architects to establish pipelines and develop spark code using Databricks platform.
• You should have sufficient knowledge and maturity to optimally take care of technical issues and help to support the broader team.
• Your success will be directly related to CGI’s and our client’s success.

Your future duties and responsibilities:

• The senior data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
• Will streamline code, rationalize datasets and tables to arrive at single source of truth
• Will build detailed architecture diagrams for all workflows and processes
• Migrate existing clients from the existing On-Prem solution to the Azure Solution

---

Qualifications:

Required qualifications to be successful in this role:

• 5 years of hands on experience with cloud automation and scripting in an Azure environment.
• Experience in writing and running Unit Tests on Databricks using Azure Pipeline is preferred
• Knowledge on hands-on agile development – experience in visualizing and breaking development work in to smaller units such as user stories, tasks etc.
• Experience in peer reviewing code – provide recommendations for improving code quality and work you're doing to the client
• Experience in stream processing tools (Spark structured streaming)
• Experience in serverless and cloud technologies like Azure functions
• Experience in data modeling and in developing data warehouse and data lake solutions.
• Experience in supervising and logging services in Azure
• Experience in development of apache Spark code using PySpark/Scala
• Experience in using Azure Databricks Platform
• Experience in using Azure Data Factory to call Azure Databricks notebook activities
• Experience in using Azure Storage (Blob/Data Lake Store), Databricks Delta
• Experience in using Git based repositories
• Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
• Exposure to Azure DevOps

 Level of Education: Bachelor of Science Degree in Computer Science or a related subject.

Skills:

• ETL
• Python
• Spark SQL
• SQL

---

How to Apply:

Apply online at https://www.cgi.com/en/careers

Visit Site to Apply