Senior Azure Data Engineer

Back

Senior Azure Data Engineer

@ CGI

Position Description:

  • Join this exciting new project as a Senior Azure Data Engineer!
  • With your required expertise in developing and deploying application on MS Azure Databricks, you will work with a strong team to implement a Cloud Modernization Project.
  • You will support and interact with the client daily in an Agile way of working.
  • You will have the opportunity to show/grow with technology in a customer-facing role.
  • You should be self-motivated to drive solutions and proactively build (in sufficient detail) the documentation required to support and describe technical solutions.
  • You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL.
  • You will work with cloud architects to establish pipelines and develop spark code using Databricks platform.
  • You should have sufficient knowledge and maturity to optimally take care of technical issues and help to support the broader team.
  • Your success will be directly related to CGI’s and our client’s success.


Your future duties and responsibilities:

  • The senior data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
  • Will streamline code, rationalize datasets and tables to arrive at single source of truth
  • Will build detailed architecture diagrams for all workflows and processes
  • Migrate existing clients from the existing On-Prem solution to the Azure Solution


Qualifications:

Required qualifications to be successful in this role:

  • 5 years of hands on experience with cloud automation and scripting in an Azure environment.
  • Experience in writing and running Unit Tests on Databricks using Azure Pipeline is preferred
  • Knowledge on hands-on agile development – experience in visualizing and breaking development work in to smaller units such as user stories, tasks etc.
  • Experience in peer reviewing code – provide recommendations for improving code quality and work you're doing to the client
  • Experience in stream processing tools (Spark structured streaming)
  • Experience in serverless and cloud technologies like Azure functions
  • Experience in data modeling and in developing data warehouse and data lake solutions.
  • Experience in supervising and logging services in Azure
  • Experience in development of apache Spark code using PySpark/Scala
  • Experience in using Azure Databricks Platform
  • Experience in using Azure Data Factory to call Azure Databricks notebook activities
  • Experience in using Azure Storage (Blob/Data Lake Store), Databricks Delta
  • Experience in using Git based repositories
  • Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
  • Exposure to Azure DevOps


 Level of Education: Bachelor of Science Degree in Computer Science or a related subject.


Skills:

  • ETL
  • Python
  • Spark SQL
  • SQL


How to Apply:

Apply online at https://www.cgi.com/en/careers

Visit Site to Apply

Location: Lafayette, LA
Date Posted: July 06, 2022
Application Deadline: September 08, 2022
Job Type: Full-time