Senior Azure Data Engineer

Back

Senior Azure Data Engineer

@ CGI

Position Description:

  • Join this exciting new project as a Senior Azure Data Engineer!
  • With your expertise in developing and deploying application on MS Azure DataBricks, you will work with a high-performing team to implement a Cloud Modernization Project.
  • You will support and interact with the client daily in an Agile way of working.
  • You will have the opportunity to show and grow with technology in a customer-facing role.
  • We are looking for someone who is self-motivated to drive solutions and proactively create (in sufficient detail) the documentation required to support and describe technical solutions.
  • You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL experience.
  • You will work with cloud architects to establish pipelines and develop spark code using DataBricks platform.
  • You should have sufficient knowledge and maturity to effectively deal with technical issues and help to support the broader team.


Your future duties and responsibilities:

  • The senior data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
  • Will streamline code, rationalize datasets and tables to arrive at single source of truth
  • Will build detailed architecture diagrams for all workflows and processes
  • Migrate existing clients from the existing On-Prem solution to the Azure Solution


Qualifications:

Required qualifications to be successful in this role:

  • 5 years of hands on experience with cloud automation and scripting in an Azure environment.
  • Experience in stream processing tools (Spark structured streaming)
  • Experience in server less and cloud technologies like Azure functions
  • Experience in data modeling and in developing data warehouse and data lake solutions.
  • Experience in monitoring and logging services in Azure
  • Experience in development of apache Spark code using PySpark/Scala
  • Experience in using Azure DataBricks Platform
  • Experience in using Azure Data Factory to call Azure DataBricks notebook activities
  • Experience in using Azure Storage (Blob/Data Lake Store), DataBricks Delta
  • Experience in using Git based repositories
  • Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
  • Exposure to Azure DevOps


Years of Experience/Skill Set:

  • 10 years of overall experience in Information Technology
  • 7 years of experience in ETL
  • 5 years of SQL experience
  • 3 years if Azure DataBricks experience
  • 5 years of experience with Spark (PySpark/Spark Scala)
  • 5 years of experience with Modern Language (Python/Scala)
  • 2 years of experience in Agile


Educational Requirements: Bachelor's Degree in Computer Science or Related Field


Skills:

  • ETL
  • Python
  • Scala
  • Spark SQL
  • SQL
  • Azure DevOps


How to Apply:

Apply online at https://www.cgi.com/en/careers

Visit Site to Apply

Location: Lafayette, LA
Date Posted: April 13, 2022
Application Deadline: June 13, 2022
Job Type: Full-time