Senior Azure Data Engineer
- Join this exciting new project as a Senior Azure Data Engineer!
- With your required expertise in developing and deploying application on MS Azure Databricks, you will work with a strong team to implement a Cloud Modernization Project.
- You will support and interact with the client daily in an Agile way of working.
- You will have the opportunity to show/grow with technology in a customer-facing role.
- You should be self-motivated to drive solutions and proactively build (in sufficient detail) the documentation required to support and describe technical solutions.
- You should be able to work independently under limited supervision and apply your knowledge of Azure-based ETL.
- You will work with cloud architects to establish pipelines and develop spark code using Databricks platform.
- You should have sufficient knowledge and maturity to optimally take care of technical issues and help to support the broader team.
- Your success will be directly related to CGI’s and our client’s success.
Your future duties and responsibilities:
- The senior data engineer will be responsible for development, deployment, maintenance, diagnostics and support of spark ETL jobs on the MS Databricks platform.
- Will streamline code, rationalize datasets and tables to arrive at single source of truth
- Will build detailed architecture diagrams for all workflows and processes
- Migrate existing clients from the existing On-Prem solution to the Azure Solution
Required qualifications to be successful in this role:
- 5 years of hands on experience with cloud automation and scripting in an Azure environment.
- Experience in writing and running Unit Tests on Databricks using Azure Pipeline is preferred
- Knowledge on hands-on agile development – experience in visualizing and breaking development work in to smaller units such as user stories, tasks etc.
- Experience in peer reviewing code – provide recommendations for improving code quality and work you're doing to the client
- Experience in stream processing tools (Spark structured streaming)
- Experience in serverless and cloud technologies like Azure functions
- Experience in data modeling and in developing data warehouse and data lake solutions.
- Experience in supervising and logging services in Azure
- Experience in development of apache Spark code using PySpark/Scala
- Experience in using Azure Databricks Platform
- Experience in using Azure Data Factory to call Azure Databricks notebook activities
- Experience in using Azure Storage (Blob/Data Lake Store), Databricks Delta
- Experience in using Git based repositories
- Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
- Exposure to Azure DevOps
Level of Education: Bachelor of Science Degree in Computer Science or a related subject.
How to Apply:
Apply online at https://www.cgi.com/en/careers
Visit Site to Apply
Location: Lafayette, LA
Date Posted: July 06, 2022
Application Deadline: September 08, 2022
Job Type: Full-time