BackAWS Data Engineer
@ CGI
Position Description:
- Want to leverage your experience and development skills in the Judicial Sector as an AWS Data Engineer?
- CGI seeks a data engineer for a high-volume data ingestion and search modernization effort.
- The data engineer will provide key features in the modern system, including: creating high-volume data ingestion pipelines for artifacts from legacy systems, automate the analysis of the ingested artifacts, and create knowledge bases using the latest artificial intelligence models.
- The data engineer will be involved in the modern system's data architecture, pipeline version control, data lineage tracking, and automated testing.
- Data quality management and traceability will be integrated into the automated pipelines.
- The data engineer will apply software engineering best practices throughout their work and leverage the latest cloud technologies to innovate and provide value to our customers.
- This position is located in our Lafayette, LA office; however, a hybrid working model is acceptable.
Your future duties and responsibilities:
- Our AWS Data Engineer will be a key contributor with the below responsibilities:
- Work with the technical development team and team leader to understand desired application capabilities.
- Work with clients and legacy systems to setup ingestion pipelines that pull new and updated artifacts into knowledge bases.
- Continuously improve machine learning models
- Design and apply data architectures that enable both field-based search and semantic search.
- Understand conflicting artifacts and prioritize accurate or more recent artifacts within the knowledge bases.
- Work within and across Agile teams to test and support technical solutions across a full stack of development tools and technologies.
- Develop applications in AWS - data and analytics technologies including, but not limited to: OpenSearch, RDS, S3, Athena; Lambda, Step Functions, Glue; Sagemaker, Textract, Comprehend, Bedrock, AI Chatbot/Lex; SQS, SNS; CloudTrail, CloudWatch; VPC, EC2, ECS and IAM.
- Application development by lifecycles, & continuous integration/deployment practices.
- Work to integrate open-source components into data-analytic solutions.
- Working with vendors to enhance tool capabilities to meet enterprise needs.
- Willingness to continuously learn & share learnings with others.
Qualifications:
Required qualifications to be successful in this role:
- Proficient in navigating the AWS console and programmatic interaction with AWS through the AWS Python SDKs and AWS CLI.
- Hands on experience with AWS services, such as: RDS, S3, Lambda, Step Functions, Glue, SQS, SNS, CloudTrail, CloudWatch, VPC, EC2 and IAM.
- Proficiency in Python: Data structures, writing custom classes/modules, object-oriented code organization, data extraction/transformation/serialization, database/API interaction, creating virtual environments, and the AWS Python SDK.
- Deep experience with troubleshooting complex end-to-end data processing issues whose causes may stem from: library code, workflow logic, API inconsistencies, network issues, corrupt data, out-of-order updates, and AI hallucinations.
- Hands on experience with high-volume data application development and version control systems such as Git.
- Hands on experience in implementing data ingestion processes incorporating ETL processes.
- Hands on experience in data modeling and relational database design of large datasets
- Knowledge of application development lifecycles, & continuous integration/deployment practices.
- 7-10 years experience delivering and operating large-scale, highly-visible distributed systems.
- Knowledge of IaC using terraform is preferred.
- Agile development experience
Desired qualifications/non-essential skills required:
- DevOps practices: IaC for pipelines, pipeline monitoring and logging, code versioning, data versioning, container writing
- Experience working with the Atlassian toolset (Jira, Confluence)
- Hands on experience with reading and writing into DynamoDB or other NoSQL databases; Redshift
- API design; API Gateway Experience
- ElasticSearch/OpenSearch Experience
- AWS Certifications
- Agile or SAFe Certificat
Skills:
How to Apply:
Apply online at https://cgi.jobs/locations/lafayette-la/jobs/
Visit Site to Apply
Location: Lafayette, LA
Date Posted: August 19, 2024
Application Deadline: September 19, 2024
Job Type: Full-time