Data engineer ssssai1339

The Customer applies technical resources to accelerate the timely, reliable, and secure delivery of open source data, information, and insights. The Customer requires developer support developing tailored solutions to tackle unique global collection problems. The work may be performed independently or within a team environment depending on the specific problem statement. Work will include developing Amazon Web Services (AWS)-based resources and the Customer needs skills spanning many compute, storage, and networking services.

Work Requirements:

• Work closely with the Customer’s technical lead to architect, deploy, and maintain multiple, fast-turnaround capabilities used to perform various highly-visible and high-priority collection efforts.
• Strategically apply AI/ML to extract, format, and expose in indexed search tools relevant content such as raw text, multimedia (audio, image, video, document), tabular (CSV, Parquet, Avro) or nested (JSON, JSONL, XML), and other structured /unstructured data types. Data is expected to be of varying formats, schemas, and structures.
• Provide Data Engineering support to include cleaning, modeling, and formatting data of unknown formats.
• Move data between different cloud storage environments for critical requests.
• Coordinate with multiple entities, including mission partners, to ensure capabilities and deliverables meet defined requirements and tradecraft needs.
• Create and maintain collection capabilities and deliverables within the Customer’s Amazon Web Services environment utilizing Customer approved AWS services.
• Validate collected data to ensure it meets data format requirements.
• Maintain all source code in Customer’s GitHub repository.
• Document all source code, including how to execute the code.
• Perform operations and maintenance on the collection capabilities and deliverables to adapt to changes in collection target, technologies, data formats, and naming conventions.

Required Skills:

1. Demonstrated experience with Python.
2. Demonstrated experience with geo-spatial software and programming packages and data formats.
3. Demonstrated experience creating and managing AWS resources, including provisioning EC2 instances, writing and deploying Lambda functions, creating and writing to S3, and managing authorization appropriately across resources with IAM policies.
4. Demonstrated experience using GitHub.

Desired Skills:

1. Demonstrated experience deploying AWS applications with AWS’s Cloud Development Kit (CDK). Ansible and Terraform are NOT a substitute for CDK.
2. Demonstrated experience building and deploying containerized applications.
3. Demonstrated experience building, programmatically working with and maintaining search engines such as ElasticSeach, Lucene, or AWS’s OpenSearch.
4. Demonstrated experience creating, programmatically working with and maintaining SQL and NO-SQL databases.
5. Demonstrated experience with other non-AWS cloud services such as, Google Cloud Platform, Microsoft Azure.
6. Certification(s) o AWS DevOps Engineer, Solutions Architect, or SysOps Administrator.