Etl modernization developer - 1645

This is a remote position.

Onsite/Hybrid/Remote: Remote

Duration: 12 months

Rate Range: $68 in W2

Work Authorization: GC and US Citizens Only

Must Have:

• IBM DataStage development
• ETL/ELT migration and modernization
• Databricks and Apache Spark on AWS
• PySpark, Python, SQL, and Shell scripting
• Delta Lake, Unity Catalog, and Photon
• AWS Glue, Lambda, and Redshift
• CI/CD and automated ETL testing
• PyTest and XML/JSON parsing

Responsibilities:

• Assess existing ETL/ELT jobs, databases, and data warehouses for migration readiness.
• Re-engineer legacy IBM DataStage jobs using Databricks Spark on AWS.
• Develop, test, and deploy scalable cloud-based data pipelines.
• Perform parity checks, unit testing, functional testing, UAT, regression testing, and performance testing.
• Validate schemas, data quality, pipeline performance, and migration accuracy.
• Build CI/CD processes for automated development, testing, and deployment.
• Support migration cutover, production deployment, and hypercare.
• Coordinate application redeployment and end-to-end testing.
• Decommission legacy DataStage jobs and related artifacts.
• Prepare technical documentation and operational readiness materials.
• Conduct knowledge-transfer and training sessions for engineering and business teams.

Qualifications:

• Strong hands-on experience developing and maintaining IBM DataStage jobs.
• Proven experience modernizing large-scale ETL/ELT and data warehouse workloads.
• Experience building Databricks and Spark pipelines in an AWS environment.
• Advanced programming skills in PySpark, Python, SQL, and Shell scripting.
• Experience with Delta Lake, Unity Catalog, and Databricks Photon.
• Experience with GitLab or Azure DevOps for CI/CD.
• Knowledge of automated testing frameworks and ETL validation methods.
• Experience supporting production cutovers, stabilization, and application teams.

Nice to Have:

• Experience with AWS Glue, Lambda, and Redshift.
• Experience with JIRA and Agile delivery.
• Knowledge of data governance and cloud security standards.
• Experience creating operational documentation and delivering technical training.