Python / PySpark Developer

Summary

We are seeking a highly skilled Python and PySpark Developer to join our IT Services team. In this pivotal role, you will design, develop, and optimize scalable data processing solutions leveraging the power of Python and the PySpark framework. This position is critical for driving our data engineering initiatives, ensuring efficient handling of large datasets, and supporting robust application architectures within our cloud environment. The ideal candidate will possess deep expertise in distributed computing and API integration to deliver high-performance software that meets complex business requirements.

Responsibilities

• Architect and implement robust data pipelines using Python and PySpark to process and transform large-scale datasets efficiently.
• Design, develop, and maintain secure and scalable RESTful APIs to facilitate seamless data exchange between systems.
• Deploy, monitor, and manage cloud infrastructure and applications on AWS, ensuring high availability and performance.
• Collaborate with cross-functional teams to define technical specifications and translate business needs into functional software solutions.
• Optimize code for maximum speed, scalability, and reliability while adhering to best coding practices and security standards.
• Troubleshoot and resolve complex technical issues related to data processing, API connectivity, and cloud deployments.

Requirements

Minimum of 6 to 9 years of professional experience in software development, with a strong focus on data engineering and backend systems. Proven expertise in Python programming, including mastery of its libraries and frameworks for data manipulation. Extensive hands-on experience with Apache PySpark for building distributed data processing applications. Solid understanding and practical application of AWS cloud services for deployment and infrastructure management. Demonstrated proficiency in designing and consuming REST APIs within enterprise-level applications. Ability to work effectively in a dynamic IT Services environment, delivering high-quality code under tight deadlines.