About the Role

This is a founding-level software engineering role focused on building reliable, AI-powered verification and release infrastructure. You will develop the systems that evaluate agent-based products, strengthen CI signals, and help ensure safe, predictable releases.

What You'll Do

• Build and maintain test harnesses that use agents to verify product behavior.
• Develop evaluation logic, including LLM graders and regression coverage for agentic systems.
• Create automated release gates that make readiness visible and prevent unsafe releases.
• Own CI health, release pipelines, artifact publication, and deployment automation.
• Improve the reliability of staging and demonstration environments.
• Investigate quality gaps and incidents, and improve systems to prevent recurring issues.

What We're Looking For

• Seven to fifteen years of experience in quality engineering, release engineering, or platform engineering.
• Recent hands-on experience using AI-native methods to test or evaluate LLM-based or agentic systems.
• Practical experience with LLM APIs, tool calling or MCP, and evaluation methods.
• Strong familiarity with GitHub Actions and GitHub-based CI/CD, including container builds and pipeline automation.
• Experience with Python, TypeScript, Docker, Kubernetes, and Helm.
• Working knowledge of at least one major cloud or data platform, such as AWS, Azure, Databricks, or Snowflake.
• A track record of taking independent ownership of engineering systems, ideally in a high-velocity startup or while modernizing established engineering practices.
• Authorization to work in the United States or Canada without visa sponsorship.

Compensation & Benefits

Base salary: $170,000 to $200,000 USD annually. Visa sponsorship is not available.

Location

Remote within the United States or Canada. Primary location: New York, New York, United States.