Who We Are
We operate beyond organizational silos, combining deep expertise in data engineering and AI-driven solutions to design and deliver scalable, user-centric applications and modern data architectures. We support a wide range of industries, including Automotive, Banking, Insurance, Telecommunications, E-commerce, and Public Services.
What You’ll Be Doing
- Define evaluation strategies, golden datasets, metrics, thresholds and release gates for ML, RAG, agent, document and vision solutions.
- Build automated evaluation harnesses, regression tests, red-team and prompt-injection tests, and human-evaluation protocols.
- Define responsible-AI controls covering explainability, bias, PII, human oversight, traceability and model risk.
- Integrate evaluation into CI/CD and MLOps or LLMOps pipelines.
- Specify monitoring, alerts, model cards and audit evidence, and investigate production quality issues.
- Work hands-on with use-case teams from baseline creation to release decisions and report quality against business measures.
- Challenge weak evidence or unsafe release decisions constructively.
- Deliver reusable, tested Python code, evaluation assets and governance templates.
What you'll bring along
- At least 10 years of professional experience preferred
- 6+ years of experience in data science, ML evaluation, model validation, experimentation or responsible AI.
- 4+ years of strong Python and SQL experience with model, log and monitoring data.
- Production experience evaluating and monitoring AI or ML systems.
- Strong foundation in statistics, experimental design, error analysis and calibration.
- Experience with RAG, LLM or agent evaluation and safety testing.
- Ability to challenge weak evidence or unsafe release decisions constructively.
- Strong technical communication, stakeholder influence, ownership, mentoring and judgment under ambiguity.
- Experience with adversarial testing, PII validation, Bedrock Guardrails, CloudWatch, New Relic, Langfuse or safety-sensitive applications is preferred.
Excellent command of English, both written and spoken