Associate Lead - Testing (QA + MLOps)
Quantiphi
Date: 2 weeks ago
City: Bengaluru, Karnataka
Contract type: Full time
While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Role: Lead/Associate Lead – QA + MLOps & Generative AI
Experience: 10+ years
Location: Mumbai/Bangalore (Hybrid)
Key Responsibilities
AI/ML & GenAI Testing Strategy (AWS Ecosystem)
Define testing approaches for AI systems built on AWS services such as:
Validate The End-to-end ML Lifecycle Including
GenAI & Agentic AI Testing
Define Quality Engineering Approaches For
Testing Expertise
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Role: Lead/Associate Lead – QA + MLOps & Generative AI
Experience: 10+ years
Location: Mumbai/Bangalore (Hybrid)
Key Responsibilities
AI/ML & GenAI Testing Strategy (AWS Ecosystem)
Define testing approaches for AI systems built on AWS services such as:
- Amazon SageMaker
- Amazon Bedrock
- AWS Lambda
- Amazon API Gateway
- Amazon Kinesis
- AWS Glue
- Amazon S3
- Amazon CloudWatch
- Model accuracy & performance validation
- Data drift & concept drift detection
- Hallucination detection for LLMs
- Prompt robustness testing
- RAG validation (retrieval accuracy + grounding)
- Bias & fairness validation
- Safety & toxicity testing
Validate The End-to-end ML Lifecycle Including
- Data ingestion & feature pipelines
- Model training & hyperparameter tuning
- Model versioning & registry
- Deployment validation
- Canary & blue/green release validation
- SageMaker Pipelines
- SageMaker Model Monitor
- SageMaker Feature Store
- Bedrock model evaluation workflows
- CloudWatch-based observability
GenAI & Agentic AI Testing
Define Quality Engineering Approaches For
- LLM-based applications using Amazon Bedrock
- Prompt engineering validation
- Multi-agent orchestration testing
- Chatbot & Voice bot conversational testing
- Intent classification validation
- Conversation drift & fallback validation
- API contract validation for LLM integrations
- BLEU / ROUGE scoring
- Embedding similarity scoring
- Response consistency
- Safety scoring frameworks
- Design reusable AI testing accelerators
- Create AWS-aligned AI test automation frameworks (Python-first)
- Develop synthetic data generation strategies
- Establish AI quality scorecards
- Build an internal AI QA Center of Excellence
- Lead AI/ML quality strategy workshops
- Perform AI risk & readiness assessments
- Present quality architecture to CXOs
- Drive QA transformation programs
- Mentor QA teams on AWS-based AI testing
- Own delivery for AI testing engagements end-to-end
Testing Expertise
- 8–12+ years in Quality Engineering
- Strong test strategy, automation & governance experience
- Experience leading QA transformation initiatives
- Experience building frameworks from scratch AI/ML & GenAI Expertise
- Deep understanding of ML lifecycle
- Experience testing ML models (NLP preferred)
- Hands-on experience validating LLM applications
- Strong understanding of:
- Prompt engineering
- RAG architecture
- Embeddings
- Bias & explainability AWS AI/ML Expertise
- Hands-on experience with:
- Amazon SageMaker (training, deployment, monitoring)
- Amazon Bedrock (LLM integration & evaluation)
- S3-based data pipelines
- AWS IAM (security validation)
- CloudWatch monitoring
- Lambda & API Gateway integrations
- AWS CI/CD (CodePipeline / CodeBuild preferred)
- Infrastructure as Code (Terraform / CloudFormation)
- Observability in AI systems
- Cost monitoring for ML workloads
- Python (mandatory)
- Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch)
- Experience with LLM frameworks (LangChain, etc.)
- API & automation testing frameworks
- Git-based workflows
- Leadership & Communication
- Strong client-facing communication
- Experience leading QA teams
- Ability to create strategy decks & solution proposals
- Strong stakeholder management
How to apply
To apply for this job you need to authorize on our website. If you don't have an account yet, please register.
Post a resumeSimilar jobs
Manager - Customer Success
Simplotel,
Bengaluru, Karnataka
5 days ago
As a Manager/Director, you will lead Simplotel's Customer Success organisation, owning customer retention, expansion, and long-term customer value. This role is responsible for building a high-performing team that partners closely with customers to drive product adoption, maximise business outcomes, and deliver an exceptional customer experience. Success in this role will directly contribute to customer growth, product adoption, and Simplotel's long-term...
Executive Assistant Business Partner 1 - Specialist
Intuit,
Bengaluru, Karnataka
1 week ago
OverviewIntuit is a mission-driven, global financial platform company that gives everyone the opportunity to prosper. We use technology to solve our customers' most important problems. We serve customers by finding more ways to put more money in their pockets, eliminating work so they can focus on their lives and what matters to them, and ensuring that every financial decision that...
Developer
IDFC FIRST Bank,
Bengaluru, Karnataka
1 week ago
Job RequirementsRole/Job Title: Senior Developer/ DeveloperFunction/ Department: Information TechnologyJob Purpose:As a Backend Developer, you will play a crucial role in designing, developing, and maintaining complex backend systems. You will work closely with cross-functional teams to deliver high-quality software solutions and drive the technical direction of our projects. Your experience and expertise will be vital in ensuring the performance, scalability, and...
