Work Location: Coimbatore
Work Mode: Hybrid/Remote
Notice Period: 0 to 30 days
Job Description:
Experience:
(4+) years’ experience with QA Automation (1+ year with Gen AI Testing)
Qualifications:
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or related field.
- Proficiency in programming languages such as Python (preferred) or Java.
- Strong understanding of testing principles and methodologies.
Job Description:
We are seeking a skilled QA Gen AI Automation Engineer to join our team. The ideal candidate will have strong hands-on experience in Python, Generative AI, and automated testing.
In this role, you will test and validate AI-driven solutions, automate regression tests, and ensure the performance and functionality of applications in an Agile environment.
Key Responsibilities:
Automation Testing:
- Develop and maintain automated regression tests using tools like Selenium WebDriver/Playwright and pytest.
- Conduct API testing using tools such as Postman.
Generative AI Testing:
- Design and execute test cases for Generative AI models and applications.
- Evaluate the performance and quality of Generative AI outputs, ensuring they meet functional requirements and quality standards.
- Validate AI systems such as Chatbots, RAG applications, and LLM-based workflows.
- Perform LLM evaluation using frameworks such as DeepEval and RAGAS.
Programming & Scripting:
- Write and maintain automated test scripts in Python.
- Work with XML, JSON, and other scripting languages to create efficient testing environments for AI systems.
Testing Tools & Frameworks:
- Utilize JMeter for performance testing and load testing.
- Implement and manage Continuous Integration (CI) pipelines using tools such as Jenkins and Bamboo.
Database & SQL:
- Hands on experience with Panda Data frames
- Execute SQL queries for database testing (inserts, updates, joins, etc.).
- Test non-GUI applications (e.g., SQL, flat files, XML).
- Work with Vector Databases such as OpenSearch or similar vector DB technologies.
Agile & DevOps:
- Collaborate within an Agile (Scrum) framework for efficient testing and development.
- Utilize Atlassian JIRA and Confluence for project tracking and documentation.
Product Evaluation & Troubleshooting:
- Identify and troubleshoot product issues, ensuring quality and stability.
- Evaluate product performance and provide feedback to development teams.
Web Application Testing:
- Conduct testing for web applications and web services (REST, SOAP).
Required Skills & Experience:
Programming:
- Proficient in Python for writing test scripts and automating processes.
Automation Testing:
- Experience with Selenium WebDriver/Playwright and pytest for automation testing.
- Hands-on experience in API testing using tools like Postman.
Generative AI & AI Testing:
- Understanding of AI systems and AI Testing concepts.
- Knowledge of Large Language Models (LLMs), RAG, and Chatbot AI systems.
- Experience designing and testing Generative AI systems and applications.
- Knowledge of LLM evaluation techniques and frameworks such as DeepEval and RAGAS.
- Ability to assess the output quality, coherence, and accuracy of Generative AI models.
Database & Search Technologies:
- Strong understanding of SQL for testing databases (including joins, inserts, and updates).
- Understanding of Vector Databases such as OpenSearch or similar vector DB technologies.
Testing Tools:
- Experience with JMeter for performance and load testing.
- Familiarity with CI/CD tools like Jenkins or Bamboo.
Agile Methodology:
- Experience working in an Agile (Scrum) environment.
- Familiar with JIRA and Confluence for issue tracking and project management.
Web & API Testing:
- Experience testing web applications and web services.
Product Performance:
- Capable of evaluating and troubleshooting performance-related issues.
Nice to Have:
Agentic AI & Observability:
- Understanding of Agents, MCP, and Agentic AI systems.
- Familiarity with observability tools such as Langfuse.
Cloud Experience:
- Familiarity with AWS services such as AWS Bedrock for cloud-based AI environments.
Development Experience:
- A background in software development is a plus.
Industry Knowledge:
- Understanding of the Property and Casualty (P&C) insurance industry is beneficial.
