Job Title: Senior QA Engineer – Web, Mobile & Conversational AI Testing
Location: Hybrid - Bangalore
Experience Level: 4+ years
Employment Type: Full Time
About the Role:
We are looking for a highly experienced Senior QA Engineer to lead and execute quality assurance efforts across our web, mobile, and AI-driven conversational products. You will play a critical role in ensuring our applications deliver exceptional user experiences, high performance, and reliability—especially in LLM-based (Large Language Model) conversational systems.
Key Responsibilities:
- Test Planning & Strategy
- Develop and maintain comprehensive test plans and test cases for web, mobile, and conversational AI applications.
- Define QA strategy for end-to-end testing of features including functional, UI/UX, performance, and regression testing.
- Execution & Automation
- Perform manual and automated testing for mobile (iOS/Android) and web platforms. (Selenium, Pytest, Python Scripts)
- Design and maintain automated test scripts using tools like Selenium, Appium, or Playwright.
- Conduct conversational flow and response accuracy testing for LLM-based chat interfaces.
- LLM & Conversational App Testing
- Evaluate dialogue quality, prompt-response relevance, tone, and fallback behaviors in GPT-like systems.
- Test for hallucinations, safety, toxicity, and coherence in AI outputs.
- Validate integrations with RAG systems, vector stores, and API workflows.
- Cross-Functional Collaboration
- Collaborate with product managers, developers, and AI engineers to ensure feature clarity and testability.
- Provide feedback early in the development lifecycle to reduce bugs and enhance quality.
- Monitoring & Reporting
- Track and report bugs using tools like JIRA or Linear.
- Analyze and report on test results, coverage, and quality trends.
Required Qualifications:
- 4+ years of experience in QA for web and mobile applications.
- Hands-on experience with both manual and automation testing tools and frameworks.
- Strong knowledge of QA methodologies, testing types, and software development lifecycle.
- Automation testing using Pytests.
- Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs).
- Proficient in writing SQL queries and working with Postman for API testing.
- Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus.
Preferred Skills:
- Experience in testing AI-driven personalization, intent recognition, or multi-turn conversation logic.
- Exposure to prompt engineering and evaluation frameworks for LLM outputs.
- Familiarity with CI/CD pipelines and version control systems like Git.
- Ability to build QA dashboards and automated quality reports.