LLM Test Engineer- Intern
Upswing Cognitive Hospitality Solutions · Pune
- Company
- Upswing Cognitive Hospitality Solutions
- Type
- Internship
- Work mode
- On-site
- Location
- Pune
- Salary / stipend
- ₹10,000 a month
- Experience
- Internship
- Posted
- 19 Nov 2026
About this internship
About the Upswing
Upswing Cognitive Hospitality Solutions Pvt. Ltd. is an AI-driven technology company focused on transforming the travel and hospitality industry. We have been building intelligent AI agents and LLM-powered solutions that automate complex workflows, enhance guest experiences, and enable smarter business decisions. Our work spans Agentic AI, Generative AI, conversational intelligence, and enterprise automation, with a mission to redefine how travel and hospitality businesses operate through intelligent, autonomous systems.
About the Role
We are looking for analytical, detail-oriented interns to test and evaluate an LLM-powered solution that uses generative AI and conversational intelligence. You will examine how effectively the system understands user queries, responds accurately, maintains context, follows instructions and handles real-world scenarios. This role offers practical exposure to LLM evaluation, prompt testing and AI quality assurance.
Key Responsibilities
- LLM response evaluation: Assess responses for accuracy, relevance, completeness, consistency and clarity; flag hallucinations, reasoning errors and misleading information.
- Prompt and scenario testing: Create prompts and test cases covering common use cases, ambiguity, edge cases, negative inputs and adversarial scenarios.
- Conversational testing: Validate multi-turn dialogue, context retention, follow-up questions, changing intent and instruction-following.
- Functional and integration testing: Verify application workflows, API-driven interactions and AI outputs against business rules and available reference data.
- Defect reporting: Record reproducible issues with prompt, steps, expected and actual output, severity and supporting evidence; retest fixes.
- Evaluation documentation: Maintain test suites, prompt datasets, evaluation results and concise quality reports; collaborate with engineers to improve reliability.
Required Qualifications
- Pursuing or recently completed BE/BTech, BSc, BCA, MCA or a related degree.
- Basic knowledge of software testing principles and test case design.
- Familiarity with ChatGPT, Gemini, Claude or similar generative AI tools.
- Strong reasoning, attention to detail, critical thinking and written communication.
- Ability to probe AI behaviour systematically and document findings clearly.
Good to Have
- Basic understanding of LLMs, prompt engineering, hallucinations, RAG or AI agents.
- Exposure to manual testing, API testing, Postman, Jira or GitHub.
- Basic Python or JavaScript skills, or familiarity with evaluation and automation tools.
What You Will Learn
- Practical LLM testing, prompt evaluation and adversarial testing techniques.
- How to evaluate AI quality across accuracy, reliability, context and instruction-following.
- How AI engineering, QA and product teams investigate and improve model behaviour.
Listed from the company's public job post; the company is responsible for it. Always apply on the official page and never pay to apply. How listings work