Senior Prompt Engineer - Data Science & Quality Analysis (India)

Checkmate is building advanced Voice AI systems for some of the largest restaurant and retail brands in the U.S. Join us at this pivotal moment of growth to shape AI products used daily by thousands of staff and customers, combining cutting-edge LLM innovation with real business impact.

Requirements

• Design, test, and optimize LLM prompts for conversational AI, text classification, and structured data extraction tasks.
• Build evaluation pipelines to analyze prompt performance using quantitative metrics, human-in-the-loop feedback, and business KPIs.
• Conduct prompt experiments and regression testing to ensure stability, accuracy, and safety as models evolve.
• Collaborate with Machine Learning, Product, and Operations teams to translate business objectives into scalable, data-driven prompt-engineering strategies that enhance model accuracy, efficiency, and real-world usability.
• Use Python/SQL to analyze model outputs, identify anomalies, and automate quality checks.
• Document best practices and contribute to internal frameworks for prompt evaluation and continuous improvement.
• Communicate findings effectively to technical and non-technical stakeholders, driving measurable business impact through insight-driven decisions.
• Proven experience in prompt engineering and working with LLMs (GPT-4, Claude, Gemini, and LLaMA) for text generation, reasoning, and structured data extraction.