Airia: Test Your HR Agent Before 60 People Use It. An Evaluation Scored 92%, and the Vacation Answer Was Still Wrong
An HR agent that answers employee questions sounds like an easy win. Sixty people stop emailing HR about vacation days and benefits, and HR gets hours back every week. The catch is that the agent is only useful if its answers are the handbook answers. One wrong vacation or benefits answer, delivered politely and confidently, creates a problem HR has to clean up. After two of those, people stop trusting the tool and go back to email. Below is the exact setup from my lesson, on Airia's free plan, testing an HR Policy Agent for a made-up firm, Northwind Partners. The scores are from that real run. This is the part that matters most, and it isn't technical. Each test case is an Input (the question) and an Expected Output (the answer HR would give). Together they're the dataset. Open Dataset and click Add test case for each one. You can also upload a CSV; Download CSV Template shows the…
