1561. Bootstrap Confidence Intervals for LLM Evaluation (engineering.indeedblog.com)
Introduction As Large Language Models (LLMs) move from research prototypes to production systems, the developers of these systems need rigorous performance evaluation. In particular, we need confidence intervals around estimates of system accuracy. However, LLMs introduce a challenge that is unusual...