Probabilities Over Prompts

Modern interaction with large language models often ends with a plea: “please don’t hallucinate.” This workshop revisits the foundations of machine learning and shows how they remain essential in the era of generative AI. Participants will learn to work with the probabilistic structure that LLMs expose, using token-level log probabilities to build classifiers, evaluate uncertainty, and detect hallucinations. Through practical examples, we demonstrate how classical ML techniques such as calibration, fine-tuning, and systematic testing can produce reliable, verifiable outcomes. Rather than treating LLMs as black boxes to be prompted, we treat them as probabilistic learners to be measured — because when a model hallucinates, the answer is not to beg it, but to calibrate and test it.