Interval-Based Decision in an Expected Utility Framework, and Epistemic Humility
Ronald Loui
Ronald Loui
Ronald Loui
Published May 28, 2026
+ Follow

You say you want to use probability intervals. Like a confidence inteval at .9. For epistemic honesty, the probability is in the interval [.5. .8]. But how do you use this if you are using expected utility to make decisions?

Obviously it's a built-in robustness study. If the threshold between A and not-A, or between A and B, sits mid-interval, like .75, you have a problem when the interval turns up [.5, .8]. Don't be too confident. Even as a gamble, maybe abstain. Do an expected value of information calculation if you have the probabilities (probably you don't but it's not a bad way of looking at things).

Problem is, at high confidence like .999, your intervals start to look like [0, 1] or [e, 1-d]. So everything looks in-between. This was the problem Kyburg gave me around 1984. So my first publication was on this. I put Henry's and Jerry Feldman's names on the UNCAI 1 paper without their review. Thought i was being respectful and considerate, but Jerry wasn't too pleased. Still, he had put in the time, and knew the idea was sound, so it was all good. We forget that 1st year doctoral students are still very young.

Hurwicz, who later won a Nobel in Econ, was famous for his optimism-pessimism "criterion." In those days, you didn't have much to work with. He suggested an alpha/1-alpha split between upper and lower bound. I learned that in Decision and Control (Engineering Science 119) from Professor Fiering as an undergrad. Elegant, but wrong.

Because [0, 1] always starts to look like .5. Ah, .5, the biggest lie in probability. Does it mean you KNOW that p and not-p are equiprobable? Or does it mean "anyone's guess"? If you are a Bayesian with indifference prior, what's the difference? This should outrage everyone who stops and thinks about it. From complete ignorance you are manufacturing complete precision at 0.500000...? Ridiculous.

So this is where one starts to get serious about epistemology and foundations of probability. Bayesian "i always have a prior to mix things" is extremely convenient. So are coin flips. When you can be principled, you should be.

Jerry Feldman's outrage at max entropy (now they call it "objective Bayesian") methods started me out here. He puzzled: you mean to tell me that if i learn the expected value of a roll of a 6-sided die is 4.5, you can calculate the probability of each face to as many decimal digits as you want? Yes, that's what Jaynes and Jeffreys say. It works in physics, as they say. Physics has a lot of unspoken symmetry.

Meanwhile, Judea Pearl had a lovely saying (a few years later). You sometimes want to be bold. Sometimes want to be cautious. Isaac Levi may have said similar in GAMBLING WITH THE TRUTH. Depending on what's at risk, sure. You think all the risk is encoded in the tree of possibilities weighted by utilities. That would be an unfortunate thought.

So the paper that appeared in UNCAI (first conference) and THEORY AND DECISION (first journal for doctoral work) said something simple: start at a high level of confidence (or acceptance); start lowering that level; as the interval narrows, it might give you a decision wrt that threshold you have in mind.

You know that interval doesn't narrow symmetrically as the confidence level changes. Why would you take a midpoint? Or a fixed ratio mix of upper and lower bound? Makes no sense. (Sorry Prof Hurwicz; I still respect your Nobel.)

If you haven't got the epistemic structure, maybe a mix is all you can do. But that's not the first option. Anyone who has intervals can generate narrower intervals. Problem is that depending on your method of covering the density, different methods narrow the intervals differently. Not in a major way, but theoretically you could always anchor your interval at the lower end, or always at the upper end. I unfortunately have no solution there. I say you choose your method (Wilson, Clopper-Pearson, Agresti-Coull), and you stick with that. You want to generate weird, custom interval covers of support whose only virtue is that the distribution over that interval adds to .9, that's your problem.

Makes sense. You even get a degenerate interval around the MLE if you do things right, so you are always decisive at a sufficiently low confidence level. You could start there and bypass the process of lowering confidence level. Ah, but in some theories, we are talking about acceptance, not confidence, and at an acceptance level of .8 you start to accept things as true that you would not at .9. Then the interval narrows not only asymmetrically, but also nonmonotonically. The lower acceptance level interval is not always contained within the higher acceptance level interval. The MLE moves around. Interesting, yes?

There are more things in heaven and earth, Bayesian, than are dreamt of in your philosophy!

Now some have asked a good question about intervals as representations of imprecision. They're better than MLE +/- delta, because they might not be symmetric around that point. OK, but why the hey do we try to represent uncertain precision about one point with two precise points? Great question. It was asked in LA at UNCAI 1. Fair question. Deep question.

The insinuation is that one should use a meta-probability distribution, like a Bayesian would. A Bayesian would hop to it. Here is my meta-prior, sir!

But that's precision at an infinite number of points! So use a band, not a function: you can use an upper and lower function, an interval for y at any x. Now you have an infinite number of intervals defined by two points! You see the problem. Wonderful philosophical problem about how best to represent uncertainty.

I do not have a killer answer here. What I tell people is that if your robustness study turns entirely on the precision of the lower bound, or the upper bound, you need to do more perturbation analysis. You need insightful process, not blind averages. Otherwise, your interval method is good enough.

What you do not want to do is manufacture knowledge or precision where you do not have any. Then pretend you do. You can always represent imprecision using ranges, or distributions, so long as the analysis does not depend on the precision of those ranges or exactness of distributions. We go meta because meta allows imprecision at meta; if we needed precision at meta, go meta-meta. My Harvard pal who died too soon, Eric Wefald, would love that answer.

What you get from an interval is a chance to pause from immediate -- you should also hear "automated" -- decision, and instead engage a knowledge-augmenting process. It's a short-circuit of the shortcut. Maybe you just train a neural net to spit out a decision AND a meta-decision whether to trust the decision. Maybe you can convert output strengths into probabilities (good luck). Maybe you build multi-class classifiers: yes, no, maybe.

However you do it, leave some room for epistemic humility. Judea would say sometimes be bold, sometimes be cautious. Hank would say intervals are your friends. Good advice, both, in an era of overly-confident AI.