How Good is Good Enough? A Statistics Lesson Gets Me Out of Jury Duty

Last week I had the privilege, duty, or burden (depending on perspective and time of day) of being part of the jury selection process for a trial in Snohomish County, WA. I had avoided jury duty in the past, feeling that I couldn’t leave work. This time, I saw it as a privilege and a duty in which I should participate. On my first day, I discovered the trial for which I had been summoned was scheduled to last four to six weeks—a potential issue given everything I had on my schedule. I cleared my plate to begin the process.

The Charges and Questioning

All potential jurors filled out a written questionnaire—a straightforward process—and were sent home. I returned the next day and learned a bit more about the charges. There were two—the first being murder in the second degree (very serious) and the second being unlawful possession of a firearm (not as serious). We then began voir dire—the process by which the defense and prosecution interview the jury pool to identify which persons could be impartial, unbiased, or simply someone they want on the jury. Interestingly, this was done as a group—i.e., the State and the defense asked questions of the entire jury pool (about 66 people) and then selected individuals to respond.

When it was my turn, I was questioned by the defendant’s attorney regarding the two charges. Specifically, the defense wanted to know if (1) I would separately judge the charges (i.e., could someone be guilty of unlawful possession of a firearm but not guilty of the murder in which the firearm was used) and (2) whether there would be a difference in the burden of proof required for the different charges.

After thinking about this, I responded to the first question by saying that I could and would separate the two charges. On the first charge (illegal possession of a firearm), I said I would expect that the State would need to show, beyond a reasonable doubt, that the defendant had possession of a firearm and that he was, under the law, not allowed to possess that firearm. On the second charge, I indicated that I would expect the State to demonstrate that the defendant committed the murder, regardless of whether they used the firearm in question or something else entirely.

Next, we got to the burden of proof question. This one was harder. But after some consideration, I thought about what is commonly called the confidence level (I am a statistician and researcher, after all). As researchers, we always hear about the margin of error (or confidence interval) being plus or minus some percentage point. The margin of error is almost always calculated using a 95 percent confidence level. So, my answer to the second question was also a yes—that is, I would expect that there would be a difference in the burden of proof required for the two different charges.

Thinking About Confidence Levels

I thought I was done. But then the prosecution (State) chimed in with a follow-up, asking me what I meant by my response. Still thinking about confidence levels, I told him that the burden of proof required would depend on the risks (or costs) of making the wrong decision based on the amount of information available. And that there are two risks or costs if we come to the wrong decision – judging a guilty person to be not guilty (In statistics, we call this a Type 1 error.) and judging an innocent person to be guilty (Type 2 error).

Looking at the one charge—illegal possession of a firearm—I said that regardless of my personal feelings about gun ownership, the consequences of simply having possession of a firearm, legal or not, were not as severe as other potentially illegal acts or felonies. It also seemed that it would be easier to demonstrate whether the person could legally own a firearm. So, for that charge, I suggested that the burden of proof might be lower. i.e., I only needed to be 90 or 95 percent confident that the State had shown beyond a reasonable doubt that the person was guilty of illegally possessing a firearm, as the risk of making the wrong decision was lower.

For the more serious charge—murder—I felt the burden of proof should be higher—i.e., I needed to be at least 95 percent confident, even 99 percent confident—that the State had demonstrated that the defendant had indeed committed the murder. I set the bar higher because the risks of making the wrong decision were significantly higher—longer prison term for the defendant, more significant impacts on the defendant’s family, the chance that someone else had committed the murder and might get away with it, etc.

After causing most of those listening to me to glaze over, I was excused from the jury pool—I think because the prosecutor was concerned that I would overthink things (likely true) and possibly be expecting too much from the State. While somewhat saddened (who gets to be part of a murder trial), I was relieved because I wasn’t looking forward to a four-to-six-week trial.

Applying This Experience

I am sharing this story because it applies to what we, as researchers, do in our daily lives. All too often, I see people expending resources and dollars to achieve lower margins of error, always sticking with the 95 percent confidence level. But there are two components to deciding how large a sample is large enough—what margin of error can you accept, and what confidence level is enough.

It is possible that for lower-risk decisions—i.e., the cost of making the wrong choice based on the information is lower—you could get the same margin of error for a lower cost if you are willing to be just a bit less confident that the actual value of the measure of interest will fall within the corresponding confidence interval. To get a better sense of this trade-off, consider the table below:

As this table shows, it takes significantly larger sample sizes to decrease the margin of error at any confidence level. However, you can always decrease your sample size (and study costs) by accepting a lower confidence level.

What does this all mean for you? Rather than immediately jumping to the common decision about sample size, always remember that:

You can be more precise if you’re willing to accept slightly more risk that you’re wrong, and
To be more confident that you’re right, you’re often forced to accept a lower level of precision or pay substantially more.

I hope we’ve cleared that up!

Engage With Us

If you would like to engage in this conversation and learn more about how ComEngage thinks about statistics and how to tailor your research to be more effective and more efficient, don’t hesitate to contact me at Rebecca@ComEngage.us.