Social scientists call this a problem with validity: the question doesn’t really measure what it’s supposed to measure. Of course, without seeing how the question was worded, we can’t know whether the chocolate milk question had validity.
Indeed, early media coverage focused on the 7 percent statistic but left out the fact that 48 percent of respondents said they don’t know where chocolate milk comes from. This gives context to the 7 percent number. While it’s conceivable that 7 percent of the population doesn’t know that chocolate milk is just milk with chocolate, the idea that a full 55 percent — over half of adults — don’t know or gave an incorrect response begins to strain credulity. This points toward a confusing survey question.