Harvard Poll of Millennials Reminds Me of Why I Hate Approve/Disapprove Questions and False Dilemmas

The Harvard Institute of Politics's 24th edition of its "Survey of Young Americans’ Attitude Toward Politics and Public Service" came out earlier this week.

What stood out to me most when reading through the poll results was how the "approve/disapprove" question format produces false binaries and meaningless results unless used for specific (and specified) policy measures.

Let's look at questions 18 to 22 in particular. Each question began with the following lead:

Now thinking specifically about Barack Obama, do you approve or disapprove of the
way he is handling

18. Syria.
Approve .......................................................... 33%
Disapprove ..................................................... 60%
Decline to answer ........................................... 7%

When the pollsters are asking about "how Obama is handling Syria," what exactly do they mean?  Do they mean the manipulation of intelligence, the incessant warmongering in late August/early September, or the covert support for the Syrian rebels?  Or do they mean the deal worked out between the U.S., Russia, and Syria on the destruction of chemical weapons?  And what about the refugee crisis? Would an answer on this question register an opinion on that?

19. The economy.
Approve .......................................................... 33%
Disapprove ..................................................... 61%
Decline to answer ........................................... 6%

Polling questions about "the economy" are a big pet peeve of mine because they are too vague. You can ask people about unemployment; poverty and inequality; job security and income security; availability, affordability, and access to goods; etc. Polling "the economy" can only register a general sense of malaise. It cannot tell us why people disapprove of how the president is handling the economy, or what aspect of the economy.

20. Iran.
Approve .......................................................... 37%
Disapprove ..................................................... 56%
Decline to answer ........................................... 7%

By "Obama's handling of Iran," do the pollsters mean the crippling sanctions imposed on Iran that are causing a humanitarian crisis? Or do they mean the diplomatic negotiations that were occurring during the time of the poll? Does disapproval convey hawkishness or a desire for a more humane, egalitarian foreign policy?

22. Federal budget deficit.
Approve .......................................................... 28%
Disapprove ..................................................... 66%
Decline to answer ........................................... 6%

What does disapproval tell us? That young people think that Obama should be cutting the deficit more? That young people disapprove of the president's deficit-cutting fetish, embrace of austerity, and prioritization of deficit reduction over restoring employment? Do questions about the "deficit" even mean anything in the first place outside of context of specific policy measures? Thankfully, the poll does ask respondents about specific measures to reduce the deficit in a later section. However, the section manifests the bias inherent in accepting deficit reduction as a normative measure.

Question #21 is similar to Question #23, so let's look at them together:

21. Health care.
Approve .......................................................... 34%
Disapprove ..................................................... 61%
Decline to answer ........................................... 5%

23. Do you approve or disapprove of [The Affordable Care Act/Obamacare], the comprehensive health reform package that President Obama signed into law in 2010?
A. The Affordable Care Act.
(n=1,042)
Approve .......................................................... 39%
Disapprove ..................................................... 56%
Refused .......................................................... 5%
B. Obamacare
(n=1,047)
Approve .......................................................... 38%
Disapprove ..................................................... 57%
Refused .......................................................... 5%

ACA polling that does not disaggregate the disapproval is disingenuous. For example, a CNN/ORC poll from May found that 51% opposed the Affordable Care Act and 43% supported it. However, that 51% disapproval consisted of 35% disapproval from the right ("too liberal") and 16% disapproval from the left ("not liberal enough"). 

 

And, with the flaws of the neoliberal parts of the Affordable Care Act, Democrats are increasingly favoring changing the law to be more progressive and more inclusive. An "approve/disapprove" binary cannot account for that and produces misleading results.

As I noted earlier, the poll, rather than just asking about the "deficit," asked about specific policy measures to reduce the deficit:

As the president and Congress work to reduce the national deficit they will be faced with a series of tradeoffs. The two options in each pair would reduce the budget deficit by similar amounts, either by reducing spending or increasing revenue. For each pair, please select the option that you most prefer.

First of all, this accepts the premise that the president and Congress should be working to reduce the budget deficit, and that is, by all means, a normative claim. However, there is another major flaw here: the poll only tests relative preferences, not absolute preferences. The poll divided respondents into two groups (A and B) and presented each group with six different measures for reducing the deficit (all of which would do so by roughly equal amounts). The results, then, can only tell us how millennials prefer an option given the slate of other options provided. Deficit reduction, if we accept it, need not proceed on a set of such false dilemmas. 

 

Group A saw the following options, which I listed in order of relative preference.
(1)    Enact the “Buffet Rule,” a requirement that people making over $1 million a year pay at least 30% of their income in taxes (69% - 24%)
(2)    Reduce food stamp levels to 2008 levels and limit growth in spending on food stamps to the rate of inflation (58% - 36%)
(3)    Reduce U.S. Navy fleet to 230 ships (from a projected 320 ships) (51% - 41%)
(4)    Raise the retirement age for Social Security from 65 to 68 (41% - 52%)
(5)    Increase the gas tax by 15 cents per gallon (32% - 61%)
(6)    Raise Medicare premiums to 35% of costs (28% - 64%)

Group B saw the following options, which I listed in order of relative preference.

(1)    Cut foreign economic aid in half (71% - 22%)
(2)    Reduce spending related to the nuclear arsenal by reducing U.S. nuclear warheads from approximately 2,000 to approximately 1,550 (70% - 23%)
(3)    Increase the gas tax by 6 cents per gallon (44% - 49%)
(4)    Significantly reduce the Earned Income Tax Credit, and offset to payroll taxes for low-income workers with children, and the Child Tax Credit (38% - 54%)
(5)    Reduce social security benefits, except for workers who earn below the 30th percentile of Earnings (33% - 60%)
(6)    Cut federal K-12 funding by 25% (22% - 71%)

These results are moderately interesting, but they don't show us absolute preference--and that matters just as much, if not much more.