"The first principle of republicanism is that the lex majoris partis is the fundamental law of every society of individuals of equal rights."

— Thomas Jefferson

"[Bear] always in mind that a nation ceases to be republican only when the will of the majority ceases to be the law."

— Thomas Jefferson

This corporeal globe, and everything upon it, belong to its present corporeal inhabitants during their generation. They alone have a right to direct what is the concern of themselves alone, and to declare the law of that direction; and this declaration can only be made by their majority. That majority, then, has a right to depute representatives to a convention, and to make the constitution what they think will be the best for themselves."

— Thomas Jefferson

The "soup principle"

"Straw" polls

The first ("scientific") polls

- Conducted face-to-face

- Used demographic quotas for representativeness

• Race, gender, age, geography

- Beat straw polls in accuracy (1936)

• By shrinking bias from demographic nonresponse

The first ("scientific") polls

- But fell short of true survey science (1948)

Polls 2.0

- SSRC says: area sampling

Polls 2.0

- SSRC says: area sampling

- Gallup implements some partisan controls

• Strata are groups of precincts by 1948 vote choice

- Use rough quotas within geography

- But, preserve interviewer bias

Polls 3.0

Technological change -> better methods

Polls 3.0

- 1970s: true random sampling (for people with phones)

- Response rates above 70-80%

- Rarer instances of severe nonresponse bias

- Cheaper to conduct = many news orgs poll (CBS, NYT)

Source: American Association of Public Opinion Research

The soup principle: satisfied?

Source: Pew Research Center

The soup principle: satisfied?

1. RDD polls are representative (at high response)

2. Availability of many different surveys allow for extra layer of aggregation to control for choices made by individual researcheers

Technological change -> worse methods?

Source: Pew Research Center

Polarized voting -> harder sampling

Source: Webster & Abramowitz 2017

- People could be very dissimilar by group, meaning small deviations in sample demographics cause big errors (sampling error)

- Or the people who respond to the poll could be systematically different from the people who don't (response error)

- Or your list of potential respondents could be missing people (coverage error)

*Polls can also go wrong if they have bad question wording, a fourth type of survey error called "measurement error"

The soup principle in theory

Source: Pew Research Center

The soup principle in practice

So, to satisfy the soup principle...

Pollsters use statistical algorithms to ensure their samples match the population on different demographic targets

• Race, age, gender, and region are most common

• Can use weighting (raking) modeling (MRP), w various tradeoffs

These adjustments make polls pretty good!

2016: Education weighting

2020: Partisan nonresponse

2020: Partisan nonresponse

• Problem reaching Trump voters overall

• And within demographic groups

• Something you cannot fix with weighting

  • Pollsters can adjust for past vote, but the electorate changes, and certain types of voters may not respond to surveys

So what are we left with?

1. Traditional polls that oscillate wildly due to intensive weighting

2. New "model-based" methods which trade lower variance for higher (potential) bias

3. Lower response rates increase chance of big misses across firms

Making polls work again

1. More weighting variables (NYT)

2. More online and off-phone data colleciton (SMS, mail)

3. Mixed samples (private pollsters)

In the pursuit of getting representative (and politically balanced) samples before and after the adjustment stage

What goes into the model?

1. National economic + political fundamentals

2. Decompose into state-level priors

3. Add the (average of) polls

2. National fundamentals?

i) Index of economic growth (1940 - 2016)

• eight different variables, scaled to measure the standard-deviation from average annual growth

ii) Presidential approval (1948 - 2016)

iii) Polarization (1948 - 2016)

• measured as the share of swing voters in the electorate, per the ANES --- and interacted with economic growth

iv) Whether an incumbent is on the ballot

2. The model is a federalist

i) Train a model to predict the Democratic share of the vote in a state relative to the national vote, 1948-2016

• Variables are: lean in the last election, lean two elections ago, home state effects * state size, conditional on the national vote in the state

ii) Use the covariates to make predictions for 2020, conditional on the national fundamentals prediction for every day

ii) Simulate state-level outcomes to extract a mean and standard deviation

• Propogates uncertainty both from the LOOCV RMSE of the national model and the state-level model

3. Add the (average of) polls

• Just a trend through points...
• Can do with any series of packages for R, other statistical languages

3. Add the (average of) polls

(...but with some fancy extra stuff)

mu_b[:,T] = cholesky_ss_cov_mu_b_T * raw_mu_b_T + mu_b_prior;  
for (i in 1:(T-1)) mu_b[:, T - i] = cholesky_ss_cov_mu_b_walk * raw_mu_b[:, T - i] + mu_b[:, T + 1 - i];
national_mu_b_average = transpose(mu_b) * state_weights;
mu_c = raw_mu_c * sigma_c;
mu_m = raw_mu_m * sigma_m;
mu_pop = raw_mu_pop * sigma_pop;
e_bias[1] = raw_e_bias[1] * sigma_e_bias;
sigma_rho = sqrt(1-square(rho_e_bias)) * sigma_e_bias;
for (t in 2:T) e_bias[t] = mu_e_bias + rho_e_bias * (e_bias[t - 1] - mu_e_bias) + raw_e_bias[t] * sigma_rho;
//*** fill pi_democrat
for (i in 1:N_state_polls){
  logit_pi_democrat_state[i] = 
    mu_b[state[i], day_state[i]] + 
    mu_c[poll_state[i]] + 
    mu_m[poll_mode_state[i]] + 
    mu_pop[poll_pop_state[i]] + 
    unadjusted_state[i] * e_bias[day_state[i]] +
    raw_measure_noise_state[i] * sigma_measure_noise_state + 
    polling_bias[state[i]];
}

3. Add the (average of) polls

i. Latent state-level vote shares evolve as a random walk over time

• "Walks" toward the state-level fundamentals more as we are further out from election day

ii. Polls are observations with measurement error that are debiased on the basis of:

• Pollster firm (so-called "house effects")
• Poll mode
• Poll population
• Bias in previous elections

iii. Correcting for partisan non-response

• Whether a pollster weights by party registration or past vote
• Adjusts for biases that remain AFTER removing the other biases

3. Add the (average of) polls

Notable improvements from partisan non-response (and other?) issues

One more lesson:

1. Traditional polls that oscillate wildly due to intensive weighting

2. New "model-based" methods which trade lower variance for higher (potential) bias

3. Lower response rates increase chance of big misses across firms

4. Aggregation is not a magic bullet

4. Aggregation is not a magic bullet

.....But luck is

The polls in 2022

Polling: fixed?

1. Death of polling is greatly exaggerated

2. Luck plays a big role in getting elections "right"

3. High nonresponse is not necessarily directional

Thank you!

Website: gelliottmorris.com

Twitter: @gelliottmorris

Questions?