Glossary of Polling Terms

Home > Poll & Survey FAQS > Glossary of Polling Terms


This glossary was developed by AAPOR as part of a comprehensive online journalism polling course created in partnership with NewsU, a project of the Poynter Institute and funded by the Knight Foundation. The course launched  September 2007.


Approval Ratings
Closed-end poll questions used to measure the extent to which the public approves of the manner in which the president (or some other elected official) is handling current domestic and foreign policy issues facing the nation. The questions employ responses that often range from "strongly approve" to "strongly disapprove." These measures have been taken several times each year for a half a century and often are used to contrast approval levels with a current elected official with previous holders of the same office.
Balanced question
A closed-end poll question with a question stem that poses both sides of an issue and/or one that uses a range of responses that has a true midpoint.
Biased question
A poll question whose wording consistently causes respondents to answer in a way that distorts their "true" opinions or preferences , thus leading to an inaccurate measurement of the topic of interest.
CASRO
Council of American Survey Research Organizations.An umbrella organization of professional groups and firms that engage in polling and other forms of survey research.
Census
Gathering data from all elements in a population, rather than just from a sample of elements as in a survey or poll. A census is sometimes called an enumeration.
Closed-end question
A poll question that provides all respondents with a predetermined set of response alternatives from which to choose, rather than allow them to answer in their own words.
Confidence Level
A statement of the likelihood that a relationship observed in a sample is not merely due to chance. This likelihood is traditionally expressed as the number of times in 100 samples that this relationship could occur, usually as 95/100 or 99/100 times.
Context Effect
The potential consequence of asking a question within the "context" of other questions such that the effect is to have the answers biased by context.
Contingency Question
A question whose answer determines which question is asked next.
Convenience Sample
A nonprobability sampling design whereby people are selected for interviews because they are easily available. As a result there is no known probability for each respondent.
Double-barreled question
Any item that includes two separate concepts that should properly be asked as two different items.
Element
Members of a population or sample. A sample consists of the selected elements of the population.
Field Period
The time that elapses between the start and stop of data collection.
Intercept Polls/Samples
Face-to-face interviewing that stops respondents in public areas such as shopping malls, downtown street corners, airports or the like and asked them to complete a poll or other research task. Used mostly for market research, intercept surveys use nonprobability samples and suffer from not knowing with any confidence what target population is represented by the sample.
Internet polls
These are currently unscientific polls that use volunteer samples because there are no good frames available that list who owns a computer and has access to the Internet. Therefore, it is incorrect to generalize the results from these polls to all adults, all consumers, or all voters.
Margin of Error
More properly termed margin of "sampling" error, this is a statistical measure that is only meaningful when a probability sample is used. It is a measure of variation, or uncertainty, associated with any finding of a poll, and is due to the fact that a poll is not a census.
Methods Box
A "sidebar" that is often printed in a newspaper or magazine to accompany an article reporting poll results. The methods box provides the reader with information about how the poll was conducted, typically including the mode of data collection, the target population that was sampled, the sample size, the dates of data collection, information about the margin of sampling error, and whether any statistical weighting adjustments were made after the data were gathered.
NCPP
The National Council of Public Polls. An umbrella organization of companies that conducts surveys and polls.
Nonprobability sample
Any of several different sampling schemes in which the elements in the "sampling frame" do not have both a known and and a nonzero probability of selection. Thus it is impossible to calculate the size of a poll's margin of sampling error with a nonprobability sample. Of note, this statistical fact does not stop some pollsters from calculating sampling error with a nonprobability sample - it just makes their calculations meaningless. Nonprobability samples are useful in the early stages of research or when a pollster needs to gain an "impression" of the preferences and attitudes of a target population but does not need to be very confident about how well the poll generalizes to the target population.
Open-end Questions
Poll questions that allow respondents to answer in their own words rather than limiting the range of responses to a predetermined set of alternatives. These answers are recorded "verbatim."
Poll
Literally, a counting of heads. When used to refer to a type of social research, a poll is a form of sample surveying that arose in the early 1800s as a way of "canvassing" the voting preferences of easily accessible subsets of the electorate in order to predict the likely election winner.
Pollster
A person who conducts polls, typically for paying clients.
Population
The group that a sample is supposed to represent.
Probability Sample
Any of several different sampling approaches that share two attributes: each element in the sampling frame (population) has a known probability of being selected, and each element in the sampling frame has a nonzero probability of being selected.
Question Stem
The part of a poll item that sets forth the substance or focus of the question being asked. The question stem does not include the response alternatives. A question stem may be balanced or unbalanced.
Random
Something that happens without purposeful choice.
Random Sampling
Any method that selects poll respondents arbitrarily and gives each possible respondent a fair chance of being selected. When used with probability sampling design, random sampling provides the basis for calculating a poll's margin of sampling error.
Representative Sample
The extent to which a sample matches the demographic, attitudinal or other characteristics of a target population it is intended to represent.
Respondents
The people selected to be interviewed for a poll.
Sample
The group, or subset, of some larger group that is selected to participate in a poll.
Sample Design
The method used to select a sample for a poll.
Sample Frame
The list from which a sample is chosen that contains all of the elements in the population.
Sample size
The number of elements, typically people, selected from a population.
SLOP
Self-selected opinion poll. These polls are frequently organized by broadcast media to assemble opinions by listeners or viewers. One significant problem is that there is no control over respondent selection because anyone can call in an opinion.
Stratified Sample
A form of probability sampling in which the sampling frame is ordered according to a relevant stratum or strata before the actual sample is selected.
Unbalanced question
A question stem that presents only one side of an issue rather than all sides. This is generally thought to bias responses to the item. An unbalanced set of response alternatives is simply on e that is not symmetrical and therefore does not contain a true midpoint. Unbalanced response alternatives do not necessarily contribute to poll error.
Weighting
Statistical adjustments to poll data that are conducted before the data are analyzed. Weighting is used to adjust for respondents' unequal probabilities of selection in probability samples. It is also used by some pollsters to try to adjust for nonresponse in a final sample whose demographic characteristics do not match those of the target population very well. Sometimes, weighting is used to count the answers of certain respondents more heavily than others, such as when formulating the prediction of an election outcome from preelection poll data. In such a case, a pollster attaches greater weight to the answers of those respondents who are most likely to vote, compared to the answers of those whose likelihood of voting is less certain.

Back to top

###