AAPOR’s Transparency Initiative is designed to promote methodological disclosure through a proactive, educational approach that assists survey organizations in developing simple and efficient means for routinely disclosing the research methods associated with their publicly-released studies.
The Transparency Initiative is an approach to the goal of an open science of survey research by acknowledging those organizations that pledge to practice transparency in their reporting of survey-based research findings. In doing so, AAPOR makes no judgment about the approach, quality or rigor of the methods being disclosed.
Membership in the Transparency Initiative will identify your organization as one that is willing to publicly disclose its basic research methods and make them available for public inspection. It marks your organization as one that values openness and has been formally recognized for your commitment to transparency in research methods.
Joining the Transparency Initiative will provide your organization with formal public recognition by AAPOR of your voluntary pledge to abide by our disclosure standards. It will also provide you with the opportunity to advertise your commitment to transparency by using AAPOR’s Transparency Initiative logo in your promotional materials. In addition, it will afford you with the opportunity to differentiate yourself from competitors who have elected not to participate.
All Transparency Initiative submissions will remain confidential and will be used only for the purpose of reviewing for membership in the TI. No submitted samples/information will be publicly released without your written permission.
No. Survey subunits of larger organizations are eligible to join the Transparency Initiative without committing other components of the organization to adhere to its requirements.
Yes, for-profit organizations are welcome and encouraged to apply for membership in the Transparency Initiative. We can also consider for membership an organization that only commissions research for private clients if they meet our requirements (see the Terms and Conditions documentation on the website).
All survey-related organizations – domestic or international – may apply for membership in the Transparency Initiative if they are willing to abide by the transparency guidelines. (See the Terms and Conditions documentation on the website).
Nothing. In 2017, the AAPOR Executive Council formally decided to waive any membership fees for organizations to join the Transparency Initiative.
Once your organization has submitted their reports and Transparency Certification Agreement, we will work to complete the initial review process as quickly as possible, but final approval and admission to the Transparency Initiative may take some time, depending on the completeness of your submission. A team of Transparency Initiative reviewers will work with your designated representative to complete the review process as quickly as possible.
If you have submitted your application, you should have received a confirmation email with your application materials. If you have questions or need assistance, please email [email protected]
Adherence to Transparency Initiative principles includes insuring that staff are trained and prepared to practice transparency when reporting survey findings and insuring that all related reports are consistent with the Transparency Initiative reporting requirements. There’s no question that compliance with Transparency Initiative principles will take some time and effort on your organization’s part. To the degree that your organization is already fully disclosing your research methods on a regular basis, less effort should be needed to become Transparency Initiative compliant.
In the majority of situations where the Transparency Initiative reviewers find problems, they will provide assistance in helping the applicant organization revise their current disclosure procedures in order to become compliant with the TI requirements. The Transparency Initiative’s goal is to work with your organization to achieve our common goal of transparency and openness.
No. Organizations will be expected to continue reporting methodological information using the modality and format they consider being most appropriate for their purposes. The Transparency Initiative requires only that the disclosure elements be made publicly available at the same time that survey findings are publicly released, and that additional methodological details be made available within 30 days of any request for that information. The specific format in which that information is to be shared is at the discretion of the organization. While organizations will always have freedom to tailor the format in which they make this information publicly available, it is expected that in the majority of cases this information can be easily posted on your organization’s website.
In those cases when your organization does not collect the survey data, you will be expected to request this information from the fieldwork subcontractor so it is available in the event the data is made public.
In those cases where your organization participates as a fieldwork contractor only, you must agree to provide the same information to the study’s sponsor. It’s understood that whether clients elect to disclose that information or not is beyond your control. But insuring that they have the information is your responsibility.
No. The Transparency Initiative disclosure requirements are prospective only. So, for example, reports that provide time trend data will only require methodological details for the new primary survey data being reported.
No. The Transparency Initiative does not attempt to judge the quality of member’s survey methods. Rather, it is concerned with insuring that research methods are consistently disclosed so that research consumers have available the information necessary to reach their own conclusions regarding survey quality.
AAPOR has long championed the importance of transparency in the public reporting of public opinion and survey research. The Transparency Initiative’s disclosure standards are those found in the AAPOR Code of Professional Ethics and Practices, the Code that all AAPOR members have pledged to uphold. Encouraging adherence to these standards is consistent with our long-held values of research transparency and openness.
Public invitations to join the Transparency Initiative were first announced in October 2014 and extended to all survey organizations.
Yes; any organization that wants to show additional support of the TI may donate funds to the Initiative and hold SPONSOR status with the TI. For more information on becoming a TI Sponsor, contact AAPOR Staff at AAPOR HQ
Describe the data collection strategies employed (e.g. surveys, focus groups, content analyses).
Name the sponsor of the research and the party(ies) who conducted it. If the original source of funding is different than the sponsor, this source will also be disclosed.
Measurement tools include questionnaires with survey questions and response options, show cards, vignettes, or scripts used to guide discussions or interviews. The exact wording and presentation of any measurement tool from which results are reported as well as any preceding contextual information that might reasonably be expected to influence responses to the reported results and instructions to respondents or interviewers should be included. Also included are scripts used to guide discussions and semi-structured interviews and any instructions to researchers, interviewers, moderators, and participants in the research. Content analyses and ethnographic research will provide the scheme or guide used to categorize the data; researchers will also disclose if no formal scheme was used.
Survey and public opinion research can be conducted with many different populations including, but not limited to, the general public, voters, people working in particular sectors, blog postings, news broadcasts, an elected official’s social media feed. Researchers will be specific about the decision rules used to define the population when describing the study population, including location, age, other social or demographic characteristics (e.g., persons who access the internet), time (e.g., immigrants entering the US between 2015 and 2019). Content analyses will also include the unit of analysis (e.g., news article, social media post) and the source of the data (e.g., Twitter, Lexis-Nexis).
The description of the methods of sampling includes the sample design and methods used to contact or recruit research participants or collect units of analysis (content analysis).
Include a description of all mode(s) used to contact participants or collect data or information (e.g., CATI, CAPI, ACASI, IVR, mail, Web for survey; paper and pencil, audio or video recording for qualitative research, etc.) and the language(s) offered or included. For qualitative research such as in-depth interviews and focus groups, also include length of interviews or the focus group session.
Disclose the dates of data collection (e.g., data collection from January 15 through March 10 of 2019). If this is a content analysis, include the dates of the content analyzed (e.g., social media posts between January 1 and 10, 2019).
Describe how the weights were calculated, including the variables used and the sources of the weighting parameters.
Describe validity checks, where applicable, including but not limited to whether the researcher added attention checks, logic checks, or excluded respondents who straight-lined or completed the survey under a certain time constraint, any screening of content for evidence that it originated from bots or fabricated profiles, re-contacts to confirm that the interview occurred or to verify respondent’s identity or both, and measures to prevent respondents from completing the survey more than once. Any data imputation or other data exclusions or replacement will also be discussed. Researchers will provide information about whether any coding was done by software or human coders (or both); if automated coding was done, name the software and specify the parameters or decision rules that were used.
All research has limitations and researchers will include a general statement acknowledging the unmeasured error associated with all forms of public opinion research.
Since its launch in fall 2014, AAPOR has welcomed numerous Members to the AAPOR Transparency Initiative (TI).