There are lots of reasons to measure stigma. Some of you are researchers who are working to develop survey instruments or scales, conducting qualitative research to understand stigma and how it works, or using measures to evaluate the impact of interventions. Others are trying to understand how stigma manifests in communities where they work in order to develop strategies to reduce stigma. Still others are implementing strategies to reduce stigma and want to understand if what they’re doing works. I was fortunate enough to participate in a recent meeting of 17 stigma researchers, convened by inroads members the Sea Change Program, which included a panel of measurement experts talking about how to apply principles of measurement in different real-world practices.
The panel included inroads members Kristen Shellenberg (Ipas) and Ushma Upadhyay (Advancing New Standards for Reproductive Health [ANSIRH] with the Bixby Center for Global Reproductive Health at the University of California, San Francisco), as well as Corinne Rocca (also with ANSIRH and the Bixby Center) and Greg Herek (University of California Davis). All are experts in developing and adapting measures, and Kristen developed one of the existing scales to measure abortion stigma. They were posed this question, which has come up at a number of inroads gatherings and conversations:
“Say you are working in the field, not in a university or research setting, and you want to measure abortion stigma. None of the existing scales are a perfect fit for your context, and you want to adapt a scale. Where should you start?”
I took detailed notes and integrated this with what I know from my background measuring things like self-efficacy to prevent pregnancy, help-seeking behavior, and attitudes towards gender among adolescents, and thought I’d distill it down to a series of steps for all of you who might be planning to or are already running an intervention and want to know if it works.
Some of the best practices included below are for those who have some resources to put toward measure adaptation, including staff to dedicate to this effort, some research background and a relationship with a statistician. I also include guidance for those who may not have the resources for rigorous research, but do want to be able to understand how stigma works so you can plan activities that make sense. Regardless of how much time or staff capacity you have to dedicate to research, it is a best practice to include people from the population that you serve or are researching along each step of the process.
STEP ONE: Identify a target population and what you want to measure. Are you working with people who’ve had abortions or are seeking abortion care? Are you working with people who provide abortion care? Are you working with a community to understand barriers to access or how community and family norms might influence attitudes and behaviors around abortion? Are you looking at how abortion is portrayed in the media?
STEP TWO: Pick a scale: Once you’ve defined the target population, it’s time to consider the existing scales that measure stigma! There’s a handy table at the end of this blog post summarizing the scales for measuring abortion stigma specifically, and all of the scale developers are inroads members. Look at what level of stigma from the socio-ecological model is measured and what population the researchers worked with to measure stigma. You can also consider the wide body of literature that exists to measure HIV-related stigma, or stigma among lesbian, gay, bisexual, transgender (LGBT) populations. (Need help with a literature review? You can request technical support from inroads here. Or if you need help accessing the full text of a key journal article you can reach out to inroads at email@example.com and we will work to connect you with that.) Based on all of those factors, pick the scale that seems to be the best fit for your context.
STEP THREE: Get feedback. Now it’s time to start identifying subject matter experts to consult with. It’s best to find people whose work and expertise is as close as possible to the population among whom you want to measure stigma. Even more important? Talk to and involve people who are FROM the population you want to work with. Talk to these people (who have been involved from step one) about what you want to measure, and go through the scale with them. What do they think? What’s missing? Do the questions make sense? How would they feel answering them?
Also, consider how you plan on collecting the data. For example, will you have clinicians asking these questions as part of an interview setting? If yes, then it’s important to learn how someone might feel answering these questions out loud. Is it through a self-administered questionnaire? If so, then it is important to consider whether the people you want to survey can read and write, or can understand the language that you want to use. Talk about both the CONTENT of the questions and the METHOD you will use to get people to answer the questions with your subject matter experts and with people who are from the population you want to survey.
STEP FOUR: Revise. Based on the work you did in Step Three, decide how much you need to revise the existing scale. Textbooks and best practices documents will tell you it is “best” to adopt the existing scale as is. And it’s true: you won’t be able to compare your results to results from other contexts precisely if you change the items. However, stigma is contextually specific, so think about the goals you have for the research. Keeping the scale as it is may not actually be “best” for your needs or goals.
Adapting the scale could be as simple as slightly changing some of the words to reflect a different population. For example, IPPF adapted the Stigmatizing Attitudes, Behaviors and Actions scale for use with adolescents. They simply specified the questions to refer to adolescents, and added a few items. (You can see the original SABAS scale here, and you can see the IPPF adaptation for young people by logging into the inroads Collaboration Workspace). You also could decide to adapt a scale for an entirely different cultural context or population than the original scale. Or you could try to build as short a survey as you can with items (questions) from already validated instruments.
STEP FIVE: Test your revision! If you are set up for research and have a statistician available (or have a relationship with a graduate student at an area university), start by administering the survey with a selection of people from your target population, and see what the results tell you! Run some statistical analyses to see what you find. Some options include running confirmatory factor analyses to see if the scale is valid in your context and check on whether what you find resonates with what the original study found. Or, run some item response modeling and introduce an interaction term to the group to see if any items don’t work.
If you don’t have a statistician available or if your goals for measuring stigma are less about stigma research and more about planning appropriate activities, then simply conducting a few rounds of cognitive interviewing can have you ready to finalize your questionnaire. The basic goal is to figure out: Are we asking what we think we are asking? Find a small number of people from the population you work with. Go through the items with them question by question, and ask them to answer this question, and talk you through what they are thinking when they answer it. This gives you a sense of whether the question is getting at what you really want to ask. This is also really useful if you are translating from one language to another and you want to test that the meaning has successfully translated. You can also ask questions about how they understand or interpret a question (“what does ‘stereotype’ mean to you?”), or ask them to paraphrase the question back to you in their own words. Do this with about five people, and you should end up with a good idea of whether your revised scale will work.
STEP SIX: Based on this pilot, you might revise again, or you might be ready to finalize your questionnaire. Finalize, go forth, and collect those data!
For the research nerds among you, the panel raised some other factors that you might consider. For example, if you want to shorten a scale or subscale, conventional wisdom suggests throwing out the items that have lower alphas and keeping items that have higher coefficients. Corinne Rocca urged us to consider, however, that we might end up cutting some of the key constructs that we want to measure. For example, the Community Level Abortion Stigma scale has a subscale that measures discrimination and guilt/shame. The lowest scored item on that subscale was “A woman without children is an incomplete woman,” but it is one of the few items that gets at norms around maternity as an “ideal” of womanhood, which might be something important to you. You might lose that if you make decisions based on statistics alone, so do consider your cultural context and what you need to measure. Depending on your measurement goals, there may be a higher importance to measuring stigma in a locally meaningful way than in ensuring comparability across settings.
One of the great things about our field is that we ARE making progress, and we WILL shift norms. Greg Herek shared that a scale developed in the United States in 2009 to measure homophobia and stigma experienced by LGBT people doesn’t work anymore. The items about homophobia are no longer a good measure because the cultural norms and policies have shifted so much. May all our scales become quickly out of date!
All of the panelists agreed: Measurement is messy. Stigma is messy. Everything is messy. So, do what you can! Are you at a point where you are thinking about adapting a scale? Do you want to connect with others? There are several ways to get engaged and get support:
Table of scales
|Name||Authors||What does it measure?||Subscales||Populations tested|
|Stigmatizing Attitudes, Beliefs, and Actions Scale (SABAS)||Shellenberg, K; Hessini, L.; & Levandowski, B.||abortion stigma at community level||Negative stereotypes
Discrimination and exclusion
Fear of contagion
|Community members in Ghana and Zambia|
|Individual Level Abortion Stigma Scale (ILAS)||Cockrill, K.; Upadhyay, U.; Turan, J. & Greene Foster, D.||Individual level stigma among women who have had abortions||Worries about judgment
|Women at abortion clinics in the U.S.|
|Community Level Abortion Stigma Scale (CLASS)||Sorhaindo, A.; Karver, T.; Karver, J.; & Garcia, S.||Community level stigma toward women who have had abortions||Autonomy
|Community members in Mexico|
|Abortion Provider Stigma Scale (APSS)||Martin, L.; Debbink, M.; Hassinger, J.; Youatt, E.; Eagen-Torkko, M.; & Harris, L.||Abortion providers’ perception of stigma||Disclosure management
Resistance and resilience
|Abortion providers in the United States|