“How do you measure self-efficacy? The answer may surprise you”

As a follow-up to my post on measuring the impact of serious games (see “8 Tips For Measuring the Impact of Serious Games”), let me give you a little quiz. It’s not as easy as it may seem.

Let’s say you made a serious game to increase the engagement of seniors in regular physical activity at a gym. One of the “research goals” of your game was to increase player’s self-efficacy to engage in regular physical activity. You included role models in the game, you separated tasks into manageable chunks, you presented players with mastery experiences, you provided encouraging feedback, and you helped players manage their stress and anxiety related to engaging in the target behaviors. The game is engaging and fun. Done!

You also designed a study in which you measure self-efficacy before and after seniors play your serious game (compared to a control group of seniors that doesn’t play your game). You hypothesize that your game will increase player self-efficacy to engage in regular physical activity at the gym (note: you may further hypothesize that increases in self-efficacy will mediate or “explain” increases in actual behavioral engagement in physical activity).

You want to make sure you have a really good measure of self-efficacy. What do you do next?

Please pick the BEST answer:

1. Use a published measure of general Self-Efficacy that has acceptable levels of reliability and validity.
2. Use a published measure of Self-Efficacy to exercise regularly that has acceptable levels of reliability and validity.
3. Generate your own measure of Self-Efficacy.
4. Forget about measuring Self-Efficacy. Just get an objective assessment of actual  engagement in regular exercise (e.g., pedometer, frequency of attending a gym). Objective measures are always better than self-report!

Believe it or not, the correct answer is…. (Please scroll down)

#3. GENERATE YOUR OWN MEASURE OF SELF-EFFICACY!

Shocking, isn’t it? Developing your own measure should be the last resort shouldn’t it? Take a deep breath and let me help you reduce your galvanic skin response by walking you through why the other responses are not ideal. I will also show you how you can develop a really good self-efficacy measure on your own.

Why 1. “Use a published measure of General Self-Efficacy that has acceptable levels of reliability and validity” is INCORRECT.

You designed your game based on self-efficacy theory to help players address specific challenges in specific situations regarding engaging in regular exercise. Although many general measures of self-efficacy exist, they will not help you fully capture increases in self-efficacy in this domain. You were not trying to help your players increase their self-efficacy across varied domains that can include public speaking, paying taxes on time, and remembering to take out the trash regularly so why measure that? You can if you want to, but I suspect you won’t fully capture the power your game has to increase self-efficacy with a general measure.

Why 2. “Use a published measure of Self-Efficacy to Exercise that has acceptable levels of reliability and validity” is INCORRECT.

This was the tricky option because it is almost correct. It could be that someone developed a measure of self-efficacy to engage in regular exercise that you could use in your study. This would happen if the measure were developed with a similar target population (e.g., older people and not school-age children), target outcome (e.g., regular physical exercise at a gym and not hiking in the woods), facing similar challenges (e.g., finding a gym that is open early enough to suit their schedule and not finding alternative means to work out if it rains).

It is usually not the case that a published measure is perfectly suited to assess the domain and specific situation that you targeted in your intervention. Therefore, #3 is the MOST correct answer.

Why 3. “Generate your own measure of Self-Efficacy” is CORRECT.

In the words of Al Bandura, “Scales of perceived self efficacy must be tailored to the particular domain of functioning that is the object of interest” (Bandura, 2006). That is his published statement and this is why it is best to generate your own measure of self-efficacy.

In addition, I also have a personal anecdote to support this claim. I was fortunate enough to be able to take a class from Al Bandura on self efficacy in the 1990’s. He was putting the finishing touches on his book, Self-Efficacy: The Exercise of Control while teaching this class. I vividly remember how he laughed when he described how he frequently received phone calls and requests for a global measure of self-efficacy.  He chuckled as he talked about how these people had a hard time understanding that he could not point them to a published scale of self-efficacy that he would recommend them to use.  They seemed incredulous that he could not offer them any help with finding a measure. They had to create one themselves.

So if you thought that you had to use a published “validated” measure of self-efficacy, you are in good company. Hopefully this information has saved you hours of combing through the literature and attempts to contact Al Bandura personally.

The good news is that there are good published guidelines for creating self-efficacy scales that you can use and cite when you write-up your research for publication. Prof. Bandura published a helpful guide for constructing self-efficacy scales that gives very specific guidelines for creating a good self-efficacy scale. Chapter 2 in his book, Self-Efficacy: The Exercise of Self-Control contains a discussion of conceptual and methodological issues involved in developing self-efficacy scales.

A final note on why 4.  “Forget about measuring Self-Efficacy. Just get an objective assessment of actual  engagement in regular exercise (e.g., pedometer, frequency of attending a gym). Objective measures are always better than self-report!” is INCORRECT.

This is another tricky one because really, who really cares if the game increased self-efficacy as long as people who play your game are engaging in more regular physical activity? Ultimately, the behavior and the health outcome is what we should be concerned about it if we want to change the world. I agree there.

However, we researchers are interested in mechanisms of action that explain what we see.  We want to go beyond the magic of waving a wand with a game and getting an improvement in health outcomes. We want to know why it happened (or not) and either replicate the findings or tweak the approach to get even better outcomes the next time. We want to learn and we want to be able to explain things.

Also, people who develop serious games, the game developers, SHOULD be interested in maximizing the power of their serious games to make a difference by building in these active ingredients (and leaving out the inert fillers) that are found in research studies.  It always surprises me to hear that there are game developers out there who aren’t interested in seeing what their games are actually doing. Luckily, that seems to be becoming more rare as the field is growing and becoming more “legitimate.”

So off you go to fearlessly develop your own self-efficacy measure for your study! People will say you are “mad” but you can defend yourself with your knowledge of self-efficacy and documented approaches to developing these scales yourself.  You are ready to level up!