URBANA, Ill. – Colleges and universities across the nation often make it part of their mission to equip students to be next-generation leaders in the workplace, offering a variety of leadership experiences and programs for students.

But just how successful these programs are in training long-term leaders can be difficult to assess.

In a new study published in Journal of Leadership Education, leadership education researchers from the University of Illinois look at the leadership capacity for students at various points in time after they have participated in a leadership development program, including up to two years after the program. They find that students’ leadership capacity doesn’t develop in a straight line, but rather in what they call “peaks and valleys” over time.

The researchers say the data they’ve collected seems to show that while students are getting good things out of programs, more follow-up has to be given to assess potential behavior. They hope their findings will help the developers of these leadership programs create experiences for students that will not only teach leadership skills, but also help students stay motivated and confident as long-term leaders. 

“Part of our research has been, to some extent, to talk about the difficulty of trying to assess these programs. You can’t do the basic, ‘give people a pre-test, give people a post-test,’ look at the fact that scores went up, and say the program was relevant. It could be the students had a good time and had a good lunch, but they’re really not any different than they were in the beginning, just happier. And that led them to have higher scores,” says David Rosch, associate professor in agricultural education at U of I and co-author of the study.

But what happens when those students go back to their student organizations, lab teams, or group-project groups after the leadership training experience? “The same conflicts exist and they don’t behaviorally do anything different than they did before. All the data that were collected in very basic, low-intensity ways shows that the program was helpful, but if you look more broadly at any hypothetical program, the program itself may not have any impact at all.”

Rosch and colleague Jasmine Collins, assistant professor of agricultural education at U of I and co-author of the study, set out to look at what happens after the students’ leadership program experience. The researchers surveyed students participating in the LeaderShape Institute, a successful, model program that Rosch says serves as the template for many other leadership development programs.

Collins explains that measures of the students’ own perceptions about their leadership skills, their confidence or self-efficacy to lead, and their sense of motivation to lead were incorporated into surveys given to students directly prior to their LeaderShape Institute experience. A post-test was given on the last day of the program, during which students answered the same questions.

The same pool of students were then surveyed with the same scale measures three to four months after program completion, and then one to two years after completion. “This is something that is very unique in the leadership education literature,” Collins says. “Most studies looking at development are not necessarily looking at development through different points in time. They are looking at development as a snapshot in time, maybe comparing incoming students to graduating students.”

Results from the surveys showed that students’ leadership capacity did not develop in a straight line, but across “peaks and valleys.”

“At the end of the program, they’ve had a great time, are often sleep deprived, there are some tears of joy, and then they fill out our survey,” Rosch says. “They’ve had the inspirational ending, so, of course, they’re going to talk about how transformational the program is.”

Rosch calls this the “honeymoon period.” But then the students go out into the real world again. “They realize all their problems aren’t going to be solved because they participated in a leadership program. That’s the valley,” he adds.

The three- to four-month follow-ups show that students, in some cases, rated their leadership capacity lower than before they started. “Honestly, I don’t think that means the program damaged them. I think it means they have a more realistic appreciation for their capacity at that time. Over time, they grow, they rebound, they engage further. We saw that in a lot of ways.”

In fact, the surveys showed by the two-year follow-up students were reporting higher leadership capacity than during their “honeymoon phase” at the end of the program.

As part of the study, the researchers also looked at campus involvement, and whether activities students participated in on campus—from just participating in an activity or organization, to holding a leadership role, to participating in group work assignments—made a difference on their reported leadership capacity. By in large, most of the factors they looked at had no significant effect on their leadership capacity across the population.

“The bottom line is development as a leader is non-linear. It’s not a straight trajectory. If you only collect at two data points, pre-test and post-test, it looks like development is a straight line. And our research is empirically showing that it’s not the case. And the way you collect data is so much more powerful in terms of driving your results, than the actual program itself,” Rosch says.

In a second paper published in the Journal of Leadership Studies, Collins and Rosch dig deeper into assessment design, by experimenting with the timing of pre-test data collection and the length of scale response sets students receive (i.e., “On a scale of 1-10”). They found that design made a difference in the results.  

“A takeaway from both papers, is ‘how’ the effects of these programs are measured can significantly impact the way the effectiveness of the program is interpreted,” Collins says.

Another takeaway Rosch stresses is that while the majority of leadership programs occur in two formats—the day-long retreat style, or class-based over time—either way, the program ends and the curriculum ends. “Are there things we can put into place to help students stay in the game of thinking intentionally about their leadership? If students can identify a mentor, that’s a really positive factor in their continued growth after they have left the program. In some cases it modulates that valley they go into.”

The paper, “Peaks and valleys: A two-year study of student leadership capacity associated with campus involvement,” is published in the Journal of Leadership Development.  [DOI: 10.12806/V18/I1/R5]. Co-authors include David M. Rosch and Jasmine D. Collins, both in the College of Agricultural, Consumer and Environmental Sciences at U of I.

The paper, “Opportunities for the gaming the system: How data collection design influences program assessment outcomes,” is published in the Journal of Leadership Studies. [DOI: 10.1002/jls.21581] Co-authors include Collins, Rosch, and Clinton M. Stephens.