The Predictive Index recruited a research group of masters-level students to investigate why it’s virtual training program was receiving low satisfaction ratings and identify potential improvements to the program.
As project lead, I led three other masters-level students through research and analysis of the program.
About The Predictive Index: The Predictive Index is a talent optimization company that businesses use to hire, inspire, and retain talent inside of their organizations. A part of their product offering are instructor-led training classes that introduce the methodology and concepts behind their workplace assessment tools.
After nine months of piloting a virtual training program, the learning team noticed satisfaction scores of the virtual program were lower than the in-person program.
The Predictive Index partnered with a group of master’s students from Boise State University to investigate why the virtual program had lower satisfaction scores ratings and identify potential improvements to the program.
Key Research Questions and Areas: The analysis focused on four areas, including, program design and delivery, learner application, completion rate, and sales/marketing with the fundamental questions being:
- Is the program meeting its intended outcomes?
- What improvements could be made to the virtual training programs?
A Mixed Methods Approach
The team used a mixed-methods approach that included learner surveys, semi-structured interviews, ad-hoc data analysis, literature review, and observations.
Learner Surveys: The project team distributed a web-based survey to learners two months after their completion of attending the virtual training.
Semi-structured interviews: I conducted interviews with key stakeholders inside of the organization to gain a better idea of the current state of the program as well as the intended business goals. Based on feedback from the interviews, I developed a graphic to show the inputs and results of the program.
To get a better picture of the training program, the project team reviewed level 1 reaction data as well as course enrollment and completion data.
Literature Review: The team conducted a literature review of 20 academic articles to identify best practices used in the delivery and design of virtual training sessions. The findings were used to create an observation checklist that was used to evaluate the current state of the program.
Observations: Using the observation checklist, the project team observed four virtual training sessions and rated various elements of the program design and delivery.
Data Synthesis: Following data collection, the project team created rubrics, scored data, and created a dimensional triangulation rubric. The team drew evidence-based conclusions and recommendations using a synthesis rubric.
Overall Themes and Opportunities
The team determined that the overall quality of the virtual program was high. Overall, the program is headed in the right direction, but some gaps should be addressed.
The team prioritized the following recommendations based on the importance weighting of the individual dimensions:
|Program design and delivery|
Determine the overall strategy of the virtual training program. For example, is it just an online version of the existing instructor-led training, or does it have another mission and purpose?
Develop a level 3 (application) survey that is delivered to learners after a specified period (3 months) after completing the program to determine whether they have applied what they learned when they return to the workplace.
Develop a customer-facing website that makes self-registration for learners an easy one-stop-shop. The site should contain a calendar of training events where customers can see availability and register for events.
|Sales and marketing strategy|
Adapt the marketing strategy to indicate the learning outcomes learners can expect to receive based on their attendance.