Training Evaluation

Most would simply define learning as the acquisition of knowledge. While that may be technically accurate, in my experience true learning is something that changes behavior. My father used to say that “people change, but not very much.” So the question is: how do we facilitate more positive behavior change in the workplace?

You may take a class on how to use a spreadsheet to track income and expenditures, but it’s not true learning unless you actually track your income and expenditures in a spreadsheet. You may understand how the American Electoral College works, but you haven’t truly learned how it works until you explain the importance of voting to someone else. Likewise, in corporate training, you may take a class on the importance of correct ergonomics in your desk setup but you didn’t truly learn anything unless you configure your desk so that it is ergonomically correct. True learning can only be quantified – and that can only be done if behavior changes.

Measuring learning is not always easy, but is almost always necessary. Often in professional learning and development we refer to Donald Kirkpatrick’s levels of training evaluation (Kirkpatrick and Partners, n.d.) – levels one through four. Over the years support for Kirkpatrick’s model of evaluation has waxed and waned, but it’s still one of the most robust models available.

• Level one refers to training participants impressions of the training (for example it asks questions like “how was the training?” and “Did you like the room?” and “Was the instructor knowledgeable?” for instructor-led training or “how easy was it to navigate?” for eLearning). This isn’t a measure of behavior change. I’ve encountered some who insert questions like “how much will this training improve your job performance.” I’m skeptical that predictive self-evaluation is something that should inform actual behavior change judgments because the most reliable way to predict future behavior is to look at past behavior (Franklin, 2013).
• Level two is a participant’s score on some sort of training test or quiz – it could be multiple guess or even some sort of application-based assessment, but usually takes place right after training. Level two evaluates knowledge and answers the question “did the participant memorize what we wanted her to learn?” But it doesn’t measure behavior. For that, one must look beyond simple assessments. Level two assessments are more effectively used as a tool to reinforce what has been learned, which requires prompts, coaching, and flexibility thus eliminating any objective statistics.
• Level three is where, I argue, the most value comes from evaluation and training assessment because it measures behavior change. For example, let’s say an insurance company is struggling because its entry-level claim adjustors continue to miss state department of insurance (often referred to as “DOI”) deadlines for communicating to their customers who are involved in an accident. The training department does some analysis and finds that adjustors miss the deadlines because they’re not aware of two thing: 1) the deadline itself and 2) the negative implication missing those deadlines has for the company and the customer. To overcome this problem, the training team designs a three-minute video that illustrates both points. In the background, they measure the adherence of that particular DOI rule after the video is pushed to the adjustors. The comparison of how well adjustors adhere to the rule after the training to compliance rates before the video is Kirkpatrick’s third level of evaluation – behavior.
• Level four measures impact to the business, based on potential behavior changes (level 3), and is useful when determining the worth of training – was it worth the effort in time, money, and labor. Ultimately, if one can prove positive level four results, it’s clear that the training was a success.

Using Kirckpatrick’s model, it’s clear that training which only addresses knowledge issues is likely a waste of time. But that is how most training is developed. The table below lists corresponding good and bad examples of how a training approach can either impact behavior or only address the basic knowledge level.