Organizations invest heavily in training and development, so it’s fair that executives want proof of results. The most widely used concept for evaluating the impact of training is Donald L. Kirkpatrick's classic four-level model (Kirkpatrick & Kirkpatrick, 2005) for evaluating training, which recommends measuring four types of outcomes.

Level 1 focuses on reaction—participants’ satisfaction with the program. End-of-course evaluations serve this purpose. Level 2 focuses on knowledge—what participants learned. Trainers administer knowledge tests that focus on concepts and principles related to course content—very much like the exams used in high school and college. While these evaluations can help trainers improve their programs, they fall short of meeting executives’ needs for evidence of results.

Level 3, however, focuses on behavior—the extent to which participants actually use their new skills on the job. Level 3 evaluations are significant because they address the transfer of training: whether employees are routinely performing the skills they learned in the classroom. While this kind of evaluation can demonstrate whether a program has done its job, most organizations haven’t settled on a simple, economical way to do it.

Fortunately, the feedback assessments used in a typical Train-to-Ingrain process are perhaps the most effective ever devised for measuring improvements in on-the-job performance. The procedure is simple. First, set up a 360-degree feedback survey with questions that describe the behaviors taught in training. Administer the survey before training, letting participants know that the survey will be administered again after training. The pre-course diagnostic helps participants set quantified, behavior-based performance improvement goals. Knowing that follow-up measurements will be taken later focuses their motivation and attention as they work with trainers—the ideal mindset for learning.

About six months after training, administer the first follow-up assessment. The pre-course assessment may be much broader than the follow-up assessments, which need only focus on the training units participants actually received. Since post-course follow-up assessments are derived from the pre-course assessment, scores may be easily compared. Improved scores will confirm improved performance. The ability to measure improvements in performance fulfills the need for ongoing feedback and accountability. The post-course assessments give learners quantitative and qualitative feedback about how they’re doing as they try to improve their skills. Also, the assessment documents whether the individual has improved on-the-job performance. For ongoing measures of performance improvement, repeat the assessment at the twelve-month and eighteen-month marks.

This commonsense approach to measuring performance improvement requires a fully customizable 360-degree assessment system such as 20/20 Insight GOLD, because the assessment items need to be tailored to exactly mirror the desired behaviors taught in training. In other words, assessment and training need to be integrated. The previously mentioned 20/20 Insight assessment system has inexpensive unlimited assessment licenses for each participant that make it possible to give them all the feedback they’ll need after training—without additional expense.

Finally, Kirkpatrick’s concept for Level 4 evaluation focuses on business outcomes— training’s impact on the bottom line. Because these outcomes are also affected by so many other variables, most organizations have abandoned efforts to measure training’s contribution to business results.

The best practical advice for calculating ROI (return on investment) is Russell Phillips’ Return on Investment in Training and Performance Improvement Programs (2003). His treatment is comprehensive, and your organization will want to be careful not to invest more in complex ROI research and calculation than the training itself.

If executives just want to know “Performance ROI”—the return they’re getting in improved performance—there are simple ways to calculate this. The key is to quantify the actual improvement in performance, translate this benefit into dollars and then determine whether this payoff is more than the cost. Conveniently, the data created by the performance improvement assessment technique described above can be used in a simple return-on-investment calculation.

For example, management may assume that leadership ability accounts for half of a supervisor’s effectiveness. As an example, let us say that a supervisor’s assessment scores show an average improvement from 6.4 (before training) to 7.7 (several months afterwards)—a 20% percent improvement. Since half of her $60,000 salary is $30,000, the organization is now getting 20% more effectiveness, worth roughly $7,500—an annual payoff several times greater than the cost of the individual’s training, which was $2,200. Philips’ formula may be used: Percent ROI = 100 x (Benefits – Costs)/Costs. Percent ROI = 100 x ($7,500 - $2,200)/$2,200 = 240%.

For “Results ROI”—the benefit the organization realizes in specific business results—Philips is a superb guide. His formula may be used, as above, but the quantifying the benefit is likely to involve a lot more research and considerably more complicated calculations. Your organization will have to supply data for specific business results. Often the difficulty is expressing this change in dollars. Even more difficult is determining how much of this result may be attributed to training. Philips explains how to research this properly, but most organizations will opt for a subjective judgment.

Measuring performance improvement (Level 3) and calculating ROI (Level 4) produce hard evidence of whether programs are changing behavior. It’s important to remember the key role players who are responsible for these results:

• The learner, who persists during the lengthy and sometimes frustrating period of reinforcement
  
  
• The direct manager, who supervises and coaches the learner while providing opportunities to apply skills in an encouraging environment
  
  
• The trainers, who present behavior-based training that is optimized for skill transfer and who coordinate follow-up programs
  
  
• Senior executives, who establish expectations, commit resources, promote an approach that can change behavior, and remain patient while employees ingrain new behavior patterns

For an immediate answer to any question about Train-to-Ingrain, email Meredith Bell, president of Performance Support Systems, or call: Toll-free: 1-800-488-6463, ext. 201 Toll phone: 757-873-3700, ext. 201 Fax: 757-873-3288