Show
According to the latest Training Industry Report, US companies have spent over $82.5 billion on training in 2020. Throughout the pandemic, companies have continued to invest heavily in the learning and development of employees. With this investment comes the expectation of measuring its impact, which is why training evaluation is so important. Every year, there has been a steady increase in spending on training to improve employees’ skills. The ability to evaluate the effectiveness of this training had been proven equally important. Due to the COVID-19 pandemic, there is also an increase in remote working, resulting in a substantial amount of remote training to replace in-person learning strategies. Subsequently, the need to evaluate the effectiveness of training has never been greater. How can you make sure that you’re evaluating training the best way possible? Contents What is training evaluation?Training evaluation is a systematic process to analyze the effects of a training program and to assess its effectiveness. It is also used to gain feedback on how to improve a training program. Furthermore, it measures whether a training program meets the objectives as to why it was created. Here’s an example of how you might use a training evaluation at your workplace. Consider you’re a software company and released version 2.0 of your product. In order to ensure that all engineers and IT employees understand the new product, you deploy a training program. A training evaluation would look at this program holistically, and evaluate it based on the business needs, performance, learning needs, impact of training, and finally, the return on investment.
Related (free) resource ahead! Continue reading below ↓
Download your free guide on using data to measure the effectiveness of your L&D programs. The outcome of an evaluation of training should give management all the information they need as to what worked, what did not work, and improvements for the future. It should also provide a clear indication if the money spent on the training program produced the desired results. Why is training evaluation important?In the last year, training evaluation has dramatically increased in its importance because of the demand for new workforce skills. The rapid change to remote work and an increased need for digital skills have changed the way we work and how business gets done. Being able to develop these skills internally for organizations is thus a priority. Furthermore, training evaluation is essential for the following reasons:
Training evaluation modelA good and practical method of evaluating training is to follow the Phillips V-model, also known as Phillips ROI model. The model, which was developed by Jack Phillips, broadly follows the Kirkpatrick model. Therefore, it is important that you understand the Kirkpatrick model, which you can read more about here: The Kirkpatrick model. In short, the Phillips model has five levels, compared to Kirkpatricks’ four levels of evaluation. Phillips added a fifth level, which is return on investment (ROI) – a metric that is incredibly valuable for HR leaders to demonstrate to decision-makers the importance of training. The ROI Institute (Canada) provides a good graphic to illustrate this model: The Phillips V-model follows the below sequence of five levels of training evaluation:
Learning & Development Certificate
Become a Strategic Level 1: Reaction The model begins with evaluating participants’ experience of the training program they’ve gone through. The most common technique is to use a post-training survey. However, as employees tend to feel ‘over surveyed’, other techniques include pulse surveys (pop-ups), AI technology to understand emotional reactions, suggestion box and review sites. This is the step to gather data for evaluation. Level 2: Learning This is to assess if learning actually took place. To put it another way, it’s a great way to measure if the training program’s learning objectives have been met and if there was attendance and attentiveness. A way of doing this is to do a pre and post-test. For example, if the training objective was to teach a group of coders a new coding language, Python, for instance, it would be good to test the learners before they start their python skills before and after the program with an assessment. Level 3: Application and Implementation At this stage, the Phillips V-model tries to distinguish if the problem is with the application of learning or if it is in implementation. Let’s, for example, say that the information collected in level 2 confirmed that the training was not successful; then level 3 would be able to determine what the issue is as to why there is no application or implementation. For instance, if the employees understood Python, but were unable to apply it because of software requirements, then the issue to be resolved is for the organization to invest in better software.
Try our need tool to determine the direction in which you want to progress based on your HR career goals and capabilities. Get StartedLevel 4: Impact At the fourth level, the Phillips V-model looks at the total impact the training has had on the organization. It not only attributes impact to the training effectiveness but also considers organizational factors that might hinder the successful implementation of the skills gained during training. It takes into consideration external factors as well, and whether that has any impact on the performance of training. As an example, let’s say you ran a training program for sales agents on the skill of door-to-door sales training. Due to COVID-19, your sales agents are unable to approach customers at their doorstep. Therefore, it can be surmised that applying the new skills gained during the training is hindered due to not being able to use them. Level 5: Return on investment (ROI) This is where the Phillips V-model distinguishes itself from the Kirkpatrick model. It measures the return on investment of the training. Although there is no direct way to attribute training to business performance, the Phillips V-model uses specific metrics and measures to give as close a viewpoint as possible. For example, using cost-benefit analysis for a training program can determine if the money invested in training has had any impact. Are there any measurable results? In other words, if the training cost $50,000 to train sales agents to sell an organization’s products online, and the net profit from this is $500,000, then that would be considered a good investment. The benefits of training are clear in this case. Of course, it would be important to take other considerations in place specific to your organization and program. Training evaluation in four stepsNow that you understand the theory behind the Phillips-V model, let’s take a look at how to implement this in practice. 1. Determine the relevant training evaluation levelGenerally, not all of your training efforts will be evaluated at all five levels of the Phillips V-model. That would be a waste of resources. In fact, only about 5% of training programs are evaluated up to level 5. As an example, you would evaluate a two-year executive leadership training program at all five levels. This is because it is high-impact, high complexity, and of high value. However, you may not evaluate, for example, a training program that only focuses on three employees in developing a new way of replying to customer emails. Phillips recommends evaluating all learning projects on the first level and then assess based on need and importance which levels the training program needs to be evaluated. For example, you may determine that you need to evaluate learning up until level 3. As a reminder of the evaluation targets of the Phillips V-model, look at the diagram below: Again, your determination of which level to measure should be based on budget and business need. The levels of evaluation happen at different times, as demonstrated in the following table:
As an example, let’s say you were to measure a training program up until level 4. The exmples of tangible and intangible outcomes of learning would look something like this (Phillips 2008):
2. Select and combine your training evaluation toolsOnce you have chosen the level at which you should evaluate your training program, you can now determine which training evaluation tool to use. You can use one tool or a combination of methods. It’s advised to collect both quantitative and qualitative data to provide an objective and holistic view of the training program. Combining the methods will help you gain valuable insights into the training experience. Training evaluation methods include:
You may use one of these tools or a combination of them, depending on the level of evaluation, on your budget and need. 3. Analyze the collected dataYou need to combine and analyze the data from different sources, which will help you come to meaningful conclusions and determine the effects of a training program. In order to give the data meaning, you need to represent it in a meaningful way. This may be through developing a report, which includes charts and comments. It could also mean compiling video analysis and texts. It’s essential that when collecting data, it is cleaned and analyzed objectively without any bias in mind. You need to ensure it is checked for accuracy and produced without any preconceived objective in mind. You need to think about how you will visualize the data because it will be necessary for decision-making, after all. 4. Report the findingsYou can report the results to key stakeholders (management as well as employees). You should always strive to connect your findings to business objectives to show what impact the training had (or didn’t have) on the performance of the organization. This is where level 5 of the Phillips V-model becomes so important to show ROI on the training that was invested in. To sum upTraining evaluation is critical to assess whether your training has the desired outcome. Understanding different training evaluation models and methods and applying the most suitable ones enables your organization to improve the effectiveness of the training and, ultimately, the organizational performance. |