If you’ve been following our blog posts, you’ll know that we recently covered the Kirkpatrick Model, a framework to evaluate the effectiveness of training programmes.
In this post, we'll be outlining another method of evaluating training effectiveness - the Phillips ROI Model. Essentially, it builds from the Kirkpatrick Model, with an extra step included in the process to consider return on investment (ROI). We’ll demonstrate exactly how it works, and give examples how training providers could be applying it.
Frameworks such as the Phillips ROI Model and the Kirkpatrick Model are great jumping-off points for training providers to evaluate whether their training quality is up to scratch. Measuring effectiveness of programmes is vital to ensure training providers are:
- Meeting learning objectives
- Improving programme quality
- Demonstrating value to stakeholders
- Identifying training programme gaps
- Aligning training with business goals
- Justifying the customer's investment in training
- Boosting participant engagement
- Ensuring compliance where needed
The Phillips ROI Model
What is the Phillips ROI Model?
What are the levels of the Phillips ROI Model?
Is the Phillips ROI Model effective?
What are the criticisms of the Phillips ROI Model?
How to evaluate training using the Phillips ROI Model
What is the Phillips ROI Model?
In the 1970s and 1980s, Don Kirkpatrick's training evaluation model, "the Kirkpatrick Model", was widely used for training evaluation. In 1980, Jack Phillips published his book, titled "Return on Investment in Training and Performance Improvement Programs". He built on the processes outlined by Kirkpatrick, and addressed several of its criticisms. His approach later became known as the Phillips ROI Model.
Although Phillips made several tweaks to the original four levels of the Kirkpatrick Model, the most impactful development made by Phillips was the addition of the fifth level. This level highlights the importance of a monetary benefit, measuring the return on investment (ROI) of training outcomes.
In short, the Phillips ROI Model helps demonstrate both the qualitative and financial value of training programmes; measuring the ROI by evaluating their effectiveness at five levels.
What are the levels of the Phillips ROI Model?
The five levels of the Phillips ROI Methodology are as follows:
- Reaction: measures participants' satisfaction with the training.
- Learning: assesses knowledge or skills gained from the training.
- Application: evaluates the extent to which participants apply the learned skills on the job.
- Impact: measures the effect of training on business outcomes (e.g., productivity, quality).
- ROI: compares the monetary benefits of the training to its cost, calculating the overall return on investment.
Is the Phillips ROI Model effective?
The Phillips ROI Model is generally considered effective for evaluating the impact of training programmes, but as with any model, its success depends on how well it is implemented - and in what context. Here are some strengths:
It's a more comprehensive evaluation
The model goes beyond basic satisfaction and learning (like the Kirkpatrick Model) to assess application, business impact, and ROI, offering a holistic view of training effectiveness.
It can provide financial justification
By measuring ROI, the model allows organisations to quantify the financial benefits of training programmes, which can be critical for securing future investment in training.
It's a practical approach to decision-making
The data from all five levels help stakeholders make informed decisions about improving or scaling training programmes.
What are the criticisms of the Phillips ROI Model?
As outlined in the previous section, the Phillips ROI Model is effective for organisations looking to justify training investments and demonstrate tangible business benefits. However, it definitely requires careful planning, sufficient resources, and accurate data collection to fully realise its potential. Here are some potential challenges and criticisms of the model:
It can be complex and time-consuming
Measuring business impact and ROI (Levels 4 and 5) can be challenging, requiring significant data collection, analysis, and time.
It can be difficult to attribute training to results
Isolating the effects of training on business results can be difficult, as many factors beyond training (e.g., market changes, leadership) influence performance.
It can be costly in itself
Implementing the full Phillips ROI Model can be resource-intensive, particularly for large-scale or long-term training programmes.
How to evaluate training using the Phillips ROI Model
Evaluating training using the Phillips ROI Model involves a five-step approach to assess various aspects of the training programme, from participant reactions to financial return. Here's a quick rundown of how it can be used in practice:
1. Measure Reaction (Level 1)
Objective: Assess participants’ satisfaction with the training.
- Method: Use surveys, questionnaires, or feedback forms immediately after training to gather participants' views on the content, trainer, materials, and delivery.
- Questions: Were the participants engaged? Did they find the training relevant and useful?
2. Measure Learning (Level 2)
Objective: Determine if participants acquired the intended knowledge, skills, or competencies.
- Method: Pre- and post-assessments, tests, or quizzes can be used to measure knowledge gain. Skill demonstrations or practical assessments can also evaluate learning.
- Focus: Compare the participants’ knowledge or skill level before and after the training to quantify what was learned.
3. Measure Application (Level 3)
Objective: Evaluate whether participants are applying what they learned on the job.
- Method: Conduct follow-up surveys, interviews, or observations several weeks or months after training to see if the new skills or knowledge are being implemented.
- Indicators: Assess behavioural changes, the frequency of skill usage, or any barriers to application on the job.
- Stakeholders: Collect feedback from participants and supervisors to gauge real-world application.
4. Measure Impact (Level 4)
Objective: Identify the impact of the training on key business outcomes.
- Method: Measure the training’s impact on relevant business metrics such as productivity, quality, customer satisfaction, employee turnover, or error rates.
- Analysis: Compare pre- and post-training performance data to isolate changes resulting from the training programme.
5. Calculate ROI (Level 5)
Objective: Calculate the financial return on investment (ROI) by comparing the benefits of the training to its costs.
- Method:
- Monetise benefits: translate the business impact (Level 4 results) into financial terms. For example, if training increased productivity, estimate the value of that improvement.
- Total costs: calculate the total cost of the training programme, including materials, trainer fees, participant time, and facilities.
- Calculate ROI: use the formula:
- Interpretation: A positive ROI indicates that the financial benefits of the training outweigh its costs.
Isolate the Effects of Training
- Objective: ensure that the observed business impacts are directly linked to the training programme and not other factors.
- Method: use control groups (employees who didn’t receive the training) or statistical methods to isolate the effects of the training from other influences.
Report and Use Results
- Objective: share the findings with stakeholders for decision-making and improvement.
- Method: provide a detailed report with data from all five levels. Highlight the ROI and any areas for programme enhancement.
Conclusion
To conclude, the Phillips ROI Model is a popular and well-established framework for evaluating the effectiveness of training programmes. Building upon the Kirkpatrick Model, with the added focus on financial return, it assesses key areas such as participant satisfaction, learning outcomes, real-world application, business impact, and ultimately ROI.
It allows training providers and organisations to not only demonstrate the value of their programmes but also make informed decisions for future improvements. It's important to note, however, that the model requires careful implementation, thorough data collection, and sufficient resources. Overall, its ability to provide financial justification for training investments makes it a useful tool for organisations seeking to optimise their training efforts and align them with business goals.
If this article has left you with the feeling of "I'd love to use this training evaluation method, but I'm already drowning in training admin!", it may be time to consider having a look into training management software. accessplanit allows you to plan, manage and sell your training courses and resources, all in one place. Helping you to get organised, be more productive and scale your training business.
Book a demo with accessplanit today to find out more.
