RETURN ON INVESTMENT, OR ROI, is a form of evaluation that allows organizations to find out if a training program has been profitable for the company. Evaluators who use this framework, such as Jack J. Phillips, do so to determine if the monetary value of the results exceeds the cost of the training. Because it builds upon information often gathered by using Kirkpatrick's 4 Levels of Evaluation Model, ROI is often considered the 5th level.

Donald Kirkpatrick (1994) developed a four-tiered evaluation model to assist program directors in determining the effectiveness of training (see Kirkpatrick's 4 Levels of Evaluation for more information). It is based on four main questions: Did they like it? Did they learn it? Will they use it? Will it matter?
ROI adds a new tier - Did we make more money than the training cost? 

Kirkpatrick's 4 Levels of Evaluation with ROI as the 5th Level.

Jack J. Phillips (1996) is well known in the area of ROI evaluations. He describes a basic formula for calculating ROI.

Step   Process
  1      Collect level-4 evaluation data by asking, "Is there a change in job performance that is positive and measurable?" 

 2       Isolate the effects of the training from other factors that may have contributed to the results. 

 3       Convert the results to monetary benefits.

 4      Total the cost of training.

 5     Compare the monetary benefits with the costs. The non-monetary benefits can be presented as additional, though intangible, evidence of the program's success.

Step #3 is probably the most difficult part of the evaluation framework as it is very subjective and based on the opinions of stakeholders. 

The costs vs. the monetary benefits of a training program of a company.

Because ROI is very subjective, there are many different pros and cons to using this form of evaluation. An evaluator must be very careful to weigh the pros and cons before deciding to use this framework. A very difficult component is trying to obtain accurate, unbiased information from stakeholders and those providing data from instruments like surveys. The following table lists some of the pros and cons.

Pros
All related tasks are assigned a specific dollar amount. Supervisors can then decide if the task is worth the price. If not, it can be cut or modified. 

Once the dollar amounts are selected, it is a simple matter of adjusting them as necessary due to increased to decreased costs. 

Internal and external experts can be used to complete this plan.   

This strategy uses estimates from a variety of resources (participants, supervisors, HR, senior managers, etc.) to create a complete picture and assign monetary values. 
There are a variety of approaches that can be used to measure ROI and, so it is possible to incorporate it into management, consumer, or participant evaluation plans.

Companies are able to see if the programs they create or the trainings they sponsor are truly having a positive fiscal impact on their profits.
Internal and external experts can be used to complete this plan.

Cons
The trainimg program in question must be in effect for some time before ROI can be calculated. 

It takes more time and money to calculate ROI as an evaluation than it does to conduct another form of evaluation.

 
It is difficult to isolate the effects of specific training or programs and provide accurate estimates.

 
Staff must be trained to calculate and/or interpret the ROI of a program. It is a very intensive strategy to implement and staff can lose steam rather quickly if there are no specific deadlines in place for them.

 
There is the fear of a negative ROI. Many do not want it proven that a training program is actually bad for the company. 
It is only a best estimate. There is no way to actually prove ROI.

Barriers to ROI. (n.d.) Retrieved June 29, 2001 from
http://www.astd.org/CMS/templates/index.html?template_id=1&articleid=21220

Sharon Adelgais
San Diego State University
Educational Technology Graduate Student
Fall, 2001