Training Evaluation

Training evaluation is a continual and systematic process of assessing the value or potential value of a training program, course, activity or event.  Results of the evaluation are used to guide decision-making around various components of the training (e.g. instructional design, delivery, results) and its overall continuation, modification, or elimination. 

To assist agencies in evaluating their training programs, OPM published the 2011 OPM Training Evaluation Field Guide. The Training Evaluation Field Guide is designed to assist agency training representatives (i.e., training managers and supervisors, training liaisons/coordinators, agency evaluators, instructional designers, training facilitators and others who have a significant role in training effectiveness) in evaluating training effectiveness and in demonstrating training value to stakeholders and decision makers. Data and information were gathered from fifteen federal agency representatives who volunteered their time to attend a one-day working meeting, participate in individual interviews and submit samples of their tools and case studies. This Field Guide reflects the input from the working group.  The OPM Training Evaluation Field Guide Supplement is an abbreviated version of the OPM Training Evaluation Field Guide.

 

View this quick video for more information on the training evaluation field guide and ways agencies can use evaluation and training data to inform decisions related to training investments.


Logic Models:

Logic models are very easy tools to help you plan your training program. A logic model provides a representation of a "theory of change" (if...then) that clearly aligns the training inputs and activities to the outputs and results.  Logic models are created in the planning phase and can be completed for programs, initiatives, individual courses, events and activities.  For more detailed information on logic models, you can watch this video or take EPA's FREE Logic Modeling Course.

 
Contact Cheryl.Ndunguru if you would like training on how to use the OPM Training Evaluation Field Guide and/or Logic Models in your agency.

Regulations

Agencies are required to evaluate their training programs annually to determine how well such plans and programs contribute to mission accomplishment and meet organizational performance goals (5 CFR 410.202).  The law authorizes OPM to require Federal agencies to report training data (Example: Sample Agency Training Report FY2010).  An important part of the evaluation process involves consideration of training costs, and other elements not directly addressed in the typical evaluation.  Agencies must track and report accurate training data on all completed training events to OPM as prescribed by the Final Rule on Training Reporting Requirements, published on May 17, 2006, in the Federal Register, The Federal Workforce Flexibility Act of 2004 (P.L. 108-411).

Agencies can reference the Guide for Collection and Management of Training Information for detailed information on reporting training data.


Other Evaluation Methods:
The Training Evaluation Field Guide uses the Kirkpatrick Model of evaluation as the basis of instruction.  However, there are other viable and well researched models available for use.  Agencies should use the model that more closely meets their needs.

 

 

 

 

 

 

 

 

Other Helpful Resources

Logic Model Template for Supervisory & Managerial Training:   This logic model provides agencies with a systematic and visual aid to identify a training program’s components: inputs, outputs, outcomes and evaluation methodologies, and illustrates the relationship among them.  The logic model has been prepopulated with Federal supervisory and managerial curriculum-specific examples for agencies to reference when creating their agency-specific logic model.  Once completed, the logic model will provide stakeholders with a roadmap that describes the sequence of related events.  Stakeholders will also have a clear understanding of the curriculum goals and evaluation. 

Note:  The Federal Managerial Curriculum Logic Model components have been color-coded to map to each of the stages included in the OPM Training Field Guide Supplement. 

Certifications: Both Kirkpatrick and Phillips offer training evaluation certificates although a certified evaluator is not necessary to evaluate the effectiveness of agency training. The 2011 OPM Training Evaluation Field Guide, books and/or training courses on performance measurement, program evaluation, and training evaluation should provide enough information to successfully evaluate your agency training.

Associations: The American Evaluation Association (AEA) is an international professional association of evaluators devoted to the application of many other forms of evaluation. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. The AEA has approximately 5500 members representing all 50 states in the US as well as over 60 foreign countries.

 

FAQs:

WHAT should you evaluate?

Training evaluations can help the organization to reach many different goals during the life cycle of a training program.  One primary reason to evaluate is to determine if the benefits derived from the training justified the costs.

Some additional reasons include:


WHY
should you evaluate?

Aside from the requirement in 5 CFR 410.202, agencies face very real demands to demonstrate training program efficiency, program effectiveness and public accountability. Use of evaluation data meets these demands in various ways:

Planning: To assess needs, set priorities, direct allocation of resources, and guide policy

Analysis of Course/Program Effectiveness or Quality: To determine achievement of objective, identify strengths and weaknesses of a program/course, determine the cost-effectiveness of a program/course, and assess causes of success or failure

Direct decision-making: To improve effectiveness, identify and facilitate needed change, and continue, expand, or terminate a program/course

Maintain accountability: To stakeholders, funding sources, and the general public

 

WHEN should you evaluate?

There are several basic questions to ask when deciding when to carry out an evaluation. If the answers to these questions are "Yes", this may be the time to evaluate.

 

HOW can you evaluate?

Once you've determined whether or not your program or course warrants evaluation, there are various methods and models agencies can use to evaluate their training courses. Here are two of the most popular: (You can find other evaluation methods above under “Other Evaluation Methods”)

 

Kirkpatrick 4 Levels

The four levels of Kirkpatrick's evaluation model essentially measure:

All these levels are recommended for full and meaningful evaluation of learning in organizations.

 

Jack Phillips' Five Level ROI Model

Building upon the Kirkpatrick model, Jack Phillips added the fifth level the Return on Investment (ROI) produced by a training course using the financial formula:

ROI(%) = (Net Program Benefits/Program Costs) x 100