The Study

Brinkerhoff Evaluation Institute (BEI) developed a survey to assess different aspects of learning, application of learning, manager engagement, accountability and other aspects of training.

BEI helps organizations achieve their objectives faster by conducting thorough, yet practical, evaluations. BEI operates under the guidance of Prof. Robert O. Brinkerhoff, an internationally recognized evaluation and learning effectiveness expert.

Read more about BEI at http://www.brinkerhoffevaluationinstitute.com/

Companies spend massive amounts each year to ensure that their employees are working to their full potential and contributing to the accomplishment of organizational goals. But are such companies really getting a return on these expenses?

Below is a very common exchange of dialogue I have with my Kirkpatrick Four Levels® Evaluation Certification Program participants:

Me: What is your training evaluation budget?

Participant: We have a training budget, but not an evaluation budget.

Me: Why not? Does the company not consider evaluation important enough to mark off a specific budget for it?

Participant: It’s definitely important. But the budget for evaluation is included in the overall training budget and not mentioned as a separate line-item.

These discussions with training budget managers (which includes CEOs, CLOs, and COOs) has helped me realize that:

• Learning evaluation is often an after-thought; a knee-jerk reaction when asked the value of conducting the training
• Evaluations are usually conducted at the Reaction and Learning levels, which unfortunately do not indicate the true value of training
• The purpose of an evaluation is most often to ensure that training budgets are not reduced, rather than to show any real value to the business
• Evaluation beyond smiley sheets is considered a vast and complex task that is simply too difficult to achieve
• In general, managers of learning budgets, learning professionals, and organizations have a very misguided understanding of what learning evaluation entails

So, what exactly is learning evaluation, rather what is it not?

1. Learning evaluation is not something you think about after the intervention comes to an end

True and complete learning evaluation begins with the end in mind. Here is what really works:

When business stakeholders come to learning managers with a requirement, the learning manager must put on their consultant hat and try to get answers to the following questions:

• Why is the intervention needed now?
• What is currently going well, and what is not going so well?
• What will happen if we do not proceed with the intervention?
• What qualitative and quantitative indicators will demonstrate training success?
• What will success look like on the job?

Most importantly, Learning and Development (L&D) managers will also reach an agreement with business stakeholders about the level of involvement expected from them. Here’s a good rule of thumb to follow for planning and executing evaluation.

2. The purpose of learning evaluation is not just to prove that L&D rocks!

The key purpose of learning evaluation is to determine whether there has been any behavioral change of the participants on the job. Although saving money and perhaps increasing business might be the end goals, these really can’t be achieved unless there is visible behavior change on the job. To achieve behavioral change, not only do we have to plan for evaluation, but we should also factor in support mechanisms (such as coaching, follow-up modules, executive modeling, etc.) that will enable the behavior change.

3. Evaluation is not just surveys and smile sheets

While smiley sheets may tell us how the participants reacted, and even whether they learned something, the surveys don’t really provide enough information to determine if there is a behavioral change on the job post-training. Some of the leading indicators you can bank on to provide this instead are:

• Observations on the job
• Conducting focus groups post-training
• Identifying and monitoring key, early indicators

This will allow us to monitor, adjust, and make corrections along the way to ensure that participants have enough support to make the transition towards new behavior seamlessly.

4. Evaluation is not just the job of L&D

Most often, it is assumed that both training and its evaluation is the sole responsibility of the L&D team. Nothing can be further from the truth. As per The Kirkpatrick Partners, “Performance on the job is the responsibility of training participants and business managers; preparing them for their roles before, during and after training is the responsibility of training professionals.”

Here is how the L&D team can be supportive in preparation, before the program:

• By understanding the business’s true needs
• By arriving on success factors
• By agreeing on critical behaviors

The preparation for evaluation during the program can be by providing a learning solution that truly resonates with the participants and makes them want to commit to change.

Provide support after the program by helping create systems and process to ensure that the post-training support activities are carried out by line managers and supervisors. Also, ensure that there are built-in course corrections if the behavior doesn’t change as expected.

5. Evaluation is super complicated

It really doesn’t need to be! Keep a watch out for my next blog in which I will reveal some quick and easy things to do for different levels of evaluation.

So, how have you experienced learning evaluation within your organization? Does this resonate with you?

Read the original article by Preethi B. Rao from C2C Organizational Development here: https://c2cod.wordpress.com/2018/03/14/what-is-learning-evaluation-really/

I am currently undertaking Brinkerhoff certification for High Performance Learning Journeys. I’d like to take this opportunity to share with you what I have learned so far. Feel free to comment and leave me a message. I’ll try and reply to everyone who does.

For many learning strategists and designers, and even those executives who invest in learning, there is often a tendency to focus on the learning event. Those events take many forms, from extended multiple-day workshops to single e-learning courses. Although, those events are often well intentioned and well designed, they usually lack a significant piece of design. Namely learning transfer.

Learning transfer is, as some of you might know, a topic close to my heart, and can be described as a strategy for increasing the likelihood that learning will be applied on the job. This is important for learners/users of learning because they can see their own personal development and the value of investing time in activities away from the workplace. It’s also important for senior executives and managers because they can see growth. Not just the growth of their people, but also the growth of their business through learning initiatives that are well matched to evolving business needs.

There are many theories about how to design and implement learning transfer. One interesting theory I’ve been looking at on this course is called a High Performance Learning Journey. The idea was developed by Robert Brinkerhoff and supported by an online platform called Promote.

So, how does a HPLJ differ from a usual learning event? Well in short, the participants go through a number of stages, which are called Core Elements:

• They commit to the learning journey to increase motivation and attendance
• They build a knowledge foundation at the right pace and intervals
• They have time to develop and practice skills in a safe and controlled environment
• They have opportunities to strengthen results through activities that support application

In order to meet those four core elements the learning is stretched through five dimensions: time, business linkage, spaces, relations, tools and structure. The idea is that if the five dimensions are addressed adequately for a particular learning and matched to a particular audience, then learning will be understood, become sticky, get adopted, and be applied.

In this article I won’t delve deeply into the theory. Instead, what I will do is describe what a HPLJ is like from the perspective of a participant, because naturally the course is designing as a HPLJ.

I’ve noticed three things that I think are important, and which I have found highly valuable.

First, because I’ve gone through the process of creating my own learning journey for this course I can see clearly the application of learning, and my business rationale for investing in this initiative. In other words, my motivation got a massive boost once it became clear how important understanding and using the content was to me.

Second, through stretching the learning over the dimensions of time and space, and using appropriate tools, I’ve been able to absorb the learning at the right amount and speed needed to build my learning scaffold. I honestly feel confident that at each stage I’m improving and developing. And, low and behold, feel like I can and will actually use what I learn.

Third, there is a big variety of attendees on the course with me. We’re spread over the world, so we have a truly global view. Talk about leveraging knowledge and experience of relations developed through a shared need and goal.

This is only the story so far. So look out for updates and other articles from me about HPLJs and Learning Transfer in general.

Thanks.

Original article by Ian Townley can be found here

Professor Robert Brinkerhoff is an internationally recognized expert in learning effectiveness and evaluation – you can find his page on LinkedIn here, or read some of his work here.

Promoteint.com is an online platform dedicated to creating high performance learning journeys and learning transfer initiatives, read more here.

Data processing and storage within the European Union is regulated, for security and privacy reasons.

However, member states may have additional regulations (e.g. Germany and the United Kingdom). Promote is designed and operated to ensure trust from our customers / users, and to adhere to legal regulations, where the service is offered. We also require, by agreement, that all usage and content generated within Promote, does not violate any privacy or security regulations.

Privacy

Sensitive personal data may not be submitted to, or stored, in Promote. The Promote basic framework does not process, or store, data about racial or ethnic origin, political opinions, religious or philosophical convictions, union membership, health status or sex life, as the European Commission has stated this to be particularly sensitive. Therefore, Promote customers and / or users, are responsible to ensure that all data submitted within the Promote system, does not violate any of the aforementioned privacy regulations.

Geographic location of data storage

All data centers, serving Promote customers targeting users in the European Union, are located within the territory of that member state.

Promote customers are themselves responsible for ensuring that the geographical location of a user, invited onto the Promote platform, does not violate any EU data transfer regulations.

Data protection

Data access control:

Promote has role based security architecture that controls the data access.

Logical access control:

All data in Promote is protected by user authentication. Promote uses well proven industry standard solutions for password protection, third party authentication as well as single sign on.

Data transfer control:

Data transfer in Promote is, by default, encrypted according to industry standards.

Data entry control:

All Promote logins are logged with time and current IP addresses.

Availability control:

All data within Promote is backed up on a daily interval basis and the complete platform can be restored, after accidental destruction or loss.

Physical protection

Production data centers used to provide the Promote services are ISO 27001 certified and offer industry standard security solutions. This includes physical access control, durability for natural conditions and power failure prevention.