Published: March 29, 2018
The Study
Brinkerhoff Evaluation Institute (BEI) developed a survey to assess different aspects of learning, application of learning, manager engagement, accountability and other aspects of training.
BEI helps organizations achieve their objectives faster by conducting thorough, yet practical, evaluations. BEI operates under the guidance of Prof. Robert O. Brinkerhoff, an internationally recognized evaluation and learning effectiveness expert.
Read more about BEI at http://www.brinkerhoffevaluationinstitute.com/
Download PDF
Read our privacy policy here.
PNS 2018
Welcome to Promote Network Summit 2018. Hosted in central Stockholm June 27-28 at the beautiful venue of Fotografiska. We are looking forward to inspiring keynotes, innovative talks, educational breakouts, hands-on lessons, and great networking.
![](https://promoteint.com/wp-content/uploads/2017/07/DSC_0094-600x401.jpg)
Inspiring keynotes
Get inspired by top speakers. Walk away with tools to do your job better and inspiration to do great things.
![](https://promoteint.com/wp-content/uploads/2017/03/brinkerhoff-speaking1-liten-liten-600x347.jpg)
Educational session tracks
Last PNS we introduced different session tracks based on interest and role. The feedback has been very positive so we are expanding the tracks for 2018.
![](https://promoteint.com/wp-content/uploads/2017/04/pns-group-600x450.jpeg)
Networking
PNS is all about the Promote Network where you play an important part. The agenda is packed with networking opportunities and social events.
![](https://promoteint.com/wp-content/uploads/2017/03/apple-600x373.jpeg)
Pre Conference Certification (June 25-26)
Get your High Performance Learning Journeys certification, when in Stockholm at the PNS pre-conference. We are offering a great bundle price. In this certification program designed and developed by Robert Brinkerhoff, you will get the tools to make your training programs create new and lasting behaviors among your participants.
Author: Preethi B. Rao
Published: March 14, 2018
Companies spend massive amounts each year to ensure that their employees are working to their full potential and contributing to the accomplishment of organizational goals. But are such companies really getting a return on these expenses?
Below is a very common exchange of dialogue I have with my Kirkpatrick Four Levels® Evaluation Certification Program participants:
Me: What is your training evaluation budget?
Participant: We have a training budget, but not an evaluation budget.
Me: Why not? Does the company not consider evaluation important enough to mark off a specific budget for it?
Participant: It’s definitely important. But the budget for evaluation is included in the overall training budget and not mentioned as a separate line-item.
These discussions with training budget managers (which includes CEOs, CLOs, and COOs) has helped me realize that:
- Learning evaluation is often an after-thought; a knee-jerk reaction when asked the value of conducting the training
- Evaluations are usually conducted at the Reaction and Learning levels, which unfortunately do not indicate the true value of training
- The purpose of an evaluation is most often to ensure that training budgets are not reduced, rather than to show any real value to the business
- Evaluation beyond smiley sheets is considered a vast and complex task that is simply too difficult to achieve
- In general, managers of learning budgets, learning professionals, and organizations have a very misguided understanding of what learning evaluation entails
So, what exactly is learning evaluation, rather what is it not?
1. Learning evaluation is not something you think about after the intervention comes to an end
True and complete learning evaluation begins with the end in mind. Here is what really works:
When business stakeholders come to learning managers with a requirement, the learning manager must put on their consultant hat and try to get answers to the following questions:
- Why is the intervention needed now?
- What is currently going well, and what is not going so well?
- What will happen if we do not proceed with the intervention?
- What qualitative and quantitative indicators will demonstrate training success?
- What will success look like on the job?
Most importantly, Learning and Development (L&D) managers will also reach an agreement with business stakeholders about the level of involvement expected from them. Here’s a good rule of thumb to follow for planning and executing evaluation.
2. The purpose of learning evaluation is not just to prove that L&D rocks!
The key purpose of learning evaluation is to determine whether there has been any behavioral change of the participants on the job. Although saving money and perhaps increasing business might be the end goals, these really can’t be achieved unless there is visible behavior change on the job. To achieve behavioral change, not only do we have to plan for evaluation, but we should also factor in support mechanisms (such as coaching, follow-up modules, executive modeling, etc.) that will enable the behavior change.
3. Evaluation is not just surveys and smile sheets
While smiley sheets may tell us how the participants reacted, and even whether they learned something, the surveys don’t really provide enough information to determine if there is a behavioral change on the job post-training. Some of the leading indicators you can bank on to provide this instead are:
- Observations on the job
- Conducting focus groups post-training
- Identifying and monitoring key, early indicators
This will allow us to monitor, adjust, and make corrections along the way to ensure that participants have enough support to make the transition towards new behavior seamlessly.
4. Evaluation is not just the job of L&D
Most often, it is assumed that both training and its evaluation is the sole responsibility of the L&D team. Nothing can be further from the truth. As per The Kirkpatrick Partners, “Performance on the job is the responsibility of training participants and business managers; preparing them for their roles before, during and after training is the responsibility of training professionals.”
Here is how the L&D team can be supportive in preparation, before the program:
- By understanding the business’s true needs
- By arriving on success factors
- By agreeing on critical behaviors
The preparation for evaluation during the program can be by providing a learning solution that truly resonates with the participants and makes them want to commit to change.
Provide support after the program by helping create systems and process to ensure that the post-training support activities are carried out by line managers and supervisors. Also, ensure that there are built-in course corrections if the behavior doesn’t change as expected.
5. Evaluation is super complicated
It really doesn’t need to be! Keep a watch out for my next blog in which I will reveal some quick and easy things to do for different levels of evaluation.
So, how have you experienced learning evaluation within your organization? Does this resonate with you?
Read the original article by Preethi B. Rao from C2C Organizational Development here: https://c2cod.wordpress.com/2018/03/14/what-is-learning-evaluation-really/
Author: Ian Townley
Published: February 18th, 2018
I am currently undertaking Brinkerhoff certification for High Performance Learning Journeys. I’d like to take this opportunity to share with you what I have learned so far. Feel free to comment and leave me a message. I’ll try and reply to everyone who does.
For many learning strategists and designers, and even those executives who invest in learning, there is often a tendency to focus on the learning event. Those events take many forms, from extended multiple-day workshops to single e-learning courses. Although, those events are often well intentioned and well designed, they usually lack a significant piece of design. Namely learning transfer.
Learning transfer is, as some of you might know, a topic close to my heart, and can be described as a strategy for increasing the likelihood that learning will be applied on the job. This is important for learners/users of learning because they can see their own personal development and the value of investing time in activities away from the workplace. It’s also important for senior executives and managers because they can see growth. Not just the growth of their people, but also the growth of their business through learning initiatives that are well matched to evolving business needs.
There are many theories about how to design and implement learning transfer. One interesting theory I’ve been looking at on this course is called a High Performance Learning Journey. The idea was developed by Robert Brinkerhoff and supported by an online platform called Promote.
So, how does a HPLJ differ from a usual learning event? Well in short, the participants go through a number of stages, which are called Core Elements:
- They commit to the learning journey to increase motivation and attendance
- They build a knowledge foundation at the right pace and intervals
- They have time to develop and practice skills in a safe and controlled environment
- They have opportunities to strengthen results through activities that support application
In order to meet those four core elements the learning is stretched through five dimensions: time, business linkage, spaces, relations, tools and structure. The idea is that if the five dimensions are addressed adequately for a particular learning and matched to a particular audience, then learning will be understood, become sticky, get adopted, and be applied.
In this article I won’t delve deeply into the theory. Instead, what I will do is describe what a HPLJ is like from the perspective of a participant, because naturally the course is designing as a HPLJ.
I’ve noticed three things that I think are important, and which I have found highly valuable.
First, because I’ve gone through the process of creating my own learning journey for this course I can see clearly the application of learning, and my business rationale for investing in this initiative. In other words, my motivation got a massive boost once it became clear how important understanding and using the content was to me.
Second, through stretching the learning over the dimensions of time and space, and using appropriate tools, I’ve been able to absorb the learning at the right amount and speed needed to build my learning scaffold. I honestly feel confident that at each stage I’m improving and developing. And, low and behold, feel like I can and will actually use what I learn.
Third, there is a big variety of attendees on the course with me. We’re spread over the world, so we have a truly global view. Talk about leveraging knowledge and experience of relations developed through a shared need and goal.
This is only the story so far. So look out for updates and other articles from me about HPLJs and Learning Transfer in general.
Thanks.
Original article by Ian Townley can be found here
Professor Robert Brinkerhoff is an internationally recognized expert in learning effectiveness and evaluation – you can find his page on LinkedIn here, or read some of his work here.
Promoteint.com is an online platform dedicated to creating high performance learning journeys and learning transfer initiatives, read more here.
![](https://promoteint.com/wp-content/uploads/2017/03/apple.jpeg)
![](https://promoteint.com/wp-content/uploads/2017/04/5-steps-to-create-higher-learning-transfer.png)
Download
Read our privacy policy here.
Attract new customers or keep the ones you have!
Download this easy to use, step by step template to enhance existing training programs or create new ones together with your customers.
![](https://promoteint.com/wp-content/uploads/2017/12/landingpage_program_improvement.jpg)
Use the template to attract new customers
(or to keep existing ones – given the competitive market)
Download template
Great for your business: If you are selling training solutions, offering a Program Improvement Workshop is a great way to attract new customers. It should also be done on a regular basis with existing customers to grow your business and avoid competition to take your place.
Read our privacy policy here.
Author: Martin Nilsson, Promote International
We could really feel that autumn was upon us when we arrived at Traefpunkt HR, Denmark’s largest HR Exhibition. Strong winds and a light shower from dark clouds. However, as soon as we entered the beautiful old meat market building spring was in the air. The venue was buzzing and the exhibitor list witnessed about quite a diversity. A good place to meet like-minded HR-professionals and talk about learning transfer.
Â
The Promote booth was of course set up to give live demos of the Promote platform. But this year we also focused on the Brinkerhoff certification – High Performance Learning Journeys. Brinkerhoff is well known in Denmark so the interest was high. There will probably be customer internal certifications in Copenhagen before Christmas and in due time open seat certifications in Denmark.
Â
 ![](https://promoteint.com/wp-content/uploads/2017/10/dansk-1-300x225.jpg)
Trends:
High pace and just in time
One apparent trend at Traefpunkt HR this year was the focus on efficiency. There is just not time anymore for inefficient onboarding, inefficient training and using “strategy of hope” for following up on results.
Many organizations were talking about micro learning, and one can argue about the definition and execution of micro learning but it’s obvious the traditional view on classroom training and tedious e-learning is challenged. Also when it comes to the concept of onboarding a clear trend was that to get new recruits up and running faster and more efficient.
This is of course what Promote is all about. Efficient training and onboarding with visible results to show for it.
Information security and data privacy
As almost all HR-processes are becoming digitized the new European data privacy legislation was very apparent at the expo. A lot of the seminars had this as a topic and almost all cloud solution providers had information about their work with GDPR.
Relocation
Another trend was workforce relocation. In this global era and despite (or maybe because of) the possibilities to collaborate from a distance today the skilled workforce is mobile. When people are moving to and from different countries and cultures that poses challenges for organizations. Both seminars and exhibitor booths presented challenges and solutions.
Health
Many of the exhibitors were dealing with health and the work environment from different perspectives. From healthy eating and quality of sleep to conflict resolution toolsets and large HR-platforms with tracking of the organization’s wellbeing.
We would like to thank everyone that we hade opportunity to talk to and are looking forward to some great new cooperations.
Author: Linnea Hakansson
During two intense and sunny days, the Promote Network met in Sweden for the annual Promote Network Summit, PNS2017. We had 100+ engaged participants and people flying in from all over the world.
We had the opportunity to listen to some really great speakers during these two days and what we have heard the event was the best Promote Network Summit ever.
Day 1 started with Jon Serrander, the CEO of Promote, welcoming all delegates and taking us on a trip down memory lane showing the earlier versions of Promote and sharing the story behind the platform. Professor Robert Brinkerhoff told the story how he first came in contact with the Promote team and why he decided to join.
The first keynote speaker was Robin Hoyle, Head of Learning Innovation from Huthwaite, talking about organizational engagement and sustainable learning.
After an “Asian-fusion lunch” and some networking mingle, all the participants then attended four breakout sessions of their choice, out of six available:
- “Introducing Brinkerhoff Certification for High Performance Learning Journeys” with Robert Brinkerhoff
- “Customer Case – Global leadership program” with Isabell Sandström from Stena Metall
- “Customer Case – Sending Execs Back to School: The Deutsche Telekom Story” with Steve Mahaley and Emma Holbrook from Duke CE
- “Driving Business: How to sell training solutions based on Promote” with Mikael Borak and Karin Plith from Promote
- “Promote Good Practice: Review your program” with Elin Lennartsson from Promote
- “Evaluating your Program using data from Promote” with Edward Boon from Promote
The venue for day 1 was Gamla Riksarkivet, a building from the 1800-century with a unique architecture. The evening was spent on their courtyard with dinner, drinks and the annual Promote Network Summit awards handed out by Jon Serrander at the stage.
The winners for the best Promote programs were:
Getinge with “Getinge Infection Control Product Sales Training”
A very well balanced program, good variation of assignments and a clear intention towards the application. Good use of announcements particularly liked the way you included a competitive element to encourage pre-work completion.
Informator with “ITIL Foundation Optimized”
The program had a clear process design, great use of resources and a very nice look and feel. This is a great example of how Promote can be utilized for more than traditional training events.
Brainard with “EXCELerate”
A well-structured program with a clear learning journey, a good mix of assignments types leading to a very high participant completion rate.
Day 2 started off in a windy yet sunny Stockholm on a boat (!)…
The first Keynote Session on Thursday was held by Peter Sheppard, Global Head of Learning Excellence at Ericsson. Peter talked about the Learning Challenges in Large Organisations.
Professor Robert Brinkerhoff followed right before the coffee break and talked about the Promote Impact Study conducted by Brinkerhoff Institute. A very interesting key note and we are excited to share this study with all of you very soon.
Robin Hoyle from Huthwaite explained why we should flip the 70-20-10 and after a lunch break with a Swedish summer theme, there were three different breakout sessions.
- “Team.bio: A simple digital tool for managers designed by millennials” with Foucauld Guerin and Nicolas Blondeau. This is a new project within the Promote organization, and if you want to read more and get some information, please visit the Team.bio website https://www.team.bio/
- “The levers of transfer effectiveness” by Dr Ina Weinbauer-Heidel from the Institute for Learning Transfer
- “Ericsson Case Study – Selling Promote within the organization” with Britt-Marie Flemström
Last Keynote was by the very inspiring and well-spoken Steve Mahaley from Duke CE who shared his views on learning in “The Yin and Yang of Learning Technologies – Uncovering the Possible”. Last but not least was Jon Serrander with some trends and a sneaky peek of what’s next for the Promote platform.
The boat set off for a dinner cruise in the Swedish archipelago and returned at 9 PM for some partying, networking and dancing.
Thank you all for a lovely time, we have had such a great year at Promote with all of our clients and partners. See you at PNS 2018!
And be sure to keep in touch to get information about all the new features and functionalities coming this year.
Looking forward to next year already!
Promote Network Summit 2017
Author: Professor Robert Brinkerhoff
Everybody knows that training is supposed to be a process, not an event. We also know that training on its own doesn’t hit the mark, it should have impact, it should drive performance improvement, 70:20:10 should be respected since informal learning on the job matters – a lot of social learning is powerful and popular managers and participants should be engaged together, training should be shorter and digitalized.
It is relatively easy to tweak our current L&D offerings to respond to one or maybe even two of these new impact-enhancing elements, such as breaking a multi-day workshop into two parts, and adding some on-the-job applications in between session. But, incorporating many of the performance improving aspects into our L&D intervention designs is a challenge, a challenge that takes a good deal of design expertise and energy.
Designing a process where the design of a training initiative combines all the things needed to create high performance is another thing entirely from designing a traditional course.
We call this new challenge High Performance Learning Journeys (HPLJ) design.
- Performance, because the focus is on improving on-job performance, not just “learning”
- High Performancebecause we are after sustained performance improvement that consistently produces results the organization needs to execute its strategy, not just an occasional application of a new skill.
- Journey because it stretches time boundaries over weeks and more typically months.
- Learning because we know that true learning requires multiple cycles of acquiring new knowledge and skill, followed by application try-outs, followed by reflection and feedback.
The research that underlies High Performance Learning Journeys is based on my extensive hands-on encounters with training participants and their managers for over 40 years determining what is actually working, and what is not, in getting the desired impact out of training programs. HPLJ is a way to put together all that stuff we know makes a difference into a practical and effective way of working that is scalable and doesn’t break your back.
We have been working for several years with the HPLJ process, we’ve studied and evaluated it to be sure that it works and to get the kinks out of it. (And there are indeed kinks, and few easy solutions). We made a decision to boil it down into a practical learning experience that others can learn about and use. We now have HPLJ certified practitioners that are out there using the process and they are finding it to be very impactful; the even better news is that this growing group of pioneers is already innovating improvements that make the HPLJ approach even stronger and more effective. We will always have a need to learn from applications for the continuous HPLJ improvement.
So our HPLJ efforts are themselves a journey a true learning journey that has changed our performance for good. We can’t go back now. We know we are on the right track, and others are picking up the learning journey concept and promoting their solutions. This tells us that we are the front edge of a wave of innovation in the L&D profession. Where this journey goes next is exciting to think about and be a part of.
The HPLJ approach has been proven to dramatically increase the impact of training – by a factor of 3 to 4 – on workplace performance.
Without recognizing the four core elements of learning design, you will create training programs that are an ongoing expense without the prerequisites to provide value. You will find that you will benefit from using technical platforms to support your programs, but it is not a necessity to have technical platforms for taking part in the program or making use of your knowledge.
The Brinkerhoff Certification Program for High Performance Learning Journeys has the right tools to improve your programs and your outcomes!
![](https://promoteint.com/wp-content/uploads/2017/03/promote-cares-about-privacy.jpg)
Data processing and storage within the European Union is regulated, for security and privacy reasons.
However, member states may have additional regulations (e.g. Germany and the United Kingdom). Promote is designed and operated to ensure trust from our customers / users, and to adhere to legal regulations, where the service is offered. We also require, by agreement, that all usage and content generated within Promote, does not violate any privacy or security regulations.
Privacy
Sensitive personal data may not be submitted to, or stored, in Promote. The Promote basic framework does not process, or store, data about racial or ethnic origin, political opinions, religious or philosophical convictions, union membership, health status or sex life, as the European Commission has stated this to be particularly sensitive. Therefore, Promote customers and / or users, are responsible to ensure that all data submitted within the Promote system, does not violate any of the aforementioned privacy regulations.
Geographic location of data storage
All data centers, serving Promote customers targeting users in the European Union, are located within the territory of that member state.
Promote customers are themselves responsible for ensuring that the geographical location of a user, invited onto the Promote platform, does not violate any EU data transfer regulations.
Data protection
Data access control:
Promote has role based security architecture that controls the data access.
Logical access control:
All data in Promote is protected by user authentication. Promote uses well proven industry standard solutions for password protection, third party authentication as well as single sign on.
Data transfer control:
Data transfer in Promote is, by default, encrypted according to industry standards.
Data entry control:
All Promote logins are logged with time and current IP addresses.
Availability control:
All data within Promote is backed up on a daily interval basis and the complete platform can be restored, after accidental destruction or loss.
Physical protection
Production data centers used to provide the Promote services are ISO 27001 certified and offer industry standard security solutions. This includes physical access control, durability for natural conditions and power failure prevention.