The Ultimate Question: Does Your Learning Program Even Matter?
I give you the the Ultimate Learning Analytics Maturity Model.
Reading Time: ~ 4 min
Morning folks!
Today, I will teach you how to measure your learning Return on Investment (ROI)!
By understanding how learning analytics actually work, you will know how to distinguish the true effect of learning versus glamour metrics. This will allow you to evaluate the ROI of learning and make decisions about your L&D strategy.
Where do you start?
With measuring learning!
Why?
Well, what gets measured gets done. Duh!
Hence, my Ultimate Learning Analytics Maturity Model has 4 horizons of learning measurement:
Learning Activity
Learning Efficiency
Learning Effectiveness, and
Learning Impact
Let’s dive in!
1. Learning Activity
This is the lowest level of maturity and is the easiest to measure. Indeed, most organizations with learning programs measure the activity associated with the learning:
Number of learning hours
Number of eLearning courses completed
Number of people enrolled in the learning program, and
L&D spend
While these metrics are important, they are missing what actually matters:
Does this learning actually do anything for me?
Or, put differently, why should I give a rat’s ass?
This is where organizations turn to Learning Efficiency.
2. Learning Efficiency
When looking at Learning Efficiency, we are really looking at how much learning has been absorbed.
Indeed, you can “watch” an hour of learning on LinkedIn, Udemy, or Coursers in the background while playing a video game in the hybrid world.
Hence, how do you know people actually retained what you are trying to train them on?
My beef with competency frameworks aside, most organizations use them to measure how much information was retained by learners. Some orgs even gate your career progression until you complete some arbitrary courses. Others create tests and assessments to determine how close you are to self-actualization in competency called “feedback.”
But being self-actualized in feedback does not mean your learning actually results in business outcomes! It just means you learned something.
Using learning something as ROI is where most organizations stop.
But how do you measure the real ROI if you really really wanted to? I am talking business results.
3. Learning Effectiveness
This is where real analytics starts.
At this stage, you are unsatisfied with self-actualized alone. You are finally asking—does any of this even matter?
To be successful at this stage, you need data, and you need some predictive analytics or, in simple statistics terms, data modelling.
Namely, you want to start with collecting data around:
Learning activity (e.g., hours spent)
Learning efficiency (e.g., competency level attained)
Individual differences (e.g., performance, potential)
Employee data (e.g., departments)
Outcomes (e.g., productivity, quota attainment)
And then, you build your model, regressing outcomes on learning metrics and other things like individual differences.
At the end of this exercise, you should be able to say the amount of time spent on learning is negatively correlated with quota attainment, for example.
Why might that be?
Maybe people are spending too much time learning and not running sales calls.
In this case, maybe axing learning is not such a bad idea. Or at least change it in some ways.
You could also find that all your competencies are really an iteration of the same thing. Maybe simplify your competency model!
All in all, the Learning Effectiveness Level is the level of insight and ROI.
But there are limits to this god-like power: the data are only correlational. So, multiple things can affect why the learning is or is not working.
4. Learning Impact
This stage is the holy grail of learning ROI measurement.
This is where you can say training people on feedback improved their sales by 50%.
And to do this, you need a proper experiment. This means measurement before and after AND random assignment of half of your sales folk into a group receiving learning and the other half waiting for learning (control).
Using an experimental design and random assignment allows you to conclude whether learning matters definitively.
Most companies probably will never get to levels 3 and 4.
But, you 100% will make better decisions for your company if you are at the higher levels of the maturity model.
See you later, alligator!
Whenever you're ready, there are 2 ways I can help you:
1. If you're still looking to get into People Analytics to build a data-driven HR function, start with this super affordable, hands-on course:
👉 Practical People Analytics: Transform your HR into a data machine with Practical People Analytics. This comprehensive course will teach you the system I used to build a strategic and data-driven HR function with minimal tech. Check out the course here!
2. Need help with people analytics? Engagement, performance, compensation, DEI? Shoot me an email at konstantin@strategicpeopleanalytics.com. Let’s discuss your challenge and see how we can solve it.