Buyers of training have started asking what changed once the course finished. Certification volume does not answer this, and neither does a satisfaction score. Research published over the last five years provides a reasonable account of why most training produces so little change in how people work, and of what must be present in a curriculum for that to be different.
Most of what gets taught goes out of use
A 2025 review in the International Journal of Training and Development gathers the current estimates. Failure to apply new knowledge and skills at work is common; unused skills decay quickly, and estimates of how much acquired learning is lost within a year of training range from about 50% to about 90%. The range is that wide because transfer is hard to measure, and the studies disagree. Even at the low end, half of what a client funded has stopped being used within twelve months.
The reasons sit outside the classroom. Nobody changed the incentives the delegate works under, their manager was not part of the decision to send them, and the first person on a team to do something differently absorbs the disruption while any improvement that follows gets credited elsewhere. None of this is something a training provider can fix, and proposals that imply otherwise are found out at the first review meeting.
What a training provider can affect is the state someone is in when they return. A person who has already had to make the decision, with a consequence they had to answer for, has something available to them on their first day back that reading about the decision does not.
What the research says about simulations
Chernikova and colleagues published a meta-analysis in the Review of Educational Research in 2020 that covered 145 empirical studies of simulation-based learning. Simulations produced a large overall effect, g = 0.85, and the result held across domains including medical education, teacher education, and management. Both technology use and scaffolding had positive effects. Prior knowledge determined which type of support worked best: reflection phases did more for learners who already knew a good deal, and worked examples did more for those who did not.
Faisal and colleagues reviewed a narrower field in Human Behavior and Emerging Technologies in 2022, covering 57 empirical studies of business simulation games published between 2015 and 2022, and reported improvements in knowledge acquisition, cognitive and interactive skills, and behaviour.
Both bodies of work were carried out in higher education with students rather than practitioners, which limits how far the figures travel. The mechanism they describe is the one a corporate simulation relies on, so the principle carries across. However, an effect size drawn from university courses does not describe a facilitated session run for a client’s operations team. A training provider quoting 0.85 in a proposal as a measurement of their own delivery is claiming more than the research supports.
Realism does less than expected
The same team returned to the literature in 2024 and expanded it to 214 studies, seeking the design features that explain why some simulations work better than others. Authenticity, which they defined as functional correspondence with the real task and physical resemblance to the real environment, accounted for only about 2% of the variance in effectiveness. The salience of the relevant information had considerably more influence.
Simulations are usually sold on fidelity, meaning the resemblance between the simulated organisation and a real one, the detail in the financial model and the credibility of the interface. On this evidence, fidelity does very little of the work. What counts is whether the thing a participant is supposed to notice is noticeable to them at the moment it happens. A simulation can reproduce an operation accurately and teach nobody anything, because the connection between a decision taken in the first cycle and the outage arriving in the fourth has been buried under convincing-looking detail.
For a training provider assessing a simulation to add to a portfolio, that moves the questions away from production values. Whether a participant learns what their own decision caused and how soon they learn will have a greater impact on the outcome than how closely the simulated business resembles a real one.
The order the evidence supports
Replacing foundational teaching with a simulation has no support in this research, and the prior-knowledge finding runs counter to it, because what someone already understands determines the kind of support they need once they are inside the exercise. Teaching first, then an applied component that puts the taught material under pressure, is the sequence the evidence backs. It is also the easier sequence for a training provider to sell, since it uses the accredited courses already in the catalogue rather than competing with them.
This sequence gives a training provider an answer in a procurement conversation that a course list cannot supply. Certification establishes the vocabulary and the reference models. The applied component asks delegates to use both while being accountable for a decision, despite limited information and depending on colleagues whose priorities conflict with theirs. Clients who have funded three rounds of certification without identifying anything different about how their teams operate are now receptive to this argument, in a way they were not five years ago.
What a training provider can measure
Completions, pass rates, and satisfaction scores are the measures most training providers hold, and a competent trainer working from good material will produce good numbers on all three without anybody’s behaviour changing at work.
The return on investment (ROI) conversation usually stops there. A training provider has no evidence of application because nothing in the delivery model generates any, and a client cannot confidently attribute an operational improvement to a course from six months earlier. An applied component changes what there is to measure, because the decisions delegates make inside a simulation are recorded and comparable: what a team did in the first cycle against what they did in the fourth, where they invested, what they cut, how they responded once consequences arrived. This describes behaviour inside a session rather than behaviour at work, which is less than a client would like. However, it sits closer to the question than a pass rate does, and it comes from delivery rather than a follow-up survey that nobody completes.
Adding one operational measure from the client’s own reporting, agreed before delivery rather than assembled afterwards, changes the character of the renewal conversation.
What we build
SXP develops and runs business simulations. Teams run a realistic organisation, make decisions about process, technology, and priorities, and see what those decisions do to service quality, cost, and customer outcomes across compressed operating cycles.
I have written about the operational side of this argument in a piece on simulation-based learning in ITSM programmes for ITSM.tools, aimed at the people running services rather than the people selling training. For a training provider, the format belongs after the certification rather than instead of it.
Clients who have paid for several rounds of certification are asking what they got for it. A course, on its own, struggles to answer, for reasons the transfer research sets out, and no amount of syllabus redesign will address. An applied component can answer, provided that consequences reach the people who caused them quickly enough to be recognised as theirs. That condition is a design question rather than a marketing one, and it is the thing to test before adding any simulation to a portfolio.
