Can We Create Experience Deliberately?
Engineering organisations preserve decades of knowledge from past programmes. What if we could use that history not only to give engineers better answers, but to recreate some of the experiences through which expertise develops?
In Everyone Wants Experience. Who Will Create It?, we asked what happens when AI performs some of the junior work through which engineers traditionally become experienced.
One answer is to create some of those experiences deliberately.
What does experience give us?
An experienced engineer does not simply know more. Experience affects how engineers interpret a problem: which assumptions attract their attention, whether a result looks physically plausible, where they expect a model to break down and when a local decision might have wider consequences.
Some of this knowledge can be preserved through lessons learned, design rationale, archived analyses and knowledge databases. AI could make this accumulated knowledge much easier to access.
But access to fifty years of engineering knowledge would not make a graduate a fifty-year aerospace veteran.
Ask an experienced engineer why they challenged a result and the first answer may simply be that something did not look right. Reconstruct the decision and we may discover an unusual combination of loads, a questionable modelling assumption or a similarity to a problem encountered years earlier.
The difference is not simply what experienced engineers know, but how that experience affects what they notice and how they interpret what they see.
If we want to create experience deliberately, we therefore need to capture more than the knowledge that resulted from it.
Preserve the problem, not only the answer
Consider a test failure.
The final report may explain what failed, why it failed and what should be done differently next time. But the engineers facing the original problem had incomplete information. Several explanations seemed plausible, specialists may have disagreed and someone had to decide what to investigate next.
The final report preserves the answer. Much of the experience was in getting there.
So preserve the situation too: what information was available, what seemed plausible, where people disagreed and what eventually changed their minds.
Instead of recording only what the organisation learned, preserve the problem through which it learned it.
Turn problems into cases
Business schools have used case studies for more than a century to let students practise decisions before they have to make them for real.
Much engineering education is built around problems where the task has already been defined and the relevant information provided. Professional engineering is often less tidy.
An unexpected test result does not arrive with all relevant inputs followed by calculate the cause of the failure. The engineer has to decide what matters, what to investigate and when there is enough evidence to act.
Instead of telling a learner that a previous programme discovered a particular modelling assumption produced an incorrect result, give them the model, the test evidence and the information available at the time and ask whether they would accept the analysis.
The first transfers knowledge. The second requires judgement.
Capture judgement and turn it into experience
To create useful cases, we need to understand what experienced engineers actually do differently.
One way is to reconstruct difficult decisions with the people who made them. Rather than simply asking what they know, ask what they noticed at the time, what they expected, when they became concerned and what a less experienced person might have missed.
Methods developed for studying expert decision-making have shown that this can reveal mental models and context-specific knowledge that experts may not otherwise articulate.
The approach is straightforward: find important engineering decisions, understand how experts approached them and turn them into cases for other engineers.
AI could make those cases interactive.
A young engineer might receive only the information available to the original team at a particular point in time and decide what matters and what information to request. The case could respond with a test result, a supplier defending the analysis or another specialist proposing a different explanation. A proposed fix might create another problem elsewhere.
AI could also vary the case and play different participants. It might produce a convincing analysis that contains a subtle error, or provide information that turns out to be irrelevant. This matters particularly if future engineers will be expected to supervise AI-generated engineering: the training environment should not teach them that AI output is necessarily correct or even useful.
Eventually the engineer has to make a decision. Only then do they see what the original team did, why they did it and what happened next.
The value is not simply in discovering whether they reached the same answer. It is in comparing how they approached the problem: what the experienced engineer noticed, which possibilities they considered and what caused them to change their mind.
Try it
To see what this might look like in practice, we built an interactive engineering case from a published NASA investigation into the failure of a spring strut during spacecraft development testing.
The case briefing pack contains the original situation, the information available at different stages of the investigation and the eventual findings, together with instructions that prevent the AI from revealing evidence before it becomes available. The learner has to decide what to investigate, when to continue or stop testing and how to interpret the evidence as it emerges.
Download the TXT file, upload it to a new Gemini conversation and ask:
Run the engineering decision case in this file.
Do not upload the original NASA paper. The briefing pack is deliberately self-contained so that the AI can reveal the historical evidence progressively rather than seeing the investigation as an ordinary document to summarise.
The case is an experiment, not a validated training method. But it provides a simple way to test the idea: can AI turn a documented engineering investigation into something closer to experiencing the problem before knowing the answer?
Which experiences should an engineer have?
Once we can create experiences deliberately, we can ask:
Which situations should an engineer encounter before we consider them experienced?
Some will occur naturally and others are rare. A deliberate curriculum could make sure important ones occur at all, with cases becoming progressively more ambiguous as judgement develops.
Real programmes would remain essential because they bring responsibility, organisational pressures and physical complexity that simulation cannot reproduce. But we would no longer have to wait for real programmes to provide every important experience by chance.
From knowledge management to experience creation
In the 1990s, Victor Basili and colleagues proposed the Experience Factory: a systematic way of collecting experience from software projects and making it available for reuse.
AI may allow us to take the idea a step further. Engineering work creates experience; important decisions and the judgement behind them can be captured and turned into cases through which other engineers practise their own judgement.
The organisation is no longer preserving only what it knows. It is using what it knows to develop people who can exercise judgement themselves.
Simulated cases cannot reproduce years of responsibility on real programmes, and some expert knowledge may be difficult to capture. But we know how to study expert decision-making, case-based learning already lets people practise decisions, and engineering organisations possess decades of real problems from which those cases could be built.
Knowledge management has traditionally asked:
What knowledge should we preserve?
Perhaps we should also ask:
What experiences should the next engineer have?
References
- Victor R. Basili, Gianluigi Caldiera and H. Dieter Rombach, The Experience Factory
- Beth Crandall, Gary Klein and Robert Hoffman, Working Minds: A Practitioner’s Guide to Cognitive Task Analysis
AI helped to research and write this article.