You have already read Keith’s complete 1984 Use of Video in Sport paper.
Recall its evaluation section. Choose one concern to carry into this encounter:
the contribution video was expected to make;what its use cost;how that contribution could be evaluated;a barrier that might prevent it helping; orwhat the answers might reveal about coaching itself.
Keep your chosen concern in view. Notice where the Clyde Street encounters reinforce it, where they make it harder to answer and where they turn evaluation back towards the analyst’s own practice.
A service can work without proving that it helped
Imagine that an analyst records every event, delivers every clip on time and produces a clear report. The technology works. The coach thanks them. The task is complete.
Was the analysis successful?
Perhaps. But those facts establish only part of the case. They do not yet show that the information was used, that anyone learned, that practice changed, that the change was worthwhile or that the result justified the cost.
This is where the evaluation questions in 1984 become demanding. Keith did not ask only whether video could be operated. He also asked about its contribution, cost, impediments and effects. Some of the questions he reproduced in this section belonged to Les Burwitz and Michael Byrne; Keith placed them within a practical evaluation that could also reveal something about the coach and the coaching situation.
Encounter 3 asked what analysis might help another person notice and judge. Encounter 4 asks what evidence would show that it actually contributed—and what the answer might reveal about the service that was provided.
What Clyde Street brings into view
Follow one practical question through the selected encounters:
How would you know that an analytical service was worth doing?
At each stop, notice:
what is being treated as a contribution;what has been counted as a cost;what the available evidence can support;what remains unknown; andwhat the evaluation reveals about the analyst, coach or setting.
This is not a straight line of progress designed by Keith. The first post looks back from 2010 to work undertaken at a 2005 event. The later posts arise in different settings—online learning, coach development, performance analysis and mentoring. Read together for learning, they offer several ways to reconsider the evaluation task that Keith opened in 1984.
2010 — When success depends on work nobody sees
In The Occupational Culture of the Performance Analyst: Providing a Video Service, Keith looked back at the video service provided at the 2005 Canoe Slalom World Championships in Penrith.
The service distributed a video feed to 30 end-user points. The team installed a wired system and a parallel backup. Bad weather caused lost connections on the first day, so new lines had to be laid and replacement equipment bought nearby. Set-up and testing took three days; dismantling took one. Keith described his aim as an “invisible service” that supplied uninterrupted augmented information to coaches and athletes.
Several achievements are visible in the post: the team responded to lost connections by laying new lines, protected its remaining backup capacity by buying replacement equipment, lost no equipment, created an archive later used for coach-education resources, and treated the event as a learning opportunity for Keith and the young service team.
The post also exposes work that a delivered video feed can hide: planning, testing, redundancy, inventories, liaison, repair, judgement under pressure and the tacit knowledge of the people providing the service. These are genuine parts of what the service required and what made its contribution possible.
But the post does not report whether coaches used the feed, what athletes learned or whether performance improved. A reliable service can make learning and decision-making possible without proving that either occurred.
What this makes visible: Operational success matters, but it is not the same as professional, educational or performance impact.
Think: If every video feed arrived when required, what else would you need to know before claiming that the service was worthwhile?
2012 — Reach is not the same as participation or learning
In #OAPS101: A Sense of Balance, Keith reflected on day 19 of a small open online course about observing and analysing performance.
The course had 450 enrolments and visits from 80 countries. Those figures showed reach. Yet Keith also reported that visible exchange on the learning platform had lessened. He hoped that people were participating at the edges and that a self-organising learning community was developing, but the post did not treat those hopes as demonstrated outcomes. He wanted summative comments from participants to inform the next phase.
Keith also followed Kent Anderson’s discussion of the physical and human costs of digital activity: infrastructure, energy and ongoing support. An online course could appear free and scalable to a participant while still depending on resources elsewhere.
This gives the learner a useful distinction:
activity evidenceshows that somebody enrolled, visited, opened or received something;participation evidenceshows how people engaged;learning evidenceshows what they became able to understand, notice or do; andcost evidenceshows what was required to create and sustain the opportunity.
One type of evidence cannot silently stand in for the others.
What this makes visible: A large number can describe access without establishing engagement, inclusion, learning or value.
Think: If a course, dashboard or analysis platform reports 450 users, which two questions would you ask before describing it as successful?
2014 — Change has layers
In Coaching and Pentimenti, Keith turned to art for a way of thinking about change in coaching.
Pentimenti are traces of earlier work that remain beneath or within a finished painting. Sometimes a previous image can be seen by looking carefully. Sometimes another method, such as X-ray analysis, reveals layers below the surface. Keith used this idea to wonder how detailed performance data might help demonstrate a coach’s impact and how earlier elements might remain within a mature coach’s practice.
Keith presented the connection as a possibility—“perhaps” an intersection of sport and art that helps us contemplate the provenance of coach development. He did not offer pentimenti as a validated evaluation method or claim that performance data could isolate a coach’s causal impact.
For this encounter, the metaphor helps us resist evaluating a journey from its final picture alone. A team’s end-of-season result does not show all the revisions, interruptions, retained habits, relationships and contextual changes through which it was produced. Equally, finding change after an intervention does not by itself prove that the intervention caused it.
What this makes visible: Evaluation may need to follow change through time and preserve earlier layers, not compare only a starting score with a final outcome.
Think: A team improves across a season. What evidence would help you judge the coach’s or analyst’s contribution without attributing every change to them?
2017 — The service is part of what must be evaluated
In Theory of mind and mental time travel: all in a day’s work as a performance analyst, Keith described performance analysis and analytics as a service and a co-production. He argued that it required sensitivity to other people and to ourselves.
Keith reached this position by connecting other writers’ work about what people care about, how we understand mental states that may differ from our own, and how remembering the past can help us imagine possible futures. Those ideas belong to the writers he encountered; Keith’s move was to ask what they meant for an analytical service.
Keith’s professional conclusion was direct. More sophisticated digital technology could tempt analysts to make the technology the essence of their work. He instead located the most powerful relationships with coaches and athletes in personal learning journeys. He also said that he should have begun these conversations with himself and coaches much earlier in his career.
For this encounter, we can use the post to evaluate more than output accuracy. We can also ask whether the service attended to what the coach or athlete cared about, whether it helped them reflect on what happened and imagine what they might do next, and whether the analyst was willing to reconsider their own assumptions. An accurate output that is mistimed, misunderstood or disconnected from the recipient’s concerns may have limited value.
What this makes visible: The analyst is not outside the evaluation. Their understanding, relationships and choices help shape what the service can contribute.
Think: If an accurate report is repeatedly ignored, what should be examined before deciding that the coach or athlete is the problem?
2019 — Challenge needs trust, context and another lens
In A Lens of a Critical Friend, Keith considered how analysts might be developed and mentored.
Arthur Costa and Bena Kallick own the definition that Keith brought into the post. A critical friend is trusted, asks provocative questions, provides data to be examined through another lens and critiques the work as a friend. The friend takes time to understand the context and intended outcomes and remains an advocate for the work’s success. Keith added that such friendship was built through trust and described long-term “unmeetings” involving conversation, pizza and coffee. He also drew on Michael Garet and colleagues’ work about sustained, high-quality professional learning.
This is not an argument that friendliness should replace evidence. For this encounter, the post offers a way to think about evaluation as more than a score issued from a distance. A different lens can reveal what the practitioner has normalised or overlooked. Trust makes it more possible to expose uncertainty, receive challenge and revise practice; attention to context prevents the evaluator from judging against an outcome the work was never intended to produce.
What this makes visible: Evaluation can be a form of professional learning undertaken with a practitioner, not merely a verdict delivered to them.
Think: What would make a colleague’s challenge credible enough to change your practice rather than simply feel like criticism?
What changed across the journey?
Read together, these encounters widen what can count in an evaluation:
Contribution:not only whether an output was delivered, but what opportunity it created, how it was used and what changed.Cost:not only the price of equipment, but preparation, infrastructure, energy, attention, repair, relationships and professional time.Evidence:not one impressive number or final outcome, but activity, participation, learning, change over time, context and remaining uncertainty.Responsibility:not only whether the coach or athlete acted on the information, but how the analyst’s choices and the service relationship shaped its use.Evaluation:not only an external verdict, but an opportunity to inspect practice through another lens and decide what should continue, change or stop.
The selected encounters leave five questions active at once:
Did the service function as intended?What did people do with it?What learning, judgement or change can be observed?What did the contribution require—and what did it displace?What does the answer reveal about the analyst’s and coach’s practice?
Calling something successful after answering only the first question is possible. It is also incomplete.
Discuss: would you count this as success?
For each situation, mark YES, NOT YET or NO. Explain what the available evidence supports and what is still missing. There is no answer pattern to discover.
Situation A:Every requested clip was delivered before the review meeting, but nobody knows whether the coach used them.- Situation B: Two analysts agreed on 92% of labelled events, but the disagreements included two rare events that changed the tactical interpretation.
Situation C:Athletes can now explain and apply a new principle in training, although their competition result has not yet improved.Situation D:A dashboard is opened every day and discussed by staff, but nobody has checked whether its main measure represents the performance problem it is being used to judge.Situation E:A costly analysis workflow produces fewer reports than before but creates more time for the analyst, coach and athlete to examine one consequential problem together.
Choose the situation that produced the most disagreement. Share:
what is being treated as the contribution;which cost or consequence matters to your judgement;what the evidence allows you to claim now;what you would need to investigate next; andwho should decide whether the service continues, changes or stops.
The linked Clyde Street posts are available if you want to inspect the fuller context or test the account presented here. They are an invitation to go further, not a prerequisite for taking part in the discussion.
Test it against an analytical service
Choose one clip, report, dashboard, conversation or technology-supported output that you have recently produced or received. Evaluate the service around it—not only the object.
Complete these six statements:
Intended contribution:It was meant to help …Observed use or change:I can see evidence that …Fuller cost:Producing and using it required …Remaining uncertainty:I still cannot tell whether …Practice revealed:This evaluation makes me notice that the analyst, coach or setting …Professional judgement:I would continue, adapt or stop it because …
Name one further piece of evidence that could change your judgement. Do not use “more data” as the answer: say which evidence, from whom, about what and for which decision.
Take the journey forward
Complete the three sentences. Name at least one Clyde Street encounter in your response.
In 1984, Keith asked us to evaluate …Across the selected Clyde Street encounters, that concern became harder or wider when …- By the end of Clyde Street, analysts still need to judge …
The learning return is not a universal formula for proving impact. It is a more accountable question to carry into practice:
Question to carry
What evidence would justify continuing, changing or stopping this analytical service?
Source note
The linked Clyde Street posts are the substantive historical sources. External ownership remains with Les Burwitz and Michael Byrne for the three 1984 evaluation questions; Kent Anderson for the digital-cost discussion; the art accounts and Christina for the pentimenti material; Harry Frankfurt and the theory-of-mind and mental-time-travel writers for the ideas connected in 2017; Arthur Costa and Bena Kallick for the critical-friend definition; and Michael Garet and colleagues for the professional-learning findings. Keith brought these sources into his own practical reflections. The sequence and activities here are an educational translation, not a curriculum authored by Keith.
Sources followed
1984 starting source
Keith Lyons, Use of Video in Sport, Resource Pack 9, National Coaching Foundation, Chapter 4.
Clyde Street sources
Keith Lyons, “The Occupational Culture of the Performance Analyst: Providing a Video Service” — 10/08/05
Keith Lyons, “Oaps101 A Sense Of Balance” — 12/11/23
Keith Lyons, “Coaching And Pentimenti” — 14/04/17
Keith Lyons, “Theory Of Mind And Mental Time Travel All In A Days Work As A Performance Analyst” — 17/09/21
Keith Lyons, “A Lens Of A Critical Friend” — 19/09/12