When the Inquiry Itself Changes — Beyond the Breadcrumbs

Intellectual Practices of Keith Lyons

An inquiry can encounter pressure without changing.

Another can become observably different because of it.

That difference became a problem for the whole-corpus study of Clyde Street.

What had to be visible before the study could say that pressure had actually changed the inquiry itself?

On 17 February 2015, Keith Lyons published a visualisation of batting partnerships from the first six games of the Cricket World Cup.

There had been 101 partnerships. Keith had arranged them into what he called a landscape.

Then Simon Gleave responded publicly.

Gleave suggested that the display would be dominated by very short partnerships and wondered whether those should be excluded or separated.

Keith recorded what happened next.

The exchange led him to produce a new landscape showing partnerships of 20 runs or more.

The important point for the study was not simply that somebody had commented on his work.

Nor was it that Keith had been given a useful idea.

The source let the study locate a before and an after.

There was an existing representation. A specific challenge acted on it. Keith then changed the representation he was using to investigate the data.

That was enough for the case to qualify for the recurrent finding the study called Lets the Inquiry Change Under Pressure.

Five days later, Keith published another post about the same World Cup data.

By then there had been 183 batting partnerships across twelve games. The post included several comparisons and again used the 20-plus-run view that had emerged after the conversation with Gleave.

The method was still there. The earlier challenge was still visible in its history.

But the study did not count this later post as another instance of the same finding.

Nothing new had yet happened to the inquiry.

The changed representation was now being used.

That difference is small, but it matters.

One post shows a challenge producing a visible change in the inquiry.

The other shows the inquiry continuing after that change.

If every continuation of a changed approach counted as another case, the finding would quickly become a story about activity rather than change.

The study therefore required more.

It had to be possible to locate both the pressure and something observably different in Keith’s current inquiry on the other side of it.

New information is not yet a changed inquiry

Clyde Street is full of encounters.

Keith read papers, followed links, received comments, noticed unexpected results, talked with students and colleagues, tried software, revisited datasets and recorded ideas that might matter later.

If encountering something new were enough, this finding would be almost impossible to constrain.

The whole-corpus study therefore distinguished between material arriving in an inquiry and material visibly altering what the inquiry was doing.

A particularly clear example came in March 2013.

Keith had been writing about small open online communities and the relationship between a course and a community. Stephen Downes responded with a sharp distinction: courses have start and end dates; communities do not.

Keith did not merely quote the objection.

He wrote that he agreed and needed to make a clarification.

He then changed the account he was building.

He clarified the relationship by describing periods of concentration and dispersal: people might gather for focused activity and then disperse, while resources created during the course remained available to support longer-term community development.

The criticism belonged to Downes.

The changed account belonged to Keith.

That ownership distinction matters. The study did not absorb another person’s intellectual contribution into Keith’s record merely because he cited or valued it. What counted was the Keith-owned change that followed.

This case also helps separate the finding from a narrower change in wording.

Keith certainly revised the way he described the course-community relationship. But the whole account was being reorganised. The source shows more than the repair of a term. It shows a changed conceptual frame for the current inquiry.

Again, the finding is not that criticism is good or that an inquiry should always yield when challenged.

The source simply lets us see that, in this case, the challenge altered the inquiry.

When experience changes the live work

Pressure did not always arrive as a contradiction.

Sometimes it came from the conditions in which Keith was working.

In May 2015, he took part in an eBreakthrough workshop at the University of Canberra. Two colleagues, Bernie Bissett and Lubna Alam, presented before him. Keith listened to their accounts of teaching, assessment, social media, curation and co-creation while also receiving a research alert about blended-learning innovation.

He then had to decide how to follow them.

The source records the change unusually clearly.

Keith had already shared his presentation with the group and had a PDF version ready. He edited his Google Slides while the workshop was still happening, decided to use the images as the focus of the presentation, and spoke briefly about community, co-operation and connectedness.

This is useful evidence because it stops “pressure” becoming a synonym for error or bad evidence.

Nothing had failed.

Nobody had disproved him.

The changed condition was the live encounter itself: what other people had contributed, what had already happened in the room and how he would follow it.

The study could still identify a before and an after.

A prepared contribution entered the workshop. A differently focused contribution emerged inside it.

That was sufficient for the case to count.

The finding therefore does not describe one single kind of pressure.

Across the approved cases, pressure could come from contradictory evidence, direct challenge, learner experience, new sources, failed expectations, professional conditions, technical failure or changing empirical circumstances.

Those are forms of variation in the evidence. They are not a taxonomy and they do not establish a general theory of adaptation.

When a failed expectation changes the inquiry

Some of the most visible cases occur when an expectation fails.

In September 2017, Keith used median scoring profiles to suggest that Adelaide would defeat Richmond in the AFL Grand Final, all other things being equal.

Richmond won by forty-eight points.

Keith then wrote a post examining what had happened.

He returned to a paper on negative evidence in social research and used the unexpected result to reconsider what the game had made visible. By the end of the post, the inquiry had moved beyond the failed prediction. Keith was using the negative evidence to reformulate a proposition about learning environments, contrasting prescriptive IF–THEN planning with a more improvisational YES–AND orientation.

The study did not count the case because the prediction was wrong. Wrong predictions and surprising results do not themselves establish a changed inquiry. The case counted because an existing line of inquiry became visibly different after the result: the source shows a changed proposition emerging from the encounter with the negative evidence.

A surprise can remain only a surprise. A counterexample can be reported without changing the inquiry. Here, the changed proposition was visible.

When the method changes because the tool does not work

The operation can also appear in much smaller technical decisions.

In November 2018, Keith was exploring data from the Women’s T20 Cricket World Cup and trying different forms of visualisation.

He attempted to use the R package ggridges.

It failed.

He recorded that he intended to try again. But for the analysis he was doing at that moment, he changed route. He used ggrepel for the comparisons and installed grid and gridExtra to lay out the plots.

This is not a story about technological resilience.

Nor does one failed software attempt establish anything about Keith’s character.

The evidential point is more modest.

The source identifies a failed analytical route and then shows a different route being used because of that failure.

The immediate analytical route changed.

The failure itself belongs to the conditions of the work. The packages belong to their authors. The choice of how to proceed in the current analysis is Keith’s.

That is enough to make the case relevant to the finding.

Saying you will change is not the same as having changed

A strong negative case appears only a month later.

In December 2018, Keith discovered an essay by Danielle Navarro on Bayes factors. He wrote that the essay had given him a holiday reading list that would help him redefine his naïve Bayes views and thinking.

That sounds almost tailor-made for a finding about inquiry changing under pressure.

The study still did not count it.

The reason is temporal.

The source contains an intention to learn and an expectation of future change. It does not yet show the changed operation.

Keith might later read the material. His thinking might later become different. The source in front of the study cannot establish that future state simply because Keith anticipated it.

This is one of the finding’s most useful boundaries.

Prospective change is not enacted change.

A person can recognise that an argument matters, assemble a reading list, announce that a method needs work or say that their thinking will develop without the present source yet showing what has actually changed.

The study preserved that distinction rather than treating the language of anticipated change as evidence that change had already occurred.

A changed idea can still belong to somebody else

Ownership creates another limit.

In April 2018, Keith wrote about Martin Lames’s research on chance involvement in football goal scoring.

The post is intellectually substantial. It describes Lames’s definition of chance, his variables, his analysis of Bundesliga and Premier League goals, and his conclusion that chance involvement may be typical rather than exceptional. Keith connected the paper to earlier work by Charles Reep, Bernard Benjamin, Richard Pollard, I. D. Hill and others.

There is clearly important material in the post.

But the whole-corpus study did not treat the substantive reframing in the cited research as evidence that Keith’s own inquiry had changed enough to qualify for this finding.

The definitions and empirical claims remain Lames’s and the cited authors’.

Keith owned the synthesis he made in the post. He did not automatically own the intellectual change contained in the sources he was discussing.

This negative case prevents a simple error in reading Clyde Street.

A writer can encounter a powerful argument without the source showing a changed writer-owned inquiry.

The presence of intellectual movement in the material is not sufficient. The study had to locate Keith’s own before-and-after operation.

Without that ownership boundary, any strong paper Keith discussed could become evidence that he had changed his inquiry, even where the source showed no such thing.

The study did not make that substitution.

What the whole corpus established

The frozen register contained 108 approved cases from September 2008 to January 2020. Sixty-nine were classified at the strongest Explicit level, thirty-two as Probable and one as Possible; six retained no separately recoverable strength classification for this finding. The study also retained 77 rivals and near-misses, including 16 explicit negative decisions, that helped constrain what could count.

Those numbers establish recurrence and temporal spread.

They do not establish that changing an inquiry under pressure was Keith’s most important practice, his most distinctive practice or an operation that became stronger over time.

Nor do they support a claim that Keith followed one general adaptive method.

The approved cases varied too much for that.

Sometimes a question changed.

Sometimes an evidential measure was used differently.

Sometimes a conceptual emphasis shifted.

Sometimes a representation or immediate analytical method changed.

And sometimes a source that looked promising remained outside the finding because it showed only continuation, reflection, intended change, another person’s reframing or new information without a visible before-and-after change in Keith’s current inquiry.

The negative cases are therefore part of the result.

A serial update is not enough.

A surprising difference is not enough.

Reflection is not enough.

A plan to change later is not enough.

Another person’s changed account is not Keith’s change.

The current inquiry itself has to become observably different under the pressure acting on it.

What the distinction leaves with the reader

There is an easy moral to attach to cases like these: be flexible, listen to criticism, change when the evidence changes.

The whole-corpus study did not establish that moral.

It did not test whether every change was wise, timely or justified. Nor did it show that refusing to change would always have been a mistake.

What the Clyde Street evidence makes available is a prior distinction: pressure can be present without a changed inquiry.

Against the near-misses, the positive cases are narrower. There is something identifiable before the pressure, something observably different afterwards, and source evidence supporting the relation between them.

That leaves a question for the reader — one enabled by the finding rather than taught by Keith as a method: