Learning at scale Using implementation science to understand critical success factors

Item: Learning at scale Using implementation science to understand critical success factors
Rating: 4
Author: PostSingularity Institute

by Thomas Rudge

Humanities, Social Sciences & Thought Education JNEducation JNQOpen learning, distance education

🎙 Thomas Rudge 👥 1K 📅 February 22, 2026 ⏱ 41 min 👁 45 📄 expert opinion 🧭 2026-08-16

Available in: English (current) Français

Keywords

implementation scienceCFIRERICsentiment analysishepatitis C

Summary

Thomas Rudge, a PhD candidate at the Kirby Institute, presents a seminar on using implementation science to understand critical success factors in scaling up the Australian Hepatitis C Point-of-Care Testing Program. He begins by defining implementation science as the study of methods to promote the adoption of evidence-based practices, emphasizing the ‘know-do gap’. The program has conducted over 60,000 tests across many sites, and the study aims to identify barriers and enablers to implementation. Data sources include 28 documents, 125 meeting transcripts, and 30 stakeholder interviews, coded using the Consolidated Framework for Implementation Research (CFIR). Factors were rated on a scale from -2 to +2, resulting in 580 factors. To manage this large dataset, the team developed an AI-assisted sentiment analysis system using a locally run large language model and a custom sentiment lexicon. The AI ratings did not perfectly align with human ratings, but discrepancies highlighted contextual tensions, such as financial incentives being effective but not always available. Results show consistent enablers like partnerships and relative advantage of point-of-care testing, while barriers include IT infrastructure, workforce capacity, and stigma. The presentation also introduces the Expert Recommendations for Implementing Change (ERIC) as a tool for selecting strategies to overcome barriers. Rudge reflects on how implementation science brings discipline to complexity and balances rigor with pragmatism. The seminar concludes with acknowledgments and a Q&A session.

224 words

Critical Evaluation

Value of the Information & Strength of the Argument

The presentation provides valuable insights into the application of implementation science to a large-scale health program. The speaker clearly explains the methods used, including the CFIR framework and the innovative AI-assisted sentiment analysis, and supports his arguments with concrete examples from the data. The argumentation is solid, as he acknowledges limitations and discusses discrepancies between AI and human ratings, demonstrating critical thinking. The value lies in the practical demonstration of how implementation science can systematically identify barriers and enablers, and how AI can assist in processing large qualitative datasets, though it cannot replace human analysis.

Scientific Rigor, Source Quality, Title Accuracy

The presentation demonstrates scientific rigor through the use of established frameworks (CFIR, ERIC) and a transparent methodology. The speaker cites relevant literature, such as the CFIR and ERIC papers, and mentions the use of the RoBERTa language model. The title accurately reflects the content, focusing on implementation science and critical success factors. The seminar is presented in collaboration with the NSW HIV PRISM Partnership, adding credibility. No comments were provided, so no analysis of public reception is possible.

188 words

Title / Content Match

The title accurately reflects the content, focusing on implementation science to understand critical success factors in scaling up a health program.

Quality & Reliability

8/10

The presentation is based on a large multi-method evaluation with systematic use of established frameworks (CFIR, ERIC) and transparent methods. The speaker is a PhD candidate at a reputable institute. Limitations include reliance on qualitative data and the novelty of AI-assisted sentiment analysis, but the approach is clearly explained and critically reflected upon.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Cited Sources

Kirby Institute event page — Official event page for the seminar, providing context and collaboration details.

Concurring Sources

CFIR Research Team — The official CFIR website, which provides the framework and tools used in the study.
ERIC implementation strategies — The original paper describing the ERIC taxonomy, referenced in the presentation.

Contribution & Novelties

The presentation offers a novel integration of AI-assisted sentiment analysis with traditional implementation science methods, demonstrating a practical approach to handling large qualitative datasets. It provides a detailed case study of applying CFIR and ERIC frameworks to a national health program, offering insights into critical success factors for scaling up point-of-care testing. The use of a locally run LLM and custom sentiment lexicon is an innovative methodological contribution.

Pour aller plus loin :

Consolidated Framework for Implementation Research (CFIR) — The framework used to code barriers and enablers.
Expert Recommendations for Implementing Change (ERIC) — The taxonomy of implementation strategies.
RoBERTa language model — The LLM used in the sentiment analysis system.
Sentiment analysis — Overview of the technique used.

120 words

Radar Profile

The radar profile shows high scores across all dimensions, indicating a well-rounded presentation with strong information content, quality, technical depth, and reliability. The balance between quantitative and qualitative aspects is notable, with a slight emphasis on methodological rigor.

Reliability 8/10