THEME: LLM-Assisted Paper Reading

The intervention aimed to prepare students for a future where AI tools are part of scholarly workflows, while fostering critical thinking and responsible use of technology.

The Teaching Activity: LLM-Paperstorm

There are three intended learning outcomes of this activity. First, students learn how to responsibly interact with modern AI technology, specifically large language models (LLMs). Second, students learn how to validate the outputs of LLMs in the context of academic paper reading – a skill that will be valuable for their careers in academia and beyond. Lastly, students gain exposure to state-of-the-art research papers, sparking curiosity and potentially deepening their subject knowledge.

How the Activity Works

The session combines modeling, group work, and synthesis:

• Modeling Phase (5 min): The teacher demonstrates the task and highlights common pitfalls when using LLMs.
• Group Work (30 min): Students work in teams, each selecting a research paper from a shared pool. They use LLMs to summarise and explain key findings.
• Synthesis Phase (30 min): Groups present their results, critically evaluating the AI outputs: “What did the language model get right?”, “Where was it wrong?”, “Could it be nudged in the right direction?”.

This last phase encourages discussion among peers and the teacher, who was familiar with all papers.

Reflections on the LLM-Paperstorm Activity

Student feedback highlights challenges, such as irrelevant summaries and the overly agreeable nature often seen in proprietary LLMs. Nevertheless, many students value how these tools speed up the process of reading papers and offer useful insights for their own work.

They also highlight the opportunity to compare different AI tools and reflect on their respective strengths and suggest extending the synthesis phase.

The teacher’s takeaways are that the LLM-Paperstorm activity allows open dialogue about AI’s risks and opportunities and shifts the focus towards judging LLM outputs.