Reflections on academic writing in the age of LLMs

Compiled by Dominik Lukeš. Last update 25 Oct 2024.

Why we need reflection on academic writing

This guide for reflection outlines some key questions we need to answer about the state and role of writing for a variety of academic purposes. The purpose of this reflection is not to discuss abstract questions but point towards the need of a deep investigation of both available knowledge and gaps in the knowledge.

The purpose is not to provide answers but to stop us assuming that the answers are obvious. The reflection then needs to focus both on the nature and role of writing and the nature and role of Large Language Models and AI in general.

Questions about Large Language Models

These are questions that we should be able to answer about the state of the technology as well as current trends.

• What is the difference between Large Language Models and previous kinds of Artificial Intelligence technologies used to assist with writing?
• What are the current capabilities and limitations of Large Language Models for producing academic writing and what can we see as prospects for the future?
• Are there the relevant differences between individual language models for academic writing?
• What are the various tools aimed at academic writing built on Large Language Models and what new affordances do they offer to writers?
• What are the recognised techniques and methods for using Large Language Models and tools for writing in general and academic writing in particular?
• What skills knowledge and skills are necessary for a successful use of Large Language Models for academic writing?

Questions about impact of Large Language Models on academic writing

These questions take as a departure point the answers we are able to give to the questions about the capabilities of Large Language Models and the landscape of their uses in the context of academic writing. But they focus on their impact on various aspects of academic writing in its full complexity.

• What are the differences in impact of Large Language Models on academic writing across its various roles:
  • Academic communication
  • Supporting cognition and thought
  • Providing evidence of learning
• What is the variation in the impact of Large Language Models across disciplines both broadly between the sciences and the humanities but also more specifically between disciplines within a larger area such as economics and sociology?
• Academic writing requires a variety of skills at different levels of control (grammar, vocabulary, composition). What is the role/impact of Large Language Models on these skills?
• A significant proportion of texts in academia are written by non-native speakers of English. What is the impact and role of Large Language Models in these contexts?
• What are the differences in the role of Large Language Models across skill levels and developmental stages of academic writers?
• Given the close ties between academic reading and writing, what is the impact of Large Language Models on writing given their potential to assist with reading?
• Technology has introduced the possibilities of new modalities into the writing process both voice (dictation, listening) and vision (visual representation) which have been amplified by both Large Language Models directly and associated developments in voice and vision models. Is there an indirect impact on academic writing due to the expansion of possibilities for both composition and consumption?
• Academic writing is a social activity both in that much of it is increasingly done in collaboration but also because of the social norms and expectations developed about it. What sort of changes in the social norms and expectations will be necessary to adapt to the new age of Large Language Models and what changes are we already seeing?