# SemanticMachines

> A repository of knowledge about AI, with guides, explainers, workshops, activities and benchmarks.

Every page listed below has a Markdown copy at <page>/index.md. The home page copy is at /index.md.

## Guides

- [Guides](https://semanticmachines.fyi/guides/index.md): A series on sharing research outputs and data with the help of AI agents.
- [Publishing research results with Cloudflare](https://semanticmachines.fyi/guides/publishing-research-with-cloudflare/index.md): Put research results, datasets and interactive views online with Cloudflare Pages.
- [Building research sites from data files: Astro and fast search](https://semanticmachines.fyi/guides/astro-and-fast-search/index.md): Build a searchable research website from data files with Astro and search that runs in the browser.

## Other public pages

- [SemanticMachines · A repository of knowledge about AI](https://semanticmachines.fyi/index.md): A repository of knowledge about AI—explainers, workshops, and interactive learning experiences for researchers, educators, and curious minds.
- [Explainers](https://semanticmachines.fyi/explainers/index.md): In-depth articles that unpack complex AI concepts for researchers, educators, and curious minds. No jargon required.
- [Workshops](https://semanticmachines.fyi/workshops/index.md): Structured learning paths that take you from principles to practical application. Each workshop combines explainers and hands-on activities.
- [Activities](https://semanticmachines.fyi/activities/index.md): Hands-on exercises to explore AI tools, test capabilities, and build practical skills. Filter by type or keyword to find what you need.
- [Evals](https://semanticmachines.fyi/evals/index.md): Prompts for testing AI models, with notes on what to look for in the answer. Copy any of them.
- [Benchmarks](https://semanticmachines.fyi/benchmarks/index.md): Published results from running the same prompts on different AI models. Each one opens its own reader.
- [Bicycle Test](https://semanticmachines.fyi/benchmarks/bicycle/index.md): Each model was asked to make an SVG of a bicycle. Three runs, newest first.
- [Model vibe run, July 2026](https://semanticmachines.fyi/benchmarks/vibe-run-july-2026/index.md): 71 prompt results from one run on one model, Gemini 3 Flash Preview, on 16 July 2026. Responses are shown as generated; no scores were assigned.
- [Gemma 4 video understanding](https://semanticmachines.fyi/benchmarks/gemma-video/index.md): Two videos were analysed by Gemma 4 12B running locally: a screencast and a meeting clip. Only the written results are shown here, not the videos or stills from them.
- [Czech speech transcription](https://semanticmachines.fyi/benchmarks/czech-transcription/index.md): Five speech-to-text services transcribed the same Czech recording. Date not recorded.
- [Writing model comparisons](https://semanticmachines.fyi/benchmarks/writing/index.md): Compare Czech poetry, Wodehouse-style prose, an academic parody about a strategic icecream stockpile, a nursery rhyme rewritten in the style of ten English poets and an English translation of a Czech poem by Jaroslav Seifert.
- [About](https://semanticmachines.fyi/about/index.md): SemanticMachines is a repository of knowledge about AI for academic research, a source of updates on the latest developments, and a place to create interactive learning experiences.
- [From prose to structure and back](https://semanticmachines.fyi/explainers/prose-to-structure/index.md): Why structured outputs like tables and lists are often more powerful than prose for learning.
- ["Training a model", I don't think those words mean what you think they mean](https://semanticmachines.fyi/explainers/training-model-meanings/index.md): A non-technical guide to the 5 layers of AI model "training" and why the distinctions matter for decision makers.
- [How does ChatGPT read your text: RAG and Context Window](https://semanticmachines.fyi/explainers/rag-and-context/index.md): Understand the difference between the Context Window and RAG (Retrieval Augmented Generation), and why it matters for reliability.
- [Reflections on academic writing in the age of LLMs](https://semanticmachines.fyi/explainers/academic-writing-reflection/index.md): A collection of questions to guide a reflection on the impact of Large Language Models on academic writing.
- [AI for Research(ers): Principles, Context, and Tools](https://semanticmachines.fyi/workshops/ai-for-researchers-workshop/index.md): A hands-on workshop designed to move researchers from basic principles to practical application of Large Language Models.
- [Beyond Deep Research](https://semanticmachines.fyi/activities/beyond-deep-research/index.md): Deep research tools write literature reviews and reports from sources they have chosen.
- [Testing the Semantic Baseline](https://semanticmachines.fyi/activities/act-semantic-baseline/index.md): This activity explores how Large Language Models (LLMs) predict text based on meaning (semantics) rather than just statistical frequency (n-grams).
- [Comparing Model Knowledge](https://semanticmachines.fyi/activities/act-model-knowledge/index.md): Understand the difference between model sizes, versions, and providers regarding factual accuracy. You will compare different Google models and then contrast them with other providers.
- [Exploring Small/Local Models](https://semanticmachines.fyi/activities/act-small-models/index.md): Explore the capabilities of 'open' or small models that could run on local devices, comparing their performance to massive frontier models.
- [Testing Reasoning Capabilities](https://semanticmachines.fyi/activities/act-reasoning/index.md): Experiment with 'System 2' thinking (slow, deliberate reasoning) versus 'System 1' (fast, intuitive prediction) using thinking models.
- [Tool Use: Counting & Math](https://semanticmachines.fyi/activities/act-tool-use-math/index.md): LLMs are historically bad at math and counting because they predict tokens. In this activity, you will see how using 'tools' (code execution) solves this problem.
- [Data Analysis with Code Execution](https://semanticmachines.fyi/activities/act-data-analysis/index.md): Use the LLM as a data analyst by having it generate mock data and then write code to process and analyze that data.
- [Multimodal Conversation](https://semanticmachines.fyi/activities/act-multimodal/index.md): Interact with the model using inputs other than text, such as video files or screenshots, to test its visual understanding.
- [Exploring Model Performance with MPI Explorer](https://semanticmachines.fyi/activities/act-mpi-explorer/index.md): Explore how to understand the contribution of different aspects of a model's capabilities using the MPI Explorer tool.
- [Long Context Window](https://semanticmachines.fyi/activities/act-long-context/index.md): Test the model's ability to 'hold' massive amounts of information in memory at once. If you need sample data, use Semantic Machines prompts .
- [Search Grounding](https://semanticmachines.fyi/activities/act-search-grounding/index.md): Connect the model to the internet to fetch up-to-date information and compare results with and without grounding.
- [Visualizing Context](https://semanticmachines.fyi/activities/act-visualizing-context/index.md): Understand how the model 'sees' conversation history and data using the Model Context Explorer tool.
- [Literature Search Comparison](https://semanticmachines.fyi/activities/act-lit-search-comparison/index.md): Compare 'Raw' models versus 'Reasoning' models for academic research tasks.
- [Specialized Research Tools](https://semanticmachines.fyi/activities/act-specialized-tools/index.md): Use tools built specifically for research (Orchestration layers) rather than raw models to conduct a literature review.
- [Building a Simple App](https://semanticmachines.fyi/activities/act-building-app/index.md): Use the LLM as a 'Tool Maker' to create software for you. For examples and guides, refer to https://edutools.fyi .
- [Cognitive Baseline: Calibrating your Mental Model](https://semanticmachines.fyi/activities/cognitive-baseline/index.md): Before using AI for complex tasks, it is crucial to establish a "cognitive baseline"—an understanding of what the raw model can and cannot do without tools (like search or code execution) or specific prompting strategies.
- [Context & Retrieval: Handling Information](https://semanticmachines.fyi/activities/context-retrieval-activity/index.md): Modern models have massive context windows (up to 2 million tokens). This activity investigates how models handle large amounts of data and external information, and where the limits of "retrieval" lie.
- [Deep Research: Comparing 5 Leading Tools](https://semanticmachines.fyi/activities/deep-research/index.md): Deep Research is a type of AI agent that takes the interaction with an LLM beyond what can be achieved by a single prompt.
- [Developing a vocabulary for thinking about AI](https://semanticmachines.fyi/activities/vocab-thinking/index.md): A short reflection activity that sets out basic terms for thinking about AI, such as large language model and context window.
- [Exploring image and multimedia generation](https://semanticmachines.fyi/activities/image-generation/index.md): Most LLMs do not generate images directly; they call other models. These models have specific limitations and control issues.
- [Reflections on academic writing in the age of LLMs](https://semanticmachines.fyi/activities/academic-writing/index.md): To stop assuming answers are obvious and investigate gaps in knowledge regarding AI in academia.
- [Personal Evals](https://semanticmachines.fyi/evals/intro/index.md): Prompts to evaluate frontier capabilities of Large Language Models. I use a selection of these prompts to evaluate a new model on the limits of performance.
- [Recognition of Vertical Text](https://semanticmachines.fyi/evals/spatial-recognition/index.md): All models fail at these tasks most of the time.
- [Spatial Relationships in Language](https://semanticmachines.fyi/evals/spatial-relationships/index.md): This prompt asks the model to come up with contexts. Come up with contexts in which the sentence The cathedral is in front of the statue makes sense.
- [Visual Reasoning](https://semanticmachines.fyi/evals/visual-reasoning/index.md): A prompt for testing AI language models: Visual Reasoning.
- [Coding Capability & Reasoning](https://semanticmachines.fyi/evals/coding-capability/index.md): This single prompt tests the models knowledge of readability formulas, its ability to implement them and output enough code to achieve this.
- [Metalanguage and Anaphora](https://semanticmachines.fyi/evals/metalanguage-anaphora/index.md): These two prompts test the ability to disambiguate anaphora. The first one is a single sentence and the second one expands.
- [Writing Output Length](https://semanticmachines.fyi/evals/writing-length/index.md): Most models cannot output text longer than about 3,000 words (even if the actual limit is higher).
- [Multilingual: German](https://semanticmachines.fyi/evals/multilingual-german/index.md): The correct translation is “You do not have to understand life” even though the literal words say “You must not understand life”.
- [Multilingual: Czech Metalanguage](https://semanticmachines.fyi/evals/multilingual-czech-metalanguage/index.md): Napiš 10 vět, které ilustrují různá použití slova totiž. Write 10 sentences illustrating the use of the word totiž.
- [Multilingual: Czech Morphology](https://semanticmachines.fyi/evals/multilingual-czech-morphology/index.md): This requires a high level of metalinguistic awareness and ability to maintain this.
- [Multilingual: Cultural Background](https://semanticmachines.fyi/evals/multilingual-cultural/index.md): A prompt for testing AI language models: Multilingual: Cultural Background.
- [Multilingual: Idiom Testing](https://semanticmachines.fyi/evals/multilingual-idioms/index.md): A prompt for testing AI language models: Multilingual: Idiom Testing.
- [Small Models: Knowledge & Instruction](https://semanticmachines.fyi/evals/small-models-knowledge/index.md): A sequence of short prompts to test knowledge, coherence and instruction following of small models.
- [Small Models: Multilingual](https://semanticmachines.fyi/evals/small-models-multilingual/index.md): This prompt tests multilingual performance linked to knowledge. Even models of 14b will struggle with this - even if they can perform relatively will in Czech.
- [Creative Writing: Wodehouse](https://semanticmachines.fyi/evals/creative-writing-wodehouse/index.md): A prompt for testing AI language models: Creative Writing: Wodehouse.
- [Creative Writing: Poetry](https://semanticmachines.fyi/evals/creative-writing-poetry/index.md): A prompt for testing AI language models: Creative Writing: Poetry.
- [Image & Video Generation](https://semanticmachines.fyi/evals/image-generation/index.md): Picture of a wineglass full to the brim with a clock on the wall behind it showing half past three.
- [Long Context: Anachronisms](https://semanticmachines.fyi/evals/long-context-anachronisms/index.md): A prompt for testing AI language models: Long Context: Anachronisms.
- [Long Context: Retrieval](https://semanticmachines.fyi/evals/long-context-retrieval/index.md): A prompt for testing AI language models: Long Context: Retrieval.
- [Vision](https://semanticmachines.fyi/evals/vision/index.md): A prompt for testing AI language models: Vision.
- [Agentic Tasks](https://semanticmachines.fyi/evals/agentic-tasks/index.md): Make a list of Oxford museums and galleries that are within 10-15 minute walking distance of Jesus College for conference attendees.
- [Arithmetic Fixed Cases](https://semanticmachines.fyi/evals/arithmetic-fixed-cases/index.md): Small computable baseline for checking arithmetic reliability and output discipline.
- [Czech Case Analysis](https://semanticmachines.fyi/evals/czech-case-analysis/index.md): Tests multilingual grammatical analysis in a language with visible case marking.
- [Heavy and Light Idiom Translation](https://semanticmachines.fyi/evals/heavy-and-light-idiom-translation/index.md): Tests whether a model notices metaphor and avoids word-by-word transfer.
- [Image Generation - Wineglass and Clock](https://semanticmachines.fyi/evals/image-generation-wineglass-and-clock/index.md): Generalised image-generation comparison prompt inspired by the `gpt-image-2-capabilities` gallery.
- [Two Turn Constraint Shift](https://semanticmachines.fyi/evals/two-turn-constraint-shift/index.md): Tests whether the model can revise a previous answer when the user adds a stricter constraint.
- [Czech Spring Poem — Three-Turn Revision](https://semanticmachines.fyi/evals/czech-spring-poem-three-turn-revision/index.md): Archived three-turn Czech spring-poem test.
- [Šla Nanynka do zelí — Five Poets](https://semanticmachines.fyi/evals/sla-nanynka-do-zeli-five-poets/index.md): Five controlled Czech poet-style transformations.
- [Terra Blind Critique — Spring](https://semanticmachines.fyi/evals/terra-blind-critique-spring/index.md): Terra comments on blind-labelled final Spring outputs. Labels: A GPT-4o, B o1, C o3, D GPT-5, E GPT-5.5, F Terra.
- [Terra Blind Critique — Nanynka](https://semanticmachines.fyi/evals/terra-blind-critique-nanynka/index.md): Terra comments on blind-labelled Nanynka outputs. Labels: A GPT-4o, B o1, C o3, D GPT-5, E GPT-5.5, F Terra.
