[EuroPython 2024 — South Hall 2A on 2024-07-11] Mutation Testing in Python with Cosmic Ray by Austin Bingham https://ep2024.europython.eu/session/mutation-testing-in-python-with-cosmic-ray Mutation testing is a technique for systematically mutating source code in order to validate test suites. It operates by making small changes to a program’s source code and then running a test suite; if the test suite ever succeeds on mutated code then a flag is raised. The goal is to check that a system’s test suite is sufficiently powerful to detect a large class of functionality-affecting changes, thereby helping ensure that the system functions as expected. While not in widespread use, mutation testing is a fascinating topic with great potential that has valuable lessons for the broader software development community. In this talk we’ll look at Cosmic Ray, an open-source mutation testing tool for Python. Mutation testing presents some difficult and fascinating challenges - both conceptually and from an implementation point of view - so we’ll look at how Cosmic Ray addresses (or plans to address) these complexities. While some of these details will necessarily be Python-specific, there are lessons in Cosmic Ray for the development of mutation in any language. Mutation testing is still a rather exotic testing technique, but it can produce genuinely useful and surprising results. To show this, we’ll look at a number of cases where Cosmic Ray has helped developers improve their test suites and tighten up their implementations. --- This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License: https://creativecommons.org/licenses/by-nc-sa/4.0/