This video was recorded at Code BEAM Europe 2022 - https://codesync.global/conferences/code-beam-sto-2022/ Know your Past to Predict your Future, Time Travel Debugging in Production | Quildreen Motta Ribeiro - Engineer at Klarna ABSTRACT You've just hit a weird corner case in your production system. Looking at the code has proved hopeless. Your system's log has so much information you feel overwhelmed. How do you debug it? And how do you go about understanding all of the implications of this issue? In this talk we'll take the learnings from building specialised time-travelling debuggers for production systems and dive into how they help in these scenarios. Then we'll look into how you'd approach building one for your system, as well as the constraints that they put on your system's design. OBJECTIVES The goal of this presentation is to show people why they'd consider building specialised time-travelling debuggers for their production system, and look at a few approaches to building them, as well as what kind of constraints that puts in their system's design, based on debuggers the author has worked with before. The idea of tracing itself isn't complicated, but it's not always clear what kind of trade-offs you need to make in production: you need to be more conscious of the required time and space overhead (which you can't always avoid, e.g.: when building a protocol parser). And it's not always clear how to correlate and present that data in a way that makes the information useful---most data in traces is noise, and just like looking at logs of modern production systems is useless without heavy filtering and correlation, debug traces for time-travelling need specialised analysis and visualisation tools. AUDIENCE This talk is aimed at engineers that run complex production systems on the BEAM, and would like better tools to understand their system's production behaviour when things go wrong. • Timecodes 00:00 - 01:05 - Agenda 01:06 - 08:44 - Act 1. Using Debuggers 08:45 - 31:24 - Act 2. Crafting Debuggers 41:25 - 35:28 - Act 3. In Closing 35:29 - 37:49 - QnA • Follow us on social: Twitter: https://twitter.com/CodeBEAMio LinkedIn: https://www.linkedin.com/company/27159258 • Looking for a unique learning experience? Attend the next Code Sync conference near you! See what's coming up at: https://codesync.global • SUBSCRIBE TO OUR CHANNEL https://www.youtube.com/channel/UC47eUBNO8KBH_V8AfowOWOw See what's coming up at: https://codesync.global