Monitorama PDX 2023 - OpenResty, My Bestie: Tracing NGINX With Lua

Conference: Monitorama PDX 2023

Year: 2023

Sam Handler's session from Monitorama PDX 2023. Shopify served 75.98 million requests per minute during Black Friday/Cyber Monday 2022, and our OpenResty deployments handled each of these requests before they hit an application server (OpenResty is a technology that lets you embed arbitrary Lua scripts into NGINX configuration files). Until recently, our routing stack was completely untraced, which left a huge blind spot in our view of our infrastructure. In 2022, we finally implemented tracing in our OpenResty deployments, and it wasn’t easy. In this talk, I’ll describe how we got a working tracing implementation. Along the way, I’ll explain the dangers of custom trace propagation formats, the joys of working in a well-specified open source project, the wonders (and challenges) of the OpenResty runtime, and the mental challenges that accompany the modification of NGINX, that famously performant HTTP server and reverse proxy.