Use frameworks to test Apache Airflow data pipelines | Tamara Janina Fingerlin | Conf42 Python 2024

Conference: Conf42 Python 2024

Year: 2024

Read the abstract ➤ https://www.conf42.com/Python_2024_Tamara_Janina_Fingerlin_frameworks_apache_airflow_pipelines Other sessions at this event ➤ https://www.conf42.com/python2024 Join Discord ➤ https://discord.gg/DnyHgrC7jC Support our mission ➤ https://www.conf42.com/support Chapters 0:00 intro 0:20 preamble 0:31 agenda 1:07 what is airflow? who is astronomer? 9:33 why test data pipelines? 10:53 local development and testing 13:33 astro cli and airflow cli testing features 16:23 dag.test() 17:45 ci/cd and testing 18:08 from dev to staging 19:16 dag validation testing 21:26 unit tests with airflow 22:44 integration tests with airflow 24:06 the cf in ci/cd - our code on the way to the cloud 25:38 code getting promoted to prod 26:26 demo 45:49 take home message