Hail Hydrate! From Stream to Lake | Tim Spann | Conf42 Machine Learning 2021

Conference: Conf42 Machine Learning 2021

Year: 2021

Tim Spann Developer Advocate @ StreamNative A cloud data lake that is empty is not useful to anyone. How can you quickly, scalably and reliably fill your cloud data lake with diverse sources of data you already have and new ones you never imagined you needed. Utilizing open source tools from Apache, the FLaNK stack enables any data engineer, programmer or analyst to build reusable modules with low or no code. In this talk we will utilize Apache NiFi, Apache Pulsar, Apache Flink and MiNiFi agents to load CDC, Logs, REST, XML, Images, PDFs, Documents, Text, semistructured data, unstructured data, structured data and a hundred data sources you could never dream of streaming before. I will teach you how to fish in the deep end of the lake and return a data engineering hero. Let’s hope everyone is ready to go from 0 to Petabyte hero. — 0:00 Intro 0:20 Talk — 🥇 Gold Sponsor AWS 🥈 Silver Sponsors ChaosNative Microsoft Restream SeMI Technologies Stream Native TypingDNA 🤝 Media Partners Bpb Infosec Conferences [ Inside Dev ] Manning O'Reilly Packt — Website 🚀🪐 https://www.conf42.com​ Reach Out 📧📭 mark@conf42.com Discord Server 🧑‍🤝‍🧑💬 https://discord.com/invite/dT6ZsFJ5ZM​ LinkedIn 👨‍💼💼 https://www.linkedin.com/company/4911...​ Twitter 🎵🐦https://twitter.com/conf42com​ Conf42Cast 🎧 http://www.conf42.com/podcast