This video was recorded at Lambda Days 2019 http://www.lambdadays.org/lambdadays2019 Get involved in Lambda Days' next conference http://www.lambdadays.org --- EFFECTIVE DATA ENGINEERING USING HIGHER KINDED TYPES by Animesh Kumar ABSTRACT In order to create a canonical framework for real-time and batch data processing in the order of 10s of TBs per day, a general purpose framework to instantiate and extend data pipelines is presented here that can be used to ingest, process, and derive ML based inferences from the base data. Read the full abstract: http://www.lambdadays.org/lambdadays2019/animesh-kumar --- THE SPEAKER - ANIMESH KUMAR Senior Software Engineer at Uber I have been a senior engineer with Uber for over 2 years. I have had a history of being part of high velocity teams, and using functional languages to build systems that perform at scale and correctness. Data engineering with functional languages is a field that I did some groundbreaking work on recently. LinkedIn: @anmshkmr More on Animesh Kumar: http://www.lambdadays.org/lambdadays2019/animesh-kumar --- LAMBDA DAYS & ERLANG SOLUTIONS Lambda Days is powered by Erlang Solutions. Lambda Days 2019 was sponsored by Brainly, Schibsted, Standard Chartered, Erlang Solutions, NoRedInk & Uber. LAMBDA DAYS Website: http://www.lambdadays.org Twitter: https://twitter.com/LambdaDays Facebook: https://www.facebook.com/lambdadays LinkedIn: https://www.linkedin.com/company/lambda-days/ Mail: info at lambdadays.org #LambdaDays #MachineLearning #DataEngineering #Spark #Flink