List of videos

"From HTTP to Kafka-based microservices [EuroPython 2019 - Talk - 2019-07-12 - Osaka / Samarkand] [Basel, CH] By Wojciech Rząsa HTTP or asynchronous communication in microservices? This question is frequently repeated and discussed. Obviously, HTTP-based communication is easier for developers and architects. Even if your developers have no prior experience with microservices, they will probably understand how to implement an HTTP service well. While asynchronous communication has a lot of advantages that allow to design and implement really robust microservices system, they also bring new challenges not so obvious for people who didn’t have a chance to work with such an environment before. In FLYR we mostly have HTTP-based Inter Process Communication (IPC) in our infrastructure. At some point, we realized that to provide the functionality required by our product we needed something more flexible and more… asynchronous. We designed and implemented a Python library to facilitate switching to asynchronous IPC, supporting one- or two-way or even single-request – multi-response communication. An important thing in the design process was to provide developers having HTTP experience a tool that would ease the process of switching to asynchronous communication. Consequently, to switch an HTTP server-side endpoint to asynchronous IPC is a straightforward task. We selected Kafka for our message broker, not surprisingly by comparing its performance reports with our very rough, but no less demanding performance requirements. But we also took care to hide the details of the broker logic from developers. Yes, we don’t use all Kafka capabilities, if you need e.g. Kafka Streams, you will have to use another solution. But we can decide what capabilities are used in our microservices and how we can make changes in the way our services communicate in a single place. There are also hooks, Kubernetes health checks and more with a lot of flexibility available out of the box. We plan to opensource our library. At the moment of writing it ‘opensourcing’ is still a work in progress and we didn’t have sufficient time to do it due to strict time constraints we have on delivering functionality to our customers, but we hope to be able to do it soon. In this talk I would like to describe how we solved particularly important problems, what solutions we developed, how we use them and what problems still need to be addressed by developers. In other words, I would like to describe you the journey we made from HTTP to Kafka-based microservices. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"import bacon 🥓 [EuroPython 2019 - Talk - 2019-07-12 - PyCharm] [Basel, CH] By Ivana Kellyerova It's often said that Python comes with batteries included, meaning that the standard library can do basically anything except for maybe conjure bacon for you (though I heard that's coming in 3.8). I don't think we fully appreciate the sheer vastness of it, though, so I went through it module by module looking for hidden gems (sorry, eggs). This is a by no means exhaustive compilation of the useful, the underrated, and the funny. When it comes to the Python standard library, chances are you use it on a daily basis -- or more likely, a more or less stable subset of it. The usual way we add things to the subset is by looking for a solution to a problem and ending up getting pointed to a standard library module. That, however, means that the odds of you finding out that there is a whole module whose sole purpose is to tell you if a string is a Python keyword are very slim. The aim of this talk is to showcase CPython libraries that are interesting in some way: mostly for their usefulness, but in some cases simply for being wonderfully weird in some way. The talk is not aimed at any particular level of Python experience - as a beginner you'll get a taste of just how many batteries Python ships with, and as a person using the language often you might end up growing your own personal subset a bit more. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"Docker meets Python - A look on the Docker SDK for Python [EuroPython 2019 - Talk - 2019-07-12 - Osaka / Samarkand] [Basel, CH] By Jan Wagner My talk aims to introduce and have a closer look on the Docker SDK for Python. I will cover: - How and where to get the SDK - How it works and how to use it in general - Possible use-cases like: Processing Container-Logs, Testing with pytest on different Python Versions, Deploy via Python Script, etc.. For my talk, you should know what Docker is and how to use it. A basic idea of pytest and server administration is nice to have, but not necessarily needed to follow my talk. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"useFlask() - or how to use a React frontend for your Flask app [EuroPython 2019 - Talk - 2019-07-12 - PyCharm] [Basel, CH] By Adrian Mönnich Many Flask projects nowadays still use mostly static HTML, WTForms and just a bit of AJAX. All the JavaScript is usually included straight from .js files or just minified with Python tools like webassets. But the JavaScript ecosystem has evolved a lot - one can now use tools like Babel to write modern JS even when targeting older browsers and a Webpack+Babel build is very straightforward and allows for advanced preprocessing of JavaScript code. In this talk we'll even go one step further and not only use Webpack to build the assets, but also integrate a small demo Flask app with a React frontend where the Flask app only serves the static HTML page and uses a RESTful API to communicate with the React app. We'll also look into how to keep convenient Flask features such as URL building in client-side code (using the js-flask-urls package) and of course React hooks will be used! License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"Parallel computing in Python: Current state and recent advances [EuroPython 2019 - Talk - 2019-07-12 - Osaka / Samarkand] [Basel, CH] By Pierre Glaser h2Parallel computing in Python: Current state and recent advances/h2 Modern hardware is multi-core. It is crucial for Python to provide high-performance parallelism. This talk will expose to both data-scientists and library developers the current state of affairs and the recent advances for parallel computing with Python. The goal is to help practitioners and developers to make better decisions on this matter. I will first cover how Python can interface with parallelism, from leveraging external parallelism of C-extensions –especially the BLAS family– to Python's multiprocessing and multithreading API. I will touch upon use cases, e.g single vs multi machine, as well as and pros and cons of the various solutions for each use case. Most of these considerations will be backed by benchmarks from the scikit-learn machine learning library. From these low-level interfaces emerged higher-level parallel processing libraries, such as concurrent.futures, joblib and loky (used by dask and scikit-learn) These libraries make it easy for Python programmers to use safe and reliable parallelism in their code. They can even work in more exotic situations, such as interactive sessions, in which Python’s native multiprocessing support tends to fail. I will describe their purpose as well as the canonical use-cases they address. The last part of this talk will focus on the most recent advances in the Python standard library, addressing one of the principal performance bottlenecks of multi-core/multi-machine processing, which is data communication. We will present a new API for shared-memory management between different Python processes, and performance improvements for the serialization of large Python objects ( PEP 574, pickle extensions). These performance improvements will be leveraged by distributed data science frameworks such as dask, ray and pyspark. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"Astro Pi: Python on the International Space Station [EuroPython 2019 - Talk - 2019-07-12 - PyCharm] [Basel, CH] By Ben Nuttall A collaboration between the Raspberry Pi Foundation and the European Space Agency put two Raspberry Pi computers augmented with sensor boards and camera modules on the International Space Station in 2015. Every year we run a series of competitions for kids in schools around Europe to design science experiments using the available sensors. Mission Zero is a low-barrier challenge where students can run a 1 minute Python program in space to display a message to the astronauts. They have access to the sensors for conditional logic but cannot record data or take photos. Mission Space Lab is a more involved challenge, including planning an experiment, writing and testing code which will run for 3 hours in space, either studying life in space or life on earth (which includes taking photos of Earth out of the ISS window). MSL teams get to keep the data and photo they record in their experiment and are to write a report analysing their findings. A small tech team at the Raspberry Pi Foundation maintain the operating system used in flight and work in collaboration with ESA and partners to keep the operation of the Pis running smoothly on the ISS LAN. As well as sharing details of the OS maintenance and devops for the Astro Pis, I'll share photos taken from space and show findings from student experiments using opencv, tensorflow, scikit-learn, ephem and more. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"Hack The CPython [EuroPython 2019 - Talk - 2019-07-12 - Osaka / Samarkand] [Basel, CH] By Batuhan Taşkaya Have you ever realized how dynamic CPython interpreter is? Maybe it is the most dynamic interpreter you may see. It gives interfaces to internal things like garbage collector or AST, allows to alter functions code, modify built-in functions etc. This talk will go beyond that dynamism. From adding a new syntax to hooking the evaluation loop, it will show how to hack parts of python. Before understanding these hacks, you will learn internals of CPython step-by-step. Steps are important because in every step we have at least one hacking option. Also it gives the audience a short brief of how python works. After learning how cpython works, we'll cover how to hack (use things that is not their main purpose) the interpreter and the interfaces it gave. For an example we will disassembly the bytecode and then assemble it again with adding our statements or adding a new syntax for python at runtime with AST. Talk will hack these steps: - AST - Bytecode - CTypes - CPython evaluation loop License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"AsyncIO in production - War Stories [EuroPython 2019 - Talk - 2019-07-12 - Osaka / Samarkand] [Basel, CH] By Michal Wysokinski My team has been running AsyncIO in production for over 2 years now and the only thing I can say about my experience with it is: ""Oh boy, what a rollercoaster of feelings"". I've experienced laughs and tears, sweat and blood but also sang songs of glory. AsyncIO is currently the biggest buzzword in the Python world advertised as a silver bullet capable of solving all Python's shortcomings in the field of performance. However, it also brings a burden of being a completely new approach with a fresh implementation which is not often mentioned and taken into consideration. In some of my team's projects we've achieved a great success thanks to AsyncIO, but there's been a few where we decided to get rid of it and replace it with a more traditional fork-join architecture. I'd like to share my experience with AsyncIO, tell some War Stories and discuss which projects it suits perfectly and which ones should avoid it. License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/

"The soul of the beast [EuroPython 2019 - Talk - 2019-07-12 - PyCharm] [Basel, CH] By Pablo Salgado Why The audience will discover one of the core pieces of the language that sits at the middle of the decisions about what new rules can or cannot be implemented in the Python programming language. They will learn how the particularities of the grammar limit what can be achieved but also serve to maintain the language consistent, powerful but straightforward. Attendants will learn how core developers solved some challenging scenarios that arise as a consequence of said limitations or how others cannot be resolved unless Python gets a significant transformation in the internal mechanism that parses the grammar. Finally, they will learn how a new rule is added to the CPython grammar, serving as a perfect example of how all the pieces come together. In summary, the audience will gain a more technical response to why people perceive the Python programming language as easy but powerful one and at the same time will gain some insight on how to understand and extend the pieces that form it. This talk will not only help members of the audience understand better the design of the language a how grammars and parser work, but will also help people wanting to contribute to CPython understanding the general structure of the compiler pipeline and how to work on it. Who This talk is for those that want to understand Python a bit deeper: not only how everything works under the hood but also what are the technical decisions in its making and what are the consequences. The talk is targeted to all Python programmers, no matter the skill level as everyone will find something for their particular level of expertise: Beginner programmers will be introduced in the topic of language grammars and will learn what a Grammar is and what are the building blocks. Also, the audience members in this level will gain insight into how everything is thread together in CPython. Medium and advanced programmers will learn some in-depth technical details and how they relate to features they already know and understand. The talk not only will try to enlight some new areas related to grammar technicalities, parser features and design and CPython implementation details but will also connect many pieces of information to explain how the small technical decisions impact the bigger picture. Outline Who am I What is the Python Grammar What is grammar? How they look like. Elements: terminal symbols, nonterminal symbols, productions. The properties of Python Grammar? Leftmost derivation 1 token lookahead No epsilon productions! (Plus what epsilon productions are) Some immediate consequences of these properties. How the Python parser generator works General structure of the parser generator. Non Deterministic Finite Automata Deterministic Finite Automata. Some examples (with cool graphs!) generated from the python grammar and the parser generator of the actual finite automatas that Python uses. Concrete syntax trees. Advantages of the grammar (or ""why Python is so easy to understand) LL(1) grammars are context-free (no state to maintain while parsing). LL(1) grammars are simple to implement and very fast to parse. LL(1) grammars are very limited, keeping the language simple Disadvantages of the grammar: Grammar ambiguity. LL(1) grammars need some hacks for very simple things. How keyword arguments were incorporated in the grammar with a hack: The grammar rule is very strange because it is ""fixed"" in the Abstract syntax tree Why parenthesized with statements cannot be implemented (with statements formed of multiple elements surrounded by parenthesis and separated by commas). Implementing a new grammar rule in CPython: the arrow operator : A complete mini-tutorial on how to introduce a new operator: A - B that gets executed as A.strongrarrow/strong(B). Altering the grammar and generating the new parser. Introducing a new token. Changing the tokenizer. Changing the Abstract Syntax Tree Generator. Changing the compiler. Implementing the new opcode. Implementing the strongrarrow/strong protocol. The future and summary of the talk: We have been discussing in the CPython discourse to change the parser generator to something more powerful. Dangers and advantages of other parser generators. What other implementations are using? Summary of the talk License: This video is licensed under the CC BY-NC-SA 3.0 license: https://creativecommons.org/licenses/by-nc-sa/3.0/ Please see our speaker release agreement for details: https://ep2019.europython.eu/events/speaker-release-agreement/