List of videos

Improving your app's performance is a complicated but essential task to handle growth. Often, the bottleneck for your system is your database. While there are many strategies on scaling your database infrastructure, there isn't a clear strategy for solving your performance issues by improving your Python code. Come to this talk to learn how a recently open-sourced library named "snapshot-queries" can help you write Python code that helps you scale without spending more!

In the contemporary world of machine learning algorithms - “data is the new oil”. For the state-of-the-art ML algorithms to work their magic it’s important to lay a strong foundation with access to relevant data. Volumes of crude data are available on the web nowadays, and all we need are the skills to identify and extract meaningful datasets. This talk aims to present the power of the most fundamental aspect of Machine Learning - Dataset Curation, which often does not get its due limelight. It will also walk the audience through the process of constructing good quality datasets as done in formal settings with a simple hands-on Pythonic example. The goal is to institute the importance of data, especially in its worthy format, and the spell it casts on fabricating smart learning algorithms.

Migrating from python 2 to python 3 is not very easy, but it can be exacerbated by needing to port a large codebase modified frequently by many different developers. Our codebase was nearly 4 million lines of code modified dozens of times a day by hundreds of total developers. It is also business critical, containing a large portion of our most important code and data. We used several tools, techniques, and patterns to achieve the migration without disrupting day-to-day development and keeping regressions a minimum. In this talk, we’ll detail our migration steps, our usage of pre-commit hooks to reduce regressions to fixes, our usage of a reverse proxy to allow granular, low risk rollout for a webapp, and our migration of pickle to rollforward safe json for caching.

In this talk, we want to make two major points: Metrics can facilitate better conversation about code quality. They help you focus more on technical problems and improvements instead of personal preferences and organizational issues. Metrics can be misused very easily. Knowing their limitations is crucial. METRICS For each metric, we'll discuss: code examples in Python how to calculate interpretation (incl. some comparison across open source Python projects) actions limitations METHOD LENGTH The simple. You can calculate it without specific tools. First step: Extract functions. It shows well some general limitations of code quality metrics. CYCLOMATIC COMPLEXITY The old. Show the formula, but don't explain it in detail. :-) Extract functions. Remove redundant if conditions. It doesn't account for nested coding constructs. It ignores some modern language patterns. COGNITIVE COMPLEXITY The human. Calculation and interpretation: see https://www.sonarsource.com/docs/CognitiveComplexity.pdf Actions: Extract functions. Use shorthand structures. More Pythonic code is also more readable. Limitations: It ignores both the length of a linear block and the complexity of the expressions used in it. WORKING MEMORY Another aspect of human understanding. Calculation: see https://sourcery.ai/blog/working-memory/ Interpretation: The 7 +/- 2 rule of the human working memory. Actions: Extract functions, some more specific refactorings this metric rewards. Limitations: It ignores the structure. LIMITATIONS AND PITFALLS GENERAL They can be gamed. They easily encourage one-sided thinking and behaviour. SPECIFIC FOR CODE QUALITY METRICS Great as warning signs, not good as "proof of excellence". COMPOUND METRICS Giving a more versatile picture than a single metric. WHAT METRICS DON'T CAPTURE naming, consistent terminology, ubiquitous language (DDD) project structure correctness

As ML/AI systems are becoming more prevalent, the need for setting quality standards and testing practices has become crucial. Testing these models goes beyond validation metrics like accuracy, precision, and recall. Instead, quality attributes like model Behaviors, Usability, and Fairness need to be tested and measured using exploratory and automated strategies. In this talk, we'll cover some of the risks and biases that can happen throughout the MLOps pipeline, demonstrate a few techniques to test a model's behaviors and fairness, and apply them against some real-world scenarios and state-of-the-art models. By the end, you will have new ideas and techniques that you can use to test your own ML/AI systems and approach testing these quality attributes from a user's perspective.

This will be a example-driven presentation. The first set of examples looks at an int which should be restricted to a value from one through ten. First I'll look at the problems in the traditional approach, passing an int to a function and checking to ensure it is within range. Next I'll encapsulate the int in a (regular) class OneToTen, which allows the movement of the test into the constructor. Although this guarantees that objects will be created correctly, such objects are mutable so they can be modified to be invalid after creation. The solution is to use @dataclass together with the frozen=True option, and add a __post_init__ function to check the validity of the object once it's been initialized. Because such an object is invariant, it cannot be later modified into an invalid state. This ensures that the new type can only ever exist as a legitimate value. Next I'll use this technique to create a Person type that is composed of FullName, BirthDate and EmailAddress fields, each of which validates itself. Finally, I'll compose BirthDate using Day, Month and Year fields.

You're all happy developing your application but when it comes the time to send it to production and have the first customers testing, you might realize that whenever a bug is found it is just too hard to understand what is going on in a production environment - you often don't have access to the user data, the user account and struggles to reproduce the error and support your customer properly. The story is too familiar and probably happened to a lot of of us. In this talk, we will walk through techniques and things to consider when writing an application that is going to be supported in a production environment with an eye for observability by having searchable, consistent, and rich log messages. At first, the problem will be presented to the audience with a use-case scenario where a developer has no way of knowing what happened with a particular customer in a production environment if an API request fails. Opening the code for this request, we will add together the log messages that would've made it possible for the development to debug this problem properly - and then, talk through strategies to keep in mind during the early development of the code. We will also walk through log levels and how to use them properly, making sure the log messages are clean and understandable, how to take take advantage of log's extra fields to have metadata about the messages that we are writing, and the best way to make log messages easily searchable.

A Python Enhancement Proposal (PEP) is the method through which the Python community debates and adopts new features to the language. The same mechanism is used to standardize interfaces and methods used by the Python Packaging Ecosystem. The main difference is that while language PEPs are written mostly by core developers, packaging PEPs are written by members of the Python Packaging Authority (PyPA). How we build Python packages was standardized in 2016 through PEP-517 and PEP-518. Editable installs, while widely used and well known through the -e flag in pip, proved to be controversial, so it got left out of those proposals. It has taken another five years to reach a consensus, and I'm happy to say that -- through PEP-660 -- we now have a way for all build back-ends to support editable installs. Join me in this talk, where I'll tell a tale explaining how having competing PEPs and exhausting discussions -- while tiresome -- led to a better solution. Plus, you'll also understand the different options we considered and the solution we developed in the end.

asyncio introduced many of us to futures, chaining, fan-out and fan-in, cancellation tokens, and other advanced control flow concepts. But Python threads were doing this stuff before it was cool! Come see Python threading techniques inspired by asyncio, Go, and Node.