WEBVTT 00:00.000 --> 00:12.000 So our next speaker is Colin Dean, he's going to talk about MacPython Devix, 00:12.000 --> 00:23.000 so it's the red green refactor of one, with an old tool, to be honest, I have no clue 00:23.000 --> 00:27.000 what he's going to talk about, so give him a warm welcome. 00:27.000 --> 00:40.000 Thank you folks, appreciate you, and thank you for coming. 00:40.000 --> 00:54.000 So I'm Colin Dean, I'm a software engineer, about 27 years, and now for about the last six years working 00:54.000 --> 00:59.000 on a major retailer in the US, to most of my experience in the past six years is in fact. 00:59.000 --> 01:04.000 So supporting team of data scientists, doing all kinds of deep learning, 01:04.000 --> 01:09.000 G-learning, A-I, and the vocation data engineer. 01:09.000 --> 01:18.000 I started out in PHP early on in 1999 or so, spent some time in the Ruby, 01:18.000 --> 01:23.000 and Java communities learning how to program and how to get better program. 01:23.000 --> 01:29.000 And then worked for about eight years in scholar before I found myself working in a little bit in roughly three 01:29.000 --> 01:36.000 person time, and then came back to Python in 2021, having not touched it since 2005. 01:36.000 --> 01:44.000 So this is a story of how I came back into Python and tooling and built some tooling in Python 01:44.000 --> 01:49.000 that suit my needs, having worked in many other businesses. 01:50.000 --> 01:57.000 I'm here from Pittsburgh, mentioned call it's an area of about two and a half million people in Norman Appalachian, 01:57.000 --> 01:59.000 or Appalachian, and I'm going to come out of it. 01:59.000 --> 02:03.000 In an area of the United States, we call it a RUSBOT. 02:03.000 --> 02:10.000 So if my accent comes out, it's okay to ask me to repeat this. 02:10.000 --> 02:18.000 I focus a lot on developer experience, I believe that good tools, and 02:18.000 --> 02:28.000 it's not muted. 02:28.000 --> 02:30.000 Can you repeat this? 02:30.000 --> 02:33.000 No, sorry. 02:33.000 --> 02:37.000 Maybe I'll just hit tunes with a little bit of Pittsburgh ease, and then you'll know what's got on there. 02:37.000 --> 02:42.000 All right, so again, my name is Colin Dean, case you missed that. 02:42.000 --> 02:45.000 That's very important. 02:45.000 --> 02:48.000 I focus a lot on developer experience. 02:48.000 --> 02:52.000 I believe that good tools and good practices enable building good software. 02:52.000 --> 02:58.000 I'm a huge fan of addressing problems through this problem diagnosis and 02:58.000 --> 03:03.000 remedy approach, consistent with medical and other scientific approaches to problem solving. 03:03.000 --> 03:09.000 If you search for Colin Dean problem diagnosis remedy on the internet, you will probably find things 03:09.000 --> 03:14.000 that I've written about using this for general proposals, as well as architecture decision records. 03:14.000 --> 03:20.000 At a high level, I'll be talking about some of the pains that I experienced and 03:20.000 --> 03:25.000 observed over the last approximately four years as an experienced developer working in 03:25.000 --> 03:28.000 Python for the first time since 2005. 03:28.000 --> 03:33.000 I'll show you how I solve those barriers to productivity using a very old tool. 03:33.000 --> 03:35.000 48 years old this year. 03:35.000 --> 03:38.000 Make. 03:38.000 --> 03:41.000 I will say that this solution is an active development. 03:41.000 --> 03:43.000 It is an active maintenance. 03:43.000 --> 03:45.000 It is evolving over time. 03:45.000 --> 03:47.000 There's some pain points here and there. 03:47.000 --> 03:50.000 But by and large, it works. 03:50.000 --> 03:53.000 Some things may have changed since I created these slides. 03:53.000 --> 03:59.000 Originally this presentation is from 2022 and I've updated it approximately 03:59.000 --> 04:04.000 yearly, including last week. 04:04.000 --> 04:09.000 This talk inspired the longest blog post ever on our corporate blog. 04:09.000 --> 04:13.000 When I turned it in, it was originally 30 pages. 04:13.000 --> 04:16.000 We squittled it down to about 14 by the time we published it. 04:16.000 --> 04:24.000 Most blog posts hadn't been more than about three or four pages at the time. 04:24.000 --> 04:27.000 It also earned a coveted open source release in my company. 04:27.000 --> 04:30.000 Eightth largest retailer in the US. 04:30.000 --> 04:35.000 So, of course, I have to thank my employer for allowing me to open source stuff. 04:35.000 --> 04:40.000 And I wish that more employers would allow their people to open source even 04:40.000 --> 04:43.000 the examples of the things that they're doing inside. 04:43.000 --> 04:46.000 It's a great portfolio work to think. 04:46.000 --> 04:49.000 Let's talk about some problems. 04:49.000 --> 04:54.000 Working with my teams of engineers and data sciences, data scientists, 04:54.000 --> 04:57.000 we encountered these problems. 04:57.000 --> 05:00.000 What Python version should we use for development? 05:00.000 --> 05:03.000 What should be used in production? 05:03.000 --> 05:08.000 How do I install Python at a very specific point version? 05:08.000 --> 05:12.000 How do I ensure that my project works on a first development and production installation 05:12.000 --> 05:15.000 based, and of course, how do I install the dependencies? 05:15.000 --> 05:20.000 Some things that too many of us in this room, these are kind of easy to answer, 05:20.000 --> 05:25.000 but to many of the people, these are really hard questions to answer. 05:25.000 --> 05:29.000 If you're getting that same feeling that all of this could be easily written down 05:29.000 --> 05:32.000 and automated, you're kind of normal. 05:32.000 --> 05:37.000 But this is some of the questions that I had in my first week of working in Python 05:37.000 --> 05:41.000 in about 15 years of not working in Python. 05:41.000 --> 05:46.000 A lot of this presentation boils down through this question. 05:52.000 --> 05:55.000 My diagnosis of this situation. 05:55.000 --> 06:00.000 My team uses macOS for development and deploy solinics. 06:00.000 --> 06:06.000 Apple tends to keep its Python 3x installation within Python and of life dates. 06:06.000 --> 06:09.000 But it's always a little out of date. 06:09.000 --> 06:11.000 A few point releases behind. 06:11.000 --> 06:21.000 Apple removed the system Python in macOS 12.3 as they weren't in the 10.15 release notes. 06:21.000 --> 06:24.000 If anybody reads it, Apple's release notes. 06:24.000 --> 06:26.000 It's still included in Xcode, though. 06:26.000 --> 06:31.000 That's how whenever you run Python and you're getting like user bin Python on macOS, 06:31.000 --> 06:33.000 it's coming through Xcode. 06:33.000 --> 06:37.000 Apple says don't rely on this and they could remove it at any time from Xcode, 06:37.000 --> 06:41.000 because it's an dependency internal to Xcode. 06:41.000 --> 06:46.000 This policy is similar to Hombrous policy on Python, 06:46.000 --> 06:49.000 which is like, yeah, you can use brewed Python, 06:49.000 --> 06:53.000 but really the brewed Python is there for us that use apps 06:54.000 --> 06:57.000 that use Python as a runtime environment. 06:57.000 --> 07:02.000 But Hombrous Python won't disappear in a release. 07:07.000 --> 07:11.000 There are a ton of different ways to install and use Python. 07:11.000 --> 07:14.000 My team is consistently bringing their own Python, 07:14.000 --> 07:19.000 doing setting up however they wanted to, or whatever they knew best. 07:20.000 --> 07:23.000 And this was proving to be unreliable and frustrating 07:23.000 --> 07:25.000 because people's environments were different. 07:25.000 --> 07:27.000 Their paths were different. 07:27.000 --> 07:30.000 They weren't consistently installing new dependencies, 07:30.000 --> 07:34.000 when they did a get poll, some basic things 07:34.000 --> 07:37.000 that we might consider to be a little bit basic. 07:37.000 --> 07:42.000 So we had to think about which Python I wanted to use. 07:42.000 --> 07:46.000 Using the one provided by macOS was unreliable 07:46.000 --> 07:48.000 because it'll change and probably go away. 07:48.000 --> 07:51.000 Using Hombrous Python was hitting a moving target, 07:51.000 --> 07:54.000 especially at the time. 07:54.000 --> 07:57.000 You could just brew and install Python 3, 07:57.000 --> 08:00.000 and you couldn't install specific minor versions. 08:00.000 --> 08:02.000 Now you can. 08:02.000 --> 08:06.000 Using Python is about as the same as choosing anaconda, 08:06.000 --> 08:11.000 but anaconda, of course, gives you some access to other dependencies as well. 08:11.000 --> 08:15.000 But anaconda is not free. 08:15.000 --> 08:18.000 Anaconda itself is open source and free. 08:18.000 --> 08:20.000 Using its repositories, configured by default, 08:20.000 --> 08:21.000 is not free. 08:21.000 --> 08:24.000 Using manyaconda is not free. 08:24.000 --> 08:29.000 And our company found that out the hard way. 08:29.000 --> 08:32.000 This suddenly there are still people who are using mac ports, 08:32.000 --> 08:35.000 and guess where to hit give a fuck in the next track. 08:35.000 --> 08:38.000 So hello to the next users out there. 08:38.000 --> 08:40.000 It's okay if you don't know where any of this is, 08:40.000 --> 08:43.000 because the developer experience in the end 08:43.000 --> 08:46.000 should abstract away most of these details. 08:46.000 --> 08:48.000 You shouldn't have to worry about it. 08:48.000 --> 08:51.000 And it should be done on a per repository basis, 08:51.000 --> 08:58.000 so there are minute differences between a corny copia of different repositories. 08:58.000 --> 08:59.000 You don't have to think about it. 08:59.000 --> 09:03.000 You can just run one command and all this works. 09:03.000 --> 09:09.000 Really, we just didn't have a clear single source of truth 09:09.000 --> 09:11.000 for what version of Python we wanted to use, 09:11.000 --> 09:14.000 or what Python, where we wanted to get it from. 09:14.000 --> 09:16.000 While many of our projects are tied to a particular version, 09:16.000 --> 09:19.000 because of a hard requirement on our cluster, 09:19.000 --> 09:24.000 Python 371 installation from 2018, 09:24.000 --> 09:28.000 we still had this unexercised freedom to choose 09:28.000 --> 09:31.000 the Python we wanted to use in other contexts, 09:31.000 --> 09:36.000 including containers and similar. 09:36.000 --> 09:40.000 Since originally created this system in this talk, 09:41.000 --> 09:44.000 we have moved to containerized deployments, 09:44.000 --> 09:48.000 so we can do anything we want inside the containers within limits. 09:48.000 --> 09:55.000 So it's very lovely no longer to have to run Python 371 in 2025. 09:55.000 --> 09:58.000 PEP is a venerable tool and under the hood, 09:58.000 --> 10:02.000 even poetry used PIP in the past doesn't anymore. 10:02.000 --> 10:08.000 And of course PIP was built outside of the time of pie project, 10:08.000 --> 10:12.000 so although that's changing soon. 10:12.000 --> 10:14.000 PIP projects take some getting used to. 10:14.000 --> 10:17.000 I think we're a little bit more used to it now in 2025, 10:17.000 --> 10:19.000 and we were in 2022. 10:19.000 --> 10:23.000 But even newer tools like Hatch, PDM, UV, 10:23.000 --> 10:26.000 all have different ways of installing dependencies, 10:26.000 --> 10:29.000 of showing dependencies, showing dependency tree, 10:29.000 --> 10:31.000 of packaging, building publishing. 10:31.000 --> 10:34.000 All these steps are ultimately automated. 10:34.000 --> 10:36.000 If a repo isn't using poetry, 10:36.000 --> 10:40.000 using something else, or needs to use PIP in some kind of intermediary step, 10:40.000 --> 10:45.000 then it's best to script these actions instead of leaving them inside of a read me, 10:45.000 --> 10:49.000 which is so many teams do. 10:49.000 --> 10:54.000 Python uses have a tendency to expect things to just work. 10:54.000 --> 10:56.000 You know, not have to think about compilation, 10:56.000 --> 11:01.000 not have to think about building packages really. 11:01.000 --> 11:04.000 Originally, it talked here about compiling 11:04.000 --> 11:09.000 because the systems that we were using were too new for the packages. 11:09.000 --> 11:15.000 But over time, I'm starting to see in the last about six months or so. 11:15.000 --> 11:18.000 The exact opposite problem, 11:18.000 --> 11:26.000 where legacy code bases that are using then current versions of things 11:26.000 --> 11:29.000 are having to recompile again because those dependencies 11:29.000 --> 11:32.000 are no longer running on the newer triplets. 11:32.000 --> 11:37.000 We've got a newer operating system, a newer Python version. 11:37.000 --> 11:42.000 We now have to go back to compiling because the wheels aren't available for those particular versions. 11:42.000 --> 11:46.000 So, there's a time when I was giving this talk when it was switching between, 11:46.000 --> 11:48.000 like, oh, you don't really have to compile anything anymore. 11:48.000 --> 11:50.000 But just over the last six months, I'm sorry, 11:50.000 --> 11:52.000 and I see things can pile more often. 11:52.000 --> 11:54.000 As a safeguard, we have to plan for this. 11:54.000 --> 11:58.000 We have to plan for the possibility that we might have to compile dependencies again 11:58.000 --> 12:00.000 in the future sometime. 12:01.000 --> 12:04.000 This is all scriptable, but, you know, 12:04.000 --> 12:07.000 any sufficiently advanced collection of scripts, 12:07.000 --> 12:12.000 just re-implements a direct, a DAG build tool, 12:12.000 --> 12:15.000 directed a cyclic graph. 12:15.000 --> 12:19.000 So why don't we just use one from the start? 12:19.000 --> 12:24.000 The remedy. 12:24.000 --> 12:29.000 We need a task runner for onboarding re-sync. 12:29.000 --> 12:35.000 Something that is going to be easily executable and not just better documentation. 12:35.000 --> 12:38.000 Because people don't read documentation. 12:38.000 --> 12:40.000 So let's install it. 12:40.000 --> 12:44.000 From something else, we're not bound to an off-benders whim. 12:44.000 --> 12:48.000 We're all using the same versions, the same tooling with the possibility for, 12:48.000 --> 12:50.000 as similar of the experience as we can get, 12:50.000 --> 12:57.000 to build a common vocabulary, common procedures for developing and debugging our code. 12:57.000 --> 13:03.000 Of course, as I explained, there's a ton of different ways to use and install Python. 13:03.000 --> 13:12.000 And it's a ton of different package managers and now, of course, with UV installing its own Python. 13:12.000 --> 13:19.000 Most of the time we were choosing either home brew or a condo on our team. 13:19.000 --> 13:26.000 So I have to return to this question, again, which Python? 13:26.000 --> 13:37.000 And the clear way forward is to explicitly say which and provide a completely mindless way to install it. 13:37.000 --> 13:41.000 I think almost all of us have seen this same as comic from 2018. 13:41.000 --> 13:48.000 From XKCD, fun fact people used to think in college when XKCD first came out, used to think that it was me. 13:48.000 --> 13:52.000 Because this is my style drawing. 13:52.000 --> 13:54.000 I did deny that. 13:54.000 --> 13:56.000 Random one rose too good. 13:56.000 --> 14:02.000 I chose Python at the time in 2022, and I probably make the same decision now, still. 14:02.000 --> 14:05.000 Mostly because it seemed to be the lowest barrier to entry. 14:05.000 --> 14:09.000 The most scripting friendly is random dash, so I knew it very well. 14:09.000 --> 14:11.000 We could choose the exact version of Python. 14:11.000 --> 14:16.000 We wanted down to the minor version and void changing it from underneath the developer, 14:16.000 --> 14:20.000 whenever there were point releases as home brew would do sometimes at the time. 14:20.000 --> 14:27.000 I allowed for the user to continue to use condo or home brew, managed by fonts, 14:27.000 --> 14:31.000 just in case what I was doing was going to mess up their productivity. 14:31.000 --> 14:36.000 It was incredibly important for developer experience for me to not mess up their workflow, 14:36.000 --> 14:40.000 even though I was trying to push a new workflow upon them. 14:40.000 --> 14:46.000 Home brew is a base requirement for all of our projects, all of our setups. 14:46.000 --> 14:54.000 A little biased because I'm a home brew maintainer and it's project leadership committee. 14:54.000 --> 14:58.000 It wasn't at the time, but I'm a little biased. 14:58.000 --> 15:04.000 Advancements in build tools feature to install Python automatically. 15:04.000 --> 15:10.000 Like UV does now, and that may significantly alter this pattern in the future. 15:10.000 --> 15:13.000 I might talk about that a little bit more later if I have time. 15:14.000 --> 15:21.000 It's settled on poetry as most new projects in 2021 and 22 were doing. 15:21.000 --> 15:24.000 It was made sense that were a pip-end. 15:24.000 --> 15:38.000 It's still a great choice in 2025, though UV has kind of got its rocket boots on towards the goal line. 15:39.000 --> 15:43.000 There's a ton of ways to install poetry too. 15:43.000 --> 15:50.000 We have this problem of which Python and how do I get Python, and now we have this problem of which poetry and how do I get poetry. 15:50.000 --> 15:54.000 The poetry maintainer strongly encouraged using their installer or pip-ups. 15:54.000 --> 15:58.000 Their installer is mostly just a wrapper around pip-ups. 15:58.000 --> 16:05.000 While poetry can be installed through home brew or via pip, upstream doesn't really want you to do this. 16:05.000 --> 16:12.000 They prefer that you're using their particular installation methods, and that's what I've always tried to do. 16:12.000 --> 16:20.000 If you don't follow that, you can break through poetry installation, and I've done that plenty of times. 16:20.000 --> 16:27.000 While running poetry, it'll create its own virtual environment, and then create a virtual environment for the code base. 16:27.000 --> 16:34.000 So if you're just installing poetry for the first time, it creates its own virtual environment, and then creates one for each project that you have. 16:34.000 --> 16:39.000 And then of course, the user can construct poetry, exactly which Python executable to use. 16:39.000 --> 16:56.000 Newer versions of poetry will figure out based on pion or some other things, which version of Python is installed that meets your requirement specified in the poetry configuration. 16:57.000 --> 17:02.000 I rejected a few commonly suggested solutions out of productivity concerns. 17:02.000 --> 17:04.000 The big one being containers. 17:04.000 --> 17:08.000 Containers are just too slow for what we do on our max. 17:08.000 --> 17:11.000 We're running Kalima. 17:11.000 --> 17:15.000 Containers on Linux on Mac, basically running it inside of the VM. 17:15.000 --> 17:19.000 It's just too slow to do real world data science work. 17:19.000 --> 17:23.000 Especially when we've cut these like $4,000 max. 17:23.000 --> 17:28.000 It seems kind of silly. 17:28.000 --> 17:33.000 I needed some glue to get all of this automated from some common tooling. 17:33.000 --> 17:41.000 So of course, the greatest common denominator is make. 17:41.000 --> 17:46.000 I've long used a minimal make setup in every ecosystem that I've touched. 17:46.000 --> 17:49.000 I've lost shell scripts of course. 17:49.000 --> 17:52.000 Java back in the day. 17:52.000 --> 17:56.000 To provide a common onboarding experience. 17:56.000 --> 18:05.000 I've wrapped it around Maven, Gradle, SPT, Rake, whole bunch of them. 18:05.000 --> 18:07.000 Make is very widely available. 18:07.000 --> 18:08.000 It's built into Mac OS. 18:08.000 --> 18:09.000 It comes with Xcode. 18:09.000 --> 18:11.000 It's never going to go away. 18:11.000 --> 18:13.000 At least enough of a make installation. 18:13.000 --> 18:18.000 There's always going to be some difference between a new make and BSD make. 18:18.000 --> 18:24.000 Most of the time, we can build a make file, configuration for make. 18:24.000 --> 18:27.000 And we can easily share it across Python projects. 18:27.000 --> 18:36.000 You may get into some copy paste, but ultimately it is very portable. 18:36.000 --> 18:40.000 It's going to fly through a introduction to make. 18:40.000 --> 18:42.000 Very simple make file. 18:42.000 --> 18:47.000 When you run make task, it will run command that does something. 18:47.000 --> 18:49.000 It will run every time that you invoke it. 18:49.000 --> 18:55.000 Sometimes you'll see dot phony above this to say, this task doesn't actually produce anything. 18:55.000 --> 19:00.000 Other basic use case for make is to produce files. 19:00.000 --> 19:03.000 Make was originally intended to compile see. 19:03.000 --> 19:06.000 And still, of course, why the use for that purpose. 19:06.000 --> 19:10.000 It also makes it really easy to add into Python projects where you have some non-python 19:10.000 --> 19:18.000 Transformations need to happen, whether it's like templating markdown or something like that. 19:18.000 --> 19:22.000 Full make tutorial is certainly out of scope for this presentation. 19:22.000 --> 19:25.000 Longer versions of this presentation, I might delve into it a little more, but not right now. 19:25.000 --> 19:28.000 It's 45 years old, well, 48 years old this year. 19:28.000 --> 19:31.000 So you can find lots of stuff out there on it. 19:31.000 --> 19:40.000 One such being this URL there where I've built a workshop that goes pretty heavily into make. 19:40.000 --> 19:45.000 Make file tutorial is another great one, not mine. 19:45.000 --> 19:47.000 So why not something else? 19:47.000 --> 19:55.000 I've used most of these and almost always end up coming back to make to wrap behaviors about these that I can't remember. 19:55.000 --> 20:06.000 Principle engineer Haroku many years ago said that every make file in the company every repo in the company 20:06.000 --> 20:10.000 needed to have a make file that had at least those six tasks. 20:10.000 --> 20:15.000 Make depth check test build help clean. 20:15.000 --> 20:22.000 Depths installed dependencies check ran lints test ran test build built a package. 20:22.000 --> 20:27.000 You can figure out what the other two mean. 20:27.000 --> 20:36.000 One of the clarifier reason why I'm so adamant about developer experience and this particular way of realizing it. 20:36.000 --> 20:42.000 It's important to have easy on ramps to code base is because it may not be somebody's new to your team. 20:42.000 --> 20:48.000 It may be you who's lost a computer and it's coming back to work that you started yesterday. 20:48.000 --> 20:51.000 It may be you six months or a year from now. 20:51.000 --> 20:58.000 Recently I picked up a code base that I hadn't that I started and hadn't touched in four years and I could remember a thing about it. 20:58.000 --> 21:06.000 But I ran make depth make rest of the command everything is worked. 21:06.000 --> 21:13.000 I'm pretty limited on time so I'm going to skip through this a little bit. 21:13.000 --> 21:25.000 We have lots of good online help built into our make files such that we have a description of each main task and what the task is. 21:25.000 --> 21:27.000 Lots of things for installing dependencies. 21:27.000 --> 21:30.000 We want to install things from home brew. 21:30.000 --> 21:35.000 Peru is a great ecosystem independent package manager. 21:35.000 --> 21:42.000 I use extensively for my slides and so on and so forth. 21:42.000 --> 21:48.000 Easily run tests build a package and publish to our internal pipi system. 21:48.000 --> 21:51.000 Then of course we have plenty of code quality checks the gamut. 21:51.000 --> 21:59.000 Nowadays we mostly just run rough but back then when we created this we had to want more tools. 21:59.000 --> 22:03.000 This some challenges in this in my last two minutes here. 22:03.000 --> 22:08.000 As the fourth generation of max rollout, fourth generation of arm max rollout to developers. 22:08.000 --> 22:13.000 We're still having to support Intel machines sometimes. 22:13.000 --> 22:18.000 We're having to compile less far less often but we still have to compile sometimes. 22:18.000 --> 22:32.000 As I said earlier I think we're going to probably maybe compile more in the future because we're probably not going to update our dependencies quite so much because that takes a person time to do that. 22:32.000 --> 22:39.000 This is an example of what we had to do in order to enable compilation inside the make file. 22:39.000 --> 22:42.000 Again if I had a little more time I would go over this in greater detail. 22:42.000 --> 22:52.000 But ultimately boils down to either running with a whole bunch of flags or just poetry. 22:52.000 --> 23:02.000 So when we would run this on Mac OS arm 64 you'd see this actually running on Linux or Intel Mac which is be poetry. 23:02.000 --> 23:16.000 Hopefully this was enough to let your whistle and make you interested in providing a really easy way to get your projects both open source and inside your companies with a good developer on ramp. 23:16.000 --> 23:24.000 It's something that anybody can approach and it just works on their machine. 23:24.000 --> 23:33.000 Some improvements most of the teams that I was working on of this banded where it was originally working on this but we've all been these. 23:33.000 --> 23:43.000 Seeds in a flower and we've now spread ourselves all over the field that's been interested to see how this changes over time. 23:43.000 --> 23:47.000 If you're interested in learning more please talk to me. 23:47.000 --> 23:55.000 I'd love to share the original version of this set up and you can also see my slides here. 23:55.000 --> 24:00.000 Check the matrix room in the left link in there as well. 24:00.000 --> 24:06.000 Thank you very much for the nice talk. 24:06.000 --> 24:14.000 If you have any questions then please go directly and then ask questions all right in the chat.