WEBVTT 00:00.000 --> 00:14.480 All right, for the next talk, we are going to tell us everything about generative 00:14.480 --> 00:23.480 immutable and the applicable safety or input in decarimages, let's go to the end. 00:23.480 --> 00:28.080 Yeah, thank you everyone, and nice for feeling this room so quickly. 00:28.080 --> 00:31.840 So what I'm going to talk about is a bit about us, so this civil infrastructure pattern 00:31.840 --> 00:37.240 project, what is this links foundation project that we created nine years ago, it's 00:37.240 --> 00:43.920 goal has to enhance Linux for long-living industrial critical infrastructure use cases, 00:43.920 --> 00:48.720 you may run it on some of your devices, but you don't know it, or some of your use 00:48.720 --> 00:50.760 cases like a train. 00:50.760 --> 00:55.000 We have a lot of things that we do in the project, maybe you heard about the CIP kernel 00:55.000 --> 00:58.680 before, something which lifts that long as the other ones we heard about in the previous 00:58.680 --> 00:59.680 talk. 00:59.680 --> 01:04.640 We do a real-time for that, or we support the real-time project on that testing, additional 01:04.640 --> 01:10.720 testing, we work on security certification for that domain we heard about before and device 01:10.720 --> 01:17.360 updating, principle for us is upstream first, own project seconds, members are coming 01:17.360 --> 01:21.440 from both suppliers of that domain as well as users. 01:21.440 --> 01:27.000 One of the users, so to say, is my employer Siemens, I'm working for Siemens Technology, 01:27.000 --> 01:33.440 foundational technology use days, doing in-house Linux consultancy and development, in 01:33.440 --> 01:38.840 the CIP projects, I'm leading the kernel team, but I'm not doing the work, other people 01:38.840 --> 01:39.840 do that. 01:39.840 --> 01:44.800 I'm maintaining the ESA CIP core project in that project as well, we will hear about more 01:44.800 --> 01:46.680 in the following slides. 01:46.680 --> 01:50.560 And yeah, there are some other projects I'm maintaining and contribute to an open source. 01:50.560 --> 01:56.120 Yeah, that's my daily life, so to say, so we are using Debian, you may wonder why, 01:56.120 --> 02:00.120 for that kind of domain, well, there are a lot of reasons, back and forth, but anyway, 02:00.120 --> 02:03.560 we consider it a mature, high quality mainstream in-line of distribution with a lot of 02:03.560 --> 02:09.680 features, you may need it runs on old hardware, and when you want, it scales down from 02:09.680 --> 02:15.120 small to big, which is also important these days, not too tiny, yes, you know, but small 02:15.120 --> 02:16.760 enough for all use cases. 02:16.760 --> 02:21.840 Yeah, it comes with security updates, surprise surprise, even these days, up to 10 years, 02:21.840 --> 02:26.080 that's also quite important for the domain, and yeah, it's a strong open source community 02:26.080 --> 02:27.720 that we like to build upon. 02:27.720 --> 02:33.920 That's why CIP, when the only CIP, but also when it was before, selected Debian as one of 02:33.920 --> 02:39.560 our feet and one of the pillars that we build upon. 02:39.560 --> 02:43.120 So what are the missing pieces in our puzzle, so to say? 02:43.120 --> 02:49.840 So we're not installing servers, we're not installing Linux systems on train control systems 02:49.840 --> 02:55.480 or on IoT gateways, we actually flashing complete images to that device normally, in the 02:55.480 --> 03:02.160 factory or in some kind of deployment process, and then these devices travel away out of 03:02.160 --> 03:07.280 reach, so there's no one typing in some passphrase for fallback, these things have to 03:07.280 --> 03:10.920 be really robust to be updateable in the field. 03:10.920 --> 03:15.400 That is different from the classical server system I would say. 03:15.400 --> 03:19.680 So we use very conservative approaches here, so you know, you don't see any kind of rocket 03:19.680 --> 03:23.920 signs in the future in the next slides, it's really really basic stuff, and some of them 03:23.920 --> 03:30.320 is probably too conservative by now, but anyway, we want to have this AB pattern for updating. 03:30.320 --> 03:35.040 We want to have the artifacts for that, obviously as well, so not packages, but images 03:35.040 --> 03:40.880 or Delta images, and all that has to be, well, these days, securely booting, obviously. 03:41.120 --> 03:46.000 There must be no gap in this boot process between kernel and file systems, where to 03:46.000 --> 03:52.000 before, what's different from classical installation on an XSX box or something like this, 03:52.000 --> 03:59.000 we sign usually ourselves, we don't trust the existing key infrastructure there. 03:59.000 --> 04:04.000 Yeah, and we also have to customize a few things still, and we try to reduce that, well, 04:04.000 --> 04:07.880 we're using a binary distribution, obviously, but there's something always like the 04:07.960 --> 04:13.360 kernel, for example, for various reasons, but also other small pieces, not hundreds, but 04:13.360 --> 04:18.040 other tens of packages that we like to customize for that. 04:18.040 --> 04:22.120 So how to do that, how to express that, they have the other talks about this before, just 04:22.120 --> 04:27.720 to give you a bit of the background, we use a build system called ESA, which is a, well, I 04:27.720 --> 04:33.480 say it's a unique combination of describing both package building for deviant as well as 04:33.560 --> 04:39.160 image creation, in one piece, so to say, in one repository, still, just like 04:39.160 --> 04:43.880 Octob build systems, you can reuse from that, a bit that we actually do, we use the bitpake 04:43.880 --> 04:49.000 task engine for that, we use the big image, for example, we use some other pieces from the 04:49.000 --> 04:53.880 way ecosystem for building up this thing, but in the end, we're generating deviant packages, 04:53.880 --> 05:00.040 the deviant way, and integrating them mostly the deviant way. We use the layering idea of bitpake, 05:00.120 --> 05:04.840 so that you have a core, then you have this ESA, because we talk about maya later, and then 05:04.840 --> 05:10.040 put your product on top. Those layers can actually weigh more, if you look about large systems, 05:10.040 --> 05:15.640 but that is possible as well, so we have better reuse of these customization bits, even if 05:15.640 --> 05:21.000 they should be few, they pile up over the time, so this is what you can stack this way. 05:22.440 --> 05:27.400 And we have a configuration management tool again, just for reference cuts that we use to pull 05:27.480 --> 05:39.400 in all these repositories together. Now, this is a normal way, classic way to do AB updating on 05:40.520 --> 05:48.680 better devices, on critical devices, so the goal for us is really to not touch the running bits and pieces 05:48.680 --> 05:54.840 for the version A, while we are preparing version B and trying out version B. That means 05:54.840 --> 06:01.560 no shared file system for the critical bits, really separate paths, and that is the way we'll 06:01.560 --> 06:07.640 talk about it before in the past, how we built up the system, we have boot system, which is 06:09.000 --> 06:15.640 immutable, we have root file system on there, and in the end, you have some consistency, 06:15.640 --> 06:21.800 some state that you keep with the device, but it is usually less, and yeah, it is, this is the 06:21.800 --> 06:26.920 other day, the only, currently the only single point where things move together, but generally 06:26.920 --> 06:31.640 you have separate passes there. You have some update agent running on the device to manage these 06:31.640 --> 06:38.520 kind of updates and switch over, but still if you look at that, probably you can build it up, 06:38.520 --> 06:43.400 shouldn't be that hard, but still some questions pop up, if you work on that, like, okay, what 06:43.400 --> 06:49.880 do we do with the boot loader? Well, you would, you if you what do we do there? How do we build the 06:49.960 --> 06:55.320 UKI's, and how do we build the UKI's, which we do for a better, for example, if you have 06:55.320 --> 07:01.320 the overlay or explains the replaced the DTV's, the device trees. By the way, overlays for your 07:01.320 --> 07:05.800 root file system, for that consistency of that, how do you describe the, how you build it up, 07:07.880 --> 07:13.000 do you do a rollback for the consistency as well, so that you can update not only the root file 07:13.000 --> 07:17.000 system, but also the consistency of this case, and how do you build the chain of trust between 07:17.080 --> 07:22.360 all the elements? Now, small pieces, small to the side, also like, for this encryption, 07:22.360 --> 07:27.640 we had a before, how to hook it up with a TPM or whatever hardware security module you have there, 07:28.680 --> 07:33.800 and obviously, which agent to use for the update management. So, all of these things, 07:33.800 --> 07:37.960 basically, have to be answered, you can answer them individually, and this is what many products 07:37.960 --> 07:43.480 in our environment, or it did before, some did it. Good enough, others did it, well, improvable, 07:43.640 --> 07:47.960 and the end, you end up with a lot of knowledge spread out in projects, and that is obviously not 07:47.960 --> 07:53.560 optimal, because one mistake in that thing, wherever, and your system is in danger, or at least 07:53.560 --> 08:00.760 it's becoming not that reliable as it should be. Let's one of the ideas behind pulling this all together, 08:00.760 --> 08:06.600 in at least one way to have some kind of best practice sharing this way, and distributed to the users. 08:06.600 --> 08:13.080 So, what is now is us here, because thing that we are here, pulling together, it brings 08:13.080 --> 08:18.120 some decisions on the table, you can follow them, then you are happy, you may not follow them, 08:18.120 --> 08:23.160 okay, then try something else. These pieces are, for example, we decided to go for your if 08:23.160 --> 08:27.800 base booting, even on the better devices, which these days becoming more commodity, if you think 08:27.800 --> 08:33.720 about system ready, for example, but anyway, this is the decision we made. That means we have 08:33.720 --> 08:39.800 also here, at this stage, not using system D boot, but using UFE boot guard, which is also 08:39.800 --> 08:46.440 UFE-based boot order, more for embedded, without any kind of menu or something like this, 08:46.440 --> 08:52.360 as a switcher between the AB path, and also currently still as the generator for the UKI 08:52.360 --> 08:58.760 images, which has historical reasons, but now it's moving forward also on system D side, 08:58.760 --> 09:02.760 we have, for example, the DB integration there, that was one of the reasons to do the ourselves. 09:03.480 --> 09:09.080 We have key logic in, for the boot paths description, in an interim of the hooks, 09:09.080 --> 09:15.800 these days, so that is about how to select the AB boot paths and how to build up the chain 09:15.800 --> 09:21.080 between the elements there, integrity for the root of S, configuring overlays, as I said, 09:21.800 --> 09:26.520 partitioning groups and all these pieces, and nowadays we are also working on these AB snapshots 09:26.520 --> 09:32.680 for the consistency. Another decision we just made, well, there's ROG, as we know, 09:32.680 --> 09:37.560 but we've grown for SW update, a long, many years ago, as device agent, 09:38.600 --> 09:44.520 we have the ROG-Dorbin overhandler for that in that context as well, so that we can switch 09:44.520 --> 09:51.720 between the AB paths here, and obviously support or integrate, so to say, what SW has 09:51.720 --> 09:57.320 supported, like encryption and signing of the artifacts. We also explore the other image 09:57.320 --> 10:02.120 update for that, so if you want to go deep on that, there's a link there, and yeah, the 10:02.120 --> 10:06.760 plumbing that you need to do a read-only route at best in this case for the ambient on top of that. 10:07.800 --> 10:14.280 So how does this plug together? So keep in mind, we are doing off-device configuration, building, 10:14.280 --> 10:19.640 and signing, so that makes a lot of things easier in the sense that we do not have to deal with 10:19.640 --> 10:24.840 the problem of a normal distribution, how do you sign something on the device, and whatever, 10:25.560 --> 10:29.240 or how do you resolve it, you don't have to sign a device, we do it anyway of device, 10:29.240 --> 10:34.440 that's the advantage here. The layer comes with a bit of help us to do the signing against HSM, 10:34.440 --> 10:40.840 so how do I model, or whatever you have there for your build procedure on that. And then it describes 10:40.840 --> 10:46.520 this bootchain. So again, the details are in other talks, just to give you an overview on that, 10:46.600 --> 10:52.920 as I said, if you boot got selects the AV path, we put an image studio ID and the 10:52.920 --> 10:58.840 clarity hash into the inner drum with S, so similar to previous talk, and sign the whole thing, 10:58.840 --> 11:04.040 and that basically change or put to the next level, so you have then, so to say, the identity 11:04.040 --> 11:13.720 of the next element, the route S, we encrypt, also on first boot, the partition, and decrypted it on 11:14.040 --> 11:19.160 following against the TPM, or an FTPM, with here on the arm side, for example, 11:20.440 --> 11:23.960 other system D, or all the systems, also use the cleavest script sets, 11:25.160 --> 11:28.280 be used to invert your hash again, to select the corresponding route of S, 11:29.560 --> 11:37.160 swanship S, or error, as these days, has an option for the persistency snapshots, 11:37.160 --> 11:42.120 we just hook it up again with the UUID and say, okay, for that route of S, and for that UKI, 11:42.600 --> 11:47.640 take that corresponding snapshot from, yeah, in this case, butter S, at the current stage, 11:47.640 --> 11:53.320 but it can also do other things, and select that thing, and then you have to say the whole chain 11:53.320 --> 11:59.560 built up and then securely identified, and then finally mount the overlays on top, coming from 11:59.560 --> 12:05.160 bar, we have to describe those few bits, or more, hopefully not the whole system, which should 12:05.160 --> 12:09.640 become, yeah, overwriterable, and then persistence across the bar updates. 12:11.720 --> 12:16.520 So again, these are here, because it provides these kind of building blocks, and it also provides 12:16.520 --> 12:21.480 some images for that, but those are for reference and for testing, not for protection. 12:21.800 --> 12:25.320 This is normally what you do, and this is what our users do, they built on top, 12:25.320 --> 12:28.840 the actual production description, how the device should look like, what it should do, 12:28.840 --> 12:33.640 what it should use from that, and what not, but this is the normal idea behind it, so it's not 12:33.640 --> 12:39.960 likely to get a final image from that call layer already, that's what you do on top for your 12:39.960 --> 12:48.840 actual design. So, no demo for me, but if you want to try it out, play around a little bit of 12:48.840 --> 12:54.040 that, go for the repository, clone it, the build system requires a privileged Docker or a 12:54.040 --> 12:59.960 apartment environment, but then otherwise just type in the command and that menu pops up and select 12:59.960 --> 13:05.160 something from that, this support x86, I'm 32 bit 64 bit and even wrist five these days, so 13:05.160 --> 13:10.520 looking forward to Pixie, to fully support it, not all features for wrist five yet, but yeah, it's 13:10.520 --> 13:17.240 coming. And you can start these things in QAmo for example, x86, you can just touch it on an x86, 13:17.240 --> 13:21.400 domain machine, it also works, and the other devices are more special in this regard. 13:21.400 --> 13:31.000 So, yeah, that's for trying out, okay, sorry because build it folks, your logo is meshed 13:31.000 --> 13:37.000 anyway, we need also reproducible images, bit identical, why is it essential? Well, supply 13:37.000 --> 13:41.560 insecurity obviously is one reason, but it also makes the data a bit smaller, so if you do not 13:41.560 --> 13:48.200 get random output just by changing one bit or not reliable output, so this is another important 13:48.200 --> 13:52.840 step, but we work together with reducible for quite a while in the context of deviant, 13:52.840 --> 13:57.640 but not only, so many packages they are already reproducible, it's helpful for us, 13:59.640 --> 14:05.480 we try to close the main gaps there and now focus for us specifically on the aspect of image building 14:05.480 --> 14:10.360 reproducibly, so like for there being, for example, it's about the ISO image which should be 14:10.360 --> 14:15.480 reproducible, but we do that also with the product images, sort of say, so many not all of the 14:15.480 --> 14:20.200 ISOs here, we call images produced right now are reproducible, between the lot on the file system 14:20.200 --> 14:25.000 production, we also have to patch a little bit on open, and better for that regard for example, 14:26.040 --> 14:31.320 we also patched some tooling which is needed for building, yeah, the ESP is in things like this, 14:31.320 --> 14:36.520 so does file system, there's something to open there, but anyway, we have a solution at least downstream, 14:36.520 --> 14:41.560 we work with stifoscope that you can actually scan complete images, disk images, do the visualization 14:42.120 --> 14:46.840 of the elders there, and we have a weekly pipeline running which checks that our images that we have 14:46.840 --> 14:54.280 currently under on the radar are kept reproducible and nothing is regretting there, yeah, in general 14:54.280 --> 14:59.640 we try to work with a deviant upstream more and more, so we or you continue to talk about all bits 14:59.640 --> 15:02.840 because we don't want to keep them all the time in our layer and we don't want to keep them as 15:02.840 --> 15:06.920 at hawk package building, so if you boot got for example, it's an upstream package plan out of 15:06.920 --> 15:13.000 the dependencies as well, we took over the maintenance for that, we worked this SW update upstream 15:13.000 --> 15:17.000 where it becomes a distribution package and not just a york do package in the sense, 15:17.000 --> 15:22.120 so it was very much focused in the past on being built on configurable, for distribution, 15:22.120 --> 15:26.040 by distribution, it obviously more flexible, runtime configurable, it's now doable, 15:27.080 --> 15:31.960 so yeah, it's an official deviant package, we use it, in some cases we have more version 15:31.960 --> 15:37.880 of our deployment for historical reasons but the future is always going for the upstream package. 15:38.920 --> 15:44.680 We also worked the step-in snapshot.org, so the infrastructure to have a reproducible package set, 15:44.680 --> 15:49.960 it can pick from, that is working way better now, thanks to Felix and the room, 15:51.720 --> 15:55.080 we are still trying to improve a little bit on, in the drummer's production there's an open issue 15:55.080 --> 15:58.440 we're discussing to have to be more complicated and there's more to come on that. 15:58.840 --> 16:06.680 So what are the future plans, so we're never done, next step is coming, as I said, 16:06.680 --> 16:11.880 we are currently working on the AB snapshot for the persistency and some good approaches there, 16:11.880 --> 16:16.920 we have something in-house already employed, which is conservative but it's working, 16:16.920 --> 16:21.400 we're going to look in something more new, it's the butter effect approach and there are some 16:21.400 --> 16:26.440 open questions and I had already discussed in these days with some folks here that could be done 16:26.520 --> 16:30.360 different, you should be done differently but we definitely have to work on is the recovery procedure 16:30.360 --> 16:36.120 and the reset procedure, also factory reset, encryption has to be looked upon again if this is actually 16:36.120 --> 16:42.360 a good stacking up here right now with the grip underneath, there's some problematic cases where 16:42.360 --> 16:45.800 you don't want to have butter fs and the mid-teen, for example, obviously you don't want to have 16:45.800 --> 16:51.000 a snapshot of your locks, that is currently, is not expressible but it can be made, it's not that 16:51.000 --> 16:55.880 hard but there are other use cases and it's not should be the only solution for us, 16:56.040 --> 17:00.520 so as I said, we have internally already the deployment with the snapshot for the block players 17:00.520 --> 17:05.400 so to say, there's always three around, we know that, we will have probably more alternative 17:05.400 --> 17:10.760 in the future to describe this pattern with different implementations depending on your demand. 17:12.360 --> 17:17.240 Delta update improvements for an Ableman actually for UKI is technically not the hard thing but 17:17.240 --> 17:22.280 that's what I was doing, we are close to that, yeah, one of the weak points I have to 17:22.280 --> 17:28.200 bit as maintain as our documentation, so it's a little bit not that easy to get started, 17:28.200 --> 17:33.720 it's the demo mission are there but if you go from there to your own product, that way may 17:33.720 --> 17:39.400 be, should be made more convenient and more clearer so we will work on documentating the interfaces 17:39.400 --> 17:44.680 that we have there and providing more light up HelloBulls currently are probably in. 17:45.880 --> 17:50.440 Already work internally but also something that's not that easy if you look at it as a measure put 17:50.840 --> 17:56.120 so that we can even go up to remote at this stage and at some stage there are some use cases 17:56.120 --> 18:00.520 fortunately currently we are getting very far the verified boot but yeah it's coming. 18:02.360 --> 18:08.840 Also discussed already was from folks here, how to do, maybe how to make things more reusable 18:08.840 --> 18:12.360 that we are currently building for the inner drama best in the turn out that we probably should 18:12.360 --> 18:16.760 look into drag-out soon for the for-devian, so yeah people have to look into if you can make some of 18:16.840 --> 18:22.840 these pieces possibly reusable even outside of the special domain we have, they will see it and again 18:22.840 --> 18:28.120 this is a collection of pattern and ideas so we will also look into further evolving yeah as the 18:28.120 --> 18:32.600 distributions for the class of use cases evolve if you can pick up something from that and reuse it 18:32.600 --> 18:38.520 ideally we don't have to have all the gluing done ourselves but rather doing well we using more 18:38.520 --> 18:45.240 what is already there out there so with that again focus on on this activity is really on the 18:45.240 --> 18:51.000 robot unattended software update in the field locked secured that's all possible with that in theory 18:51.000 --> 18:55.400 in practice you have to do a lot of plumbing and if you do it wrong you have problems possibly 18:55.400 --> 19:00.360 we really want to provide these kind of best practices that we are using, share it this way 19:01.000 --> 19:05.720 and evolve it it could be blueprint you can just in getting inspired from that you can use this also 19:05.720 --> 19:11.640 maybe in a context of a yup to build a built-out system but well if you're using it on a 19:11.720 --> 19:16.520 deviant system then you have a lot of pieces that you can reuse we wanted according to test them 19:17.320 --> 19:23.400 certify them and maintain them where needed look at the repositories and if you have any follow-ups 19:23.400 --> 19:29.240 on that there's a meaning this as well so we happy to take input and feedback on that as well 19:29.240 --> 19:33.240 with that thank you for your attention 19:42.520 --> 19:48.280 is the device software or device programmer also open source and they still need to trust in proprietary 19:48.840 --> 19:52.040 so the question is if the firmware is open source it clearly depends 19:53.240 --> 19:59.400 so obviously we try to have it open source so some again for a cement some of our products actually 19:59.400 --> 20:05.800 use co-about others don't we use u-boot then it's open source but that could also be proprietary bits 20:05.880 --> 20:11.720 it really depends it's not part of c-i-p so we keep the firmware so to say out of scope for us 20:11.720 --> 20:17.720 it comes with a device at least this is the vision and that's why it's not in clear answer to that 20:20.200 --> 20:21.800 yes please 20:21.800 --> 20:27.080 I want you to know is there a bit of a busy fetching the feedback from the computer 20:28.520 --> 20:33.160 repository or is it fetching the sources from the package so the question is 20:33.160 --> 20:38.520 if we are fetching for the build procedure the binaries from deviant upstream or if you're just 20:38.520 --> 20:45.960 fetching the sources yes yeah so both and but the normal case is as I said we want to use the binary 20:45.960 --> 20:50.440 parts of it so we are fetching the binaries and we are relying on reproducibility ideally 20:51.080 --> 20:55.240 we are patched fetching from snapshot if we want to have a specific version you pinned it for 20:55.240 --> 21:01.640 example if you have to customize something or if you have to let's say use the package which is 21:01.640 --> 21:07.000 going to be in the next release of deviant already now then we go for salsa for example and pull 21:07.000 --> 21:12.440 the depinization and the sources from there or we even go for upstream differently and pull our own 21:12.440 --> 21:18.200 deviantization on top and build it at hawk during the image build process as well that is the 21:18.280 --> 21:22.760 as I said the unique feature of the user build procedure that you don't have to do it separately you 21:22.760 --> 21:27.560 can but you don't have to you can also describe if it's a single repository the whole procedure 21:27.560 --> 21:37.800 from customizing to integrating an image in more questions what is slides which kind of versions 21:37.800 --> 21:45.880 get the CIP like the SLPS so the question is about our kernel okay so which version do we pick 21:46.520 --> 21:53.400 so we started with four or four some nine years ago eight years ago and so far the regular 21:53.400 --> 21:58.840 schedule is every two years we pick an LTS kernel and declared to be an SLTS kernel that just happened 21:58.840 --> 22:05.320 again for six or twelve and that's currently the schedule the decision is done by the members of 22:05.320 --> 22:10.440 the project because we in the end we have to carry the effort and the commitment for that so it's 22:10.520 --> 22:15.720 happening within the CIP workgroup and the CIP technical steering committee in the end 22:26.440 --> 22:32.040 for the question is if we want to use system the what extension so it's extension yeah so 22:32.920 --> 22:38.120 if we want to use more of system the extension for that the concrete plan doesn't exist yet 22:38.280 --> 22:42.440 but as I said we want to evolve that over the time so we currently went for the 22:43.320 --> 22:47.960 pragmatic simple approach of plugging a lot of things out together ourselves and again it grew 22:47.960 --> 22:54.360 and now the point is actually to revise and think about again if it grew to far if you can start to 22:54.360 --> 22:59.640 reuse so if you look for example we started this cleavace because at that time system the 22:59.640 --> 23:04.920 CIP Dendol wasn't available for the version that we were running there been 11 nowadays obviously 23:05.080 --> 23:09.240 we used the newer deployment system decrypt and roll and further things will probably 23:09.240 --> 23:15.080 the time yeah come automatically if the step for example towards breakout is happening then 23:15.080 --> 23:19.400 we will also look obviously more into what can be reused from there we don't want to reinvent 23:19.400 --> 23:24.120 wheels if it's not really necessary so we rather want to add then the bits and pieces to that but 23:24.120 --> 23:28.840 yeah that happens over the time as you know you may also miss the point where you should 23:29.320 --> 23:39.880 better take something which is all you existing by now okay okay that sounds very interesting we should 23:39.880 --> 23:41.160 discuss that 23:59.480 --> 24:10.200 one star one star one star one set so the question is about how do we what do we do with the 24:10.200 --> 24:16.920 watchdog or how do we do the monitoring of the AP yeah so that's what I think I stepped over 24:16.920 --> 24:23.000 because of the time yes one of the reason we have our own bootloader here is that again how to 24:23.000 --> 24:28.920 sell watchdog drivers to if he boot got sorry to it was system reboot and we actually use the 24:28.920 --> 24:33.160 watchdogs from your boot where available so when we're booting from your boot you boot is 24:33.160 --> 24:37.800 responsible for setting up a watchdog to monitor the complete boot procedure up to the point 24:37.800 --> 24:44.920 where your system is up and running again where we don't have that on X86 for example we use our own 24:44.920 --> 24:51.080 watch watchdog starting system so to say our watchdog driver in an if you boot got which 24:51.080 --> 24:55.640 starts the watchdog and hands it and that's it then hand over to Linux or Linux can continue that 24:55.640 --> 25:01.800 and finally yeah stop it so to say or or take over the running of that watchdog this is part of 25:01.800 --> 25:06.440 the concept yes so that we can actually detect if something gets stuck kernel booting in the 25:06.440 --> 25:12.520 drum with a starting over whatever looping later on before the final system is up and running that 25:12.520 --> 25:18.040 would be detected by this watchdog pattern let's go pan biops so we're spanky all 25:21.080 --> 25:33.080 that