WEBVTT 00:00.000 --> 00:15.000 My name is Hartmut. I'm leading a design team at Canonical. I also have a passion project and 00:15.000 --> 00:21.000 I'm going to talk about the passion project today. If you think design systems is a 00:21.000 --> 00:26.000 daunting task, I think CLI design is actually for most designers, quite a daunting task. 00:27.000 --> 00:33.000 My colleagues told me I should start with this. This is what I'm going to talk about 00:33.000 --> 00:38.000 after I finish the motivation of it, but I'll start with the motivation of it. 00:38.000 --> 00:46.000 So I would love to talk about CLI's and design. These are just like Wikipedia, of course, 00:46.000 --> 00:53.000 and a famous non-designer who made a quite nice statement about design. 00:53.000 --> 00:59.000 And what I find really interesting is that CLI's, even though nowadays I might see 00:59.000 --> 01:04.000 as something that's always been there, were actually invented to improve the user experience. 01:04.000 --> 01:09.000 So before that you had punch cards, and then suddenly you had something that was interactive 01:09.000 --> 01:15.000 and it was user-friendly. And this is something that I think we can double down on. 01:15.000 --> 01:24.000 And I think design is a perfect support in doing so, because design for me is not so 01:24.000 --> 01:33.000 much about how you arrange graphics. It's more about how something should work. 01:33.000 --> 01:39.000 And of course that includes some graphic elements. And this is just to show. 01:39.000 --> 01:46.000 There is a need for CLI tools to improve because they can get pretty complex. 01:46.000 --> 01:54.000 And it can create critical situations. And if someone is able to explain to me all of those statements, 01:54.000 --> 02:00.000 I had to look them up. Please find me after the talk and I'll buy your coffee. 02:00.000 --> 02:07.000 So this is just to say, you can have fatal consequences. A lot of these things are not trivial to understand. 02:08.000 --> 02:15.000 And for designers, I think to challenge is even greater than for designers who want to involve themselves 02:15.000 --> 02:21.000 with open source, to involve themselves with CLI's. And this is really trying to motivate you. 02:21.000 --> 02:28.000 You should be doing that even if today's CLI's are sometimes really nicely, I would say, 02:28.000 --> 02:37.000 using colors and layout and feel very complex. Or there is, I would say, a lot of things going on, 02:37.000 --> 02:42.000 so that you have to really think about the state that you are dealing with. 02:42.000 --> 02:49.000 So this is something that I think makes it really fun to work with CLI's. 02:49.000 --> 02:53.000 And so I want to start with building some bridges, maybe. 02:53.000 --> 02:59.000 So developers should definitely be involved when talking about CLI's. 02:59.000 --> 03:07.000 And I think developers can contribute to CLI design and to really meaningful ways. 03:07.000 --> 03:14.000 Because they have been working with the material, how what you can actually do for a long time. 03:14.000 --> 03:20.000 So they know what is possible. And sometimes they are also really interested to what might be possible. 03:20.000 --> 03:26.000 Because that's what this is a little bit about, how to extend what is currently possible. 03:26.000 --> 03:33.000 So they can actually share the limits and help you explore something if you have an idea. 03:33.000 --> 03:40.000 But they also have the users of CLI's. Most developers work with CLI's all day long. 03:40.000 --> 03:47.000 And that does not mean they are the user for the specific CLI that you are trying to design, 03:47.000 --> 03:55.000 they have the CLI mindset and they can actually also share with you the joys of using a CLI. 03:55.000 --> 04:04.000 So things that like a CLI can feel make you feel extremely powerful, which is something that I think is a really good quality in one that we want to keep. 04:04.000 --> 04:11.000 But on the other side of the bridge designers, most designers I know, bring a curiosity with them. 04:11.000 --> 04:15.000 So that they really try to understand what is the goal that someone tries to achieve. 04:15.000 --> 04:21.000 And that can be really useful because if you want to build something not just for yourself, 04:21.000 --> 04:25.000 I know a lot of open stories is building things for yourself. 04:25.000 --> 04:32.000 But if you want to reach others, it makes sense from time to time also to look into the context of what other people are trying to achieve. 04:32.000 --> 04:35.000 And then help them do that. 04:36.000 --> 04:43.000 Also designers get taught skills that provide structure and flow. 04:43.000 --> 04:47.000 And both of these things are something that we want to have in CLI. 04:47.000 --> 04:57.000 So that you can actually try to redefine the solution space to meet the mental models that users might have. 04:57.000 --> 05:04.000 So now I'm getting to the part where I'm talking about how the magic actually happens. 05:04.000 --> 05:13.000 Before I do that, just to cover it, CLIs are different and also have this open source joke. 05:13.000 --> 05:20.000 But there is no graphical UI, which makes it really hard for you as a designer to say, let's just do something. 05:20.000 --> 05:25.000 I'll send you a scribble because what will you scribble in CLI? 05:25.000 --> 05:28.000 They're also used by experts. 05:28.000 --> 05:37.000 And that might feel like you don't need design because experts actually like really complex things that are hard to do because then they can only, they're the only ones who can do it. 05:37.000 --> 05:39.000 And that's kind of a good thing. 05:39.000 --> 05:41.000 But also experts want to do their job. 05:41.000 --> 05:47.000 So it makes sense to have things that are simple and that they don't have to relearn every time they use them. 05:47.000 --> 05:57.000 And one thing that is particularly interesting for you when looking at CLIs is when you're looking at a graphical user interface, you see state. 05:57.000 --> 06:03.000 You're looking at something and what you're looking at represents what the system state is. 06:03.000 --> 06:10.000 If you're working with a CLI, you're giving a command and then something happens, but there's no state. 06:10.000 --> 06:17.000 So that's fundamental difference and one that makes designing for CLIs extremely interesting. 06:17.000 --> 06:21.000 And this is what you can do as a designer. 06:21.000 --> 06:23.000 You can work on accessibility. 06:23.000 --> 06:30.000 And it's actually accessibility for CLIs is a very thankful topic because a lot of it is built in. 06:30.000 --> 06:34.000 You can work on visual hierarchy because spacing and typography matter. 06:34.000 --> 06:48.000 You can work on colors, which is hard, even harder than for graphical user interfaces because sometimes most of the time you actually can determine what the color will look like, you just have a rough idea of it. 06:48.000 --> 06:56.000 You can work on iconography, which, again, means you show the icons that someone else will have made. 06:56.000 --> 07:05.000 And you can work on copy, which means you can really try to be clear when something is communicated. 07:05.000 --> 07:09.000 And there are actually really nice talks about this. 07:09.000 --> 07:12.000 So I will try to focus on the last three bits. 07:12.000 --> 07:16.000 And that is, you can work on information architecture. 07:16.000 --> 07:21.000 And I would call it conceptual architecture and I hope to be able to explain in the minute why. 07:21.000 --> 07:25.000 You can work on flows and you can work on error handling. 07:25.000 --> 07:28.000 So let's start with the conceptual architecture. 07:28.000 --> 07:31.000 And this is something I find really, really fascinating. 07:31.000 --> 07:41.000 When working with a CLI, you have to be extremely clear what are the things that you work with, the object and what are the actions that you want to execute. 07:41.000 --> 07:49.000 And these are things that, in a graphical user interface, you also will have to decide what are these things. 07:49.000 --> 07:52.000 But it doesn't necessarily force you to do so. 07:52.000 --> 08:02.000 And with the CLI, this is a really important question to answer because it will create the structure of your CLI. 08:02.000 --> 08:13.000 And the structure or the shape of your CLI will be what users see and what will help them understand what is your command actually doing. 08:13.000 --> 08:21.000 And one example for that is, or how to deal that is, we are working on an internal standard and canonical. 08:21.000 --> 08:26.000 And we are trying to define what is the grammar that we use for commands. 08:26.000 --> 08:31.000 So this is not the only way that you can do it, even though it says canonical grammar. 08:31.000 --> 08:39.000 This is how we want to do it. But what we want to do is we want to be very clear about what kinds of words do we use for commands. 08:39.000 --> 08:47.000 And in our case, we said, let's use verbs. And then we have something that is like the primary thing that a tool does. 08:47.000 --> 08:54.000 And as long as you're doing something that is like the primary purpose of a tool, let's keep it extremely simple, just use verbs. 08:54.000 --> 09:03.000 And then if you're building more and more stuff into a draw, which we tend to do, then you're allowed to say, okay there's other things that this tool can do. 09:03.000 --> 09:09.000 And in that case, you can get a little bit more complex, but really be clear about what your primary purpose is. 09:09.000 --> 09:20.000 And these are rules that we can hand out to engineers so that they can actually have something where they can say, okay, if I'm doing it like this, I'm aligning. 09:20.000 --> 09:32.000 So we are writing a standard and I also want this to become a design system at some point. And this is one example of an output that we want to achieve, where 09:32.000 --> 09:47.000 being canonical, we build our own tool to install software on Ubuntu. And what's nice about this is that there is a really small set of core commands. 09:47.000 --> 09:56.000 And this is what you like look at first and then you understand, oh, this thing is doing five things or six things and I can deal with that. 09:56.000 --> 10:10.000 And then at some point you might find out this is not enough. I want to do more. And then we have these 50 other commands that are nicely grouped that help you do more advanced things with the same tool. 10:10.000 --> 10:19.000 So it's kind of a layer to approach, but one where we try to really highlight the primary purpose of tool. 10:19.000 --> 10:34.000 Second flows, use states and feedback. And here I find myself and my designers often asking the question, what is important, so for views. 10:34.000 --> 10:42.000 How can we show what is important? And there are some things in CLIs that are different than graphical user interfaces or in web designs. 10:42.000 --> 10:54.000 For example, there's nothing like the above the folding and web. It's actually the last thing that you put on a command line is often the thing that is most visible because it's just above the next command that someone will enter. 10:54.000 --> 11:01.000 It's like the last thing that you said to someone is maybe something they will remember. 11:01.000 --> 11:08.000 And then there's also structure that you want to bring into your output. So it's extremely important to be consistent about tabular data. 11:08.000 --> 11:13.000 And we are making effort to differentiate between tabular data that we showed humans. 11:13.000 --> 11:23.000 And tabular data that is piped into another program. Because the second needs to be exactly formatted the same every time. 11:23.000 --> 11:29.000 And the first needs to be as clear as possible. So we're trying to actually differentiate between those. 11:29.000 --> 11:39.000 We're necessary. For states and feedback, again, what's important is actually really hard with command line tools because you have so little space to show something. 11:39.000 --> 11:47.000 It's really important to think hard about what would a user need to know and only give that feedback. 11:47.000 --> 11:56.000 But don't omit anything that they might want. And that means that when you're building a CLIs, you're actually not building one interface, you're actually building a couple of interfaces. 11:56.000 --> 12:04.000 Because you need to choose the user say, I want this to be quiet, I want this to be verbose, I want the back input output. 12:04.000 --> 12:12.000 So you're actually not building one piece of feedback, you're building seven or six or five modes in parallel. 12:12.000 --> 12:18.000 And last progress is extremely important. This can be very simple. 12:18.000 --> 12:26.000 And it's been there actually since the 80s. There's a very nice video I found. Like the invention of the progress bar. 12:26.000 --> 12:32.000 And we did some research how to actually build like the for us optimal progress bar. 12:32.000 --> 12:44.000 And we also have some standards how to make progress really small and compact. And how to have like a hierarchy of different things that are progressing and then something that subsumes that. 12:44.000 --> 12:46.000 And this is something that we also want to standardize. 12:49.000 --> 12:58.000 And finally, our handling. And in our handling, this is something that most CLIs don't do very well. 12:58.000 --> 13:08.000 So when you are having a dialogue with someone, that's fine. It's actually really like, I mean, there was this. 13:08.000 --> 13:14.000 Surgeons of command lines, I think when chatGPD came out and people suddenly laughed having like a conversation with someone. 13:14.000 --> 13:21.000 You can also have conversation with a command line tool, but it can also and really badly if there is something that goes wrong. 13:21.000 --> 13:29.000 Because in that moment, you want to know what actually went wrong. And when you followed my advice and said, don't give too much output. 13:29.000 --> 13:33.000 It might be that there's not enough to understand what is actually happening. 13:33.000 --> 13:40.000 So what you might want to consider adding is suggestions for first time users, but also for infrequent errors. 13:40.000 --> 13:46.000 Because yes, if someone has seen an error a thousand times and kind of expected that happens maybe a dozen teenagers explanation. 13:46.000 --> 13:52.000 But if you have a tool that is rarely used or an error that is not very frequent. 13:52.000 --> 14:00.000 Feel free to give a lot of information about that. And finally, you're describing the shape of the command. 14:00.000 --> 14:06.000 But that doesn't mean you have to limit yourself to understanding just the words that you said you want to understand. 14:06.000 --> 14:12.000 So what you can do is when you build a CLIs, you can be a lot more forgiving and say, oh, you said this word. 14:12.000 --> 14:15.000 Did you perhaps mean this one? 14:15.000 --> 14:25.000 And you don't have to train people to use this other word, but you can be like a little bit more friendly than saying command not found in that case. 14:25.000 --> 14:30.000 And there's a nice example for that last one in Ubuntu. 14:30.000 --> 14:39.000 This was done a long before my time. If you run a command in the command, something you can't do it will actually help you install the application that you wanted to run. 14:39.000 --> 14:43.000 So this is kind of a really nice standard feature. 14:43.000 --> 14:49.000 We also in Ubuntu have both extremely short version of feedback. 14:49.000 --> 14:53.000 So snap gives you the minimum amount of feedback and apt. 14:53.000 --> 15:00.000 The other install tool for Ubuntu gives you like all the feedback that you never wanted. 15:00.000 --> 15:11.000 So finally, again, back to the motivations. What's in it for you as a designer? I think as a designer limitations are a really good thing because they help you focus. 15:11.000 --> 15:24.000 So if you are trying out CLI design, if you are more lighting as a CLI designer, that will make you more aware of the things that you can do with CLIs. 15:24.000 --> 15:30.000 And that CLIs ask you basically to do, and that can make your better designer overall. 15:30.000 --> 15:36.000 So that is everything I wanted to present today. 15:36.000 --> 15:41.000 There's also an open design group at Canonical that's the NEMA address. 15:41.000 --> 15:46.000 And one of my colleagues has built prototyping tool for CLIs. 15:46.000 --> 15:51.000 And if you want to try that out, you can scan the QR code and he'll get an email from you. 15:51.000 --> 15:56.000 Thank you. 15:56.000 --> 16:00.000 Do you have time for questions? 16:01.000 --> 16:04.000 Yeah. 16:04.000 --> 16:09.000 Yes, please. 16:09.000 --> 16:17.000 Some people are told the unaware of the good or bad user interface. 16:17.000 --> 16:26.000 So the comment was some people are not aware of not able to differentiate between a good and bad interface. 16:26.000 --> 16:30.000 You mean people that are developing a tool? 16:30.000 --> 16:40.000 I think if you have someone that is for themselves not able to differentiate what sometimes helps is if you bring them together with the people who are using the tool. 16:40.000 --> 16:45.000 Because then they might be able to develop a sense of, okay, there's actually a problem. 16:45.000 --> 16:49.000 Might work, might not work. 16:49.000 --> 16:51.000 Yes, please. 16:52.000 --> 16:59.000 One, you know, a fringe, but not that it's important for you that interacting with terminals is screen readers. 16:59.000 --> 17:07.000 And real chemical of maybe you see them could use guidelines on how to structure your CLIs output. 17:07.000 --> 17:09.000 You know, to play well with computers. 17:09.000 --> 17:12.000 That's for example, you know, well, it's really very rough. 17:12.000 --> 17:19.000 So we have to use the asking, you know, control codes to produce like colors or links. 17:19.000 --> 17:20.000 Yeah. 17:20.000 --> 17:23.000 You have to send me to the data that is textual. 17:24.000 --> 17:31.000 So the question was, will someone maybe my team work on accessibility for CLIs? 17:31.000 --> 17:35.000 And the answer is to a certain degree, yes, I would say. 17:35.000 --> 17:41.000 So we're already including like everything that I showed, like the transient communication. 17:41.000 --> 17:44.000 Can only be something that you add to make it nicer. 17:44.000 --> 17:47.000 It cannot be something that you rely on, same with color. 17:47.000 --> 17:50.000 Color can only be something that you add to make it nicer. 17:50.000 --> 17:55.000 You cannot rely on it because you are not like sure that everyone will be seeing that. 17:55.000 --> 17:58.000 You're actually sure that not everyone will be seeing that. 17:58.000 --> 18:04.000 So that element is baked into like the guidelines that we are currently writing. 18:04.000 --> 18:15.000 The addition of like if you want to help screen readers understand what is actually happening at the moment and give them hints. 18:15.000 --> 18:24.000 Let's a topic that would need, I would say, further investment that we haven't done so far. 18:24.000 --> 18:27.000 But if you're willing to help us, please reach out. 18:27.000 --> 18:30.000 We are definitely interested in making that. 18:30.000 --> 18:31.000 Yeah. 18:31.000 --> 18:36.000 So my question was treated by the fact that you spray dry patches to like modular progress bar. 18:36.000 --> 18:37.000 Yeah. 18:37.000 --> 18:40.000 And then you know, you don't know someone to listen to over. 18:41.000 --> 18:53.000 Yeah, but at the moment our approach for screen readers would be to respect of no color flag and respect like so if we do that in every single eye that we build. 18:53.000 --> 18:55.000 So that we don't make it harder for the screen. 18:55.000 --> 18:58.000 We also would like to make it easier if possible. 18:58.000 --> 19:00.000 Yes, please. 19:00.000 --> 19:01.000 Okay. 19:01.000 --> 19:04.000 So the question was, there might be some. 19:04.000 --> 19:08.000 So I don't know if you should focus on. 19:08.000 --> 19:10.000 What do we do anymore, maybe? 19:10.000 --> 19:11.000 Short one. 19:11.000 --> 19:12.000 Okay. 19:12.000 --> 19:17.000 So the question was, there might be some. 19:17.000 --> 19:20.000 So I don't know if you should focus on. 19:20.000 --> 19:22.000 What do we do anymore, maybe? 19:22.000 --> 19:23.000 Short one. 19:24.000 --> 19:27.000 Okay. 19:27.000 --> 19:38.000 So the question was, there might be some like some problem with people sharing a command online probably to some others where they're using what we would call a short flag. 19:38.000 --> 19:41.000 So one letter. 19:41.000 --> 19:45.000 And I'd say short flags are there for a reason. 19:45.000 --> 19:47.000 They are extremely convenient. 19:47.000 --> 19:52.000 So what we do is we say if you have a short flag, always have a long flag. 19:53.000 --> 19:55.000 So people can use the long flag. 19:55.000 --> 19:58.000 And only have short flags for things that people use frequently. 19:58.000 --> 20:01.000 So we encourage the use of long flags. 20:01.000 --> 20:04.000 But we do not enforce them for everything. 20:04.000 --> 20:06.000 And you want to answer to that. 20:16.000 --> 20:17.000 Okay. 20:17.000 --> 20:19.000 Then we. 20:19.000 --> 20:23.000 If we remove the short versions, people are forced use of long versions, which makes the sharing better. 20:23.000 --> 20:26.000 And I would say. 20:26.000 --> 20:33.000 I do not completely disagree with that, but I'm a little bit careful with taking convenience away where it does make sense. 20:33.000 --> 20:40.000 And for some of our commands that are extremely often used and actually type manually, I would say, I would be careful with doing that. 20:40.000 --> 20:46.000 As I said, like, for 90% of our flags, we are using long flags for the ones. 20:47.000 --> 20:48.000 That are short. 20:48.000 --> 20:51.000 They are usually so frequent that people know them. 20:51.000 --> 20:54.000 And you can always find out by using help. 20:54.000 --> 20:56.000 So it's, I would say, a trade off. 20:56.000 --> 20:58.000 But one where we lean on. 20:58.000 --> 21:00.000 We have ever possible. 21:00.000 --> 21:02.000 Take a long flag. 21:02.000 --> 21:06.000 And then leave the rest to the designers of the tool themselves. 21:07.000 --> 21:09.000 I'm going to come from that. 21:09.000 --> 21:11.000 We're trying to share some of the question. 21:11.000 --> 21:12.000 Just come on. 21:12.000 --> 21:18.000 Also have some documentation regarding the arguments of the format. 21:18.000 --> 21:19.000 Okay. 21:19.000 --> 21:21.000 Let's say I have a certain cloud. 21:21.000 --> 21:23.000 And I will create secrets. 21:23.000 --> 21:24.000 With the. 21:24.000 --> 21:25.000 Okay. 21:25.000 --> 21:26.000 Cloud points. 21:26.000 --> 21:28.000 Create secret or secret. 21:28.000 --> 21:29.000 Create. 21:29.000 --> 21:30.000 Yes. 21:30.000 --> 21:31.000 I would prefer. 21:31.000 --> 21:33.000 Has been separated by the main. 21:33.000 --> 21:34.000 So like. 21:35.000 --> 21:36.000 Yeah. 21:36.000 --> 21:37.000 Yes. 21:37.000 --> 21:38.000 We do have an internal. 21:38.000 --> 21:39.000 Sorry, the question. 21:39.000 --> 21:40.000 Thank you. 21:40.000 --> 21:42.000 The question was, would we use like an inverse. 21:42.000 --> 21:52.000 Notation for using like if we have a subcommand using a noun and then the verb. 21:52.000 --> 21:55.000 And at canonical we decided not to do that. 21:55.000 --> 21:57.000 It's I think it's a perfectly valid. 21:57.000 --> 22:02.000 Like approach and one that I've seen other people use to to to success. 22:02.000 --> 22:03.000 I would say. 22:03.000 --> 22:06.000 The canonical we said we want to focus on the action. 22:06.000 --> 22:08.000 The action is the most important bit. 22:08.000 --> 22:11.000 So in that case if we have a commanded. 22:11.000 --> 22:13.000 Coincidentally sometimes great clouds. 22:13.000 --> 22:15.000 We would say it's create cloud. 22:15.000 --> 22:17.000 If it is the main purpose. 22:17.000 --> 22:19.000 Then we would find a word that. 22:19.000 --> 22:22.000 I'm not sure create is a good word for clouds. 22:22.000 --> 22:24.000 That would be a good verb. 22:24.000 --> 22:25.000 That helps you understand. 22:25.000 --> 22:26.000 You're now. 22:26.000 --> 22:29.000 Like creating that cloud. 22:30.000 --> 22:32.000 That's the second thing that that's been helpful. 22:32.000 --> 22:35.000 I used to use a somewhat different. 22:35.000 --> 22:36.000 With all of these stops. 22:36.000 --> 22:37.000 So. 22:37.000 --> 22:38.000 Yeah. 22:38.000 --> 22:43.000 The other thing that makes it accessible to me for the. 22:43.000 --> 22:44.000 The one cloud. 22:44.000 --> 22:45.000 It's okay. 22:45.000 --> 22:47.000 And the secret creates. 22:47.000 --> 22:48.000 And if I just like, 22:48.000 --> 22:49.000 Secret and prestige. 22:49.000 --> 22:50.000 Yeah. 22:50.000 --> 22:51.000 All of which is really. 22:51.000 --> 22:52.000 Yeah. 22:52.000 --> 22:54.000 So you, VPN notation would be easier for auto completion. 22:54.000 --> 22:55.000 That's definitely true. 22:55.000 --> 22:57.000 That's also an argument we discussed with our engineers. 22:57.000 --> 23:03.880 engineers, but what we also will be doing as we can control or to complete, we can also 23:03.880 --> 23:10.600 give you just if you press cloud, we can give you the option of completing that to create cloud. 23:10.600 --> 23:16.600 So that's how we will deal with that. That's kind of not saying type cloud first, but 23:17.720 --> 23:21.320 like accepting you type in cloud first and then helping you. 23:22.280 --> 23:25.640 We have time for one very quick question. 23:27.160 --> 23:27.880 This one in. 23:34.680 --> 23:40.440 If there's a design pattern that links the CLI to the API layer, we have done, 23:41.240 --> 23:43.160 yeah, that was a question, I think. Yeah. 23:43.480 --> 23:49.800 We have thought about that, and our feeling was no, 23:50.440 --> 23:56.920 like the answer that we have currently is no because when you design an API, 23:56.920 --> 24:02.520 what you want to do is you want to have a separation of domains. 24:03.880 --> 24:09.240 That is very, very clear. And that would be closer to the command line structure where you 24:09.240 --> 24:14.440 start with the noun and then have the verb. I think in that case you can have an alignment 24:14.440 --> 24:19.640 of the API and the CLI and our case we can't because we start with the verb. 24:19.640 --> 24:24.120 Is that a bad or a good thing? We don't think it's necessary a bad thing because if you want to keep 24:24.120 --> 24:28.120 alignment that would mean if you change the API, you have to change the CLI, if you change the CLI, 24:28.120 --> 24:34.360 you have to change the API. And that would include, I think, a connection between the two, 24:34.360 --> 24:37.160 that would create more problems than solve things. 24:39.640 --> 24:46.520 Okay, I'm sorry, I'm out of questions, but I can wait outside if you have more questions. 24:46.520 --> 24:48.520 Thank you.