WEBVTT 00:00.000 --> 00:09.000 Let's welcome on next speaker Sophia the floor is yours. 00:09.000 --> 00:10.000 Thank you. 00:10.000 --> 00:11.000 Thank you so much. 00:11.000 --> 00:12.000 I'm so excited to be here. 00:12.000 --> 00:15.000 I've been in this Debra Malat and it's first time presenting. 00:15.000 --> 00:18.000 So thanks to the organizers for taking a chance on this talk. 00:18.000 --> 00:20.000 It's a little bit of a weird one. 00:20.000 --> 00:24.000 Fraising the question of do we need another open source taxonomy. 00:24.000 --> 00:26.000 Why am I talking about this? 00:26.000 --> 00:29.000 I've spent my career in market research consulting 00:29.000 --> 00:33.000 analysis doing research and analysis projects now around open source software 00:33.000 --> 00:34.000 community ecosystems. 00:34.000 --> 00:39.000 And through all of that I have found myself building taxonomy. 00:39.000 --> 00:43.000 Either design comprehensive service questions or to create groupings 00:43.000 --> 00:46.000 and hierarchies in ways to group and analyze data. 00:46.000 --> 00:50.000 And now increasingly data about open source projects and communities. 00:50.000 --> 00:54.000 And mainly I guess if you want to use case, 00:54.000 --> 00:56.000 why am I doing this to begin with? 00:56.000 --> 00:58.000 If we work on a project we gather a bunch of data. 00:58.000 --> 01:04.000 There's always the question about what about this project is influencing the data or the behavior 01:04.000 --> 01:07.000 or the risk or the metrics that we're calculating. 01:07.000 --> 01:13.000 Is there some nuance or context that we can look at in our analysis and our grouping 01:13.000 --> 01:18.000 to compare how different projects may be a language versus something supporting container 01:18.000 --> 01:19.000 run times? 01:19.000 --> 01:21.000 Is that going to look different in our data? 01:21.000 --> 01:22.000 And how do we group that? 01:22.000 --> 01:25.000 So to do that we're going to need a taxonomy. 01:25.000 --> 01:27.000 So where do we find them? 01:27.000 --> 01:33.000 I started really proud just looking everywhere and anywhere on the internet from things like 01:33.000 --> 01:37.000 our government administration use and the standard industry classification. 01:37.000 --> 01:39.000 This was designed in 1937. 01:39.000 --> 01:41.000 So a little bit old, a little bit more technology started. 01:41.000 --> 01:43.000 There was a couple of revisions here. 01:43.000 --> 01:46.000 And it has a nice split between software and hardware. 01:46.000 --> 01:50.000 But if you look at the categories, this is not particularly helpful in terms of grouping open source software 01:50.000 --> 01:51.000 and packages. 01:51.000 --> 01:52.000 So I kept going. 01:52.000 --> 01:53.000 They actually revised this. 01:53.000 --> 01:58.000 And now the technology services are now even more less granular. 01:58.000 --> 02:00.000 So not necessarily helpful. 02:00.000 --> 02:04.000 Note that that original classification is still being used today in the screening stage commission 02:04.000 --> 02:06.000 in the United States. 02:06.000 --> 02:07.000 I kept going. 02:07.000 --> 02:09.000 I went through public administrations. 02:09.000 --> 02:10.000 I went through other types of governments. 02:10.000 --> 02:12.000 I went through the UN data. 02:12.000 --> 02:16.000 And what I didn't do because I don't have a group of researchers that my back in call 02:16.000 --> 02:18.000 is a systematic literature review. 02:18.000 --> 02:20.000 So I realized that's a big gap in this approach. 02:20.000 --> 02:22.000 But you'll see why I didn't necessarily get there. 02:22.000 --> 02:25.000 So then I started with the internet. 02:25.000 --> 02:26.000 What does the internet say? 02:26.000 --> 02:29.000 Anything that was in front of a not behind a paywall. 02:29.000 --> 02:34.000 How are people grouping different types of software and technologies in and around open source? 02:34.000 --> 02:36.000 Clearly, we know hardware, software split. 02:36.000 --> 02:37.000 That's an easy one. 02:37.000 --> 02:40.000 And within software, we saw a split between application and system. 02:40.000 --> 02:45.000 More application is loosely defined as interacting with the end user and system is loosely defined 02:45.000 --> 02:50.000 as everything that sits on top of the machine to interact with the machine from operating systems, drivers, 02:50.000 --> 02:52.000 from where is language processing. 02:52.000 --> 02:56.000 Utilities, I snuck in at the end here because this is where I started to see some conflict, 02:56.000 --> 03:01.000 where some taxonomies had that included in the application layer and some had it included in the system. 03:01.000 --> 03:04.000 And the question I had really is, well, what is a developer? 03:04.000 --> 03:05.000 Are they a user? 03:05.000 --> 03:07.000 Are they interacting with the system? 03:07.000 --> 03:11.000 And it's ambiguous because they could be anywhere in a full-sac development environment. 03:12.000 --> 03:15.000 What was also missing was that there's a whole bucket of other stuff, 03:15.000 --> 03:20.000 especially in open source software communities that aren't software or hardware. 03:20.000 --> 03:23.000 It could be protocols, frameworks, networks, standards. 03:23.000 --> 03:27.000 You notice from my slide deck, I work with the open source project chaos, 03:27.000 --> 03:31.000 which designs metrics around measuring open source health and sustainability. 03:31.000 --> 03:36.000 We also make software, but a lot of our assets are actually content and not necessarily software. 03:36.000 --> 03:38.000 So, where would this fit in this taxonomy? 03:38.000 --> 03:43.000 In our other bucket, which happens a lot if you're familiar with these sorts of exercises. 03:43.000 --> 03:49.000 The thing that actually found the most helpful was a Wikipedia page about ontology engineering. 03:49.000 --> 03:51.000 So, not actually software development at all, 03:51.000 --> 03:57.000 but actually really liked the categories that they were distilling to describe the engineering process. 03:57.000 --> 04:00.000 Starting with an organization, the mission requirements and goals, 04:00.000 --> 04:04.000 what is the function of the thing, what are the individual components that are needed in the function, 04:04.000 --> 04:07.000 how do they communicate with each other, how do you connect them together, 04:07.000 --> 04:12.000 what is needed for information to process and to work through these, 04:12.000 --> 04:18.000 to work through the system, and where does it actually get a run in terms of the physical environment and the attributes of that? 04:18.000 --> 04:24.000 So, again, not designed for software at all, but probably the most applicable generic taxonomy I could find 04:24.000 --> 04:32.000 that could generally describe groups of functional things that could be applied to say open source software, open source software packages. 04:33.000 --> 04:37.000 So, the more I started looking at taxonomies, I kind of went down a little bit of rabbit hole, 04:37.000 --> 04:40.000 and I realized, wait, there's kind of different kinds of taxonomies. 04:40.000 --> 04:47.000 I started making a taxonomy effect taxonomies, and so, again, getting a little bit over over the top here, 04:47.000 --> 04:51.000 but I started noticing certain characteristics about them, whether or not they were functionally designed, 04:51.000 --> 04:57.000 like that ontology engineering page, where we're really just looking at the core function of the thing, 04:57.000 --> 05:03.000 or is there some sort of organizational umbrella in context that's describing the need or the function of the thing, 05:03.000 --> 05:08.000 which kind of goes all the way to the other side of that, which is a completely context-driven taxonomy, 05:08.000 --> 05:14.000 which an example would be, and maybe more often the case for researchers, is you're given a bucket of data. 05:14.000 --> 05:19.000 You might describe a taxonomy in general and say, this would be the perfect theoretical way to group the data, 05:19.000 --> 05:25.000 and you apply it to the data, and you have 90% of the things in one category, and maybe a couple of random others. 05:25.000 --> 05:32.000 That is not an effective way to analyze your set because statistical significance requires you to have more distribution in your bucketing, 05:32.000 --> 05:37.000 and to have enough, or at least an interesting sample or breakdown to look at. 05:37.000 --> 05:43.000 So, often you end up maybe breaking down categories so you can have more interesting groupings, or more granular groupings, 05:43.000 --> 05:47.000 or lumping things together when your sample sizes are too small. 05:47.000 --> 05:49.000 So, examples. 05:49.000 --> 05:51.000 This is a functional taxonomy. 05:51.000 --> 05:55.000 I apologize, the source is internal, this is done for internal exercise, and I have a couple of these, 05:55.000 --> 06:00.000 and I realized it couldn't share all the data, but I could share the metadata because it's not sensitive, 06:00.000 --> 06:04.000 where a colleague of mine attempted to propose a functional taxonomy for open source software packages, 06:04.000 --> 06:10.000 framework, language, library, and database utility operating system, and I tried to apply this to an exercise this year, 06:10.000 --> 06:16.000 and I found, I was still missing things, particularly infrastructure, so that kind of physical environment. 06:16.000 --> 06:20.000 So, again, this person doesn't work on my team anymore, so it couldn't go back and ask them, 06:20.000 --> 06:24.000 why didn't you have infrastructure in here, maybe they were focusing just on developers. 06:24.000 --> 06:29.000 So, possibly something we could apply, but there were still some shortcomings to this approach. 06:29.000 --> 06:35.000 This is the results from an actual project that I completed earlier this last year, 06:35.000 --> 06:41.000 where I looked at a set of packages in our environment, and I was trying to categorize them in a functional taxonomy, 06:41.000 --> 06:47.000 to again evaluate our open source infrastructure portfolio, what we're using, we're re-depending on, 06:47.000 --> 06:53.000 where there's certain characteristics of these projects that change the way that we evaluated the risk, 06:53.000 --> 06:58.000 how critical they are to our environment, what kind of resourcing and staffing do we have upstream versus internally 06:58.000 --> 07:03.000 that are working on these kinds of projects? Again, kind of questions that we have that might be nuanced or related 07:03.000 --> 07:08.000 to the individual project at hand. So, I went through a whole bunch, I read through a lot of readmees, 07:08.000 --> 07:15.000 and started to apply individual labels to each of them, and then tried to group those into logical functional categories. 07:15.000 --> 07:22.000 So, what started as a functional taxonomy, if you look at it more closely, I would say might actually be an organizational taxonomy, 07:22.000 --> 07:29.000 because some of these groupings and labels actually looks a lot like the departmental names and groupings inside of my organization. 07:29.000 --> 07:34.000 We have a security team, we have a team that looks at our physical environment, we have a team that looks at observability, 07:34.000 --> 07:40.000 a team that focuses specifically on QA, or data processing protocols, policies. 07:40.000 --> 07:46.000 And so, I realized while I was trying to design a functional taxonomy, actually made of design and organizational taxonomy, 07:46.000 --> 07:53.000 but in my case, that was actually more helpful because in the organizational context, I wanted to understand where we were using open surf software, 07:53.000 --> 08:02.000 how was that critical or less critical to specific teams, to specific use cases inside of our environment? 08:02.000 --> 08:09.000 And then here's a weird one. This is an example of a mixed taxonomy that was actually a survey question that we use when we were serving 08:09.000 --> 08:13.000 a whole bunch of open-source software projects and organizations. 08:13.000 --> 08:20.000 And you can notice that the categories are mixture of stuff, they're a mixture of tooling, they're a mixture of applications, 08:20.000 --> 08:24.000 and even some industries kind of stuck in there. Why does it look like this? 08:24.000 --> 08:28.000 Well, we had less categories in our prior survey in the last year, 08:28.000 --> 08:34.000 and we had a whole bunch of people checking the same boxes. And so, again, if you think about the distribution of the data, 08:34.000 --> 08:38.000 it wasn't actually telling us that much about the distribution of the entire sample. 08:38.000 --> 08:42.000 So we essentially looked at the categories that had the most responses and said, 08:42.000 --> 08:48.000 can we break this up even further? Can we provide a little bit more context that can allow us from an analysis perspective 08:48.000 --> 08:54.000 to really understand the group and make up of the people taking our survey and the projects that they represent? 08:54.000 --> 08:58.000 So, let's say this is a contextual taxonomy. 08:58.000 --> 09:05.000 Now in practice, have I usually use these things to refine significant results in my analysis projects? 09:06.000 --> 09:13.000 Surprisingly, not that much. What I've compared different types of open-source projects from a functional perspective 09:13.000 --> 09:20.000 and try to compare them, is there any difference in their community dynamics and their growth and their engagement trajectories and their use cases? 09:20.000 --> 09:25.000 And yes, there's difference in use cases, but everything else, it was kind of muddled. 09:25.000 --> 09:31.000 There was so much nuance and context that the actual taxonomy is the end of using an end of being most helpful and functional in my analysis 09:31.000 --> 09:39.000 where more are directed about the people, say the users, if it was like, okay, here's the system and application infrastructure 09:39.000 --> 09:44.000 and end user, but also developer as an individual person or user use case. 09:44.000 --> 09:51.000 And then looking more at contributors and contributor breakdowns, whether or not they were full time contributors, part time, 09:51.000 --> 09:58.000 what how they were contributing, whether or not it was code or other types of tasks that they were taking on and sort of the general level of contribution. 09:59.000 --> 10:01.000 So I ended up using that a lot more. 10:01.000 --> 10:08.000 Here's one of the examples that I love was a project that was looking at trying to find records of non-code contribution in open source software 10:08.000 --> 10:14.000 and they looked at a whole bunch of different artifacts and from the artifacts designed to taxonomy of things that they found and then grouped it. 10:14.000 --> 10:20.000 So here I would say is a contextually built taxonomy based on the data that they had in order to make sense of what they were looking at. 10:20.000 --> 10:26.000 And again, look at levels and types of contributions happening outside of just code. 10:27.000 --> 10:34.000 The other segment that I ended up using a lot was again still looking at the people, except for this time around how we engage with each other as a community. 10:34.000 --> 10:43.000 So looking at say the life cycle of the project, is it brand new, is it years old, is it in growth or decline, general community size? 10:43.000 --> 10:50.000 What kind of processes are in place, governance models, say for example, and particularly technology platforms in use. 10:50.000 --> 10:56.000 If you're on GitLab versus GitHub or in a private Git server, you're interaction and engagement patterns are going a little bit different. 10:56.000 --> 10:59.000 And so that's going to color what your research looks like. 10:59.000 --> 11:03.000 And so these actually ended up being much more effective ways to group projects. 11:03.000 --> 11:08.000 Again, then any of those individual functional technology categories. 11:08.000 --> 11:10.000 Here's another one that I found. 11:10.000 --> 11:12.000 I know I just stuck a lot of taxonomy examples here. 11:12.000 --> 11:18.000 And if you look at the full deck on the, there's actually even more in here that I cut out because I realized just talking about taxonomy examples. 11:18.000 --> 11:22.000 Could eventually get a little bit boring, but there's a lot out there. 11:22.000 --> 11:24.000 And I'm just been trying to collect as many as possible. 11:24.000 --> 11:27.000 Just so we have more references and things to build from. 11:27.000 --> 11:28.000 Here's an example of project status. 11:28.000 --> 11:35.000 This individual proposed a taxonomy to apply badges to your GitHub project to communicate where it was. 11:35.000 --> 11:36.000 Is this something brand new? 11:36.000 --> 11:37.000 Was it a concept? 11:37.000 --> 11:41.000 Is it continually being developed or is it something that you've abandoned? 11:42.000 --> 11:50.000 So as it was realizing and putting all this together that while as a technology researcher, I really wanted to apply. 11:50.000 --> 11:58.000 Functional technical categories in my open source project research finding that it was less actually useful than all of the social characteristics. 11:58.000 --> 12:06.000 That was yet again reminded that open source software is really defined as a social technical ecosystem and model. 12:06.000 --> 12:16.000 The social elements seemingly are the ones that are standing out and grouping us more than these technology categories. 12:16.000 --> 12:19.000 Another taxonomy example because I have to. 12:19.000 --> 12:29.000 Another one of my former colleagues Julie Ferrioli proposed the social model open source, which proposes that we should look at projects by classifying their intent or original purpose. 12:29.000 --> 12:35.000 Because we have to recognize that not every open source software project was intended to be a collaborative growing community. 12:35.000 --> 12:41.000 This of one is a release of demo code or experiment something or prove that something could be done. 12:41.000 --> 12:53.000 And in theory this would be an amazing thing to test and research can we group projects by these things and see maybe that should impact their project trajectory, their sustainability, their engagement patterns. 12:53.000 --> 12:55.000 However, in practice, this is really hard to do. 12:55.000 --> 12:59.000 It's not particularly scalable or on something you can automate necessarily. 12:59.000 --> 13:05.000 I haven't been able to do this outside of reading rabies and still not really being sure. 13:05.000 --> 13:09.000 So I talked about a lot of examples, but I also want to talk about some of the pain. 13:09.000 --> 13:15.000 If I get anyone here has done research like this, you've probably tried to do this and you've probably run into some challenges. 13:15.000 --> 13:21.000 Here are the ones that I've come across the most, particularly my own bias. 13:21.000 --> 13:27.000 It's really hard to get out of your own bias, which is why my general recommendation is don't do this alone. 13:27.000 --> 13:33.000 This is why you have to look for examples, work with your colleagues, work with people who have different perspectives. 13:33.000 --> 13:36.000 Because it's really hard to build a comprehensive view of the world. 13:36.000 --> 13:41.000 One of my favorite examples of where I failed, I was designing a survey for an event feedback. 13:41.000 --> 13:45.000 I'm mechanism we were asking people how the event was, what sessions they liked. 13:45.000 --> 13:52.000 And so we had the question on unemployment, are you currently employed, are you not, are you contractor, are you looking for work, and we put it in the survey. 13:52.000 --> 13:56.000 And I'm looking at the results, and 25% of the other say, student. 13:56.000 --> 13:58.000 I didn't put student in there. 13:58.000 --> 14:00.000 I haven't been a student for 15 years. 14:00.000 --> 14:01.000 I wasn't thinking about that. 14:01.000 --> 14:04.000 That is completely an oversight of my part. 14:04.000 --> 14:08.000 And maybe if I shared this list with someone else before I published a survey, we would have noticed that. 14:08.000 --> 14:12.000 So there could be silly reasons why you forget things, but also the less the ability things in the vacuum, 14:12.000 --> 14:15.000 the more likely you can improve things that you're missing. 14:15.000 --> 14:20.000 We acknowledge that mostly things are built for purpose, so they're not necessarily extensible, 14:20.000 --> 14:26.000 which is possibly why we keep redoing this exercise, because we keep building it for our particular problem, 14:26.000 --> 14:32.000 our particular data set, our particular research question, and we have to keep building them again. 14:32.000 --> 14:36.000 And maybe that's not necessarily a thing that we want to do. 14:36.000 --> 14:39.000 We acknowledge that there's a lot of overlap in taxonomies. 14:39.000 --> 14:41.000 Things don't neatly fit into categories. 14:41.000 --> 14:46.000 I don't know if favorite examples is where would you put a Python library in that functional taxonomy? 14:46.000 --> 14:47.000 It's a language. 14:47.000 --> 14:48.000 It's a library. 14:48.000 --> 14:50.000 It might even be some utilities in that library. 14:50.000 --> 14:52.000 I don't even know what it does. 14:52.000 --> 14:53.000 So what is it then? 14:53.000 --> 14:55.000 Do we put it under all three categories? 14:55.000 --> 14:57.000 Or do we try to group it under one? 14:57.000 --> 14:59.000 And technology keeps changing. 14:59.000 --> 15:05.000 So maybe we have to keep refreshing our taxonomy, and you're always going to have an other bucket. 15:05.000 --> 15:08.000 I still haven't gone around the other problem. 15:08.000 --> 15:12.000 So my general proposal is that it's okay if things have multiple categories. 15:12.000 --> 15:16.000 Perhaps we should start looking at things and acknowledging that we need multiple kinds of things. 15:16.000 --> 15:21.000 Multiple kinds of taxonomies to really understand what the thing is and how we might analyze it. 15:21.000 --> 15:26.000 And just a reminder for a Melvin Conway that a lot of the times are organizational designs and systems. 15:26.000 --> 15:28.000 Mirror on communication structure. 15:28.000 --> 15:32.000 So implying that even if we think we're building something completely logically and systematically, 15:32.000 --> 15:38.000 there's some contextual bias at play, which is probably was causing a lot of these problems. 15:38.000 --> 15:41.000 So can we do this better? 15:41.000 --> 15:48.000 First is trying to just shove a large gooey cat into a small box that he doesn't fit it. 15:48.000 --> 15:51.000 I got really inspired by the Open Demographics project. 15:51.000 --> 16:00.000 If you're not familiar with it, Mickey Stevens proposed this as a way to better write demographic questions for open source surveys of community members. 16:00.000 --> 16:08.000 Because demographic questions can be really sensitive, especially if you start asking about under representation or different types of identity categories. 16:08.000 --> 16:11.000 You really generally don't want to offend your community members. 16:11.000 --> 16:14.000 You want to make sure everyone feels represented and heard but not exposed. 16:14.000 --> 16:20.000 And so ensuring you have the right kind of questions and the right balance of questions is a very delegate exercise. 16:20.000 --> 16:28.000 And so because of that, she proposed this project where other people have been able to come in and provide suggestions for how to write demographic questions. 16:28.000 --> 16:34.000 And I really like that project, which the website might be down so I really hope it's still happening, but they get a page is still active. 16:34.000 --> 16:38.000 So we had an idea in the Chaos community. 16:38.000 --> 16:46.000 I've been working with the data science working group, which works through lots of analysis projects that we propose that we work on together, that we talk about. 16:46.000 --> 16:52.000 And we said, is this something that we could crowdsource? Is this something that we never have to do again? 16:52.000 --> 16:54.000 So what if we shared? 16:54.000 --> 16:58.000 This is sort of, this is again an experiment. I don't know if this is going to work. 16:58.000 --> 17:05.000 But I started to upload some of the tax honomies that I found on to GitHub as just a way to increase more visibility of this. 17:05.000 --> 17:12.000 Open source researchers are going to keep building tax honomies to try to describe open source software in different ways. 17:12.000 --> 17:17.000 And again, if maybe if I were able to apply a literature review, we could start to pull in some from research as well. 17:17.000 --> 17:27.000 But just making it more discoverable, I don't know if this is going to work, but if it does, then hopefully we can stop the next person from building a taxonomy in a vacuum. 17:27.000 --> 17:29.000 Thank you. 17:29.000 --> 17:37.000 Okay. 17:37.000 --> 17:41.000 Okay. 17:41.000 --> 17:45.000 Yes. 17:45.000 --> 17:49.000 The question or what are the cat's names? 17:49.000 --> 17:52.000 The black and white cat is Newton. 17:52.000 --> 17:57.000 And the orange cat is Moby after Mobius. 17:57.000 --> 18:04.000 Notice there are two themes here around mathematics. My partner and I are really into naming our pets after famous mathematicians. 18:04.000 --> 18:09.000 Moby makes a lot of fun shapes in Euclidean geometry and Newton. 18:09.000 --> 18:14.000 She's just smarter, I don't know. 18:15.000 --> 18:17.000 That was really, really interesting. 18:17.000 --> 18:26.000 And it also, a language that we all do research in the economy and the things that only found that there is no sense of the community. 18:26.000 --> 18:29.000 It's really great that this is electing all of these economies. 18:29.000 --> 18:38.000 Because one thing, so my question is, imagine there is an expert in coming in and saying, I need an economy. 18:38.000 --> 18:40.000 I need to be filled with that. 18:40.000 --> 18:43.000 There are a lot of those standards of that exist. 18:43.000 --> 18:46.000 We've got to standardize that. 18:46.000 --> 18:48.000 Do we have that economy? 18:48.000 --> 18:53.000 Where people can throw it and shop for the specific economy that works out that. 18:53.000 --> 18:59.000 And what's the end of coming from that space that we could never do up from this economy? 18:59.000 --> 19:01.000 Yes. 19:01.000 --> 19:04.000 I guess there's, I'm not quite sure what the question was there. 19:04.000 --> 19:05.000 I guess there was a kind of example. 19:05.000 --> 19:15.000 Is there a place that's a standard hub? 19:15.000 --> 19:16.000 I hope so. 19:16.000 --> 19:23.000 The question is the place that we're gathering tax economies could be interactive and place where people can kind of browse and pursue or peruse tax economies. 19:23.000 --> 19:25.000 That's the end goal for this. 19:25.000 --> 19:33.000 So I think part of the structure of it is also designing a data card if you're familiar with other types of open source data sets or data that's being opened, 19:33.000 --> 19:35.000 whether or not it's open source. 19:35.000 --> 19:40.000 There's sort of this model the data card that provides metadata about what the thing is before you even open it. 19:40.000 --> 19:45.000 And so we're hoping if this structure works well as a way to collect information about tax economies, 19:45.000 --> 19:49.000 then we can turn that into a way to search for tax economies. 19:49.000 --> 19:51.000 So that would essentially build on itself. 19:51.000 --> 19:54.000 This is all depending on whether or not we get enough data. 19:54.000 --> 19:56.000 But again, experiment. 19:56.000 --> 20:02.000 So that would be the end goal that we could just start to build this as a growing reference for anyone to use. 20:02.000 --> 20:05.000 Okay. 20:07.000 --> 20:10.000 Okay? 20:10.000 --> 20:12.000 You have permanent question? 20:12.000 --> 20:13.000 Me? 20:13.000 --> 20:14.000 No? 20:14.000 --> 20:17.000 No. 20:17.000 --> 20:19.000 I'll have one. 20:19.000 --> 20:24.000 I think I heard everything. 20:24.000 --> 20:29.600 whether it is more important, but one of the reasons 20:29.600 --> 20:35.200 is to think there is something that, at the end of the day, 20:35.200 --> 20:38.200 it's one taxonomy of TV. 20:38.200 --> 20:43.400 Don't you see the usefulness of the contextual use 20:43.400 --> 20:46.800 or creation of it, and whether it could be, 20:46.800 --> 20:49.800 then, something important not to pay less, 20:49.800 --> 20:53.800 just to get to the information, because just to keep in mind, 20:53.800 --> 20:58.400 that it was for one purpose, and it's good to pay for that. 20:58.400 --> 21:01.880 Yeah, so the question is around that a lot of the contextual 21:01.880 --> 21:04.520 Bill talks on andies are really designed for that purpose, 21:04.520 --> 21:08.280 and maybe that's just, they continue to be relevant in that purpose. 21:08.280 --> 21:09.600 I can completely agree with that. 21:09.600 --> 21:11.320 I think, especially the ones that I've built, 21:11.320 --> 21:15.400 they're very much reflective of the problem at hand. 21:15.400 --> 21:19.760 I think my inclination for this is that someone can still learn from that. 21:19.760 --> 21:22.680 So I think even if you're never used it again for anything else, 21:22.680 --> 21:27.000 it's still, here's a wonderful example of a taxonomy built for this particular case. 21:27.000 --> 21:30.600 The one that comes to mind is actually Daniel Katz here with me, 21:30.600 --> 21:33.720 a research paper that he might talk about later today, 21:33.720 --> 21:39.160 around research for science, open source research for science, 21:39.160 --> 21:42.680 and I'm blanking on the exact naming, and I apologize for that. 21:42.680 --> 21:44.360 But it actually proposes another taxonomy, 21:44.360 --> 21:50.360 that's specifically for research software versus just open source software. 21:50.520 --> 21:54.200 And I think that is an example of one that is particularly context-focused, 21:54.200 --> 21:58.280 and it's very effective in where it is, but it isn't generally applicable. 21:58.280 --> 22:05.160 So I agree, and I think, again, just more visibility of these things would benefit the general case. 22:05.880 --> 22:07.960 I see on next speaker here. 22:07.960 --> 22:09.560 Thank you. Thank you. 22:09.560 --> 22:11.560 Thank you.