But this is going to be a talk about using NYXOS for the
deterministic distributed system benchmarking.
And the speaker here is Bruce Gain.
It's a really cool topic.
I think there's a lot of applications.
NYX can be leveraged for and this is a particular
interesting one, so take it away.
Okay.
A lot of applause for this speaker, please.
Yeah, hi.
My name is as Martin just said, I'm Bruce Gain.
I'm an analyst with consulting firm and it called
the Revcom.
And what we do among other things is benchmark testing.
And we try to really drill down and perform its characteristics
of certain software packages and tools.
And we're looking at the runtime basis as much as possible.
And as you could probably understand that it's quite difficult to do,
especially when trying to benchmark commercial applications.
And by the way, about myself, I'm a huge Linux advocate.
I've been using Linux for over 25 years and I love open source.
And we ran into a wall recently with really trying to gauge runtime
performance of certain applications.
I just mentioned.
And we looked at a few things.
A few alternatives to do that.
And we saw geeks.
I don't know.
Is anybody here familiar with geeks?
Kix?
Yeah, cool.
Yeah, great.
I love geeks.
But right now, we're not getting it to function as we'd like.
It's really hard for us.
So what we started out with next.
We're doing a lot of work with next now.
And we're initially looking at trying to gauge the performance
of skill-a-db and Cassandra.
You know, these database applications and platforms.
Who's here?
Who here is familiar with skill-a-db?
That's database.
OK, Cassandra, probably.
Yeah, Cassandra.
Yeah, everybody is Cassandra almost right.
So it became notoriously hard for just for some reason.
That might be obvious.
The skill-a-db versus Cassandra for the performance based on the different
benchmarks for that, latency, et cetera.
It was just very skewed.
That's not the right word.
But as far as the performance and the benchmarks go of skill-a-db,
those on file are very, the performance and skill-a-db,
according to those benchmarks, are like 2 to 3x.
And we'll go to that later, specific benchmarks.
But compared to Cassandra.
So what we wanted to do is just compare the two and a more apples
way, nicks are sorry, excuse me.
A Cassandra versus skill-a-db.
And we didn't, it's difficult because right now we don't have access
to a skill-a-db package to do that to put it on nicks.
So we're asking, we're trying to get the skill-a-db folks to help us with that.
So hopefully we'll be able to do that soon one day.
But at the meantime, we were looking at how to do this, how the gauge
of benchmarks with Cassandra.
And we covered this with, for example,
Docker, oh yeah, the other issue is we wanted to share our work,
to share our benchmarks so you can reproduce those and see for yourself.
With the president with Docker, I love Docker as much as everybody else.
We use it every day.
But reproducing that with Docker is problematic.
It's not accurate as you know.
The performance of St. Cassandra differs according to the,
you know, the runtime, the operating system, et cetera.
I mean, you just can't port directly replicate those environments
with Docker.
We know that.
So nicks, I think, is everybody else.
Here knows that there's one of the beautiful things that nicks does
is that reproduce ability aspect.
And I find it quite amazing actually.
So yeah, here are the benchmarks I was referring to.
Cassandra versus a skill of DB.
These are provided by skill of DB.
Now, you know, these different, you know,
latency, et cetera, the rewrite frequency, et cetera.
Yeah, skill or Cassandra gets killed.
But, you know, this is probably.
Yeah, I mean, I don't know.
I would like to put this on nicks.
That's what I want to do.
And hopefully we can make that happen.
Make that comparison happen.
And going back to the Docker issue, you know,
the leaky abstractions, the containers on the host, et cetera.
You know, those are, you know,
and not Docker is just not, we're often said,
okay, just look at Docker.
Just how we share a work.
We'll put it on a Docker container.
No, that's not really.
That doesn't work as far as reproducibility goes.
You know, for a different, a number of different reasons.
I mean, it's even contingent on the, you know, operating system.
Of course, your laptop, whatever.
With, you know, Docker for those, you know,
reproducing results or reproducing environments.
It's just not cut, cut out for that.
So that's, again, that's what I was just mentioned.
Well, at the beginning was that, you know, we're frustrated
in our journey.
I hate that word, but for lack of a better word,
our journey to figure out, you know,
how are we going to act early, you know, compare,
not just Cassandra and, and, and skill of DB,
but, you know, other runtime performances or different applications.
And, you know, for right now, it just looks like Nick's
is the way to go for that.
If anybody has any alternatives,
I don't know, geeks shows promise, but right now,
I can't think of anything better than with Nick's.
And it's pretty fun to set up.
I mean, but that won't get to that in a second.
I think most people are probably familiar already
with, you know, why, you know, the next functionality,
you know, how that works.
You know, I've learned recently that, oh, yeah,
sorry, to go back to one thing about Nick's,
I've learned recently, is that Debbie and actually,
for a while, was trying to get over that reproducibility hump
with, you know, with that doctor, you know,
to solve that doctor issue for that quote,
drift.
It's another word I hate using, but, you know,
adding, you know, reproducibility to something like
doctor and Debbie and stop that.
In fact, they started using Nick.
I learned that recently.
I thought it was interesting.
Everybody here does Debbie and Linux.
Yep.
Yeah.
I don't know.
Anyway, so, so, going back to Nick's, you know,
the reproducibility, which I found was fascinating,
it was, you know, on the level with the output,
it was the hash functionality of, you know,
the flake configuration, FL, AKE,
and that, for me, just on a computational level,
it's fascinating, I thought, because that,
the reproducibility hinges on that hash functionality,
where if that hash sees one single digit in the code
that is different from the build, it will not function.
It just stops.
It's just not, will not read to that.
And so, that's what I thought was, you know,
computationaly, interesting,
main aspect of Nick's, which I found fascinating.
I guess you could call it the power of hash, if you'd like.
So, so again, you know, we put this through, you know,
set this up with, well, an engineer he did at first.
One of, one of the engineers on our team,
who's credited at the end of this, he's in the US.
And he, you know, we, you know, this configuration
evolved, you know, Sanders, I mentioned, you know,
we loaded up Nick's, we, you know, looked at,
we did, we pulled it off GitHub, you know,
we did the standard thing of putting Nick's onto the machine,
getting that flake file and integrating the package,
the, um, the standard package with Nick's.
And that proved a little difficult sometimes.
You know, the, um, sometimes the, you know,
it was at one point, it said that we,
if I remember correctly, the cache was not
considered correctly, whether the Java was reading
into the wrong file or the wrong place.
And that, um, and it kept failing.
So that was, um, I had a look and dig into the documentation
that took a few hours just that one part.
But we figured it out, um, and just the big,
quite candid, I just cut and pasted from the documentation
and it works now.
All right.
I hope it will.
And I do the demo.
So again, going back to the, you know,
summary of the, um, you know,
you know, for the, you know, how Nick's works with the
standard, particularly, you're just in general,
you know, you have to tick at the hash, um,
it's calculating, you know, ensuring that the,
the code is, um, completely has not changed.
It's immutable as, as a term, uh,
checks everything.
You can put, you know, thoughts, say, some fridge.
Download the, the pre-build environment,
which I did once.
So at the end of this, I'll show you the GitHub link.
You should just be able to clone the GitHub repository
and run this benchmark.
Uh, it'll save you probably,
according to my engineer.
He spent 10 hours getting this set up.
So that way you can just do that.
Um, I hope.
Let me know if it doesn't work.
Or put it in a pull request and get up.
If you like.
So anyway, going back to this workflow, uh,
you know, a download environment,
and you spin it up and you start looking at your benchmark.
Uh, that's, that's essentially it.
Um, you know, until now,
let's see if I can have any questions right now.
Nope.
Okay.
Great.
Okay.
I see how this works.
Okay.
So, that's like what I'm getting.
I see you.
Okay.
I apologize.
We had a problem with that cable.
Yeah.
Is well, the old cable.
I'm not sure what we can do about it.
Well, excuse me, sir.
Okay.
I apologize.
Okay.
Actually we already had this one cable.
Okay.
Yeah.
That seems to be a bit flagging.
Where did it start working when I was pulling it or when it was?
I don't have a different screen.
Oh yeah.
I understand the impulse of trying to, you know, have, like, chatting a bit, but let's not lean into that too much because the room gets really, you know, a bit too energetic.
Can I go?
Okay.
Sorry.
So we got that figured out.
Alright, everybody, please quiet down.
We're continuing after just a bit of technical difficulty.
Okay, now it's not going to work.
It's not going to work.
Anyway, so if you pull this down, if you have, I've seen you have Python.
You have Java.
You have what's necessary, you know, to do this.
Just pull it off and get up and I'll show you the link at the end of this.
And once you're in, you're just going to the directory.
Are you cloned into and of just hopefully a work.
Alright.
Just takes like 20, 15, 20 seconds.
Anybody have any questions so far?
Nope.
It's always great to have talks with live demos.
I'm sorry.
It's always great to have talks where you really see something happening, mate.
Tick, tick, tick, tick.
Alright, there we go.
Okay, let's run the benchmark.
Here we go.
That's it.
We got our benchmarks.
Yep.
That's not the easy part actually.
Are two things.
If you look at the statistics, so I mean, if you look at the other benchmarks we did,
there's a variation of 20 to 30%.
So you would say we failed, but we didn't because this is running my laptop.
So as this is scaled, if we were using a very, if we scaled at thousands of, you know,
to thousands of X, that variation would be maybe one to two percent.
So the fact I'm doing in my laptop, there's a lot of Paris Titoz or Paris Heights,
which would contribute to that 20 to 30% difference between the different benchmarks we ran.
So that's it.
And then what I found particularly interesting too,
is that we just, it's very, you know,
state was, I don't think it's the right word, but we just, you know,
killed all and start over.
That's it.
It's done.
It's gone.
That's, that's out of the hard part.
So hopefully my slide will be working again.
Okay, so here's the shout outs to, you know,
here's, if you want to, if you want to use the, you know, do this.
Again, the easy part is just should be pulled this off from the GitHub and clone it.
You can start doing your benchmarks on Cassandra.
Hmm.
The, you know, the acknowledgements, obviously Nick's next to us.
And then Shaiid Khan, he's facing the US, he was working for us.
Now he's working for Deloitte.
And that's not so great, but anyway.
And come give us a shout if you like.
We love doing science and testing.
That's what we like to do.
Like to do.
Thank you.
Thank you very much for this game.