merkle DAGs

[00:00:00]
>> So hopefully all of you got a chance to poke around a little bit with the clor80API. Gonna switch gears a little bit, but we're gonna circle back to doing some signing and verification to talk about Merkle DAGs. So, you've probably, all of you, have already used Merkel DAGs.

[00:00:19]
You just haven't known it because Merkle DAGs are the basic data structure that get uses in other decentralized version control systems. So how a Merkel DAG works is whenever you have some content, you take the hash of that content and the hash is sort of like the key for the content.

[00:00:39]
So we did this a bit earlier with content addressable blob store, but what makes it a DAG, directed acyclic graph, is that when we wanna refer to other documents, we do that by their hash. And we put those hashes that we wanna use to refer to other documents inside of the document that we're hashing, itself.

[00:01:06]
When we do this, it means that you can sort of establish this chain of custody. So if anyone ever tampers with any one of those documents, the hash for that document will change, but so will the hash of every other document. So you get this sort of cascading property that's very useful for building secure data structures, for building verifiable structures.

[00:01:32]
So that's gonna save you a lot of problems in terms of data integrity and also it means that you can get content from untrusted peers on a network. It's a very handy property on the public Internet, especially if you're building a peer-to-peer system, for example. So some other examples of Merkle DAGs in the real world is a tool called ipfs.

[00:01:57]
This is similar to BitTorrent, but it uses hashes to sort of build up these richer data structures. There's also, I mentioned it already, secure scuttlebutt. This is a social database that also uses Merkel DAGs to kind of build tools like a peer-to-peer social network, a peer-to-peer GitHub replacement.

[00:02:25]
It's a very useful general purpose data model using Merkle DAGs and especially if you add signing into the mix. So if you have a signed Merkle DAG, you can do a lot of things that, normally you would have to write a bunch of software to verify these properties, but if you just are kind of clever about how you structure your data, with hashing and signatures, you get a lot of these properties just for free, which is very cool, I think.

[00:02:55]
Yes. So the kind of network where you get these sorts of signed documents, like where you can get them Merkel DAG, is sometimes called a gossip network. Gossip networks are really cool because all of the peers don't have to be online at the same time. So you don't need to build complicated things like global consensus.

[00:03:14]
You can just make kind of these point-to-point connections like maybe, riding my bicycle through town and I've got a little Raspberry Pi in my backpack that's configured to hop on the WiFi when it sees it but the WiFi is not hooked up to the public Internet. And so I can sync my data as I'm riding around through town to whatever the nearest node is and then be on my way to the next village and do the same thing.

[00:03:38]
And now, both of those villages, if I come back again, will have a consistent copy of the data set, even though they're not necessarily online except through travelers with computing devices on their person. Anyways, that's just a fun example, that people actually do. So part of my contracting work has been building a system like this using USB drives to build a gossip network for offline communities in the Amazon Rainforest, mostly.

[00:04:09]
Just to make maps. Very fun project.