IPFS: The Permanent Web
Juan Benet (@juanbenet) spoke at the Sourcegraph Hacker Meetup about his project, “IPFS: The Permanent Web” (slides and video). IPFS is a bold attempt at evolving the Internet’s infrastructure. Here’s how Juan describes it to Sourcegraph:
IPFS is a global, versioned, peer-to-peer file system. It combines good ideas from Git, BitTorrent, Kademlia, and SFS. You can think of it like a single BitTorrent swarm, exchanging Git objects, making up the web. IPFS provides an interface much simpler than HTTP, but has permanence built in. You can also use it to mount the world at /ipfs.
So far, IPFS has 2 implementations: go-ipfs and node-ipfs.
Let’s dig a bit deeper into IPFS and how it could be used.
The Permanent Web
IPFS envisions a world where any resource is available via a locally mounted filesystem at paths like:
# a mutable path /ipns/my.host.com/some/file.txt
# or a permanent path /ipfs/QmaKtFK3wiq9z2LmhMKgoXvuB4XDeY5Xrac8vVBLPjc9CX/some/file.txt
But this file doesn't necessarily reside on the local disk. Instead, it exists in IPFS, a global distributed storage system. IPFS makes files in this namespace:
- highly available, so they can be fetched from any host that stores and is willing to provide the data, not just their originator or owner
- fast to access, so you can treat the /ipns and /ipfs filesystems as local storage instead of remote servers, as in HTTP. In fact, nodes chose which files they store locally
- trusted, like in git where you can trust files in a commit if you trust the commit ID
- signed, so you can attest that you published a certain file at a certain path
- Merkle-linked, so you can version and back up everything, and so you can create a link structure like the web
To achieve these goals, IPFS synthesizes many successful techniques from the last 15 years of distributed systems research. Central to the IPFS design is the Merkle DAG, a data structure that represents all the files. It’s like Git’s blob, tree, and commit, except IPFS has a more flexible model: you can define what your link structure is and how it works. This means you can implement Git on top, or a Blockchain like Bitcoin, or linked web pages.
Juan also discussed a list of use cases for IPFS:
- Mounted global filesystem, under /ipfs (permanent) and /ipns (mutable)
- Mounted personal sync folder that automatically versions, publishes, and backs up any writes (like a versioned, distributed Dropbox)
- Encrypted file or data sharing system
- Distributing mutable content (BitTorrent + Git)
- Blockchain data structures
- Versioned package manager for all software (see this discussion)
- Booting a virtual machine from the network
- Booting a VM from a hash (trust your system is at that particular state)
- Databases (applications write directly to the Merkle DAG data model, getting all the versioning, caching, and distribution that IPFS provides)
- Linked (and encrypted) communications platform
- Integrity-checked CDN for large static files (without SSL)
- Encrypted CDN
- The Permanent Web, where links do not die
Watch the talk, and check out the IPFS paper for more details.
Getting involved with IPFS
IPFS is an open source, MIT-licensed project. You can use the ideas and the implementations for whatever you want. It’s still early, so the implementations are not complete. You should follow @juanbenet and check out the IPFS spec and open source implementations. Watch or star these repositories on GitHub to follow along:
Also, you can join #ipfs on irc.freenode.org.
Juan has a compelling vision for a future where the Internet is more efficient, robust, and safe. He has been thinking about these things for almost a decade, and he’s now turning all that thought into action. We’re excited to follow IPFS!
