IPFS is a peer-to-peer network written using libp2p libraries. Its goal is to make anyone able to make any file available to the network via “content addressing” (“I want this hash => I get it from anywhere that’s the closest, it doesn’t matter from where”). IPFS can be seen as a developer-friendly generalised BitTorrent.
Content addressing is based on verifiable data, implemented via a technology called “Content IDentifier” (CID).
Technically, a CID is nothing but a glorified hash, with the exception that the selected hash algorithm is prepended before the payload hash itself.
By prepending the hash that’s used, CIDs are self-describing, and that is why IPFS can allow users to request data directly by their CID rather than by their location. After downloading data, the client would re-calculate the CID and verify if it effectively hashes to the same CID as the one the client initially requested. This way, there is no need to have prior trust in any peer of the network, and a permissionless network is possible.
Each file “added” or “pinned” to the IPFS network is in fact subdivided into chunks, each represented by their CID, using the IPLD technology, which is a set of algorithms that divide any input file into an IPLD DAG. An IPLD DAG is a Merkle DAG, with the exception that hashes are replaced by CIDs.
Fetching a file in IPFS is fetching each CID of the file’s IPLD DAG.
On the IPFS network, data is not persistent. Filecoin brings this persistence by allowing clients to pay storage providers in exchange for cryptographic proofs of storage of data, addressed by CIDs.
Filecoin recently brought Compute to its blockchain (smart contracts called “actors” through the Filecoin Virtual Machine), allowing various use-cases such as Data DAO: a DAO that’s responsible for the governance & stewardship of any data that verifies certain cryptographic conditions.
Advantages
- The PL stack is a great building block for addressing verifiable and self-describing data.
- In particular, IPLD and content addressing can be used in a variety of tools, independently of IPFS and Filecoin. We’ve seen various successful applications already in the Web3, SSI, Local-First, and DWeb spaces (e.g., NFT, DWN, AT Protocol, UCAN, IPVM). As a result, one can say that the “PL” stack is the bridge between these various ecosystems. From our experience, the PLN crowd is indeed very diverse, from Bitcoin or Ethereum maximalists to DWeb & Web Standards aficionados.
- Filecoin and IPFS are great ways to decentralize data stewardship and related APIs.
- libp2p is a useful library to build a custom peer-to-peer network, if need be. It makes building one incredibly faster and safer. As an example, Ethereum Beacon Nodes are built on libp2p.
Limitations
- To fetch data by CID, one must know the CID first, which is not often the case. Searching for the right CID is re-centralizing and prone to scams and malware propagation. The same issue is observed with the BitTorrent network, which is also content-addressed (via torrents): this problem is due to the very nature of content addressing and as a result, it cannot be easily circumvented.
- IPFS and Filecoin work well with immutable (“static”) data. But mutable (“dynamic”) data keeps changing its associated CID, which makes it difficult to fetch immutably. As a workaround, clients can use IPNS to fetch “whichever data that pub key XYZ is publishing”. It would be better if one could fetch “whichever data that verifies cryptographic condition XYZ”, but to our knowledge, this is not directly possible via existing tools.
- Filecoin only provides proof of storage and proof of retrieval, not proof of constant availability. A Filecoin Storage Provider could very well get paid to store data and retrieve it from time to time but never make it accessible for retrieval at any other time.
- The Filecoin ecosystem, while one of the earliest and most funded ICOs, is still far from providing the level of maturity one could expect. For example, at the pure protocol level, clients have to manually find Storage Providers that would accept their deal request, while the role of Storage Provider (== Filecoin miner) is by far one of the most complex in the whole blockchain mining ecosystem.
- IPFS and libp2p have issues working within web browsers and often face performance issues in general. To be fair, this is not specific to these technologies; it is a limitation that peer-to-peer networks face in general.
Could it be useful for our requirements?
- IPLD and content-addressable data are loosely related to semantic data, and for the same reasons, they are fundamental to provide interoperability, even across protocols.
- The Filecoin-based Data DAO and its related incentivization mechanism can be an interesting way to fund our public good.