Send in your ideas. Deadline October 1, 2024
Stay up to date
Theme fund: NGI0 Discovery
Start: 2019-04
End: 2022-10
More projects like this
Verticals + Search

Search engine for the Interplanetary File System is a Free and Open Source (FOSS) search engine for directories, documents, videos, music on the Interplanetary Filesystem (IPFS), supporting the creation of a decentralized web where privacy is possible, censorship is difficult, and the internet can remain open to all.

Why does this actually matter to end users?

On the web it can happens that someone who is hosting a small website that is valuable to you no longer wishes operate it. Or is unable to bear the costs. Or is censored. She turns off the server without warning (or someone does it for her). And all the information ever submitted is lost. If that information is for instance not just some web app, but an original resource that is valuable from a personal, cultural or historical point of view, that feels like a waste. We often learn a lot while browsing through old books, brochures, news papers and advertisements - but also from resurfacing our own notes from the past. Surely, valuable institutions like the Internet Archive can have potentially archive anything. But they won't be able to capture everything, and furthermore an archive is a frozen state. What if this was actually a community resource that you a user contributed a lot of time and content to, and have a stake in wanting to keep it alive? Wouldn't it be great if you as a contributor had the "source" all along, and were able to bring it back to life?

IPFS, or the InterPlanetary File System, is different from the original web that assumes and thus assigns all the rights to a single server and owner. It is like a giant virtual storage cluster spread across the computers of all the people that use it. You can use it to publish things, such as web pages. One thing is different: as long as there are people that retain a copy of those pages, it will remain available. IPFS does not per se have a central place of storage, any copy is enough to give others access to this data by sharing links to the content. That means you as a user can help curate the cultural, social, technical heritage you care about. The approach is similar to torrenting, but IPFS offers more transparency and security since users can look at past versions of edited data and content is uniquely identified and permanently stored.

One significant issue though for users is that conventional search engines at present do not engage with the InterPlanetary File System just yet. This is because it is challenging to work with such a distributed system, especially for a search engine. It is a constantly changing, complex and vibrant space, very much like the real world is. Easily finding things you need is vital to making any information systm usable, especially if the system in question is huge (and "interplanetary" definitely has some ambitions there).

This is where the project comes in. It aims to let users search the Interplanetary File System through a regular search engine interface, that is nevertheless distributed over the IPFS-system itself. The actual indexing of files, which is automatically done by something called a crawler, can also be decentralized. This would ensure that search and discovery on IPFS would not become a single point of failure that can get taken down, attacked or censored. But actually belongs to and is operated by the same people that make the IPFS possible in the first place, namely its users.

Run by ipfs-search

Logo NLnet: abstract logo of four people seen from above Logo NGI Zero: letterlogo shaped like a tag

This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 825322.