Sonar: a modular peer-to-peer search engine
Modular peer-to-peer search engine
Sonar is a project to research and build a toolkit for decentralized search. Currently, most open-source search engines are designed to work on centralized infrastructure. This proves to be problematic when working within a decentralized environment. Sonar will try to solve some of these problems by making a search engine share its indexes incrementally over a P2P network. Thereby, Sonar will provide a base layer for the integration of full-text search into peer to peer/decentralized applications. Initially, Sonar will focus on integration with a peer-to-peer network (Dat) to expose search indexes securely in a decentralized structure. Sonar will provide a library that allows to create, share, and query search indexes. An user interface and content ingestion pipeline will be provided through integration with the peer to peer archiving tool Archipel.
- The project's own website: https://sonar.arso.xyz
Why does this actually matter to end users?
Search and discovery are some of the most important and essential use cases of the internet. When you are in school and need to give a presentation or write a paper, when you are looking for a job, trying to promote your business or finding relevant commercial or public services you need, most of the time you will turn to the internet and more importantly the search bar in your browser to find answers. Searching information and making sure your name, company or idea can be discovered is crucial for users, but they actually have little control over this. Search engines decide what results you see, how your website can be discovered and what information is logged about your searches. What filters and algorithms are are used remains opaque for users. They can only follow the rules laid out for them, instead of deciding on their own what, where and how to find the information they are looking for.
This project takes a radically different approach to privacy-friendly and decentralized search and discovery by ensuring that the search engine is decentralized by design. Even the current search engines don't run the work on a single machine - the web is way to big for that. So it is already a distributed task. By harnessing the combined power of many small systems made available by their users, such a collective approach can measure up to the services traditionally delivered by monolithic search companies. The reverse also holds: without enough participants working together and contributing, the use will likely be limited. For now, the first concern is the availability of the essential technical building blocks. Of course, a decentralized search engine should still provide users with relevant results that are at least on par with what proprietary search algorithms can offer, which this project aims to do with machine learning.
The P2P nature has attractive features. It ensures users that there can be no central point of control, not now nor in the future. It is also potentially extremely robust and resilient: the computers contributing capacity are potentially widely spread across the internet. If for some reason like a major disaster, the European internet is cut off from the rest, P2P search may still work.
Run by arso project
This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 825322.
