SensifAI
AI driven image tagging
Billions of users manually upload their captured videos and images to cloud storages such as Dropbox, Google Drive and Apple iCloud straight from their camera or phone. Their private pictures and video material are subsequently stored unprotected somewhere else on some remote computer, in many cases in another country with quite different legislation. Users depend on the tools from these service providers to browse their archives of often thousands and thousands of videos and photo's in search of some specific image or video of interest. The direct result of this is continuous exposure to cyber threats like extortion and an intrinsic loss of privacy towards the service providers. There is a perfectly valid user-centric approach possible in dealing with such confidential materials, which is to encrypt everything before uploading anything to the internet. At that point the user may be a lot more safe, but from now on would have a hard time locating any specific videos or images in their often very large collection. What if smart algorithms could describe the pictures for you, recognise who is in it and you can store this information and use it to conveniently search and share? This project develops an open source smart-gallery app which uses machine learning to recognize and tag all visual material automatically - and on the device itself. After that, the user can do what she or he wants with the additional information and the original source material. They can save them to local storage, using the tags for easy search and navigation. Or offload the content to the internet in encrypted form, and use the descriptions and tags to navigate this remote content. Either option makes images and videos searchable while fully preserving user privacy.
- The project's own website: https://sensifai.com
Why does this actually matter to end users?
Our smartphones and tablets are filled to the brim with photographs and videos we take of everything we see around us, so when we reach the memory limit of our devices, we need to put our vacation pictures, baby photos and nature videos somewhere we can easily access and sift through all of it. Many users rely on cloud storage to safely store these memories in our own personal vault, secured by a password (or two), handily synchronized across devices and easily accessible. But in practice, that is not always what cloud storage really is. What users in fact do, is store their own pictures and videos in some undefined location, as the cloud service provider rarely explains where data is kept (and under what local legislation), with little to no explanation about what access that service provider actually has to your private images.
One of the ways to keep your online pictures and videos safe and private, is to encrypt them before you save them online. But how then can you find that one picture of you and your friends out on the town that you want to put on the wall, or delete and select badly taken photos from your cloud space? Encryption in general is a proven technology to protect your privacy and strengthen your security, but can be hard to manage and maintain for users. The same thing goes for encrypting and storing photos and videos: users do not want to end up with a massive vault of unrecognizable data that they cannot search through or interact with in any meaningful way.
This project combines privacy protection and searchability of photos and videos in a user-friendly way. Visual recognition software is increasingly capable of accurately recognizing who and what can be seen in pictures and videos, which can be automatically tagged to the files as they are saved on your phone or tablet. The user can then decide where they want to store this content and whether they want to keep it safe and encrypted, all the while keeping the content searchable. Users do not have to give up agency over their own (very personal) photos and videos to any service provider that indexes and categorizes files: now they can do that themselves.
Run by SensifAI bvba
This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 825322.
