Facial Recognition

[tekstrand]

As an administrator of a Piwigo site, I want to be able have automatic facial recognition run across all of my photos.

• After the facial recognition has completed I want to be able to assign names to the recognized faces.
• As more data is entered by users, the facial recognition should be able to be run again in order improve it's accuracy.
• When new photos are uploaded, automatically run the facial recognition suite and make recommendations

[tekstrand]

I've created this issue in the interest of capturing conversation around this feature. Have any libraries been investigated or previously worked with by the maintainers? Are there any other key features that we should add to the story above?

[ivanbalashov239]

@tekstrand https://github.com/ageitgey/face_recognition
this is the only library we need for this. It works perfectly and should be relatively easy to use.

[tekstrand]

@seniorivn why that library? Seems that using a Python library in a PHP app probably adds complexity for most users

[ivanbalashov239]

@tekstrand because it's the best one, and python is not gonna be a problem there are cli and example of http api
if i decide to use piwigo for my photo collection, i will implement this.

[tekstrand]

because it's the best one

What is this based on? I don't have experience with any of them so I would love some more info

[ivanbalashov239]

@tekstrand i've googled and decided it's the most accurate and easy to use and used it for another project.

Unfortunately for php fans, most academia related libraries are implemented in different languages.

[ivanbalashov239]

also there is a https://github.com/hooram/ownphotos
which is quite good, but not as mature as piwigo, and uses this lib for facial recognition.

[ivanbalashov239]

papers behind it are great, and accuracy near 100%

[m3eu]

@seniorivn any update on this conversation? TIA

[ivanbalashov239]

@m3eu unfortunately haven't gotten to even digitalize my photos(family archive) so no updates from me.

[m3eu]

@seniorivn too bad, was hoping for some better news ;)

[cccraig]

Checkout DeepFace. The 2.0 release crops the photos and saves them within the plugin directory in such a manner that DeepFace can be easily trained to recognize people. What I don't have is a good process for,

1. periodically updating the model as new individuals are added.
2. dynamically evaluating who is in an image when it's uploaded. This would include getting the bounding box and determining the scaling ratio for when the image is resized.

[cccraig]

Additional thoughts for how to do this well.

1. When images are uploaded we don't try to immediately tag the faces. That would be problematic if there were hundreds of photos. Instead we create another table to serve as a queue and add the photos to that. I don't think that Piwigo has a queuing system (thinking something like what Laravel has). This site has a good example of how this might be done.
2. It would be a two part process. One cycle extracts the tags and faces and the other actually process the faces to label them. This second part should be done during off peak hours as it could be very intensive.
3. How to handle net new faces? Should probably have some sort of threshold configured to prevent lots of erroneous data being saved.
4. Should probably have a way to have a person verify who is in the photo later. Say we have a confidence of 95% and we're good to label that but unless we hit a confidence of 100% we have some sort of prompt for users to verify who someone is.
5. For item 4, there should be a way to associate people. So if I tag George Jones, then I should be associated with that name and only be prompted to verify images that were automatically tagged by the system as George Jones.