shotwell

There is some sample code for using the Python bindings to OpenCV to highlight faces in images

http://creatingwithcode.com/howto/face-detection-in-static-images-with-python/

This can be a useful reference for an initial implemetation.

faces is a must for the best photo manager

I would be nice to have a DBus face detection (and eventually recognition) service. That might be a nice project to do with libface, keep things language agnostic to some extent, and separate from Shotwell.

Hello all,

I'm starting to have a look to implement a new feature that is linked with this ticket.

In a first step, I want to implement the possibility to set who is on the picture and where manually (with a shape).

If everything go fast and good, I will have a look to all existing library for face recognition and maybe have a try to implement it.

So, wait and see :)

Also if you test it now, there are a lot of bugs (faces detected but not well reported) but I will work on it with good test pictures like this one !http://www.ists.dartmouth.edu/images/ISTSpeople.jpg.

I don't understand the advantage of implementing this as a separate service process. Why not simply call the OpenCV shared library from the Shotwell process directly? A separate process will add an additional dependency, will complicate installation and startup, and will potentially slow things down since we'll need to cross process boundaries with each request.

I'm only guessing here but I guess the idea is that if you tag someone as being person X in Shotwell, then other apps could reuse that info and suddenly your browser or e-mail program could also recognize that person in photos?

I can see that it may not be the fastest way to do face recognition, by as mnemo pointed out, it may be a better way to share face recognition and detection across applications. Then again, if the IPC overhead is too large, Shotwell could implement its own face recog/detect and then share the processed information with whatever OS service may be available.

My main motivation for the suggestion was to decouple the development of this feature from Shotwell, and let the Shotwell developers concentrate on other more important features.

marcus.ps, thanks for the explanation. I think there's no need to try to decouple this feature's development from Shotwell. On the contrary, if you or anyone else if interested in working on this I'd like to encourage communication between you and the Shotwell team. Rather than building an external service which we have to glue on later, I'd be inclined to start thinking today about how to call the OpenCV shared library from Shotwell (do we need to build a Vala binding, for instance?) and about how we want face detection to work in the Shotwell user interface.

I'm doing it in that way because it will be done one time, and then, nobody else will have to do it again and again (re-writing the wheel).

Replying to [comment:9 adam]:

A separate process will add an additional dependency, will complicate installation and startup, and will potentially slow things down since we'll need to cross process boundaries with each request.

I don't agree with you. Face recognition will be disabled until the person install the dependence.

It's a cool thing, because in that way, users that don't want face recognition just have to do nothing more.

For person who want it it's just a search in the packet manager, and press install…

I also imagine (always if everything work great etc…) that the service could be integrated in the OS directly.

I still don't understand the precise benefit you're looking for in implementing a separate server process. You said that “it will be done one time, and then nobody else will have to do it again and again”. Shared libraries have that exact benefit: I can write a shared library once and then many programs can use it. And, conveniently, OpenCV is already a shared library which we can call directly from Shotwell. If you want to write extra functionality to be shared among programs that perform face recognition, you could either write another shared library which calls OpenCV (in fact, I think libface does exactly this), or you could simply submit patches to OpenCV itself.

Shotwell already uses shared libraries for lots of functionality: for decoding RAW photos and for displaying Web pages as part of publishing, for example. Do you think those capabilites should have been implemented via separate server processes instead?

I agree that it might be nice if people can optionally enable certain features, and we'd like to have a plugin system that allows exactly that (this is #1603). Once we've implemented that, we could conceivably implement face detection as a plugin. I think that face detection will be useful enough, however, that once it's implemented it should be part of the core Shotwell program and available to all users.

Vala bindings for OpenCV would be nice, I think, and I wish I had the time to contribute to a project like that. I did look into, but I don't think it would be something I could take a lead on at this time.

I understand your point adam.

I think that shared library are here to give you a, as developer, a lot of classes/methods that you can use in many possible ways to do a lot of things. ( Like OpenCV. It don't only detect faces. )

But here, I'm implementing a specific feature that use some of these classes/methods to do it.

Now, another application want to have face detection but to do another thing. It can use my work. And having a DBus daemon or a shared library is not big difference.

I will continue to do my DBus daemon for now, because I'm learning a lot of thing about picture manipulations, that I never did before.

I plan to do a try, implement a Dbus client to use my daemon and make benches.

After that, I can write the same code, but directly in Shotwell in Vala.

Is this ok for you ?

Just for give to all of you a status, I want to post a result of my daemon :)

To do my tests I'm taking picture of people on the net.

Here is a source: !http://www.dfas.mil/careers/acareeratdfas/comminvolv/communitypeople.jpg

And here the end result of my daemon: http://a.yfrog.com/img684/7900/communitypeoplewithface.jpg

It's not clean but cool 14/17 :)

Yes, that plan sounds OK with me. That's great that you already have code which is detecting some faces!

Ok, we have a deal :)

I have improve the garbage detector and now in my previous example I have 17/17 !! :)

http://img33.imageshack.us/img33/7900/communitypeoplewithface.jpg

Now I'm working on other sources to improve face detection.

Here another example where the first try work great:

Source: http://lac.web.psi.ch/Pictures/LAC_people_20080403.jpg

With faces areas: http://img682.imageshack.us/img682/5030/lacpeople20080403withfa.jpg

Replying to [comment:21 zedtux]:

Hi, nice to hear that you're getting good accuracy. In the last example, can you tell me:

1. The parameter values for detection (search increment, minimum face size used, etc)

2. The number of cascades used. Are you using a custom-trained cascade or opencv's?

3. Time taken for recognition. Does it increase a lot if the image's pixel count increases?

Source code would be good :)

Thanks.

I'm the developer of libface, btw.

Hello all.

A short message to tell you that I'm in holidays for 3 weeks o/

@adityabhatt: I already have post the project's url (https://launchpad.net/quiestla) where you can find the Python code.

You will also find all you answer in it.

But, basically, I'm using OpenCV to do face detection, with 5 haarcascades XML files.

(Here is the core method where opencv face detection method is called http://bazaar.launchpad.net/~zedtux/quiestla/trunk/annotate/head:/quiestla/common/opencvclient.py#L160)

The execution time isn't very short until now, because it wasn't the highest priority.

Here they are:

• Discover and test opencv Done

• Discover how to implement a D-Bus service Done

• Have first good results on random pictures from google Done

• Increase queue and thread use to do more than one task at the same time Done

• Improve D-Bus service (method and signals) Done

• Improve execution time In progress

• Improve results Waiting

• Implement into Shotwell Waiting

My last commit was about execution time improvement by resizing pictures. And it work great !

Execution time is about 1 second, and my other code to clean results is also about 1 second.

(If you execute the project, you will see in logs execution time)

But I've planned to improve it more.

Thanks for your questions, and do not hesitate if you have others or suggestions :)

P.S: It's a very good project for my second project in Python ! :)

Hi all,

After 2 commits, I've decided to write a little documentation about how to test my project. You will find it on the page of the project on Launchpad: https://launchpad.net/quiestla

If you find bugs or have any questions, please Launchpad tools for that, thanks ! :)

I'm asking me a question about faces areas positions and associated name storage.

I've browsed the SQLite Database, and I was thinking that we should have to create a new table like FaceTable that should be like this:

CREATE TABLE FaceTable (id INTEGER PRIMARY KEY, primary_photo_id INTEGER, name TEXT, x INTEGER, y INTEGER, width INTEGER, height INTEGER)

Am I wrong ?

This looks reasonable, but stepping back, what are you proposing as far as implementing this? I have an idea of what you're storing in the FaceTable, but how does this interact with the rest of the system (i.e. PhotoTable)?

The idea is quite simple:

I've create an external project that do all that Shotwell need to know where are faces. So, after importing a folder of photos, in order to show faces and associated names, a FaceManager should connect through DBus to my external project, given the path of pictures where faces should be found. Then, my project emit a signal when stuff is done for a photo, returning the result (faces areas).

When Shotwell will receive that result, it should store then in a row of the FaceTable table, containing x and y positions and width and height. More after, the user should update a face area to give the name of the detected face.

In the future, I hope to implement the face detection feature to recognize a face and give it's name directly.

If you have future questions, do not hesitate to ask me ! :)

Hi all,

I'm currently facing a problem that stop me in dev.

I've start to write a Dbus class like this:

[DBus (name = "org.zedroot.QuiEstLa.PeopleLocalisation")]

private class PeopleLocalisation : GLib.Object {

public signal void a_file_detection_finished (Results[] results);

}

But the compilation failed with this error messages:

cc -c !pkg-config --cflags atk gdk-2.0 gee-1.0 gtk+-2.0 glib-2.0 libexif sqlite3 gexiv2 gconf-2.0 libgphoto2 libsoup-2.4 libxml-2.0 unique-1.0 webkit-1.0 gudev-1.0 dbus-glib-1 gdk-x11-2.0 gthread-2.0 -I./vapi -D_PREFIX='“/usr/local”' -D_VERSION='“0.7.1+trunk”' -DGETTEXT_PACKAGE='“shotwell”' -D_LANG_SUPPORT_DIR='“/usr/local/share/locale”' !./libraw-config --cflags -O2 -g -pipe -fPIC -DG_UDEV_API_IS_SUBJECT_TO_CHANGE -o src/!FaceManager.o src/FaceManager.c

FaceManager.c: In function face_manager_finalize:

FaceManager.c:635: error: tmp1 undeclared (first use in this function)

FaceManager.c:635: error: (Each undeclared identifier is reported only once

FaceManager.c:635: error: for each function it appears in.)

make: *** [src/!FaceManager.o] Erreur 1

Compilationwork fine if I remove the line

[DBus (name = "org.zedroot.QuiEstLa.PeopleLocalisation")]

What's wrong ?

P.S: I've alsotriedexample likethose compilation neverfail.

I'm not sure what the problem is here, but it appears to be with the DBus implementation and not with Shotwell. I would recommend asking your question on the Vala mailing list or the Vala IRC channel.

Hello all,

Since this issue, I haven't worked on because here priorities have changed.

I don't want to give up.

So I suggest that someone that have a biggest knowledge than me in vala write the vala code to send requests to my QuiEstLa service.

I'm available for help, and will watch this ticket to answer to all questions and do quick modifications to my project if necessary.

Please, do not hesitate to request my help, and I will take time to continue to work on it.

If the descriptions found under http://launchpad.net/quiestla are correct quiestla still isn't very fast but is already usable…

Am interested in this feature too, and am ready to help out testing (at least).

Requested downstream: https://bugs.launchpad.net/ubuntu/source/shotwell/bug/693808

Replying to [comment:27 zedtux]:

I'm asking me a question about faces areas positions and associated name storage.

I've browsed the SQLite Database, and I was thinking that we should have to create a new table like FaceTable that should be like this:

> %(=caps)CREATE% %(=caps)TABLE% FaceTable (id %(=caps)INTEGER% %(=caps)PRIMARY% %(=caps)KEY%, primary_photo_id %(=caps)INTEGER%, name %(=caps)TEXT%, x %(=caps)INTEGER%, y %(=caps)INTEGER%, width %(=caps)INTEGER%, height %(=caps)INTEGER%)

> 

Am I wrong ?

I suggest RegionTable. A region may be used for something other than a face.

RegionTable will have a field tag_id (not “name”), so a region could be associated with a tag.

A tag may represent a person.

This ticket was titled "Face detection and people view," but has been retitled "Face detection" because the "people view" functionality has been factored out into ticket #3894. "People view" is the ability to manage faces/people in Shotwell, while "Face detection" is the ability to automatically recognize faces using a pattern recognition/computer vision library.

Thanks, Valentin - we'll look forward to your patches.

I'm trying to generate the patches but they are created on top of previous modifications I did in my local repository to create patches that have not been revised yet, so if you try to apply them to Shotwell master they will fail.

If you don't mind, I prefer to wait until you review #3910, #3916, #3939 and #3991 (listed in the order I've sent them) to fix my local branch and get the correct patches.

I'm working on this, too.

My code is based on: https://github.com/Sanva/OpenCV-test-programs

I only need to add the Boxes detected by OpenCV. Is there any open source tool which is able to do face recognition and I can read some code?

So no chance to use a decentralized system like my QuiEstLa project (https://launchpad.net/quiestla) instead of creating multiple implementations?

I could use your daemon instead if the shotwell devs are ok with it.
I need a reply soon, because I don't want to invest time on sth which will be rejected.

Agree.

If the Shotwell project want to implement it, I will be available for fixes and improvements.
To give a quick status of the development (stopped since a long time ago because nobody was interested by the project), QuiEstLa is only able to locate faces for now, not recognize faces.

As I mentioned in comments 8, 11 and 13 above, we'd prefer to have Shotwell call OpenCV in process rather than by accessing an external service over DBus. zedtux, I know that you've been working on a DBus service, but in comment #15 you wrote "After that, I can write the same code, but directly in Shotwell in Vala." I thought that "the same code" meant both the DBus server and client code, so that DBus would no longer be used. Perhaps you meant only the DBus client code and were intending that Shotwell would still talk to a DBus server. If so, we had a misunderstanding.

Again, if someone can point out some technical advantage of using a DBus service over a shared library then we could consider it. But so far I haven't heard anyone say what the advantage would be.

@adam: When I said "After that, I can write the same code, but directly in Shotwell in Vala." it means that I was thinking of writing the same code (to detect faces locations) in Vala into Shotwell (as you're expecting).

I tried to do it, but Vala is a bit hard to learn. (I wrote C++ code years ago, and I'm not Java/C#/all other complex code syntax developer).
So, today I'm not going to implement it into Shotwell myself.
I'll could do it if someone can give me some screencasts or good guides to learn Vala.

In the other hand, I would like to know why you don't want to use a DBus service? (Just to understand your point).

@zedtux: See my comments #8 and #13 above. Calling a shared library is simpler and more efficient than using inter-process communication. It sounds to me like you may want to use a separate process because you don't want to write code in Vala. I think that the decision whether to use a shared library or separate process should be made for architectural reasons not related to a choice of programming language. Also, if you've coded C++ in the past then I don't think Vala should be so hard to learn - it's a simpler language. There's unfortunately no Vala book yet, but the Vala web site has plenty of sample code and tutorials.

@adam: The first reason I suggested a DBus service is because it could be used by any other applications, not only Shotwell, and the the second one is like you said to don't write in Vala. But before to try Vala, I had in my mind to implement this DBus service in Shotwell in Vala. (Write in Shotwell a client code)

@zedtux: As I pointed out in comment #13 above, a shared library can also be used by any other application, not only Shotwell. I don't see that a DBus service is any different in this respect.