Update: v4l2loopback

This is a loopback device that can act as a webcam for any program that can access a webcam. Essentially it takes in an input and that same input is its output for a program like Skype or Windows messenger. This essentially means that i can modify the input from a "real" camera with an overlay and transmit that to the v4l2loopback device which then Skype uses.

If i have time after the first prototype, i will try to integrate this with my prototype and possible show a live demonstration.

Links

http://code.google.com/p/v4l2loopback/

Lack of Blog Updates

I apologise for the lack of updates on this blog. Essentially i am in a coding phase. So at the present time i am just coding and am trying to put forward a prototype to show at the next meeting.

There is an update on my environment however. I have changed my coding environment from windows to linux for the sole reason that it is easier to install and use libraries.

Issues in Windows

Currently i am having some issues coding with GStreamer in Windows. It seems that i get a few errors that relate to creating a pipeline for the program to work with. Not sure if it is my environment or some other issue. So currently getting a linux distribution working on a computer for easier coding environment setup and should have a prototype done in about 1 week or 2.

If anyone viewing this blog has any experience with GStreamer on windows (www.gstreamer.com - sdk), and has any information for me it would be really appreciated.

For a little more information about the prototype. If i get the time for a UI then it will be done with the QT Framework

Current Modifications and What im Doing

Over these past few weeks, i have decided on a few changes to my system.

UI

The previous UI that i had prepared, now seems to be a little too "clunky". So i have decided to mainstream the UI to make it easier to use. Essentially there will be no video screen on the actual program, but instead will have options that the user will be able to set. I decided on this because this program, put simply is changing a webcam stream and replacing the face with a 2d caricature or 3d avatar, and because of this the UI does not need any screens of its own. The settings for the user to modify will include the choice of type of avatar (2d or 3d), then the avatar itself, along with a checkbox with the name activate which is checked when the apply button is pressed which then activates the program overlay, and others with if i find that more options would be useful.

System Structure

The system itself will be :

• GStreamer on the frontend that the UI code deals with
• OpenCV on the backend which does the algorithmic functions for face detection etc, called only via GStreamer functions made.

Essentially, the UI will call the GStreamer functions when the checkbox for activate (explained in the above paragraph) is on when the user clicks apply. At the moment i am trying to find the best method to pass the screen to OpenCV so that the OpenCV libraries can do the algorithmic functions. 

My current view is that i send via GStreamer a mat (or the relevant format) to OpenCV, which then calls functions for face detection and feature detection. OpenCV will then return a 2 dimensional array which are the coordinates of the features on the face, in reference to the mat that was sent to OpenCV. Then GStreamer will draw on the webcam video an overlay with the drawing according to the coordinates given by OpenCV.

Update: 03/07/2012

During these last couple of days i have been preparing my computer environment for proper coding which would include GStreamer.

My current environment now has :

• Windows SDK (which took longer to install than i originally thought)
• GStreamer Binaries (Built using CMake)

Currently i am doing code testing with GStreamer to verify its capability in my project.  If GStreamer works for what i intend to use it for then i shall continue with doing project code and updating this blog when i can

Resuming Work

As i have finished my exams, progress with this project will now continue.

I will update my progress on Monday 2nd July

Post-poned Work

As of now i am currently in my study period for final exams. Because of this, minimal to no work will be done on this till after my last exam on the 27th of June. My work on the project will then continue after my final exam.

Sorry for the delay in the project.