OpticalFlowForHMI/diary/

week 1

Goals:
-Time planning of the project.
-Get OpenFrameworks working on Android.
-Read the theory about optical flow and study the "prototype code".

Problems:
-OpenFrameworks for Android is not working on the Windows platform (only in Linux and Osx). Eclipse has to be configured in order to compile the OF libraries for Android.

week 3

After a little bit more than one week, the openFrameworks library has been compiled for Android and the examples can be run in the Android device. It has taken more time than expected, and I still have some problems with editing the code and making it work in de device. Eclipse also seems to compile the openFrameworks library every time that I compile an aplication, and this makes it annoying to compile and run the new code (specially when you are trying a lot of new things to get used to this way of programming). For next week:

-Avoid eclipse compiling the OF library every time.
-Find an emulator to try to speed up the programming of the device.
-Get used to the OF way of programming on android: Develop an aplication mirroring the camera image and with a bouncing ball to show that I control the image and graphics libraries.

week 4

During last week I worked with images using OF in the Android Device. I also implemented an edge detector using laplacian filters, and it is working on the device, altough it is not fast enough. The application works with double numbers, so I will try to speed up the process using integers instead. The work I should do for next week is:

1. Get some optical flow results working in Android.
2. Start reading theory, and make a summary of each of the next papers:

  - Lukas and Kanade

  - Farnebäk

  - Horn and Schunk

  - Steffan Karlsson/Josef Bigun

  - SURF and SIFT

3. Communication in OpenFrameworks: Bluetoth or Wifi.

I must choose two of this three points, so I will start with point number 1 and 2.

week 6

At this point, the optical flow algorithm is working on the Android device. A ball moves in the screen of the device depending on the flow. For next week I have to make some improvements on it:

1. Averaging of the velocity vecotrs for moments, not for flow.
2. The application must have the back light always on, and it should not shut down to save energy.
3. The algorithm also has some problems with brightness changes and the auto adjust of the camera. I will try to solve this problem by changing the format of the images processed from grayscale to LAB.
4. Before calculating the optical flow in the frames, the image is downsampled to half its width and height. This should be for the user to choose, and in order to make it faster, the scale selection will be implemented by ignoring pixels.
5. Study the "restrict" c keyword and check compiler directives for optimizing code.
6. Comunication in openFrameworks: Bluetoth or WiFi.

week 7

After doing the improvements commented in the previous week work, the optical flow algorithm is running really fast (looks real-time), so the ball in the phone is moving really fast and smooth. No more work is needed regarding the restrict keyword for now. But there is an anisotropy of motion: The ball moves faster in the vertical direction than in the horitzontal. For next week:

1. Fix anisotropy of motion.
2. Check that the scale selection is working properly by executing the modified code in matlab.
3. Try to reduce the auto-scale effect of the camera when there is illumination change, by checking the average of the m002 moment.

week 8

The mouse in the android phone is working perfectly (No anisotropy, smoothness, speed...), but the work with illumiation invariance did not go so good. It is not so easy to detect this illumination changes (only the peaks are detected), so I will try to fix this in some other way. Next week I am going on vacation, and in two weeks there is the midway presentation. I will do the communication between the phone and the PC in order to show it during the presentation.

week 10

After holidays and the midway presentation, the communication between device and computer is kind of working. OpenFrameworks does not support bluetooth for android, so it is working by WiFi. The protocol used now is TCP/IP, but I don't know why the speed in the communication is not enough, so it still has to be improved. Before next week's meeting I will work on:

1. Improve the connection phone-PC (by checking firewall communication, or changing the protocol to UDP...).
2. Illumination invariance work:

  - Moments generated by derivating Hue or some combination of Hue and Sat of the pixels.

  - Start studying "Photometric Invariants".