Calibration

Camera calibration done with Camera Calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/. Uses a variety of checkerboard poses to determine intrinsic and extrinsic parameters.

Light source calibration to triangulate the position of the light source in World space. Convert to camera space with Rc_1*pt1+Tc_1. Important points for the light calibration are that two points are known to be on the ground plane, the other is on a ray orthogonal to the ground plane, and starting at one of the known points.

Filtering

Determine the contrast of a pixel. Keep Max and Min for each pixel over all images. For low contrast pixels, we can not determine when it is actually shadowed, so it should be ignored. For pixels with enough contrast, set the trigger for being shadowed as the mean of the min and max values of that pixel. This helps correct for the lack of a perfect point light source on different backgrounds.

Problems

Only the shadow edge is used determine points, but we can't put the shadow edge at every possible point we want to scan. The solution is to linearly interpolate the data.

What we want is to know for each pixel, where the shadow is on the image plane, where the light source is, and the ray from the camera, through the image. The 3d space plot of where the pixel is corresponds to the intersection of the ray through the image from the camera with the shadow plane for that specific point in time where the pixel is first shadowed. To do this we build an image of when each pixel is first shadowed. To find the exact time we do a linear approximation between the first frame when it is shadowed, and the the previous frame. We find out at what t the temporal brightness function (image-maskedmean) intersects the I=0 line (time-axis).

For all images
	For all pixels
		if shadowed
			if firstshadow
				lineartimeapproximation

Triangulation

Once the timemap is made of when each pixel has first seen the shadow, we then look up where A and B linearly interpolate to at that frame. They will show where the shadow plane intersects with the ground plane (table).

For all pixels
	t = pixeltime
	t0 = floor(t)
	t1 = ceil(t)
	A = LookupPt(HorizontalRef1,t0)
	A' = LookupPt(HorizontalRef1,t1)
	B = LookupPt(HorizontalRef2,t0)
	B' = LookupPt(HorizontalRef2,t1)

	alpha = mod(t,1)

	Amix = (1-alpha)*A' + alpha*A
	Bmix = (1-alpha)*B' + alpha*A
	
	plane = world2camera(Amix,Bmix,Ls)
	ray = normalize([x,y]',fc,cc,kc,alpha_c)

	% 3dpt in camera space. 	
	3dpt = isect(ray,plane)
	

Todo

Compute the LightSource position based on calibration data.
Output a depth image with the t's of the ray/plane intersections and display as a heightmap.