Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible improvements for dense matching #108

Open
3 of 8 tasks
paulinus opened this issue Oct 25, 2016 · 3 comments
Open
3 of 8 tasks

Possible improvements for dense matching #108

paulinus opened this issue Oct 25, 2016 · 3 comments

Comments

@paulinus
Copy link
Member

paulinus commented Oct 25, 2016

The current dense matching implementation uses the PatchMatch variant from Shen13

Here is a list of possible improvements to the current implementation.

  • Init scores scores properly. Right now scores are initialized to 0. They should be initialized by computing the score of the initial depth map.
  • Init depthmaps from sparse point cloud. Depthmap is currently initialized using random values to explore the search space. An alternative would be to initialize it using the SfM points. For example, interpolating the depth of those points for all pixels.
  • Ignore homogeneous pixels. Pixel on the sky or on white walls have very little texture information and usually produce no match or an erroneous one. It would be better to not use them at all. Should try removing patches with constant color or constant gradient.
  • Sample views instead of checking them all. Currently, if more than one view are used as a reference, the score of a given depth plane is best correlation score amount the reference views. All views are checked in order to compute the best correlation. A faster alternative would be to sample the reference view randomly in a similar way we do for depth and normal. After each all random samples of depth and normal are done, we can sample a random view and check if matching score is better on that other view.
  • Filter out depth values without supporters on their neighborhood. When cleaning a depthmap, additionally to checking that other depthmaps agree with a depth value, check that the neighborhood of the point on its depthmap and on the others has enough points/depths nearby. That should remove many more outliers.
  • Average matching points instead of keeping just one of them when merging.
  • Striding depth computation. Since we use a 7x7 correlation kernel, very little change between one pixel and the next one. One could try computing the depth only every each other pixel to save time and then interpolating. Note that this is different than reducing the image size.
  • Approximate the plane induced homography by an affinity. We only use the homographies locally around one pixel so an affinity would be a good approximation. Given the affinity, the warpping for every pixel can be computed by adding vectors.
@xialang2012
Copy link

How can I run the dense module? Could you give some directions?

@paulinus
Copy link
Member Author

you can run

bin/run_all path_to_data
bin/opensfm undistort path_to_data
bin/opensfm compute_depthmaps path_to_data

And you should get the resulting point cloud depthmaps/merged.ply

@xialang2012
Copy link

got it, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants