🏛️ SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization

Upload a video or a set of images to create a 3D reconstruction of a scene or object. SAIL-Recon takes these images and generates a 3D point cloud, along with estimated camera poses.

Getting Started:

1. Upload Your Data: Use the "Upload Video" or "Upload Images" buttons on the left to provide your input. Videos will be automatically split into individual frames (one frame per second).
2. Preview: Your uploaded images will appear in the gallery on the left.
3. Reconstruct: Click the "Reconstruct" button to start the 3D reconstruction process.
4. Visualize: The 3D reconstruction will appear in the viewer on the right. You can rotate, pan, and zoom to explore the model, and download the GLB file. Note that the visualization of 3D points may be slow for a large number of input images.
5. Adjust Visualization (Optional): After reconstruction, you can fine-tune the visualization using the options below
   (click to expand):

   • Confidence Threshold: Adjust the filtering of points based on confidence.
   • Show Points from Frame: Select specific frames to display in the point cloud.
   • Show Camera: Toggle the display of estimated camera positions.
   • Filter Sky / Filter Black Background: Remove sky or black-background points.
   • Select a Prediction Mode: Choose between "Depthmap and Camera Branch" or "Pointmap Branch."

Please note: SAIL-Recon typically reconstructs a scene at 5FPS with full 3D attributes. However, visualizing 3D points may take tens of seconds due to third-party rendering, which is independent of SAIL-Recon's processing time. Using the 'demo.py' can provide much faster processing.

NOTE: due to the limitation of zerogpu provided by Huggingface, this web application can only process 50-200 images depending on the dynamic GPU resources. Please run `demo_gradio.py` LOCALLY if you want to process the video input.