add demo code

Author: pesser

License.txt

@@ -0,0 +1,19 @@
+Copyright (c) 2020 Robin Rombach and Patrick Esser and Björn Ommer
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
+OR OTHER DEALINGS IN THE SOFTWARE./

README.md

@@ -7,6 +7,8 @@
 [Björn Ommer](https://hci.iwr.uni-heidelberg.de/Staff/bommer)<br/>
 \* equal contribution
 
+[arXiv](https://arxiv.org/abs/2104.07652) | [BibTeX](#bibtex)
+
 ### Interactive Scene Exploration Results
 
 [RealEstate10K](https://google.github.io/realestate10k/):<br/>
@@ -17,4 +19,81 @@ Videos: [short (2min)](assets/realestate_short.mp4) / [long (12min)](assets/real
 <a href="assets/acid_short.mp4">![acid](assets/acid_preview.gif)</a><br/>
 Videos: [short (2min)](assets/acid_short.mp4) / [long (9min)](assets/acid_long.mp4)
 
-Code coming soon.
+### Demo
+
+#### Installation
+
+The demo requires building a PyTorch extension. If you have a sane development
+environment with PyTorch, g++ and nvcc, you can simply
+
+```
+pip install git+https://github.com/CompVis/geometry-free-view-synthesis#egg=geometry-free-view-synthesis
+```
+
+If you run into problems and have a GPU with compute capability below 8, you
+can also use the provided conda environment:
+
+```
+git clone https://github.com/CompVis/geometry-free-view-synthesis
+conda env create -f geometry-free-view-synthesis/environment.yaml
+conda activate geofree
+pip install geometry-free-view-synthesis/
+```
+
+#### Running
+
+After [installation](#installation), running
+
+```
+braindance.py
+```
+
+will start the demo on [a sample scene](http://walledoffhotel.com/rooms.html).
+Explore the scene interactively using the `WASD` keys to move and `arrow keys` to
+look around. Once positioned, hit the `space bar` to render the novel view with
+GeoGPT.
+
+You can move again with WASD keys. Mouse control can be activated with the m
+key. Run `braindance.py <folder to select image from/path to image>` to run the
+demo on your own images. By default, it uses the `re-impl-nodepth` (trained on
+RealEstate without explicit transformation and no depth input) which can be
+changed with the `--model` flag. The corresponding checkpoints will be
+downloaded the first time they are required. Specify an output path using
+`--video path/to/vid.mp4` to record a video.
+
+```
+> braindance.py -h
+usage: braindance.py [-h] [--model {re_impl_nodepth,re_impl_depth}] [--video [VIDEO]] [path]
+
+What's up, BD-maniacs?
+
+key(s)       action                  
+=====================================
+wasd         move around             
+arrows       look around             
+m            enable looking with mouse
+space        render with transformer 
+q            quit                    
+
+positional arguments:
+  path                  path to image or directory from which to select image. Default example is used if not specified.
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --model {re_impl_nodepth,re_impl_depth}
+                        pretrained model to use.
+  --video [VIDEO]       path to write video recording to. (no recording if unspecified).
+```
+
+## BibTeX
+
+```
+@misc{rombach2021geometryfree,
+      title={Geometry-Free View Synthesis: Transformers and no 3D Priors}, 
+      author={Robin Rombach and Patrick Esser and Björn Ommer},
+      year={2021},
+      eprint={2104.07652},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```

environment.yaml

@@ -0,0 +1,15 @@
+name: geofree
+channels:
+  - pytorch
+  - defaults
+  - conda-forge
+dependencies:
+  - python=3.8.5
+  - pip=20.3
+  - cudatoolkit=10.1
+  - cudatoolkit-dev=10.1.243 # for nvcc
+  - gcc_linux-64=7.3.0 # for nvcc compatibility
+  - gxx_linux-64=7.3.0 # for nvcc compatibility
+  - pytorch=1.7.1
+  - torchvision=0.8.2
+  - numpy=1.19.2

geofree/__init__.py

@@ -0,0 +1 @@
+from geofree.util import pretrained_models