Interaction Database

This is the project page for creating a database of interactions between a character and an object/objects. This work is ongoing: expect more files to be added in the future. This work was performed by Peter Sandilands in collaboration with Myung Geol Choi.

Related publications:

Using the Polhemus Liberty magnetic 6 degree of freedom sensors, alongside the Microsoft Kinect, we have been able to capture complex, close interactions involving a character and various objects. We present these captured interactions here in a variety of formats, and intend to expand this database with further interactions in the future.

The objects we interact with are scanned using the Kinect, to get the oriented pointcloud data. The mesh for the object is then reconstructed using the Poisson surface reconstruction approach (via Meshlab). This ensures the mesh is an approximation of the real object we use for the interaction.

Markers are applied to both the actor and the object.

The overall data is stored in Autodesk FBX format, to easily be imported into Autodesk products. The free FBX SDK also allows for quick import of the motion (including skeleton and surface data) into your own programs. Currently we also provide BVH format data for some takes of a simple skeleton of the character and the markers on the object. This does not support geometry, surface animiation or changing bone lengths however. FBX data is the main format does, and is provided for the object and actor in all scenes, but this is uncleaned, and so has some holes (from reconstruction) and complex geometry. A skinned character needs to be fitted to the motion based on the given actor motion for full reconstruction.

We acknowledge the EU TOMSY project in the creation of this database.

The following captures show the style of interactions and approximate quality:

Single character, single rigid object interaction

FBX (multiple takes) BVH (if available) Video Example  
Spray Bottle N/A  
Container N/A  
Screwdriver N/A  
Coffee Jar Take1  
Cup (medium) Take1 Take2  
Cup (small) Take1  
Pistol Take1  
Rifle Take1 Take2 Take3  
Ball(into Box)
Many New Interactions  

Single character, deformable object interaction - note that the deformable object geometry isn't captured

FBX (multiple takes) BVH (if available) Video Example  
Shirt Dressing 1
Shirt Dressing 2
T-Shirt and Scarf Dressing
Jeans Dressing
Full Dressing
N/A Captured motion as compared to original motion.
Folding Clothes N/A  
Interaction Between Bag and Object N/A  
Paper Folding N/A  
Knotting Motion

Knotting Take 1 (Rendered)
Knotting Take 1 (Video Capture)
Knotting Take 7 (Rendered)
Knotting Take 7 (Video Capture)
Knotting Take 8 (Rendered)
Knotting Take 8 (Video Capture)
  Captured motion as compared to original motion.
Additional Clothing Interaction Data
All Data
Putting On Shorts
Putting On Sock
Taking Off Sweater 1
Taking Off Sweater 2
Taking Off Sweater 3

Further full body motions contributed by others

Xi Zhao: A Selection of Full-body Interactions with Non-Scanned Objects
Broom 1
Broom 2
Broom 3
Tube 1
Table 1
Chair 1
Chair 2
Big Box 1
Small Box 1
Big Bin 1
Small Bin 1
Hula Hoop 1
A Selection of Environment Interactions
Climbing Down From An Object
Climbing Down From An Object (Mirrored)
Climbing Up Onto An Object
Climbing Up Onto An Object (Mirrored)
Crawling (Mirrored)
Transition: Run To Crawl
Transition: Run To Crawl (Mirrored)
Transition: Crawl To Run
Transition: Crawl To Run (Mirrored)
Walking While Ducking
Walking While Ducking (Mirrored)
Transition: Run To Ducking
Transition: Run To Ducking (Mirrored)
Transition: Ducking To Run
Transition: Ducking To Run (Mirrored)
Jumping Over Obstacle
Jumping Over Obstacle (Mirrored)
Swimming (Mirrored)
Running (Mirrored)

Valid XHTML 1.0 Transitional