Interaction Database
This is the project page for creating a database of interactions between a character and an object/objects.
This work is ongoing: expect more files to be added in the future.
This work was performed by
Peter
Sandilands in collaboration with
Myung Geol Choi.
Related publications:
Using the Polhemus Liberty magnetic 6 degree of freedom sensors, alongside the Microsoft Kinect, we have been able to capture complex, close interactions involving a character and various objects. We present these captured interactions here in a variety of formats, and intend to expand this database with further interactions in the future.
The objects we interact with are scanned using the Kinect, to get the oriented pointcloud data. The mesh for the object is then reconstructed using the Poisson surface reconstruction approach (via Meshlab). This ensures the mesh is an approximation of the real object we use for the interaction.
Markers are applied to both the actor and the object.
The overall data is stored in Autodesk FBX format, to easily be imported into Autodesk products. The free FBX SDK also allows for quick import of the motion (including skeleton and surface data) into your own programs. Currently we also provide BVH format data for some takes of a simple skeleton of the character and the markers on the object. This does not support geometry, surface animiation or changing bone lengths however. FBX data is the main format does, and is provided for the object and actor in all scenes, but this is uncleaned, and so has some holes (from reconstruction) and complex geometry. A skinned character needs to be fitted to the motion based on the given actor motion for full reconstruction.
We acknowledge the EU TOMSY project in the creation of this database.
The following captures show the style of interactions and approximate quality:
Related publications:
- Sandilands, P., Choi, M. G. and Komura, T. (2012), Capturing Close Interactions with Objects Using a Magnetic Motion Capture System and a RGBD Sensor, Motion in Games, Lecture Notes in Computer Science Volume 7660, 2012, pp 220-231
- Sandilands, P., Choi, M. G. and Komura, T. (2013), Interaction capture using magnetic sensors. Comp. Anim. Virtual Worlds. doi: 10.1002/cav.1537 (to appear)
Using the Polhemus Liberty magnetic 6 degree of freedom sensors, alongside the Microsoft Kinect, we have been able to capture complex, close interactions involving a character and various objects. We present these captured interactions here in a variety of formats, and intend to expand this database with further interactions in the future.
The objects we interact with are scanned using the Kinect, to get the oriented pointcloud data. The mesh for the object is then reconstructed using the Poisson surface reconstruction approach (via Meshlab). This ensures the mesh is an approximation of the real object we use for the interaction.
Markers are applied to both the actor and the object.
The overall data is stored in Autodesk FBX format, to easily be imported into Autodesk products. The free FBX SDK also allows for quick import of the motion (including skeleton and surface data) into your own programs. Currently we also provide BVH format data for some takes of a simple skeleton of the character and the markers on the object. This does not support geometry, surface animiation or changing bone lengths however. FBX data is the main format does, and is provided for the object and actor in all scenes, but this is uncleaned, and so has some holes (from reconstruction) and complex geometry. A skinned character needs to be fitted to the motion based on the given actor motion for full reconstruction.
We acknowledge the EU TOMSY project in the creation of this database.
The following captures show the style of interactions and approximate quality:
Single character, single rigid object interaction
| FBX (multiple takes) | BVH (if available) | Video Example | |
| Spray Bottle | N/A | ||
| Container | N/A | ||
| Screwdriver | N/A | ||
| Coffee Jar | Take1 | ||
| Cup (medium) | Take1 Take2 | ||
| Cup (small) | Take1 | ||
| Pistol | Take1 | ||
| Rifle | Take1 Take2 Take3 | ||
|
Ball Ball(into Box) Book Bottle Box Broom Chair Hat Pen Table Racquet |
Many New Interactions | ||
Single character, deformable object interaction - note that the deformable object geometry isn't captured
| FBX (multiple takes) | BVH (if available) | Video Example | |
|
Shirt Dressing 1 Shirt Dressing 2 T-Shirt and Scarf Dressing Jeans Dressing Full Dressing |
N/A |
Captured motion as compared to original motion. |
|
| Folding Clothes | N/A | ||
| Interaction Between Bag and Object | N/A | ||
| Paper Folding | N/A | ||
|
Knotting Motion Videos: Knotting Take 1 (Rendered) Knotting Take 1 (Video Capture) Knotting Take 7 (Rendered) Knotting Take 7 (Video Capture) Knotting Take 8 (Rendered) Knotting Take 8 (Video Capture) |
Captured motion as compared to original motion. |
||
|
Additional Clothing Interaction Data All Data Putting On Shorts Putting On Sock Taking Off Sweater 1 Taking Off Sweater 2 Taking Off Sweater 3 |