Transcript The Office of the Past Jiwon Kim Steve Seitz Maneesh Agrawala

The Office of the Past
Jiwon Kim
Steve Seitz
Maneesh Agrawala
Where is my W-2?
Unify physical and
electronic desktops
Video camera
• Recognize video of
paper on physical
desktop
Desktop
– Tracking
– Recognition
– Linking
Demo – Remote desktop
System overview
Video camera
Computer
Desk
User
System overview
Video of desk
System overview
Video of desk
Images from PDF
System overview
Images from PDF
Video of desk
Track &
recognize
System overview
Images from PDF
Video of desk
Internal representation
Track &
recognize
Desk
Desk
T
T+1
System overview
Images from PDF
Video of desk
Internal representation
Track &
recognize
Scene Graph
Desk
Desk
T
T+1
System overview
Where is
my W-2?
Images from PDF
Video of desk
Internal representation
Track &
recognize
Desk
Desk
T
T+1
System overview
Where is
my W-2?
Images from PDF
Video of desk
Answer
Internal representation
Track &
recognize
Desk
Desk
T
T+1
Document tracking example
before
after
Document tracking example
before
after
Document tracking example
SIFT [Lowe 04]
before
after
Document tracking example
before
after
Document tracking example
Motion: (x,y,θ)
before
after
Document Recognition
• Match against PDF image database
…
…
File1.pdf File2.pdf File3.pdf File4.pdf File5.pdf File6.pdf
Demo – Paper tracking
Photo sorting example
Photo sorting example
Demo – Photo sorting
Future work
• Enhance realism
– Handle more realistic desktops
– Real-time performance
• More applications
– Support other document tasks
• E.g., attach reminder, cluster documents
– Beyond documents
• Other 3D desktop objects, books/CD’s