Transcript Oculus Crane - Intelligent-visuals | AR & VR products

Oculus Crane
Crane Control System
Operated by an Oculus Rift and a controller
Barth, Thomas
Dayani, Ahmad Yama
Sauter, Christoph
Outline
1.
2.
Introduction
Project Description
1. Motivation, Task, Goal
2. Architecture
3. Usage
3. Hardware
4. GUI
5. Software Challenges
6. Summary/Conclusion
OCULUS CRANE
Donnerstag, 16. Juli 2015
2
Introduction
OCULUS CRANE
Donnerstag, 16. Juli 2015
3
Introduction
Project
Manager
Yama Dayani
Test
Manager
Quality
Manager
Thomas Barth
Architect
Cristoph Sauter
OCULUS CRANE
Integrator
Thoamas Barth
Yama Dayani
Tester
Yama Dayani
Developer
(Oculus Rift &
Juho)
Thomas Barth
Developer
(Crane &
Controller)
Cristoph Sauter
Donnerstag, 16. Juli 2015
4
Project Description
• Motivation
• Inovative system to revolutionize the human machine interactions
• Make complex task easier to understand and handle
• be a pioneer and give thought-provoking impulses for new inventions and development
• Task
• Develop and construct a model system which simulates the real world environment and
in where a user is able to control a crane with a controller and the Oculus Rift
• Goal
• establish a rudimentary but working version of the required crane system
•
control the crane conveniently
• Universal, not proprietary solution
OCULUS CRANE
Donnerstag, 16. Juli 2015
5
Project Description
Architecture
•
system depends on running
application
•
All information exchanged
with computer
• Oculus Rift controls the
respective camera system
indirectly
•
OCULUS CRANE
Crane gets operated by the
controller indirectly
Donnerstag, 16. Juli 2015
6
Project Description
Usage
• Configure system with Juho
• Inputs, parameters, profiles
• Oculus Rift
• Sight, movements, widgets
• Crane and controller
• movements
OCULUS CRANE
Donnerstag, 16. Juli 2015
7
Hardware requirements
• Evaluation of methods to implement control
• Rotation of boom
• Position of crane trolley
• Position of crane hook
• Preconditions
• None
• „Control crane and communicate to PC“
OCULUS CRANE
Donnerstag, 16. Juli 2015
8
Architecture
• No strict preconditions / reqirements
peripheral
PC
Juho
peripheral
µC
peripheral
peripheral
Scope of hardware part
OCULUS CRANE
Donnerstag, 16. Juli 2015
9
Process
• Theory
Evaluate
Test
Develop
• Reality
Evaluate
Test
Develop
OCULUS CRANE
Donnerstag, 16. Juli 2015
10
Evaluation
Power Supply
Additional functionalities
Choose µC
Estimate resources
Others ?
Analog IO
PWM IO
IO functionalities
Standard IO
Others ?
Plan PCB
Plan Software
DC Motor driver
Evaluate sensors and actors
Measure rotation
Measure trolley
Measure hook
OCULUS CRANE
Donnerstag, 16. Juli 2015
11
Chose µC
• STM32F4 based processor
•
•
•
•
•
Free tool chain available
Tool chain already setup
Available
(Some) experience
LCD/Touch nice to have
PCB needed
OCULUS CRANE
Donnerstag, 16. Juli 2015
12
Evaluate sensors and actors
• Sensor: Rotation of boom
• Install potentiometer mounted to axis of rotation
• Sensor: Position of crane trolley
• Install light barrier counting holes in gear wheel
• Sensor: Position of crane hook
• Install light barrier counting holes in gear wheel
OCULUS CRANE
Donnerstag, 16. Juli 2015
13
Evaluate sensors and actors
• Actor: DC motor drivers
• Change Rotation direction by
reversing polarity
 H-bridge like relay circuit
OCULUS CRANE
Donnerstag, 16. Juli 2015
14
Estimate Resources
OCULUS CRANE
Donnerstag, 16. Juli 2015
15
Plan PCB
• Failure in PCB planning would lead to major time delay
• IO shortage due to favored stepper solution possible
• Reduce PCB planning risk by
• Creating more universal solution
• Introducing additional µC as peripheral board
 Plan and produce prototype PCB while evaluation still in progress
Sensible decision. Stepper motors discarded later on
OCULUS CRANE
Donnerstag, 16. Juli 2015
16
Plan Software
•
•
FreeRTOS als OS
•
•
(Some) Experience
•
OS offers nice features (ITC)
OS handles implementation overhead to
execute (pseudo) parallel processes
• Peripherals drivers
•
•
•
•
•
Analog
PWM
USB
Serial
IO
Implementation as different tasks
•
•
•
•
Communication
Control rotation
Control trolley
• ASCII based Protocol
@value,…,vaue<EOL>
Control hook
OCULUS CRANE
Donnerstag, 16. Juli 2015
17
Development & Test
Install sensor rotation
Implement Code
Install trolley
Produce Baseboard
Install hook
Gather Parts
Build DC Motor Driver
Test
OCULUS CRANE
Donnerstag, 16. Juli 2015
18
GUI
•
•
•
•
•
•
•
•
•
•
•
Output Window
HUD
Input devices
Cameras
OOR
Servos
Interface
Screen
Oculus Rift
Shader
Profiles
OCULUS CRANE
Donnerstag, 16. Juli 2015
19
Juho
• Output Window
OCULUS CRANE
Donnerstag, 16. Juli 2015
20
Juho
• Output Window
OCULUS CRANE
Donnerstag, 16. Juli 2015
21
Juho
• Output Window
OCULUS CRANE
Donnerstag, 16. Juli 2015
22
Juho
• HUD
OCULUS CRANE
Donnerstag, 16. Juli 2015
23
Juho
• Input device
OCULUS CRANE
Donnerstag, 16. Juli 2015
24
Juho
• Cameras
OCULUS CRANE
Donnerstag, 16. Juli 2015
25
Juho
• OOR
OCULUS CRANE
Donnerstag, 16. Juli 2015
26
Juho
• Servos
OCULUS CRANE
Donnerstag, 16. Juli 2015
27
Juho
• Interface
OCULUS CRANE
Donnerstag, 16. Juli 2015
28
Juho
• Screen
OCULUS CRANE
Donnerstag, 16. Juli 2015
29
Juho
• Oculus
OCULUS CRANE
Donnerstag, 16. Juli 2015
30
Juho
• Shader
OCULUS CRANE
Donnerstag, 16. Juli 2015
31
Juho
• Profiles
OCULUS CRANE
Donnerstag, 16. Juli 2015
32
Software Challenges
• Distortion
• OOR
OCULUS CRANE
Donnerstag, 16. Juli 2015
33
Distortion
• Lenses in Oculus Rift cause distortion
OCULUS CRANE
Donnerstag, 16. Juli 2015
34
Distortion
Pincushion Distortion
OCULUS CRANE
Barrel Distortion
Pincushion + Barrel Distortion
Donnerstag, 16. Juli 2015
35
Distortion
• CPU based distortion
• Very slow
• Need to be done for every image component
• GPU based distortion
• Almost no delay (high fps)
• Can be applied to WPF element
OCULUS CRANE
Donnerstag, 16. Juli 2015
36
Distortion
• Output
Factor 0
OCULUS CRANE
Factor 1
Factor 2
Donnerstag, 16. Juli 2015
37
OOR
• Detect objects based on color
• Step 1: filter out unwanted color
• Step 2: search for “blobs”
• Step 3: give back position of “blob”
OCULUS CRANE
Donnerstag, 16. Juli 2015
38
OOR
• Filter colors
Original Picture
OCULUS CRANE
Filtered Picture
Donnerstag, 16. Juli 2015
39
OOR
• Search for blob
OCULUS CRANE
Donnerstag, 16. Juli 2015
40
Summary/Conclusion
• Task fulfilled
•
•
Working rudimentary solution to manipulate a toy with Oculus Rift
enhanced the controllability and perception
• Further enhancements
•
•
•
•
•
Adaptable to a lot of configuration
Tons of thinkable applications
• Teaching inexperienced personnel
• Using a moving base/vehicle to reach hazardous areas
Usage of more hardware
• LCD, LED, lights etc.
Wireless connections
Mobile applications
OCULUS CRANE
Donnerstag, 16. Juli 2015
41
Backup
OCULUS CRANE
Donnerstag, 16. Juli 2015
42
PCB
OCULUS CRANE
Donnerstag, 16. Juli 2015
43
Steppers
OCULUS CRANE
Donnerstag, 16. Juli 2015
44
Motor Driver
OCULUS CRANE
Donnerstag, 16. Juli 2015
45
Control loops
• Control Rotation
•
•
•
•
Read current position (analog value, mean over 128 samples)
Median over past 21 cycles
Calculate difference
Set velocity and direction
• Median and mean needed, reason may be voltage swings in supply voltage
 No further investigation
OCULUS CRANE
Donnerstag, 16. Juli 2015
46
Control loops
• Control trolley
• Two-Point control
• Control hook
• Control go up, go down
• Position is not measured due to mechanical difficulties
• Estimation over time not possible
OCULUS CRANE
Donnerstag, 16. Juli 2015
47