Transcript SpeechClipse v 1.0 - University of Alabama

SpeechClipse
v 1.0
“An Effective Plug-In for the Eclipse IDE”
Team Members:
Faizan, Javed
Qasim, Ijaz
Rajesh, Sudarsan
Raymond, Corvin
Shairaj, Shaik
Suman, Roychoudhury
Advisors:
Dr.Jeff Gray & Dr.Barrett Bryant
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
Computer and Information Sciences Department
University Of Alabama at Birmingham
Motivation

Speech plug-in to positively affect individuals who are
physically challenged or diagnosed with Carpal Tunnel
Syndrome (CTS)

Increased productivity and ease of IDE navigation by
concurrent control

Customizable and Taylor-Made speech control

Contribution towards Eclipse community
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
2
Background
How does a Typical Speech Engine works ?
Picture - Courtesy Microsoft Corporation
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
3
Implementation

The user defined “Speech-Robot” Class to translate
speech commands into keyboard events

Mapping Eclipse events to speech commands through
rule based grammar rules

Extensible design to accommodate future IDE events

Complete control of keyboard through used defined
KeyMap Class

JSGF (Java Speech Grammar Format) to specify rules
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
4
Control Flow Diagram
RECOGNIZER
Rule-Based
Grammar
NAVIGATOR
SYNTHESIZER
TOKEN
SpeechClipse
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
5
CloudGarden’s Speech Engine

JSAPI based implementation

Consists of both Recognizer and Synthesizer

Supports both Rule-Based and Dictation grammar
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
6
Grammar

SpeechClipse Control:


Grammar rules and event properties are the primary
ways of controlling the speech recognizer’s activity
JSAPI Specifications:


20-Jul-15
JSGF (Java Speech API Grammar Format)
Currently Used In SpeechClipse
JSML (Java Speech API Markup Language)
Will be used in future versions of SpeechClipse
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
7
Advantages of SpeechClipse

Increase productivity to users by allowing multi-tasking

Speech enabled programming environment for various users
groups

Better recognition by adapting Speech-engine with user voice

Highly Customizable

Rich set of Speech Commands:

Hand’s Free Refactoring

Hand’s Free Debugging

Code templates

Speech Wizards
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
8
Current Status


Efficient navigation of Eclipse IDE
Code-Templates:


Speech-Hot-Keys:





Automate the typing of standard programming constructs
Accomplish the effect of multiple keyboard or mouse clicks
Access to standard external applications
An extensible design
Ability to dynamically update the SpeechClipse
grammar
Harness the powerful refactoring and debugging
capabilities of Eclipse through Speech commands
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
9
Video Demonstration






Project creation
Standard keyboard capabilities
Project compilation and building
Hand’s free Debugging
Hand’s free Refactoring
Generating Java-Documentation
SpeechClipse
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
10
Future Work

Improve the Grammar using of all the capabilities of
rule-based grammars.

Implement Lazy-Typing using Dictation grammar

Emulate mouse movements

Ability to effectively code and store identifier-names

Speech synthesis (Futuristic) for visually impaired
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
11
Thank You
20-Jul-15
http://www.cis.uab.edu/info/Eclipse/SpeechClipse/
12