Transcript Document

T/TAC 2 Local Improvement Plan Project
Education for a Lifetime
Presented by T/TAC Group 2
December 11, 2003
Project lead: Dr Michael Behrmann
Introduction
 As a result of a grant from the Virginia
Department of Education, we are working on
automating the Local Improvement plan process.
 LIP process will be incorporated into the
T/TAC online site under “School Improvement”.
 The main goal of this online process is to
disseminate success stories and to function as a
performance tool for the Technical Assistants
and the Local Education Administrators.
Project Outline
Background to the LIP process
Analysis:
• Performance Analysis
Design:
Development:
• Wire Frames
• Prototype
Conclusions: Findings and
Recommendations
• Personas
• Use cases
• Interface Content Modeling
• Site Diagrams / Databases
Background
 Community Forum
 ‘No Child Left Behind’
 LIP Grants  Proposals and Reports
T/TAC Online: As Is
No Child Left Behind
 Three targets:
– School Personnel
– Service Providers
– Administrators
 January 8th, 2002
– Goal: every child meet state standards by the
2013-2014 school years
•SOL’s (Standards of Learning)
•AYP: ‘adequate yearly process’
LIP Grants
LIP (Local Improvement Plan) Grants
– “Sliver”
– Federal flow through money
• $3.5 million
• Fairfax receives the largest amount of any division: $285,088
since it has the most students in the state
LIP is comprised of Proposals and Reports
 Seven section document
– Focused on one or more of the five goals that are
related to ‘No Child Left Behind’ program
– Relates to 3 strategic directions and goals of
the VDOE
LIP Plan Map
http://immersion.gmu.edu/ttac/fall2003/group2/work/LIP/lipprocess.htm
Instructional Design Model we used
- Analysis
- Design
- Development
- Implementation
- Evaluation
ADDIE (Analysis)
Performance Analysis
“Partnering with clients and customers to help them
define and achieve their goals. Performance analysis
involves reaching out for several perspectives on a
problem or opportunity, determining any and all
drivers toward barriers to successful performance,
and proposing a solution system based on what is
learned, not on what is typically done.”
Allison Rossett (1999)
Why Performance Analysis?
 Preliminary study
 Prior to a needs assessment
 Task related:
– Optimals
– Actuals
Our Clients and Stakeholders
 Clients
– Dr. Pat Abrams: Associate Director, Special Education, VDOE
– Dr. Michael Behrmann: Professor, Director Helen A. Kellar Center
– Mr. Ken Olsen – Federal Technical Specialist - Mid-Atlantic region
 Stakeholders
– Dr. Pat Abrams
– Dr. Michael Behrmann
– Mr. Ken Olsen
– Ms. Lucinda Zimmerman - T/TAC Online Administrator
– VDOE Technical Assistance Staff (TAs)
– Local Education Agencies (LEAs)
– Dr. Shuangbao Wang – T/TAC Online programmer
Information Sources
 Information Sources
– Dr. Pat Abrams- Associate Director, Special Education, VDOE
– Dr. Michael Behrmann - Professor, Director of the Helen A. Kellar Center
– Ms. Lucinda Zimmerman - T/TAC Online Administrator
– Ms. Mary Wilds - Old Dominion University T/TAC (Region 2)
– Dr. Shuangbao Wang – T/TAC Online programmer
– Ms. Lisa Carson (College of William & Mary)
– Ms. Carol David (McLaughlin & Associates)
– Mr. Jeff Schuyler (McLaughlin & Associates)
– Educational professionals involved with grants proposal/reporting:
 School District Administrators
 Special Education Specialists
 Virginia Department of Education Technical Assistants
Our Research Strategy
 “Triangulation” – getting input/feedback on the
same material from different points of view
 Research methods
– Internal discovery
– Contextual Inquiry/Task Analysis
– Focus groups, Interviews, Surveys
Allison Rossett (1999)
Research Phase – 1
 Gathered background data
– Previous TTAC websites (Phases 1 through 6)
– Presentations related to GMU T/TAC & T/TAC Online
– Web-based community technologies such as Webinars,
chat groups, Discussion forums, Online communities
– VDOE website
– Grant Process - Developed Concept Map
– LIP Process – Developed Concept Map
Research Phase – 2
 Interviewed and gathered information from the
following sources:
– Dr. Michael Behrmann- Professor, Director Helen A. Kellar Center
– Dr. Lynn Wiley - Project Coordinator T/TAC Region 4
– Dr. Shuangbao Wang – T/TAC Online programmer
– LEAs and VDOE TAs
– VDOE Project Coordinator
– ‘No Child Left Behind’ Background
– Past Grant proposals/reports
– LIP program
– Rubric for proposal/reporting
– Goals
Research Phase – 3
 VDOE meeting in Richmond
– September 24, 2003, interviewed Dr. Michael Behrmann and began
questionnaire.
– Attended VDOE meeting in Richmond On September 29, 2003.
– Questionnaire distributed to participants at the Richmond meeting.
– Asked for responses by Friday, October 2, 2003.
– Sent thank you notes to primary participants.
– Conducted face-to-face interviews.
– Attended focus groups.
– Reviewed LIP evaluation report from McLaughlin group
(McLaughlin group performed preliminary survey of the LIP
grant process).
Questionnaire to participants at Richmond
Findings – Drivers for LEAs / TAs
– VDOE personnel could use a systematic compilation of
data across projects.
– Realization of geographical constraints.
– Need to share best practices in a timely manner.
– Frequent staff turnover, an organized system online
would require little or no training for the new hires.
– A repository that would enable users to access old data
easily and in a timely manner.
– Access to quantifiable data.
– The ability to submit or access interim reports.
Findings – Barriers for LEAs / TAs
– Operating systems vary.
– Internet connection speed low in some counties.
– No face-to-face interaction.
– Due to time constraints, unable to complete a
proposal / report in one go.
– Comfortable in using Microsoft Word / Excel
for LIP proposal and reporting purposes.
Findings – LEA Needs / Requirements
– Keep the LIP process simple.
– Make the LIP process sustainable.
– Additional staff.
– Local school administration/school board
support to promote the LIP project to the
individual schools.
Findings – TA Needs / Requirements
– Reporting process should be part of grant
application.
– Need interim reports.
– Formally track problems.
– TAs need training/support.
– Want the ability to create and evaluate longterm plans.
– Want to quantify knowledge learned from LIP
grant process to apply to state goals/priority
projects.
Recommendations
 Stakeholders see value in an online process for
Local Improvement Plans.
 Any process needs to be carefully designed to
ensure it meets user needs.
– Considerations are: limitations on time, connectivity, and user
knowledge.
• Simple and well supported with help functions
• Effectively use with minimal learning time.
• Flexible to enable users to adapt easily
• Enable users to receive and retrieve data using programs
that they are familiar with and have readily available.
• Word and Excel templates to streamline data
reception.
A
DDIE (Design)
Personas
“A user role is an abstract collection of
(common) needs, interests, (shared) expectations,
(common) behaviors, and responsibilities
characterizing a relationship between a class or
kind of users and a system”.
Constantine & Lockwood (1999)
Persona – Public
Characteristic/Background
 Parent of special needs child
 Married
 Loundon County
 Literate
 39 years-old
Michelle Gallager
Work Environment
 Accesses internet via dial up from home or work
 Special needs volunteer
Persona – Public
Responsibilities
 Raising their child to the best of her ability
- Wants most information possible
Michelle Gallager
Goals / Wants / Needs
 Desire to select best school for child
 Develop child’s abilities
 Wants to know how schools are doing
- District
- States
Persona – Public
Avoid
 LEA jargon & technical terms
Frequency of Use of
“School Improvement” site
 Infrequent : every 6 months
Quote
“My child deserves the best.”
Michelle Gallager
Persona – Local Education Administrator
Characteristic/Background
 Busy
 Overworked
 Understaffed
 Not interested in becoming
technological experts
 45 years-old
Katherine Cox
Work Environment
 Access to computer at work operating on Window 98
 Access to internet via dial up
 Dose not have access to latest software/browsers
 Only special education administrator in her
work location
Persona – Local Education Administrator
Responsibilities
 LIP Proposal
 LIP report
- Final and Interim data for reports
 Gathering baseline data for LIP proposal
 Evaluate project against their goals
Goals / Wants / Needs
 Need to get administrative buy in
- Accountability
 Exchange best practices with other LEA’s
 Store data as it is collected for LIP reports
 Simplify the whole process
- Avoid the ‘tax return’ syndrome
 Utilize past report-do not start from scratch
Katherine Cox
Persona – Local Education Administrator
Avoid
 Creating more work due to the process being online
 Last minute reporting
Frequency of Use of
“School Improvement” site
 Infrequent:
Write proposal once every year
Entering in data for the report
Write final report once every year
Quote
“K.I.S.S.”-driving design
(keep It Sweet & Simple)
Katherine Cox
Persona – Technical Assistant
Characteristic/Background
 Married with children
 Busy
 Not interested in becoming
technological experts
 47 years-old
Work Environment
 Centrally located with other TA’s
 Less than one year on the job
 High speed internet access
Justine Braxfield
Persona – Technical Assistant
Responsibilities
 Helping LEA’s with LIP proposals
 Evaluating LIP proposals using checklist/rubric
 Keep the LEA’s on schedule with LIP
 Responsible for 1 of 8 T/TAC regions
 Issue status of LIP proposal-approval or not Justine
 Review and disseminate reports
Goals / Wants / Needs
 Want ‘how to’ on the site for LEA’s
 Ability to track progress of LIP project
 Exchange best practices with other TA’s and their LEA’s
 Rubric for evaluating proposals online
 Easily disseminate the LIP reports
- Quantify the data and produce reports from that data
Braxfield
Persona – Technical Assistant
Avoid
 Last minute reporting
 Complexity
 More work because of an online process
Frequency of Use of
“School Improvement” site
Justine Braxfield
 Intermittent Frequent Use:
- Monitoring of LIP process:
Verifying that proposals/reports are submitted on time-for
1 month-4hrs./day
 Sending reports to Pat Abrams
Quote
“ I just started working here, I don’t know what
has been done in the past.”
We then created
Use Cases…
Essential Use Case
 Describes an interaction that is complete,
meaningful and well–defined to a particular
user.
 Based on purposes or intentions of users rather
than concrete steps or mechanisms.
Constantine & Lockwood (1999)
Purpose of Use Cases
We Want to know…
 What System users
• Are trying to accomplish (goals).
• Need from the system to accomplish their goals.
 How the system can provide what they need?
Use Case Process
 Described the Tasks that needed to be
performed.
 Included Extension Use Cases as well.
 Mapped relationships amongst use cases.
Essential Tasks
 Read
 Communicate
 Write
 Save
 Edit
 Review
 Track
Use Case Mapping
Use system
Write
Review
Communication
connected to All
Save
Read connected
to All
Track
Edit
Use Cases
Description
Enter site to
READ ONLY
Interaction
System Response
Enter website address
Description
User wants to create a new
document
Interaction
System Response
User selects template
Site Displays
Enter Region
Welcome page displays
Click on School
Improvement
displays School Improvement page
Print
Download
Work online
Selects and acts on
prompts for finding the
information
displays list of LIP reports matching
selections
Selects desired
information
displays LIP report allows user to read
it on-line, print, or download file
Print
Email
Exit
Download
Notify TA
Chooses to read LIP
report on-line
Description
Interaction
User wants to review
a document
System Response
User modifies document
User selects their document
document is displayed
displays prompt with newly received
documents
user acknowledges prompt
sends receipt to sender
User selects a document
document is displayed
Print
Download
Work Online
Save
Print
Email
Exit
Download
Notify LEA
User chooses either to print(*),
download(*) or work online to modify
document
User chooses to print(*)
User chooses to download(*)
User chooses to work online
user modifies document
User saves data
User chooses either to print(*),
download(*), notify TA, email(*) or
exit(*)
User chooses to print(*)
User chooses to Email(*)
User chooses to Exit(*)
User chooses to download(*)
user chooses to notify LEA
print(*)
download(*)
Print
Download
Work Online
Save
Save(*)
print(*)
Email(*)
Exit(*)
download(*)
Print
Email
Exit
Download
Notify TA
User chooses either to print(*),
download(*) or work online to
modify document
User chooses to print(*)
User chooses to download(*)
User chooses to work online
user modifies document
User saves data
User chooses either to print(*),
download(*), notify TA, email(*)
or exit(*)
User chooses to print(*)
User chooses to Email(*)
User chooses to Exit(*)
User chooses to download(*)
user chooses to notify TA
print(*)
download(*)
Save(*)
print(*)
Email(*)
Exit(*)
download(*)
system notifies TA via prompt
system notifies LEA via prompt
User chooses to Exit(*)
template is displayed
User chooses either to print(*),
download(*) or work online
print(*)
User chooses to print(*)
download(*)
User chooses to download(*)
User chooses to work online
User enters data
User finishes entering data
User saves data
Save(*)
User chooses either to print(*),
download(*), notify TA, email(*)
or exit(*)
print(*)
User chooses to print(*)
Email(*)
User chooses to Email(*)
Exit(*)
User chooses to Exit(*)
download(*)
User chooses to download(*)
user chooses to notify TA
system notifies TA via prompt
User chooses to Exit(*)
User chooses to Exit(*)
Use Case: TA Review
Description
Interaction
User wants to review
a document
System Response
displays prompt with newly received
documents
user acknowledges prompt
sends receipt to sender
User selects a document
document is displayed
Print
Download
Work Online
Save
Print
Email
Exit
Download
Notify LEA
User chooses either to print(*),
download(*) or work online to modify
document
User chooses to print(*)
User chooses to download(*)
User chooses to work online
user modifies document
User saves data
User chooses either to print(*),
download(*), notify TA, email(*) or
exit(*)
User chooses to print(*)
User chooses to Email(*)
User chooses to Exit(*)
User chooses to download(*)
user chooses to notify LEA
print(*)
download(*)
Save(*)
print(*)
Email(*)
Exit(*)
download(*)
system notifies LEA via prompt
User chooses to Exit(*)
Which leads to…
 Interface content models
 Site diagrams / Database
Interface Content
Models
Interface Content Models
An abstract representation of the contents of
the various interaction spaces for a system
and their interconnections.
Some of the simplest modeling technology
- paper and Post-it notes-works best.
Constantine & Lockwood (1999)
Purpose of Interface Content Models
 In implementation, each interaction space
becomes a recognizable collection comprising
part of the user interface.
 Denotes specific interface components, such
as toolbars, command buttons or selection lists.
Constantine & Lockwood (1999)
Interface Content Models
Some of the simplest modeling technology
-paper and Post-it notes-works best.
Constantine & Lockwood (1999)
Database Set – Up
Reasons for considering database design
Poor database design leads to…
 Data integrity issues
• Loss of data
• Slow queries
– User confidence, usage dies
 System dies and is not used
– Money is wasted
Facts
 The LIP data will be stored in a MS SQL
server database.
 The MS SQL Server can manage scalable
databases (Data can grow).
 The database must provide accessible, reliable
and accurate information for the customers of
VDOE, LEA and the public.
Database Tasks
 Read
 Write
 Save
 Edit
 Delete
Save, Write, Edit Data
How will data captured from web site be stored…
 System users who save information
• Query string is sent when “save” is clicked to
tell the database to save a set of data.
• MS SQL Server communicates via query
language using an Insert or Update script.
(i.e.. Update main_table or
Insert data into main_table.)
 Saved information can be retrieved
for future use.
Read or View Data
 How will captured data from the web site be
viewed?
• System users can use the “search” feature
• A query string is sent when “search” is
clicked to tell the database to find a set of data.
(i.e.. Select * from main_table
where Lea_id = Jdoe.)
• The data is the displayed as results for the user.
Delete Data
 How will data be removed from the web site…
• System users can use the “Delete” feature
• A query string is sent when “Delete” is clicked
to tell the database to find a set of data.
(i.e.. Delete * from main_table
where Lea_id = Jdoe.)
• The data is then deleted for that specific user.
LIP Database Sample
Key
Database = Stores a set
of relational tables
LIP
Table
LEA Table
TA Table
Field1
Field1
Field2
Field2
Main Table
LIP Table
Field1
Field1
Field2
Field2
Tables = Store a data for
related a subject or fields.
Field
Stores data for subjects
in the form of bytes (8
bits).
Bit
Stores one unit or
character.
Site Diagrams
Site Diagram - T/ Taconline User
Begin:
Welcome T/TAC User
Public
User?
Yes
Choose
Report
No
User Selects
Primary
Search
User Selects
Secondary
Search
User Login
TA
User?
Yes
TA
Displays
Results
End
No
No
LEA
User?
User:
Reads/Prints/
Download/ Save
Report
Yes
Yes
LEA
Search
Again?
No
Site Diagram - T/ Taconline User - LEA
LEA
New
Document?
Yes
Selects
Template
No
No
Reports
Proposal
User
Selects
Document
Enter
Demographic
Data
User
Writes
Updates
Fill out
Section 1
User
Saves/Prints
Data
Selects
Section
Notify
TA?
Write Data
Select
Interim
Report
Template
No
Notify TA?
Yes
Notification
Sent to TA
Yes
User
Prints/
Download/
Save
Notification
Sent to TA
TA
TA
Next
Section?
Yes
End
Select Final
Report
Template
No
End
Site Diagram - T/ Taconline User - TA
TA
No
Any Regional
documents to
review?
Yes
User
selects
document
End
User Write
Comments
User
Reviews
Document
Approval
Sent to
LEA
Comments
Save and
Sent to LEA
No
Yes
Approve?
End
LEA
AD
DIE (Development)
Wire Frames
What is a Wire frame?
 Identifies navigation
 Does not include content
 Offers an opportunity for stakeholders to give
input into functionality
 Translates the interactions of the flowchart into
a frame-by-frame process
Systems Wire Frames
Public User
Systems Wire Frames
Public User – search page
Flowchart
Public User – search page
Enter t/tac
online site
Select search
Yes
Yes
Public User?
Search by
category?
No
Yes
No
User Login
No
Yes
TA User?
TA
No
No
Choose
category
Yes
LEA User?
LEA
Submit
Enter
keyword?
Systems Wire Frames
Public User – results page
Systems Wire Frames
Public User – chosen results page
Flowchart
Public User – results page
Open chosen
Results page
Yes
Print page
Results page
Print?
Yes
No
No
Email
Yes
Choose result?
Email?
No
Yes
Download
Download?
No
Yes
Search again?
End
Systems Wire Frames
TA User – TA Home Page
Systems Wire Frames
TA User – Notification Page
Systems Wire Frames
LEA User – LEA Homepage
Systems Wire Frames
LEA User – LEA Proposal Page
Systems Wire Frames
LEA User – LEA Home Page
Systems Wire Frames
LEA User – LEA Reports Page
Systems Wire Frames
LEA User – LEA Home Page
Systems Wire Frames
LEA User – LEA Archived Materials
Page
Systems Wire Frames
LEA User – LEA Home Page
Systems Wire Frames
LEA User – Notification Page
Wire Frames: lessons learned
 Not Pretty
 Content? What content?
 Harder than it seemingly looks!!!
Prototype
Prototype
This is the fun part. For many designers, the
real fun finally begins when it is time to
produce the visual design for the user
interface.
Constantine & Lockwood (1999)
Active low Fidelity Prototype
 Connects all the wire frames.
 Vaguely looks like the final product.
 Opportunity to show how the user would
navigate the system.
 Shows interactivity.
Getting Started…
Michelle
Gallager
Processes Completed
-Analysis
Fall 2003
Performance Analysis
-Design
Personas
Use Cases
Interface Content Modeling
Databases / Site Diagrams
-Development
Wire Frames
Prototype
- Implementation
- Evaluation
Fall 2003
Fall 2003
Findings…
Systems Wire Frames
TA User – TA Home Page
Questions for SME (Technical Assistant)
1. How do you currently give feedback on a particular proposal or report.
for instance, do you e-mail the entire proposal/report with rephrased
sentences? Do you send via U.S. Mail? Or if e-mailing, do you simply write
your comments independently?
2. Do you edit reports? If yes, how? Could you give us your step-by-step proces
3. Do you have a template that you've designed or have inherited for reviewing
reports or proposals? If you don't have one, would you want one online? If yes,
what are your "must have's" on the template that you use?
4. Are there only the the 7 sections that need to be answered in a proposal? Is
there any additional information that needs to be sent along with a proposal?
5. Are there only the 4 questions that need to be answered in a report?
Is there any additional information that needs to be sent along with this report?
6. What happens after a proposal approved? Please
outline the steps.
…Surprise!!!
Annual Plan
Implications from
Findings…
LIP Database Sample
Key
Database = Stores a set
of relational tables
LIP
Table
LEA Table
TA Table
Field1
Field1
Field2
Field2
Main Table
LIP Table
Field1
Field1
Field2
Field2
Tables = Store a data for
related a subject or fields.
Field
Stores data for subjects
in the form of bytes (8
bits).
Bit
Stores one unit or
character.
Annual Plan Database Sample
Annual Plan
1. Medicare Fund
2. Policies & Procedures
3. LIP
4. Personnel Development
5. Local advisory committee
6. Local jails
7. Previous Years Plan Report
Grant Approval
Immediate Suggestion:
 Revisit current infrastructure of LIP in
supporting future requirements (Annual Plan)
– 7 un-identified elements of a VDOE Annual
Plan
– Finances are not released until Annual Plan
is satisfied.
Recommendations…
We propose:
 Analysis of the 7 unidentified elements
and a database design for all
or
 Revisit client intention of LIPs online
References
 Constantine, L.L. & Lockwood, L.A.D.
(1999). Software for use: A practical guide to
the models and methods of usage-centered
design. New York: Acme Press.
 Allison Rossett (1999). First Things Fast. A
Handbook for Performance Analysis. CA:
John Wiley & Sons, Inc.
 http://immersion.gmu.edu/immsite/student
Questions