Transcript File PowerPoint
© 2012 Boise State University
Data Warehouse User Engagement Readout
Daniel Gold, Project Manager
1
Agenda:
• • • • • • •
Executive Summary Study Methodology and Participants
Tool Needs and How They Align to Pyramid Common Reporting Tasks Process Opportunities What is Needed for Success Summary and Next Steps 2 © 2013 Boise State University
Executive Summary
Goal of User Engagement • Understand the key features and functions that users need from our ProClarity replacement • Understand barriers that exist today that create complexity when obtaining or using data • Understand what training and support will be needed as we roll out our ProClarity replacement Outcome • Based on features requested by users, Pyramid is a suitable replacement for ProClarity • Key reporting areas were identified that if improved would return immediate value to end users • Hands-on training based on knowledge level (i.e. Novice, Power User, Expert), robust self-help resources, BIRS point of contact and peer groups will be needed to support new tool adoption • Process changes are needed to streamline report intake and prioritization • Process changes are needed to build trust in the data warehouse and improve usability © 2013 Boise State University 3
• •
Methodology
Two user engagement sessions were held – 12 participants on 7/15 – 18 participants on 7/18 Participants were broken into small groups and led through a series of activities including: 1.
Brainstorm and categorization of data needs 2.
Completing a “reporting story” form 3.
4.
Extracting tool features and process enhancements from their stories Prioritization of features and process changes 30 50 43 12
• Participants • Reporting stories collected • Unique tool feature needs identified • Unique process changes requested
© 2013 Boise State University 4
Participant Population
Areas Represented:
– – – – – – – – – – – – – – – – – College of Arts and Sciences Institutional Research College of Engineering eCampus Extended Studies College of Health Registrar College of Health Sciences Provost College of Education College of Social Sciences and Public Affairs Admissions Honors College University Financial Services BSU Foundation STEM OIT HR and Finance under-represented! Further investigation to needs of these teams required.
Roles Represented:
– – – – – – – – – – – – – – – – – – – 5 directors 3 associate directors 3 business managers 2 administrative assistants 2 data analysts 1 adjunct faculty coordinator 1 advisor 1 associate registrar 1 business operations manager 1 chair 1 coordinator 1 financial technician 1 management assistant 1 management systems coordinator 1 program information coordinator 1 project coordinator 1 project manager 1 senior business manager 1 vice provost © 2013 Boise State University 5
Agenda:
• • • • • Executive Summary Study Methodology and Participants
Tool Needs and How They Align to Pyramid
Common Reporting Tasks Process Opportunities What is Needed for Success Summary and Next Steps 6 © 2013 Boise State University
© 2013 Boise State University
Requested Tool Features
Top 10 Prioritized Features: 1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Drill to Detail (that I need) Snapshots (compare points in time) Dashboards Build in Calculations Easy export to Excel and PDF Trend over time Ability to combine multiple reports Sort/filter data in web tool Data “Table of Contents” and improved logical structure Ability to add charts/graphs in web tool Support for custom reports (MyViews) Distribute reports to email audience from tool 7
Pyramid Can Address Our Tool Needs
Requested Feature Available in Pyramid Suite Demo links
Drill to detail (that I need) Basic Analytics ; Cascading Slicers ; Custom Parameters Support for Snapshots Dashboards Trending over time Combine multiple reports Built in calculations Data “Table of Contents” Sort/Filter data Ability to add charts/graphs Distribute reports via email Easy export to excel/PDF Support for custom views BIRS Process Controlled BIRS Process Controlled Building Dashboards Basic Analytics Data Mash up with Power Pivot Built in and Custom Calculations Basic Analytics Multi/Combo Charts © 2013 Boise State University 8
Agenda:
• • • • Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid
Common Reporting Tasks
Process Opportunities What is Needed for Success Summary and Next Steps 9 © 2013 Boise State University
Report Frequent Flyers
•
4 report categories made up nearly 2/3rds of all reporting stories written by participants.
• Short Term: Understanding the usage of reports in these categories and improving them could have a significant impact to user satisfaction and eliminate needless effort • Long Term: Consider prioritizing the creation of dynamic dashboards in Pyramid for these common reporting tasks
© 2013 Boise State University 10
• • •
Recommendations:
Recognize the “core four”
reporting needs and prioritize enhancements for these areas
Form tiger teams
for each area and investigate reporting enhancements
Identify standard reporting tasks
for the “core four” that will be good candidates for dashboards when Pyramid is released
© 2013 Boise State University 11
Common Data Manipulation Tasks
• • • • • Pain Points:
Lack of complete data
(HR, Finance, shadow systems) in data warehouse leads to users having to combine DW reports with queries from other sources
Comparing data for particular points in time
(aka snapshots) requires running multiple reports
Drill to detail
often provides significantly more data than is needed and excess rows/columns must be deleted
Lack of built in calculations and sort/filter options
in web tool makes exporting data to excel mandatory
Lack of native graph building capabilities
makes exporting data to excel mandatory User comment: All this would be better if… “we were able to gather point in time comparison
and make the comparisons in a more efficient manner. Right now this takes 2 people approximate 6-8 hours every week to complete.”
© 2013 Boise State University 12
• • •
Recommendations:
Utilize tiger teams to
understand why reports frequently need to be combined.
Make snapshot data available
to users where it exists Review report frequent flyers to
determine how simple changes in formatting can reduce manual manipulation
© 2013 Boise State University 13
• •
Use of Production Queries
52%
– of participants rely on queries against the production PeopleSoft system Expected for data that is not in the warehouse (HR, Finance, and Financial Aid) – Expected for data that needs to be real time
35%
– of participants rely on queries for student data that is in the warehouse Unexpected since this data is available
Potential Reasons for production queries:
1. Historical point in time comparisons (snapshots) 2. Validate data obtained through MyInsights or ProClarity 3. Familiarity with querying production over using DW tools © 2013 Boise State University 14
• •
Recommendations:
Utilize tiger teams to
understand why production queries are needed
to support the “core four” – Is data missing in the warehouse?
– What kind of snapshots are needed for comparison?
– Are we missing key data elements like type/rank of faculty?
Ensure that Pyramid is marketed well and
encourage users to begin using the warehouse
to meet reporting needs © 2013 Boise State University 15
Agenda:
• • • Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks
Process Opportunities
What is Needed for Success Summary and Next Steps 16 © 2013 Boise State University
Data Integrity
During group discussions two types of data integrity / validation issues emerged : 1.
2.
Trust in the accuracy of the data in the warehouse (
checking for warehouse errors
) Trust that manual manipulation performed on data had not corrupted the report (
checking for user errors
) User Comments:
“I think the most complex part of the reporting is validating the accuracy of the data. So much of what we do is this it takes the most time and is the most frustrating.” “[M]anual manipulation in excel is time consuming and there is risk of error. Data validation.”
All of this would be better if… “[It] didn't take
extensive manipulation each time. Confidence that the data was correct.”
1 in 4
participants cited data validation as a time consuming and required step.
“[N]umerous proclarity reports are referenced in order to create the summer report. Difficult to identify the criteria used to create the ProClarity report and difficult to validate the data. Time consuming to create report.”
© 2013 Boise State University 17
• •
Recommendations
Look for ways to
limit the need to combine multiple reports
in excel through enhancements to current reports Build trust in data warehouse reports by
consistently validating data
with data stewards and
stating explicitly on reports when validation took place
in each report © 2013 Boise State University 18
•
Data Definitions
Users reported difficulty interpreting the data due to
irregular naming conventions
and
inconsistent, unavailable, or out of date definitions
.
•
Knowing which fields should be used in a report
was often difficult when data from multiple sources had to be combined.
User Comment:
All this would be better if… “You could get a clear definition for what "active" means (e.g. not discontinued on PS, or what?!)”
User Comment:
“have you seen the terminology? there are five categories for each line. Things like STEM secondary non-STEM major”
© 2013 Boise State University 19
• • •
Recommendations
Determine
what information needs to be stored in each data definition
to aid users in making decisions when building reports Ensure that
updating data definitions is built into the development teams “definition of done”
any time a report is altered or created
Improve data definition discoverability
by creating links or embedding definitions in reports © 2013 Boise State University 20
•
Data Access and Integration
•
Access:
Access to data was not a wide spread issue, and
most users who spoke about access were concerned with limiting it
(security).
A small number of users stated that they needed access to HR data to perform their job functions • •
Integration:
Integrating HR, Finance and Student data
into the data warehouse was a frequent request
Many non-BIRS supported reporting systems
are used on campus. Frequently these are used for storing snapshot data for comparisons. © 2013 Boise State University 21
• • •
Recommendations
Leverage the Identity and Access Management Project to help
define role based data access authorization
Continue roadmap work to
bring in HR and Finance data into the warehouse Investigate commonly used reporting systems
and determine if it would be reasonable to bring the data into the warehouse © 2013 Boise State University 22
© 2013 Boise State University
Process Enhancements
Top 10 Prioritized Process Enhancements 1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Data validation Data integration Data definitions Data access Clear report structure Standardize reports for wider usage Make the data current (where is a student now) Migrate custom views to new tool Improve report request workflow Map tools to user needs 23
• • • • • • • •
Recommendations
Continue with roadmap work to
bring HR and Financial data into the warehouse
Establish a
holistic “definition of done”
for reports that includes data validation and data definitions for every report Utilize Identity and Access Management project to help
define roles for data access
Leverage tiger teams to
standardize reports for wider use
Develop a
clear and logical hierarchy for reports
Revise and evangelize
report request intake process
with a simple workflow that that takes into consideration requirements gathering and prioritization Begin working to
establish a “data doctor” program
that includes support from subject matter experts as well as BIRS Begin developing a rollout strategy for Pyramid that will help
map training and tool sets to groups of users
based on their reporting needs © 2013 Boise State University 24
Agenda:
• • Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks Process Opportunities
What is Needed for Success
Summary and Next Steps 25 © 2013 Boise State University
• • • • • • •
Success Requires…
Hands on training for users – Levels of training (novice, power user, expert) Robust documentation and tutorials An accessible reporting expert – – Data Dr. from BIRS Peer groups Migrate existing ProClarity reports into new tool Clearly define what tool should be used for what task Early adopter program for new tool More report building resources on the BIRS team © 2013 Boise State University 26
Agenda:
• Executive Summary Study Methodology and Participants Tool Needs and How They Align to Pyramid Common Reporting Tasks Process Opportunities What is Needed for Success
Summary and Next Steps
27 © 2013 Boise State University
User Rollout Track •Early adopter program for IR and key users •BIRS developer training Create Experts
Tool Implementation
Develop Self Help Tools •FAQ and how-to documents •Tutorials •Tool documentation •Classroom based training •Training based on data tasks and complexity Train Users Provide Ongoing Support •Data Dr.
•Peer groups •Ongoing training opportunities Development Track Install Pyramid Migrate ProClarity Reports Early adopter rollout Develop Report Hierarchy End User Rollout Develop Core Reports and Dashboards © 2013 Boise State University 28
• • •
Recommended Next Steps
Stay the Course for Roadmap Activities
– Continue to move forward with Pyramid purchase and implementation – Continue to integrate HR and Finance data into the warehouse
Pause Report Development and Modernize Process
– Define and evangelize project intake process – – Refine the team “definition of done” Respond to user engagement requests by generating specific enhancement projects for “core four” report issues identified in sessions
Identify Goals and Participants for Tiger Teams
– Develop specific and measurable goals for tiger teams – Identify subject matter experts, power users, and developers to participate – Establish a timeline for tiger teams to provide actionable recommendations (likely post-roadmap) © 2013 Boise State University 29
Daniel Gold, Project Manager Email [email protected]
THANK YOU
© 2013 Boise State University 30
BACKUP
© 2013 Boise State University 31
Breakdown of the Core Four
Enrollment • Out of State Students – Currently Enrolled • Class Enrollment Report • Fall Enrollment by Plan • Enrollment Activity Report • Student Enrollment • Enrollment Profile • Credit and Enrollment for Extended Studies • Summer Enrollment and Credit Data Student Success • Subsequent Course Success • Graduation Rate and Retention • # of Graduates • Count of Graduates • Cohort Graduation • Cohort Retention and Graduation • Student Progress through Curriculum • Retention and Graduation Success Rates • Academic Progress Report • Honors Students Graduated from Honors College Admissions • Funnel report • Admissions comparative data over time • Admissions reporting • Daily snapshot of applications and registration • Student registrations Course/Class Management • Class management • Time utilization • Class availability and utilization • Course statistics for BAS students • Course schedule The above items represent the report or report goal that participants identified in their reporting story forms. Duplicates intentionally included.
© 2013 Boise State University 32
Tiger Teams
Tiger teams work best when they are small hand-picked groups focused on a specific task (e.g. simplifying enrollment reporting). The team should include not only subject matter experts and power users, but also report and back-end database developers to speed up the process of weighing out potential solutions. These “tigers” are easily identified by being the go-to people in their area when a problem arises. 2-3 key report users The activity should be time-boxed and the output should be a list of enhancements the team can prioritize and implement with the goal of delivering on the specific task assigned.
Student Records SME Simplify Enrollment Reporting ETL Developer © 2013 Boise State University Report Developer 33