Transcript The Politics of Performance Engineering

The Politics of
Performance Engineering
At Mercury World 2004
By Wilson Mar
(wilsonmar.com)
Organizational Challenges
•
•
•
•
•
“We’re not told about projects”
“We’re left out of discussions”
“We’re not given enough time to test”
“We’re not sought out for advice”
“Our warnings are ignored”
(Notice: These are passive statements)
Resources
• Joel R. DeLuca, Ph.D. http://www.politicalsavvy.com/
Political Savvy: Systematic Approaches to Leadership Behind-theScenes ISBN: 0966763602x
• Eugene Kaluzniacky
Managing Psychological Factors in Information Systems Work : An
Orientation to Emotional Intelligence (Hershey PA Idea Group
Publishing, 2004) 101771 ISBN 1591401984
•
Definitions
• The mission of performance engineering
is to identify and measure potential
problems!
• We’re paid to be proactive (adjective)
– taking the initiative by acting rather than
reacting to events (Encarta)
– taking action by causing change and not only
reacting to change when it happens (Freesearch)
9 Ways to Be Proactive
1.
2.
3.
4.
5.
6.
7.
8.
9.
Deliver (build trustworthiness)
Open information flows
Satisfy Stakeholders - know who wants what
Follow the money (workflow)
Speak their language, not your own
Disclose fully
Stay within your role
Walk in their shoes
Don’t gloat or gossip
1. Deliver
• Performance Test Plan
–
–
–
–
Application Architecture, Use Case, Data Flows
Application Risks and Test goals
Test scripts & Run parameters
Application & Data Configurations
• Performance Test Results
– Response Times and Error Patterns
• Performance Profile Spreadsheet
– Predicted Upgrade Points & Timing
– Costs of Growth
• Project Status and Completion Projections
1.1 Performance Technical Profile
• The time it takes to start & stop the app.
• Speed timings:
– for first use by single user
– for subsequent uses and users
– Degredation as volume of data increases for
a single user
– Degredation as additional users are added
• Longevity:
2. Open Information Flows
•
•
•
•
•
Before – Objectives, Milestones, Timelines
Starting – Roles, Schedules, Metrics
During – Progress, Projections, Drafts
Ending – Sign-offs, Thank You’s
After – Actual vs. Planned Comparison
– Lessons Learned
• Throughout – Lunch, Snacks, Parties,
Gifts
Knowledge vs. Wisdom
Satisfying Stakeholders
• Marketing
– Ability to handle expected and possible loads.
• Finance
– Projected Timing and Costs of Growth.
• Data/Network Operations Center
– Expected bandwidth use (by network segment)
• Configuration Management (Archives)
– Content of Packages
• Deployment Manager
– Installation instructions and cautions
3.b Satisfy Stakeholders
•
•
•
•
•
•
Developers using the app under test
Trainers, Documentation Specialists
Development Manager
Development Lead
Developers
Functional Testers
4. Follow the Money
• Map people & projects
– Organize to Org. charts (from Phone Book)
– Find them before they hunt you down:
• Who are in the cubicles around you?
• Introduce yourself. Mingle at office parties
• Offer eats, magazine, website
• Map Technology Connections
• Map your own Use Cases
4.1 People & Projects
• Performance engineers often bridge more
technologies and organizational units than
others.
Director
Manager Dept/
Project
Need
Status
4.2 Technology Connections
• Have production equivalent
• Have reference services and data
available before need. Examples:
– User authentication service
– Security service to provide SQL connection
strings.
– Legal documents in applications
4.3 Performance Use Case
5. Speak Their Language
a. Identify application components and fields
using names assigned by developers, not
identifiers created by testers.
b. Cross reference:
Test Path
Component TestScript
0. Invoke URL Index.asp
Invoke
Speak visually - Illustrate
6. Disclose fully
• Disclose full test conditions:
“1 out of 1,000 invoke transactions”
“encountered an error when”
“under a load of 15 simultaneous users”
“with 0.5 seconds between transactions”
• Disclose full statistics:
“Invoke transactions averaged 20 seconds,
with a maximum of 41 and a minimum of 2
seconds.”
7. Stay within your role
• Let executives make judgments:
– “It’s not many errors”
– “It’s fast enough”
– “Users won’t care”
– “The change is not worth the risk”
• Is it your place to argue?
• Is it your place to enforce change?
• Who gave you a sheriff’s badge?
Exercise: If you were King
• Write a policy statement delegating
authority to the Performance Engineer
• Consider:
– Responsibility for reporting what?
– Authority to delay what under what
circumstances?
– Should be notified of what when by whom?
Mutually Exclusive Roles
An analogy:
Are you …
– a family doctor whom a teenager can safely
discuss issues she doesn’t feel comfortable
discussing with her parents.
OR
– a lab technician who takes urine samples for
pre-employment drug testing, with whom one
does not usually converse.
The Assumptions Minefield
• Assumptions about roles and
responsibilities colors perceptions and
communications.
• Inconsistency in assumptions can secretly
hinder trust, credibility, communication,
cooperation, and effectiveness.
8. Walk In Their Shoes
• Don't ask questions you can answer
yourself.
• Focus on solutions rather than problems
• Be skilled (certified) in the tools
developers use.
• Read their books. Attend their classes.
• Get VPN access for evenings & weekends
Hidden Agendas
• “My system is perfect”
– Response: “We’ll make it even more perfect”
– Response: “I’m required to test it”
Methods of Communication
A. Within HTML text, extra blanks, tabs, and
carriage return characters among text
will have a slight impact on performance.
B. After user logs off, server continues to
respond to previous URLs (obtained from
browser’s “Back” button).
Offer Possibilities
• Number of file handles
• Time before used handles are released
9. Don’t gloat or gossip
• “We’re in the same boat”
• Don’t say “You” or label/nickname people
• Instead of asking “why don’t you…
Ask “Is it possible to …
• Communicate minor issues quietly (grouped in a
list along with other items, not in a
separate/individual email)
• Deliver bad news in person and in private
• Don’t tell others about test results
• Preview status reports with the team before the
management
Remember: Be Proactive
“You can get everything you want if
you give enough other people
what they want.”
– Zig Zigler
“Get close to your friends.
Get even closer to your enemies.”
–?