Transcript Powerpoint - The Fighting Pi

FRC Scouting and
Competitive Assessment
Jim Zondag - Chrysler
Isaac Rife – BAE Systems
4/11/2013
About the Presenters
Jim Zondag:
Apps Development Manager - Chrysler
Team 33 Team Leader
FRC mentor since 1998.
Isaac Rife:
Sr. Mobility Systems Engineer - BAE Systems
Team 33 Mechanical Mentor, Chief Scout.
FRC mentor since 2002.
Quote of the Day
“If you know the enemy and know yourself, you
need not fear the result of a hundred battles. If
you know yourself but not the enemy, for every
victory gained you will also suffer a defeat. If you
know neither the enemy nor yourself,
you will succumb in every battle”
- Sun Tzu – The Art of War
Why Scout?
 Scouting is very important, and is often undervalued.
 Good quality, accurate scouting is one of the key
ingredients to a successful FRC competition weekend.
 Successful teams put considerable effort into scouting.
 Without scouting, proper strategy development is
almost impossible.
 Good scouting info will allow all members of your
alliances to make the most out of every match.
 A good pick list is very important for Alliance Selection.
 Even if you think your team will not be picked, ALWAYS
make a pick list.
What is Scouting?
 Scouting is as much about your own team as it is about
the other teams.
 The goal of scouting is to figure out how your team can
best work with your two alliance partners against your
three opponents, and to help produce a well thought
out Pick-List.
 Scouting is the process of collecting objective and
subjective data on all of the teams at the event
(including your own team).
 There are several different types of scouting




Pre-scouting – before the event.
Pit Scouting - static review at events.
Match Scouting – observation and data collection of matches
Analytical methods – Statistical analysis of match results / data
feeds.
Pre-Scouting
 Advance preparation before the event.
 This can often be accomplished by one person.
 Software/Macros will help tremendously, if you know
what you want, you can automate almost all of this task.
 Compile results for all teams:
 Match scores, Win/Loss, Ranking position, Draft position,
Eliminations results, Judged Awards.
 Despite the fact that we build new robots each year,
results from prior years can be amazingly accurate.
 There is very strong correlation between past
performance and future results for almost all teams.
Pit Scouting
 Also known as “Static Review”
 Tour the pits and review all the machines.
 Take notes on key attributes: Type of drivetrain,
features, wheel type, etc.
 Talk to the teams: Most teams love to tell you all about
their robots. Ask them if they have made recent
improvements.
 Always check the facts teams tell you against actual
performance on the field. Most teams have a tendency
to be optimistic about their own performance.
 Always involve your students in these efforts. They will
learn a lot about the different ways that teams solve
problems.
Match Scouting
 Proper match scouting takes considerable effort and
organization. You must watch EVERY match.
 The exact data you collect will change each year with the
game design.
 We typically use a group of 7 students to do our data
collection; one for each robot per match, plus a scouting
leader / coordinator.
 Goal is to collect objective and subjective data on each robot
in each match.
 Be sure to collect penalty data.
 A good data set allows you to evaluate teams for both
capability AND consistency.
 Some teams collaborate together to do data collection.
Match Scouting
 Many teams have trouble with scouting….students hate it
and thus do a poor job. They have to want to do it.
 Match scouting is a very tiring and sometimes boring task.
Despite this, NEVER treat scouting like a lower tier task, it is
one of the most important things your team will do at an
event.
 Recommend that students work in shifts.
 We give scouting a lot of esteem on our team. We praise
the scouts often for the quality of their data and pick list.
 Our student scouting leader is an earned position, and
students actually compete for this title.
 This greatly improves the willingness of our students to do a
good job.
Data Entry
 Methods vary from pencil and paper lists, to databases, to
iPad apps.
 As with robot design, choose your scouting method based on
your team’s ability and comfort level.
 There is no one right answer, you must do what will work
best for your team.
 Personally, I find computers to be somewhat of a distraction
in the match scouting process. Technology is great, but
often comes with additional failure modes and new avenues
for goofing off.
 The batteries never die on a paper list.
 We use a customized scouting sheet, redesigned each year.
The key to paper scouting is a compact, easy to read and sort
data set.
 We use computerized methods in both the pre-scouting and
analytical
Match Scouting
Prev. M Match #
Partner
Auto
Tele-op
Bonus
Partner
Auto
Tele-op
Bonus
Opponent
Auto
Tele-op
Bonus
Opponent
Auto
Tele-op
Bonus
Opponent
Auto
Tele-op
Bonus
Go through the schedule and highlight your matches. Then watch partners
and opponents to help formulate a match strategy.
Data Acquisition Sheets
Inspired by FRC 3929
Starting a Pick-List
Team List
1
17
33
2
18
34
3
19
35
4
20
36
5
21
37
6
22
38
7
23
39
8
24
40
9
25
41
10
26
42
11
27
43
12
28
44
13
29
45
14
30
46
15
31
47
16
32
48
• Yes: Definitely a team that
will make Elims.
• Maybe : A team you are
not 100% sure of early on.
• No: A team that will not
make elims.
• Yes+Maybe>24 (23 + your
team)
Always have a full team list handy when making a pick-list. Sort team by Yes, Maybe, or No.
These three levels make it pretty easy to sort. You need at least 24 Yes and Maybes
combined, or you have to move some over from the No pile…
Making a Pick-List
Always have a full team list handy when making a pick-list. It is important to have a full list of
24 teams (23 plus your team) as well as any possible additional teams (in case you and your
first pick have conflicts). We also use DNP as a section for Do Not Pick. This can be because
they are broken, get penalties, or just have a conflicting strategy.
Analytical Methods
 Numerical Analysis is a big part of most successful
teams’ scouting efforts.
 By using the published match results from past matches
and events, future match results can be predicted with
over 80% accuracy in most cases.
 For a team with limited resources, these methods can
go a long way to reduce the burden of manual data
collection.
 Several automated methods exist for obtaining this data
in real-time:




The Team 2834 Scouting Database
The FRC Tracker App
The FIRST OPR App.
FRC Twitter Feed
OPR and CCWM
 What is OPR? (Offensive Power Rating)
 OPR is a computed number for the offensive contribution of a
given team. The higher your OPR number, the more points
your robot is probably scoring.
 The math assumes that your robot’s potential is the same in
every match at an event.
 The OPR value range is game specific, depending on the points
available in the game.
 OPR will generally not indicate defensive specialists well.
 CCWM (Calculate Contribution to Winning Margin) does a
better job at highlighting good defense teams.
 The Twitter Feed now provides separate results per match for
Auton, Teleop and Climbing, allowing individualized OPRs for
each section of the game to be computed for each team.
 Never forget that these numbers are leading indicators, and
should never be trusted as being an absolute measure of team
performance.
“It’s not all about the Robots”
 Most scouting is focused on the capabilities of the
machines.
 Robots are designed by people, built by people,
maintained by people, and operated by people. The
human factor in all of this is huge.
 If you do not know the people on the other teams, your
ability to do comprehensive competitive assessment
will be limited.
 Get out and meet the other coaches and drivers. Every
team is unique. Once you understand how they
operate, you can better determine the best ways to play
with and against them.
Q&A
 We are happy to answer any and all questions on this
or other FRC topics.
Good Luck to all teams at the
2013 Michigan State Championship!
Thanks to teams 1114 and 2834 for some of
the material in this presentation .