Transcript Presentation PowerPoint

ASSET - Analysis of Spacecraft Qualification
Sequence and Environmental Testing
Authors:
P. Messidoro, M. Pasquinelli– TAS-I
Lorenzo P. - POLITO/TAS-I
F. Vergès – Astrium SAS
B. Laine, J. Candé - ESA
© TAS-I and Astrium SAS 2012 - all rights reserved.
20/JUN/2013
This document is the common property of Thales Alenia Space Italia and Astrium SAS.
It has been generated under the ESTEC contract N° 4000105946/12/NL/RA
and its use shall be governed by the terms of such contract.
83230350-DOC-TAS-EN-001
FADAT – June 20th, 2013, ESA ESTEC
Summary
2
Introduction
ASSET General Objectives
Study Logic
Objectives and Methodology
Overview of Selected Cases
Conclusions and Next Steps
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Introduction
The Analysis of Spacecraft Qualification Sequence and Environmental
Testing study (ASSET) is an ESA TRP Study in progress since September
2012
Thales Alenia Space (TAS) is prime contractor (Turin, Rome and Cannes
sites), and Astrium is subcontractor (Toulouse and Friedrichshafen sites)
TAS and Astrium keep a co-prime spirit for all the technical matters,
with a full share of contents and decisions
ASSET is aimed “at improving the effectiveness of the testing campaign to
detect anomalies before flight on the basis of lessons learned from the past
and in view of future project needs”
20/JUN/2013
Study boundaries
Environmental testing
Space Segment
Hardware anomalies
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
3
ASSET Study Principles
4
Objective: to improve the effectiveness of test program to detect
anomalies before flight on the basis of lessons learned from the past and in
view of future project needs, identifying areas of improvement in the test
program
Effectiveness of a test program = ability to minimize the number of
anomalies (causing mission failure or mission performance degradation)
discovered after launch, maximizing their discovery during ground testing
activities.
ASSET takes into account the effectiveness dependency on the class of
the program and acceptable risk.
The study uses past projects data to understand effects of choices on
the test programs in the basis of each program history (AIV plan,
discovered anomalies on ground, failures during flight)
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
ASSET Objectives
5
ASSET final objectives
(to answer to or to bring elements to answer to the following questions)
A
Which problems that occurred in previous programs could have been avoided
through a different test campaign (limited to environmental tests)?
B
Considering the lessons learned of previous programs in terms of test
effectiveness, are there any activities related to environmental testing
(including test result analysis) that in future programs could be deleted,
reduced, aggregated or optimized without impacting mission success?
C
What could be effective test sequences and model philosophies according to
mission type, product type and acceptable risk, to reduce hardware related
anomalies?
D
Which type of methodologies and database(s) should be developed/improved to
follow-up the test effectiveness topics?
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
ASSET Study Logic
6
KO
Task 1
Task 2
Data Requirements and Survey
Study
Management
T2R
Task 3
Assessment of
in-orbit anomalies
Task 4
Assessment of
on-ground anomalies
Task 5
Assessment of as-run
Test programs
T3-4R
Task 6
Synthesis and
Recommendations
FR
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
We are
here
Methodology
7
Objectives are detailed and translated into questions for the analysis
Analysis questions are then provided with:
Explanation of terms (to avoid ambiguity)
Analysis methods
Empirical: derived from experience of experts and with feedback related to a
specific problem and/or project
Systematic: transversal between projects, mission type, product types
and/or activities, in order to sustain specific analyses with the support of
trends, statistics and transversal categorization
Data requirements: data needed for the analysis
24 questions were defined and analyzed
19 questions related to general problems
5 questions coming from the project experience, aimed to specific
tests/activities
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Detailed Study Questions 1/6
8
completeness of the test program (are we covering enough / too much),
1
Is the test program
sufficient to detect all
the mission critical and
major anomalies?
2
What is the correlation between
completeness of the verification by test
approach and presence of flight
anomalies?
Feedback from Flight Anomalies to Test
Program (i.e. list of FA and associated AIV
activities)
Which test would have allowed to
detect this FLA? Are there FLA that
could have been discovered on
ground? How the related test plan
should have been performed?
List of FLA related to causes that may be
anticipated on ground including reason of
non-discovery before (incl. undertesting or
wrong test), according to previous question
results. List of test related to the previous
list, and link to specific AIV program (the test
was not sufficiently performed/not
performed)
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Detailed Study Questions 2/6
consistency (e.g. with lower level tests, with model philosophy or with activities objective)
Did we find at system level NCR that should
have been seen at equipment level? Did we
find FLA that should have been seen at
equipment level ?
List of NCR at system level and FLA related to causes
that may be anticipated on equipment testing
4
Is there any test duplication among
equipment testing and Spacecraft level
acceptance? What may go wrong if we skip
Spacecraft level testing on already tested
equipments/refurbished hardware?
Analysis of specific cases (if any) where Spacecraft
level testing discovered hardware problems on
already tested equipments with similar
activities/levels/conditions.
5
Are there any anomalies that occurred onground and again in flight and why?
List of FLA similar to NCR found on ground testing,
according to experts judgment. Identify not
adequate disposition or corrective action.
Are there anomalies that may have been
anticipated with different measurements or
exploitation of test data? i.e would better
instrumentation/monitoring allow to detect
anomalies?
Analysis of cases of FLA that may have occurred
during a performed ground testing activity, but
unsufficient exploitation of test data caused its non
discovery.
3
Are there missing tests at
equipment level?
Is the test outcome
exploited correctly?
6
What is the link between test effectiveness
and the development/verification philosophy
(i.e. model philosophy)?
7
What is the most effective
model philosophy?
8
20/JUN/2013
In a product line (satellite, equipment), on
which model can a test be removed? Analyse
decrease rate of a type of anomaly across
the product line.
Comparison of (at least) three classes of system
model philosophy: STM+EM+PFM, only PFM,
QM+FM. Experiences of major findings on the STM
leading to significant redesign.
List of AIV activities w.r.t.list of tested products (in
case of equipments) and flight model number (for
S/C), associated with a number/severity of NCR &
FLA (expected decreasing rate of occurrences).
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
9
Detailed Study Questions 3/6
correctness/relevance (are we doing the right things).
9
Are there FLA not seen on ground although
the test has been performed, because of the
low likelihood of occurrence (probabilistic
anomalies)?
List of FLA that would have been discovered by test
performed with different conditions (in terms of
durations, cycles, etc.). (e.g. probabilistic anomalies)
10
Would different test conditions allow to
detect issues? E.g. solar simulation, different
boundary conditions..
Identify FLA that could have been detected through
different test conditions
What is the relationship between the test
effectiveness and the severity (number of TV
cycles, levels…) of testing, the type of
anomaly, characteristics of related
equipments?
1) Cross-projects analysis of NCR and related AIV activities
related to specific category (e.g. specific test activity as "System
PFM TVTB", related to D49the Thermal Vacuum category) for
the identification of potential link between mission type and
test conditions (e.g. higher number of TV cycles for LEO
missions w.r.t. interplanetary missions)
2) Cross-projects analysis of NCR, related AIV activities related
to specific categories and FLA for the identification of:
- test conditions (e.g. number of cycles, margins, other "levels")
- equipment characteristics (e.g. propulsion type, solar panels
dimension and I/F type, thermal system type (active, passive) ,
secondary structure complexity)
able to represent the potential impact on the effectiveness (i.e.
that can be linked to the occurrence of specific type of
anomalies).
Are there FLA where a parameter is out of
spec, but disposed easily because the
allowable range for the parameter was too
limited (margins too high) and the design
works well outside of those boundaries?
Analysis of specific minor flight anomalies where the
question is applicable, i.e. where the disposition is
related to testing requirements too conservative
w.r.t. design margin.
11
12
Are different test
conditions required?
List of FLA/NCR associated to AIV activities
20/JUN/2013
13
Which type of risk does exist associated to
performed on Flight Model (e.g. fatigue stress
Can we detect dangerous
caused by random vibration test and subsequent
specific test, in terms of flight hardware
test to improve them
structural failure during launch).
degradation?
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
Is the test method the root cause of the NCR?
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
10
Detailed Study Questions 4/6
11
Test sequence (are we doing the things in the right order ?)
14
What is the most
effective test sequence
at Spacecraft (and
lower levels)?
Impact of having Mech/TV ot TV/Mech
on the number of anomalies detected
in orbit/on ground, to answer:
Between Mech/TV and TV/Mech, is
there a better sequence or is it
indifferent?
Comparison among AIV programs with the
two different solutions, identification of
similar classes of anomalies discovered in
both cases and singularities. If possible,
identification of defects raised during the
first test on the second.
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Detailed Study Questions 5/6
12
How to consolidate the standards
What is the relationship between the test
effectiveness and the ECSS test standard?
Deviances of test programs w.r.t test standard and
related effectiveness (divided by verification stages Qualification, Acceptance, pRotoqualification, Prelaunch)
What is the typical test program per mission
type, in comparison also with standards?
List of most common deviations from standards
according to mission type
17
How can deviations from standards be
justified, in different mission types and
products?
Assess the reasons of most common deviations from
standards and relate them to the mission type and to
the effectiveness (if it is kept high, the deviation may
be justified).
18
What is the added value (effectiveness) of
each test (in particular for environmental
tests) per type of mission/product?
Analyze effectiveness by type of mission (LEO, GEO,
etc.) and type of product (scientific, telecom,
manned, etc.)
How the test effectiveness varies with the
verification stage (i.e. acceptance,
qualification) for different types of products
types?
1) Comparison between the theory of verification
stages (e.g. qualification to find design problems and
acceptance to find workmanship/material defects)
and reality as derived from experience, NCR and
related FLA (e.g. statistical analysis coped with
analysis of specific anomalies, e.g. design problems
found in acceptance). If possible, analysis of
secondary objectives related to each stage (e.g.
comparison of flight model with qualification model
characteristics in acceptance) and of hybrid stages
(e.g. protoqualification) impact on effectiveness.
2) Evaluation of specificities related to type of
product (e.g. scientific, telecom, etc., one-of-a-kind,
series... ).
15
16
19
20/JUN/2013
How to consolidate
standards through
feedback/advices?
How to customise ECSS in
function of the product
(one-of S/C, product line,
constellation)?
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Detailed Study Questions 6/6
Specific questions (this list is related to potential final answers. It will be extended during analysis)
20
21
Can we cancel shock test?
Analysis of anomalies discovered during
shocks tests. Or anomalies induced by shocks
tests.
Can we cancel tests after
transportation?
Did we find NCR on launch site due to
transportation (if accelerometers detected
nothing)? Did we discover any NCR due to
transportation during test sequence?Identify
all NCR related to transport and analyse
those caused by transport. And analyze
NCR's related to the "pre-launch" (P) stage
and related cause, and, if not related to
transportation, in which test it should have
been discovered.
Can the duration of the Thermal
Vacuum (TVAC) be reduced?
Analysis of anomalies raised during TVAC.
Link with the number of cycles. On programs
where fewer cycles have been performed,
did the test effectiveness changed ?
Can we limit the number of Axis for
sine?
Analysis of anomalies raised during
Sinusoidal Vibration test. Link with the
number of axis. On programs where only one
axis was done, did the test effectiveness
changed ?
What risk mitigation does STM bring?
Analysis of anomalies found on STM. On
programs where no STM was used, did the
test effectiveness changed ?
What are the
questions derived from
lessons learned from
previous programs ?
22
23
20/JUN/2013
24
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
13
Selection of cases
14
Screening across all the available projects is performed, to select relevant
anomalies for the ASSET study
Screening is performed using selection criteria
Additional cases may be added during the analysis, or some case may be
excluded, according to the relevance w.r.t. study questions
Cases list is provided with essential fields (name, description, cause, severity,
type of test)
New fields will be added to trace the results of the analysis, especially
those required for systematic analysis
Only for ESA programs
For non-ESA programs currently only statistics are provided (e.g. number
of screened anomalies, per type of test). Anonymous cases will be added, with
the agreement of the related programs.
Current tasks are analyzing in detail each selected case to retrieve findings to
answer the study questions
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Selection Criteria
Criterion
ASSET-related
To be discarded*
Affected Item
(item failed during
test/operations)
Space segment system
Space segment element/module
Space segment subsystem
Space segment equipment
Part and Materials
Launch Segment
Ground segment
Software
GSE/TSE
Verification level
(at which the test activity is
performed)
Space segment system
Space segment element(/module)
Space System
Space segment Subsystem
Space segment Equipment
Severity
(potential impact on mission)
Critical (loss of function, loss of mission)
Major (switch to redundant, delay to operations)
Minor (no potential impact on mission)
Cause
(Process)
Design
Workmanship
Part and Material
Excessive Testing
Unknown
Not Reported
Operator Error
Failure to Follow Procedure
Defective GSE
Cause
(Category of anomaly, i.e. nature
of the phenomenon)
Electrical
Mechanical
Thermal
Others (e.g. Optical/Fluidic)
Software (if unrelated to hardware)
System (Space+Ground Segment)
Type of test that would have
discovered it (preliminary
screening)
See Table 3
Functional and Performance
Integration
Alignment
End-to-end communications
Leakage
Life
Mission simulation
Pressure
Spin
Verification Stage
-
Qualification
Protoqualification
Acceptance
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
15
Only NCR
Only FLA
Both
Considered Projects
16
A total of 48 models, of which 34 are flight/protoflight models,
are considered for 22 ESA programs
Science (e.g. Telescopes, Interplanetary), Telecom, Human
Rated, Earth Obs, Navigation missions
Also SM/STM’s and EM’s are considered
Equipment level data are retrieved where necessary to answer to
system-level questions
A total of 86 flight/protoflight models are considered for nonESA programs
Statistics and anonymous cases will help to answer to study
questions
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Overview of Cases – Current Status
17
Program
Name
Tot.
NCR
Selected
NCR
Tot
FLA
Selected
FLA
NOTES
ESA
Programs
10251
127
357
11
Collection still in
progress, see next
slides
Non-ESA
Programs
28783
293*
70
26
(*) of which 184 to be
further screened
Total
39034
FLA Anomalies to be
completed, related to
less than 50% of
programs
420
427
37
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Overview of Cases – Thales Alenia Space - Italy
18
Program Name
Tot.
Selected NCR
NCR
Tot
Selected FLA
NOTES
FLA
Integral
159
8
25
2
From MAT€D
Artemis
241
12
10
0
From MAT€D
Node 2
257
0
6
0
From MAT€D. No
Environmental tests
performed. Specific cases
will be collected for specific
questions.
ATV ICC
86
0
N/A
N/A
GOCE
70
3
66
2
From MAT€D and from
anomaly DB’s
Cupola
25
1
8
0
From anomaly DB’s
Node 3
1295
0
54
0
No Environmental tests
performed. Specific cases
will be collected for specific
questions.
Galileo IOV
791
6
(*)
(*)
FLA not available for the ICC
From anomaly DB’s.
FLA data to be retrieved
from CNES and with AST-D
involvement
ASI programs
1250
26
56
24
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
From MAT€D and from
anomaly DB’s
Overview of Cases – Thales Alenia Space - France
19
Program Name
Tot.
Selected NCR
NCR
Planck
Tot
Selected FLA
NOTES
FLA
1260
13*
TBC
TBC
From anomaly DB,
FLA data collection
to be concluded
192
9*
TBC
TBC
From MATED, FLA
data to be retrieved
with EUMETSAT
support
132
1*
TBC
TBC
From anomaly DB,
FLA data to be
retrieved from
CNES; +1 NCR not
in selection criteria
8650
184
14
2
(3 models)
MSG
(5 models)
SMOS
(1 model)
Commercial
TBC = Collection To be completed
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Overview of Cases – ASTRIUM-F
20
Program Name
Tot.
Selected NCR
NCR
MEX
310 (M)
Tot
Selected FLA
NOTES
14 (M)
1 (M)
44 (D)
1 (D)
Retrieved from
MAT€D (M) from
TAS-I and own
DB’s (D) from ASTF
FLA
4 (M)
VEX
152
5
36
0
ATV
N/A
N/A
N/A
N/A
Summary results
will be contributed
MetOP
1505
19
TBC
TBC
3 S/C.
28 NCR if not
applying severity
filter
Commercial
18883
83
TBC
TBC
TBC = Collection To be completed
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Overview of Cases – ASTRIUM-D
21
Program Name
Tot.
Selected NCR
NCR
Tot
Selected FLA
FLA
Cluster 1
55 (M)
10 (M)
0 (M)
-
Cluster 2
84 (M)
4 (M)
42 (M)
2 (M)
XMM
511 (M)
5 (M)
25 (M)
0 (M)
971 (N)
Herschel
654 (M)
(*)
TBC
TBC
13(M)
14 (M)
3 (M)
25
0
Rosetta
ERS2
116 (M)
tbc
16 (M)
1 (M)
CryoSAT2
413 (N)
14(N)
TBC
TBC
CryoSAT1
972 (D)
NOTES
Tbc (*) (D)
n/a
n/a
TBC = Collection To be completed
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
(*) Screening on
progress
(M) Preliminary
provided by TAS-I
from MAT€D
screening
(*) Screening on
progress on 226
NCR on
environmental tests
Cases statistics
NCR by nature of the root cause
18%
29%
Mechanical
NCR by test activity type
Electrical
Acoustic
Thermal
Others
17%
7%
18%
EMC conducted
5%
4%
2%
4%
EMC Radiated/autocompatibility/RF
36%
Vibration
FLA by nature of the root cause
Thermal ambient
20%
Thermal vacuum
60%
Electrical
Others
Thermal
55%
25%
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
Others
Conclusions and Next Steps
23
ASSET Detailed Objectives and Related methodology have been defined
between the Industrial Consortium and ESA
Past programs data have been retrieved to proceed with the study
The “complete picture” has been difficult to retrieve for all the programs:
there are few databases where the complete lifecycle (especially phases
C/D/E) and the different levels (System/Element/Equipment) are included.
complete access to data requires to interface with many companies and many
departments
Analysis of In-flight / On-ground anomalies and As-run test programs is in
progress, on specific programs and cases.
Any contribution is welcome: anonymous (e.g. without program name or with few
details) cases, statistics and lessons learned may help the team to provide sounder
recommendations:
→ please contact us if you are interested to contribute
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space
ASSET Study Team
24
ESA: B. Laine, J. Candé
with the involvement of O. Brunner and C. Lahorgue
TAS-I: P. Messidoro, M. Pasquinelli, P. Giordano, S. Voglino,
L. Pace (POLITO)
TAS-F: J. P. Chessel
Astrium-F: F. Vergès, P. Desiré, G. Caillard
Astrium-D: J. Hendrikse
… and all the program experts interviewed (thanks!)
20/JUN/2013
This document is not to be reproduced, modified, adapted, published, translated in any material form in whole or in part nor disclosed
to any third party without the prior written permission of Thales Alenia Space -  2012, Thales Alenia Space