Transcript NUMERICAL METHODS IN ENGINEERING ENGR 391

CURVE FITTING
Describes techniques to fit curves (curve fitting) to
discrete data to obtain intermediate estimates.
There are two general approaches for curve fitting:
• Regression:
Data exhibit a significant degree of scatter. The
strategy is to derive a single curve that
represents the general trend of the data.
• Interpolation:
Data is very precise. The strategy is to pass a
curve or a series of curves through each of the
points.
Introduction
In engineering, two types of applications are encountered:
– Trend analysis. Predicting values of dependent
variable, may include extrapolation beyond data points
or interpolation between data points.
– Hypothesis testing. Comparing existing mathematical
model with measured data.
Mathematical Background
• Arithmetic mean. The sum of the individual data
points (yi) divided by the number of points (n).
y

y

,i
1
,
,n
i
n
• Standard deviation. The most common measure
of a spread for a sample.
S
2
t
S

,
S

(
y

y
)
y
t 
i
n

1
Mathematical Background
(cont’d)
• Variance. Representation of spread by the square
of the standard deviation.
2
(
y

y
)
i
2 
S
or
y
n
1


y
y
/n

S
2
y
2
i
2
i
n

1
• Coefficient of variation. Has the utility to quantify
the spread of data.
Sy
c.v. 100
%
y
Least Squares Regression
Chapter 17
Linear Regression
Fitting a straight line to a set of paired
observations: (x1, y1), (x2, y2),…,(xn, yn).
y = a0+ a1 x + e
a1 - slope
a0 - intercept
e - error, or residual, between the model and
the observations
Linear Regression: Residual
Linear Regression: Question
How to find a0 and a1 so that the error would be
minimum?
Linear Regression: Criteria for a “Best”
Fit
n
n
i

1
i

1
min
e
(
y
a

a
x
)


i
i
0
1
i
e1
e2
e1= -e2
Linear Regression: Criteria for a “Best”
Fit
n
n
i

1
i

1
min
|e

|y
a

a
x


i|
i
0
1
i|
Linear Regression: Criteria for a “Best”
Fit
n
min
max
|e

|y
a

a
x
i|
i
0
1
i|
i

1
Linear Regression: Least Squares Fit
n n
2
r
i
i
i

1 i

1
n
2
2
i 0
1
i
i

1
S

e

(
y
,
measu

y
,
mod
)

(
y

a

a
x
)



i
n
n
2
2
min
S
e
(
y
a

a
x
)


r
i
i
0
1
i
i

1 i

1
Yields a unique line for a given set of data.
Linear Regression: Least Squares Fit
n
n
2
2
min
S
e
(
y
a

a
x
)


r
i
i
0
1
i
i

1 i

1
The coefficients a0 and a1 that minimize Sr must satisfy
the following conditions:
 S r
 a  0
 0

 S r  0
  a 1
Linear Regression:
Determination of ao and a1
Sr
2(yi ao a1xi ) 0
ao
Sr
2(yi ao a1xi )xi 0
a1
0yi a0 a1xi
0yi xi a0xi a1xi2
a na

na
xa
y
yx 
ax
ax

0
0
0
i
i i
1
0i
i
2
1i
2 equations with 2
unknowns, can be solved
simultaneously
Linear Regression:
Determination of ao and a1
n
x
x
y


iy
i
i
i
a

1
2
2
x
n
x

i
i
a0 ya1x
Error Quantification of Linear
Regression
• Total sum of the squares around the mean for the
dependent variable, y, is St
2
S

(
y

y
)
i
t
• Sum of the squares of residuals around the regression
line is Sr
n
n
i

1
i

1
2
2
S

e

(
y

a

a
x
)
r 
i 
i
o 1
i
Error Quantification of Linear
Regression
• St-Sr quantifies the improvement or error reduction due
to describing data in terms of a straight line rather than
as an average value.
St Sr
r 
St
2
r2: coefficient of determination
r : correlation coefficient
Error Quantification of Linear
Regression
For a perfect fit:
• Sr= 0 and r = r2 =1, signifying that the line explains 100
percent of the variability of the data.
• For r = r2 = 0, Sr = St, the fit represents no improvement.
Least Squares Fit of a Straight
Line: Example
Fit a straight line to the x and y values in
the following Table:
xi
yi
xiyi
xi2
1
0.5
0.5
1
2
2.5
5
4
3
2
6
9
4
4
16
16
5
3.5
17.5
25
6
6
36
36
7
5.5
38.5
49
28
24
119.5
140
 xi  28  yi  24.0
2
x
 i 140
.5
xi yi 119
Least Squares Fit of a Straight Line:
Example (cont’d)
n
xiyi 
xi yi
a
1
2
2
n
xi (
xi)
7119
.528
24

0.8392857
2
7140
28
a0 ya
1x
3.428571
0.8392857
40.07142857
Y = 0.07142857 + 0.8392857 x
Least Squares Fit of a Straight Line:
Example (Error Analysis)
xi
1
2
3
4
5
6
7
yi
0.5
2.5
2.0
4.0
3.5
6.0
5.5
28
24.0
(y
y
)
i
8.5765
0.8622
2.0408
0.3265
0.0051
6.6122
4.2908
22.7143
2
^
2

(
y
e
y
)
i
2


S

y

y

22
.
714
t 
i
0.1687
0.5625
2
S
e
2
.9911

r
i 
0.3473
0.3265
0.5896
S
2 S
t
r
r


0
.868
0.7972
S
t
0.1993
2.9911
2
2
i
r
r0
.
868

0
.
932
Least Squares Fit of a Straight Line:
Example (Error Analysis)
•The standard deviation (quantifies the spread around the mean):
S
22
.
7143
t
s



1
.
9457
y
n

1 7

1
•The standard error of estimate (quantifies the spread around the
regression line)
S
2
.
9911
r
s
 

0
.
7735
y
/
x
n

2 7

2
Because Sy / x  Sy , the linear regression model has good fitness
Algorithm for linear regression
Linearization of Nonlinear Relationships
•
•
The relationship between the dependent and independent
variables is linear.
However, a few types of nonlinear functions can be
transformed into linear regression problems.
 The exponential equation.
 The power equation.
 The saturation-growth-rate equation.
Linearization of Nonlinear Relationships
1. The exponential equation.
ya1eb1x 
ln
yln
a
b
1
1x
y* =
a o + a1 x
Linearization of Nonlinear Relationships
2. The power equation
ya2xb2 
log
y

log
a

b
log
x
2
2
y* =
ao + a1 x*
Linearization of Nonlinear Relationships
3. The saturation-growth-rate equation
x
ya

3
b
x
3
1 1 b

31
   
y a

3 a
3x
y* = 1/y
ao = 1/a3
a1 = b3/a3
x* = 1/x
Example
Fit the following Equation:
y  a2 x
b2
to the data in the following table:
xi
1
2
3
4
1
15
yi
0.5
1.7
3.4
5.7
8.4
19.7
X*=log xi Y*=logyi
0
-0.301
0.301
0.226
0.477
0.534
0.602
0.753
0.699
0.922
2.079
2.141
b
2
log
ylog(
a
x
)
2
log
y

log
a

b
log
x
2
2
*
*
let
Y

log
y,

log
X
x,
a

log
a
, 1
a
b
0
2
2
Y* a0a1X*
Example
Sum
Xi
Yi
X*i=Log(X)
Y*i=Log(Y)
X*Y*
X*^2
1
0.5
0.0000
-0.3010
0.0000
0.0000
2
1.7
0.3010
0.2304
0.0694
0.0906
3
3.4
0.4771
0.5315
0.2536
0.2276
4
5.7
0.6021
0.7559
0.4551
0.3625
5
15
8.4
19.700
0.6990
2.079
0.9243
2.141
0.6460
1.424
0.4886
1.169
n
x
y
xy

1
.
4
2
4

2
.
0
7
9

2
.
1
4
1



i
i
i
i 5
a



1
.
7
5

1
2
2
2
5

1
.
1
6
9

2
.
0
7
9
x
(
x
)
 n


i
i

a

y

a
x

0
.
4
2
8
2

1
.
7
5

0
.
4
1
5
8
4


0
.
3
3
4
0
1

Linearization of Nonlinear
Functions: Example
log y=-0.334+1.75log x
y 0.46x
1.75
Polynomial Regression
●
●
●
Some engineering data is poorly represented by a
straight line.
For these cases a curve is better suited to fit the data.
The least squares method can readily be extended to fit
the data to higher order polynomials.
Polynomial Regression (cont’d)
A parabola is preferable
Polynomial Regression (cont’d)
• A 2nd order polynomial (quadratic) is defined by:
y

a
a
x

a
x
e
o
1
2
2
●
The residuals between the model and the data:
e
y
a

a
x
a
x
i
i
o
1
i
2
2i
●
The sum of squares of the residual:


S

e

y

a

a
x

a
x


r
i
o
1
i
2
i
2
2
2
i
Polynomial Regression (cont’d)
Sr
2
(yi aoa1xi a2xi2)0
ao
Sr
2
(yi aoa1xi a2xi2)xi 0
a1
Sr
2 2
2
(yi aoa1xi a2xi )xi 0
a2
2
y

n

a

a
x

a
x



i
o
1
i
2
i
2
3
x
y

a
x

a
x

a
x




i i
o
i
1
i
2
i
2
2
3
4
x
y

a
x

a
x

a
x




i i
o
i
1
i
2
i
3 linear equations
with 3 unknowns
(ao,a1,a2), can be
solved
Polynomial Regression (cont’d)
●
A system of 3x3 equations needs to be solved to
determine the coefficients of the polynomial.
2
n 
a
x
x
y


i 
i 
0
i




2
3
x
x
x
a

x






i 
i
i
1
iy
i

2
3
4
2





x
x
x
a
x



i
i 
i
2
iy
i



●
The standard error & the coefficient of determination
Sr
sy/x 
n3
St Sr
r 
St
2
Polynomial Regression (cont’d)
General:
The mth-order polynomial:
2
m
y

a

a
x

a
x

.....

a
x
e
o 1
2
m
●
A system of (m+1)x(m+1) linear equations must be solved for
determining the coefficients of the mth-order polynomial.
●
The standard error:
sy/x
●
S
r
m

n


1
The coefficient of determination:
St Sr
r 
St
2
Polynomial Regression- Example
Fit a second order polynomial to data:
xi
yi
xi2
xi3
x i4
xiyi
xi2yi
0
2.1
0
0
0
0
0
1
7.7
1
1
1
7.7
7.7
2
13.6
4
8
16
27.2
54.4
3
27.2
9
27
81
81.6
244.8
4
40.9
16
64
256
163.6
654.4
5
61.1
25
125
625
305.5 1527.5
15
152.6
55
225
979
585.6
2489
x 15
i
.6
y 152
xi3 225
i
2
x
 i 55
4
x
 i 979
xy 585
.6

x y2488
.8

i i
2
i i
Polynomial Regression- Example
(cont’d)
●
The system of simultaneous linear equations:
6 1555
a
152
.6





0
 



15
55
225
a

585
.6




1







55
225
979
a
2488
.
8


2 

a

2
.
47857
,
a

2
.
35929
,
a

1
.
86071
0
1
2
2
y

2
.
47857

2
.
35929
x

1
.
86071
x

2
i
3
.74657


S
y
y
2513
.
39S
r e 

t
i
2
Polynomial Regression- Example
(cont’d)
xi
yi
ymodel
ei2
(yi-y`)2
0
2.1
2.4786
0.14332
544.42889
1
7.7
6.6986
1.00286
314.45929
2
13.6
14.64
1.08158
140.01989
3
27.2
26.303
0.80491
3.12229
4
40.9
41.687
0.61951
239.22809
5
61.1
60.793
0.09439
1272.13489
15
152.6
3.74657
2513.39333
•The standard error of estimate:
3
.74657
s

1
.12
y
/x
6

3
•The coefficient of determination:
2513
.
39

3
.
74657
2
2
r


0
.
99851
,
r

r

0
.
999
2513
.
39
Credits:
●
●
Chapra, Canale
The Islamic University of Gaza, Civil Engineering Department