Transcript Nonlinear Regression

Nonlinear Regression
Chemical Engineering Majors
Authors: Autar Kaw, Luke Snyder
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
7/7/2015
http://numericalmethods.eng.usf.edu
1
Nonlinear Regression
http://numericalmethods.eng.usf.edu
Nonlinear Regression
Some popular nonlinear regression models:
bx
1. Exponential model: ( y  ae )
( y  axb )
ax 

y

3. Saturation growth model: 

b

x


2. Power model:
4. Polynomial model:
( y  a0  a1x  ... amx m )
Nonlinear Regression
Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn) best fit y  f (x)
to the data, where f (x) is a nonlinear function of x .
( xn , yn )
( x2 , y2 )
( xi , yi )
( x1 , y1 )
y  f (x)
yi  f ( xi )
Figure. Nonlinear regression model for discrete y vs. x data
Regression
Exponential Model
Exponential Model
Given ( x1, y1), ( x 2, y 2), ... , ( xn, yn) best fit y  aebx to the data.
( xn , yn )
( x2 , y2 )
( xi , yi )
( x1 , y1 )
y  aebx
yi  f ( xi )
Figure. Exponential model of nonlinear regression for y vs. x data
Finding constants of Exponential Model
The sum of the square of the residuals is defined as
n

Sr   yi  ae
i 1

bx 2
i
Differentiate with respect to a and b
n
S r
bxi
bxi
  2 yi  ae  e  0
a
i 1





n
S r
  2 yi  aebxi  axi e bxi  0
b
i 1

Finding constants of Exponential Model
Rewriting the equations, we obtain
n
  yi e
bxi
i 1
n
 yi xi e
i 1
bxi
n
 ae
2bxi
i 1
n
 a  xi e
i 1
2bxi
0
0
Finding constants of Exponential Model
Solving the first equation for a yields
n
a
 yi e
i 1
n
e
bxi
2bxi
i 1
Substituting a back into the previous equation
n
n
 y i xi e
i 1
bxi

 yi e
i 1
n
e
bxi
n
 xi e
2bxi
2bxi i 1
0
i 1
Nonlinear equation in terms of b
The constant b can be found through numerical methods such
as the bisection method or secant method.
Example 1-Exponential Model
Many patients get concerned when a test involves injection of a
radioactive material. For example for scanning a gallbladder, a
few drops of Technetium-99m isotope is used. Half of the
techritium-99m would be gone in about 6 hours. It, however,
takes about 24 hours for the radiation levels to reach what we
are exposed to in day-to-day activities. Below is given the
relative intensity of radiation as a function of time.
Table. Relative intensity of radiation as a

function
of time.
t(hrs)
0
1
3
5
7
9
1.000
0.891
0.708
0.562
0.447
0.355
Example 1-Exponential Model cont.
The relative intensity is related to time by the equation
t
  Ae
Find:
a) The value of the regression constants A and 
b) The half-life of Technium-99m
c) Radiation intensity after 24 hours
Plot of data
Constants of the Model
  Aet
The value of λ is found by solving the nonlinear equation
n
n
f      i t i e
ti
i 1

ti

e
 i
i 1
n
2ti
e

i 1
n
A
t

e
 i
i 1
n
2 t i
e

i 1
i
n
2ti
t
e
0
 i
i 1
Setting up the Equation in MATLAB
n
n
f      i t i e
i 1
ti

ti

e
 i
i 1
n
e
n
2ti
t
e
0
 i
2ti i 1
i 1
t (hrs)
0
1
3
5
7
9
γ
1.000 0.891 0.708 0.562 0.447 0.355
Setting up the Equation in MATLAB
n
n
f      i t i e
i 1
ti

 ie
i 1
n
ti
2ti
e

n
 ti e
i 1
2ti
0
  0.1151
i 1
t=[0 1 3 5 7 9]
gamma=[1 0.891 0.708 0.562 0.447 0.355]
syms lamda
sum1=sum(gamma.*t.*exp(lamda*t));
sum2=sum(gamma.*exp(lamda*t));
sum3=sum(exp(2*lamda*t));
sum4=sum(t.*exp(2*lamda*t));
f=sum1-sum2/sum3*sum4;
Calculating the Other Constant
The value of A can now be calculated
6
A
 e
i 1
6
ti
i
2 ti
e

 0.9998
i 1
The exponential regression model then is
  0.9998e
0.1151t
Plot of data and regression curve
  0.9998e
0.1151t
Relative Intensity After 24 hrs
The relative intensity of radiation after 24 hours
  0.9998 e
0.115124 
 6.3160 102
6.316 102
 100  6.317%
This result implies that only
0.9998
radioactive intensity is left after 24 hours.
Homework
1.
2.
3.
What is the half-life of technetium
99m isotope?
Compare the constants of this
regression model with the one
where the data is transformed.
Write a program in the language
of your choice to find the
constants of the model.
THE END
http://numericalmethods.eng.usf.edu
Polynomial Model
y  a  a x  ... a x m
Given ( x1, y1), ( x 2, y 2), ... , ( xn, yn) best fit
0
1
to a given data set.
(m  n  2)
( xn , yn )
( x2 , y2 )
( xi , yi )
( x1 , y1 )
y  a  a x   a xm
0
1
m
yi  f ( xi )
Figure. Polynomial model for nonlinear regression of y vs. x data
m
Polynomial Model cont.
The residual at each data point is given by
Ei  yi  a0  a1 xi  . . .  am xim
The sum of the square of the residuals then is
n
S r   Ei2
i 1
n

  y i  a0  a1 xi  . . .  a m xim
i 1

2
Polynomial Model cont.
To find the constants of the polynomial model, we set the derivatives
with respect to a i where i  1, m, equal to zero.
n
S r
  2. yi  a0  a1 xi  . . .  am xim (1)  0
a0 i 1


n
S r
  2. yi  a0  a1 xi  . . .  am xim ( xi )  0
a1 i 1






n
S r
  2. yi  a0  a1 xi  . . .  am xim ( xim )  0
am i 1


Polynomial Model cont.
These equations in matrix form are given by


 n

 n
   xi 
  i 1 
. . .

 n m 
  xi 
 i 1 
 n

  xi 
 i 1 
 n 2
  xi 
 i 1 
. . .
. .
. .
. .
 n m1 
  xi  . .
 i 1



 n m 
.  xi  a
 i 1    0

 n m1  a1
.  xi 
 i 1
 . .
. . .  a
 m
n


.  xi2 m  
 i 1


 n

   yi
  ni 1

xi y i


.  i 1
 . . .
 
n
 xim yi
 i 1











The above equations are then solved for a0 , a1 ,, am
Example 2-Polynomial Model
Below is given the FT-IR (Fourier Transform Infra Red) data of a 1:1 (by weight)
mixture of ethylene carbonate (EC) and dimethyl carbonate (DMC). Absorbance
P is given as a function of wavenumber m.
Absorbance P, as a function of Wavenumber, m
Wavenumber, m
( cm 1 )
Absorbance, P
(arbitrary unit)
804.184
0.1591
827.326
0.0439
846.611
0.0050
869.753
0.0073
889.038
0.0448
892.895
0.0649
900.609
0.1204
Absorbance, P (arbitrary unit)
Table. Absorbance vs Wavenumber data
0.18
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
780
800
820
840
860
880
900
920
Wavenumber, m (cm -1)
Figure. Absorbance vs. Wavenumber data
http://numericalmethods.eng.usf.edu
25
Example 2-Polynomial Model cont.
Regress the data to a second order polynomial
2
where P  a0  a1m  a2 m
and find the absorbance at m  1000cm1
The coefficients a0 ,a1,a2 are found as follows

 n


 n
m

  i

 i n1
 m 2 
i
 

 i 1
 n

  mi 
 i 1

 n 2
  mi 
 i 1

n


  mi3 
 i 1

 n 2 
 n
  mi  
 Pi
 i 1
  a   i 1
0
 n 3     n
  mi   a1    mi Pi
 i 1
     i 1
a
n
 n 4   2   m2 P
  mi  

i i

i

1

 i 1









http://numericalmethods.eng.usf.edu
26
Example 2-Polynomial Model cont.
The necessary summations are as follows
Table. Necessary summations for calculation of polynomial model constants
Absorbance, P
(arbitrary unit)
m2
m3
1
Wavenumber, m
( cm 1 )
804.18
0.1591
6.4671×105
5.2008×108
2
827.33
0.0439
6.8447×105
5.6628×108
3
846.61
0.0050
7.1675×105
6.0681×108
4
869.75
0.0073
7.5647×105
6.5794×108
5
889.04
0.0448
7.9039×105
7.0269×108
6
892.90
0.0649
7.9726×105
7.1187×108
7
900.61
0.1204
8.1110×105
7.3048×108
6030.4
0.4454
5.2031×106
4.4961×109
i
7

i 1
http://numericalmethods.eng.usf.edu
27
Example 2-Polynomial Model cont.
Necessary summations continued:
Table. Necessary summations for calculation of
polynomial model constants.
i
m4
m×P
m2 × P
1
4.1824×1011
127.95
1.0289×105
2
4.6849×1011
36.319
3.0048×104
3
5.1373×1011
4.233
3.583×103
4
5.7225×1011
6.349
5.522×103
5
6.2471×1011
39.828
3.5409×104
6
6.3563×1011
57.948
5.1742×104
7
6.5787×1011
108.43
9.7655×104
3.8909×1012
381.06
3.2685×105
7

i 1
http://numericalmethods.eng.usf.edu
28
Example 2-Polynomial Model cont.
Using these summations we have
 7.0000
6.0304103 5.2031106  a0   0.4454 

   381.06 
3
6
9 
6
.
0304

10
5
.
2031

10
4
.
4961

10
a

 1  

6
9
12
5.203110 4.496110 3.890910  a2  3.2685105 


Solving this system of equations we find
a0    8.6623 
 a    2.079810 2 
 1 

5
a2   1.236510 
The regression model is then
P  a0  a1m  a2 m2
 8.6623 2.0798102 m  1.2365105 m2
http://numericalmethods.eng.usf.edu
29
Example 2-Polynomial Model cont.
With the model is given by
P  8.6623 2.0798102 m  1.2365105 m2
Figure. Polynomial model of Absorbance vs. Wavenumber.
http://numericalmethods.eng.usf.edu
30
Example 2-Polynomial Model cont.
1
To find P where m  1000cm we have
P  a0  a1m  a2 m 2
 8.6623 2.079810 2 m  1.2365105 m 2
 8.6623 2.079810 2 1000  1.2365 105 1000
2
 0.22970
http://numericalmethods.eng.usf.edu
31
Linearization of Data
To find the constants of many nonlinear models, it results in solving
simultaneous nonlinear equations. For mathematical convenience,
some of the data for such models can be linearized. For example, the
data for an exponential model can be linearized.
As shown in the previous example, many chemical and physical processes
are governed by the equation,
y  aebx
Taking the natural log of both sides yields,
ln y  ln a  bx
Let z  ln y and a0  ln a
We now have a linear regression model where z  a0  a1 x
(implying)
a  eao with a1  b
Linearization of data cont.
Using linear model regression methods,
a1 
n
n
n
i 1
i 1
i 1
n xi z i   xi  z i


n xi2    xi 
i 1
 i 1 
n
_
n
2
_
a 0  z  a1 x
Once ao , a1 are found, the original constants of the model are found as
b  a1
a  e a0
Example 3-Linearization of data
Many patients get concerned when a test involves injection of a radioactive
material. For example for scanning a gallbladder, a few drops of Technetium99m isotope is used. Half of the technetium-99m would be gone in about 6
hours. It, however, takes about 24 hours for the radiation levels to reach what
we are exposed to in day-to-day activities. Below is given the relative intensity
of radiation as a function of time.
t(hrs)

0
1
3
5
7
9
1.000
0.891
0.708
0.562
0.447
0.355
1
Relative intensity of radiation, γ
Table. Relative intensity of radiation as a function
of time
0.5
0
0
5
Time t, (hours)
10
Figure. Data points of relative radiation intensity
vs. time
Example 3-Linearization of data cont.
Find:
a) The value of the regression constants A and 
b) The half-life of Technium-99m
c) Radiation intensity after 24 hours
The relative intensity is related to time by the equation
  Aet
Example 3-Linearization of data cont.
Exponential model given as,
  Aet
ln   ln A  t
Assuming z  ln  , ao  ln A and a1  
we obtain
z  a0  a1t
This is a linear relationship between z and t
Example 3-Linearization of data cont.
Using this linear relationship, we can calculate a0 , a1 where
n
a1 
n
n t z   t
i 1
i i
i 1
n
i
z
i 1


n t12    ti 
i 1
 i 1 
n
and
a0  z  a1t
 a
1
a0
Ae
n
2
i
Example 3-Linearization of Data cont.
Summations for data linearization are as follows
Table. Summation data for linearization of data model
ti
i
1
2
3
4
5
6
0
1
3
5
7
9

25.000
i
zi  ln  i
1
0.891
0.708
0.562
0.447
0.355
0.00000
−0.11541
−0.34531
−0.57625
−0.80520
−1.0356
0.0000
−0.11541
−1.0359
−2.8813
−5.6364
−9.3207
0.0000
1.0000
9.0000
25.000
49.000
81.000
−2.8778
−18.990
165.00
ti zi
t
2
i
With n  6
6
t
i 1
 25.000
i
6
z
i 1
 2.8778
i
6
t z
i 1
i i
6
t
i 1
2
i
 18.990
 165.00
Example 3-Linearization of Data cont.
Calculating a0 , a1
a1 
6 18.990  25 2.8778
2
6165.00  25
 0.11505
a0 
 2.8778
25
  0.11505 
6
6
 2.6150104
Since
a0  ln A
A  e a0
e
2.6150104
 0.99974
also
  a1  0.11505
Example 3-Linearization of Data cont.
0.11505t
Resulting model is   0.99974 e
1
  0.99974 e 0.11505t
Relative
Intensity
0.5
of
Radiation,
0
0
5
10
Time, t (hrs)
Figure. Relative intensity of radiation as a function of
temperature using linearization of data model.
Example 3-Linearization of Data cont.
The regression formula is then
  0.99974 e 0.11505t
1



b) Half life of Technetium 99 is when
2
1
0.99974 e 0 .11505 t  0.99974e 0 .11505 0 
2
e 0 .11508 t  0.5
 0.11505t  ln0.5
t  6.0248 hours
t 0
Example 3-Linearization of Data cont.
c) The relative intensity of radiation after 24 hours is then
  0.99974e 0.1150524
 0.063200
6.3200102
100  6.3216%
This implies that only
0.99983
material is left after 24 hours.
of the radioactive
Comparison
Comparison of exponential model with and without data linearization:
Table. Comparison for exponential model with and without data
linearization.
With data linearization
(Example 3)
Without data linearization
(Example 1)
A
0.99974
0.99983
λ
−0.11505
−0.11508
Half-Life (hrs)
6.0248
6.0232
Relative intensity
after 24 hrs.
6.3200×10−2
6.3160×10−2
The values are very similar so data linearization was suitable to
find the constants of the nonlinear exponential model in this
case.
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/nonlinear_r
egression.html
THE END
http://numericalmethods.eng.usf.edu