Engineering Computation Part 2 E. T. S. I. Caminos, Canales y Puertos Roots of Equations Objective: Solve for x, given that f(x) = 0 -orEquivalently, solve.
Download ReportTranscript Engineering Computation Part 2 E. T. S. I. Caminos, Canales y Puertos Roots of Equations Objective: Solve for x, given that f(x) = 0 -orEquivalently, solve.
Engineering Computation Part 2 E. T. S. I. Caminos, Canales y Puertos 1 Roots of Equations Objective: Solve for x, given that f(x) = 0 -orEquivalently, solve for x such that g(x) = h(x) ==> f(x) = g(x) – h(x) = 0 E. T. S. I. Caminos, Canales y Puertos 2 Roots of Equations Chemical Engineering (C&C 8.1, p. 187): van der Waals equation; v = V/n (= volume/# moles) Find the molal volume v such that f(v) = ( p + a v 2 )(v - b) - RT = 0 p = pressure, T = temperature, R = universal gas constant, a & b = empirical constants E. T. S. I. Caminos, Canales y Puertos 3 Roots of Equations Civil Engineering (C&C Prob. 8.17, p. 205): Find horizontal component of tension, H, in a cable that passes through (0,y0) and (x,y) H wx f(H) = cosh -1 + y 0 - y = 0 w H w = weight per unit length of cable E. T. S. I. Caminos, Canales y Puertos 4 Roots of Equations Electrical Engineering (C&C 8.3, p. 194): Find the resistance, R, of a circuit such that the charge reaches q at specified time t f(R) = e-Rt 2 1 R 2L cos - LC 2L q t =0 q0 L = inductance, C = capacitance, q0 = initial charge E. T. S. I. Caminos, Canales y Puertos 5 Roots of Equations Mechanical Engineering (C&C 8.4, p. 196): Find the value of stiffness k of a vibrating mechanical system such that the displacement x(t) becomes zero at t= 0.5sec. The initial displacement is x0 and the initial velocity is zero. The mass m and damping c are known, and λ = c/(2m). x(t) = x0e-t cos t + sin t = 0 in which k c2 μ= - 2 m 4m E. T. S. I. Caminos, Canales y Puertos 6 Roots of Equations Determine real roots of : • Algebraic equations (including polynomials) • Transcendental equations such as f(x) = sin(x) + e-x • Combinations thereof E. T. S. I. Caminos, Canales y Puertos 7 Roots of Equations Engineering Economics Example: A municipal bond has an annual payout of $1,000 for 20 years. It costs $7,500 to purchase now. What is the implicit interest rate, i ? 1 1 i n Solution: Present-value, PV, is: PV A i in which: PV = present value or purchase price = $7,500 A = annual payment = $1,000/yr n = number of years = 20 yrs i = interest rate = ? (as a fraction, e.g., 0.05 = 5%) E. T. S. I. Caminos, Canales y Puertos 8 Roots of Equations Engineering Economics Example (cont.): We need to solve the equation for i: 1 1 i 20 7,500 1, 000 i Equivalently, find the root of: 1 1 i 20 0 f (i) 7,500 1,000 i E. T. S. I. Caminos, Canales y Puertos 9 Roots of Equations Engineering Economics Example $10,000 f(i) = PV - g(i) $5,000 $0 0% 20% 40% 60% 80% 100% -$5,000 -$10,000 Excel -$15,000 Interest Rate, i E. T. S. I. Caminos, Canales y Puertos 10 Roots of Equations • Graphical methods: – Determine the friction coefficient c necessary for a parachutist of mass 68.1 kg to have a speed of 40 m/seg at 10 seconds. – Reorganizing. E. T. S. I. Caminos, Canales y Puertos 11 Roots of Equations Two Fundamental Approaches 1. Bracketing or Closed Methods - Bisection Method - False-position Method 2. Open Methods - One Point Iteration - Newton-Raphson Iteration - Secant Method E. T. S. I. Caminos, Canales y Puertos 12 Bracketing Methods f(x) x a) f(x) x b) f(x) x c) f(x) xl d) xu x In Figure a) we have the case of f(xl) and f(xu) with the same sign, and there is no root in the interval (xl,xu). In Figure b) we have the case of f(xl) and f(xu) With different sign, and there is a root in the interval (xl,xu). In Figure c) we have the case of f(xl) and f(xu) with the same sign, and there are two roots. In Figure d) we have the case of f(xl) and f(xu) with different sign, and there is an odd number of roots. E. T. S. I. Caminos, Canales y Puertos 13 Bracketing Methods • Though the cases above are generally valid, there are cases in which they do not hold. In Figure a) we have the case of f(xl) and f(xu) with different sign, but there is a double root. f(x) a) x b) x f(x) f(x) xl c) In Figure b) We have the case of f(xl) and f(xu) With different sign, but there are two discontinuities. In Figure c) we have the case of f(xl) and f(xu) with the same sign, but there is a multiple root. xu x E. T. S. I. Caminos, Canales y Puertos 14 Bracketing Methods (Bisection method) Bisection Method f(x) xl xu xr 2 f(x) f(xu) f(xu) f(x1) f(xr) > 0 (x1) f(xr) (xu) x xr => x1 f(xr) (x1) (xr)(xu) x f(x1) f(x1) E. T. S. I. Caminos, Canales y Puertos 15 Bracketing Methods (Bisection method) Bisection Method: Given lower and upper bounds, xl and xu, which bracket the root: f(xl) f(xu) < 0 x1 x u 1) Estimate the Root by midpoint: x r 2 2) Revise the bracket: f(xl) f(xr) < 0, xr –> xu, f(xl) f(xr) > 0, xr –> xl 3) Repeat steps 1-2 until: old x new x a) |f(xr)| < k, b) a < s , with a = r new r 100% xr c) | xl – xu | < E. T. S. I. Caminos, Canales y Puertos d) maximum # of iterations is reached. (Always do this in iteration algorithms.) 16 Bracketing Methods (Bisection method) Engineering Economics Example: We need to solve the equation for i: 1 1 i 20 7,500 1, 000 i Equivalently, find the root of: 1 1 i 20 0 f (i) 7,500 1,000 i Make conservative guesses at the upper and lower bounds: 100% interest rate, f(1.0) = 6,500 0% interest rate, f(0.0) = -12,500 E. T. S. I. Caminos, Canales y Puertos 17 Bracketing Methods (Bisection method) Engineering Economics Example $10,000 f(i) = PV - g(i) $5,000 $0 0% 20% 40% 60% 80% 100% -$5,000 -$10,000 Excel -$15,000 Interest Rate, i E. T. S. I. Caminos, Canales y Puertos 18 Bracketing Methods (Bisection method) Engineering Economics Example: Score Sheet for Rootfinding Example: Method Bisection Initial Est(s). s = 2 E-2 (0.00, 1.00) (0.05, 0.15) Convergence guaranteed: E. T. S. I. Caminos, Canales y Puertos 9 6 s = 2 E-7 26 22 1 Eimax 0.5 Eimax 19 Bracketing Methods (Bisection method) One important advantage of this method is that one can calculate the number of required iterations for a given error. E. T. S. I. Caminos, Canales y Puertos 20 Bracketing Methods (Bisection method) Parachutist Example: E. T. S. I. Caminos, Canales y Puertos 21 Bracketing Methods (Bisection method) Parachutist Example: E. T. S. I. Caminos, Canales y Puertos 22 Bracketing Methods (Bisection method) Bisection Method Advantages: 1. Simple 2. Good estimate of maximum error: E max 3. Convergence guaranteed xl xu 2 1 Eimax 0.5 Eimax Disadvantages: 1. Slow 2. Requires two good initial estimates which define an interval around root: use graph of function, incremental search, or trial & error E. T. S. I. Caminos, Canales y Puertos 23 Bracketing Methods (False-position Method) False-position Method f(x) f(x) xr xu f (x u )(x1 x u ) f (x1 ) f (x u ) (xr) f(xu) f(x1) f(xr) > 0 (x1) (xu) x f(xr) f(x1) E. T. S. I. Caminos, Canales y Puertos x1 = x r (x1) f(xu) (xu) f(x1) f(xr) 24 Bracketing Methods (False-position Method) Similar to bisection. Uses linear interpolation to approximate the root xr: f (x u )(x1 x u ) 1) x r x u f (x1 ) f (x u ) 2) Revise the bracket: f(x1) f(xr) < 0, xr –> xu, f(x1) f(xr) > 0, xr –> x1 3) Repeat steps 1-2 until: a) |f(xr)| < k, b) a < s , with a = new x r x old r 100% c) |xu – x1 | = d new xr d) maximum # of iterations is reached. (Always do this in iteration algorithms.) E. T. S. I. Caminos, Canales y Puertos 25 Bracketing Methods (False-position Method) Engineering Economics Example $10,000 f(i) = PV - g(i) $5,000 $0 0% 20% 40% 60% 80% 100% -$5,000 -$10,000 Excel -$15,000 Interest Rate, i E. T. S. I. Caminos, Canales y Puertos 26 Bracketing Methods (False-position Method) Engineering Economics Example $0 7% 9% 11% 13% 15% f(i) = PV - g(i) 5% -$5,000 Interest Rate, i E. T. S. I. Caminos, Canales y Puertos 27 Bracketing Methods (False-position Method) Score Sheet for False-Position Example: Method Initial Est(s). s = 2 E-2 s = 2 E-7 Bisection (0.00, 1.00) (0.05, 0.15) 9 6 26 22 False-pos. (0.00, 1.00) (0.05, 0.15) 11 3 28 14 E. T. S. I. Caminos, Canales y Puertos 28 Bracketing Methods (False-position Method) Parachutist Example: E. T. S. I. Caminos, Canales y Puertos 29 Bracketing Methods (False-position Method) Parachutist Example: E. T. S. I. Caminos, Canales y Puertos 30 Bracketing Methods (False-position Method) There are some cases in which the false position method is very slow, and the bisection method gives a faster solution. E. T. S. I. Caminos, Canales y Puertos 31 Bracketing Methods (False-position Method) Summary of False-Position Method: Advantages: 1. Simple 2. Brackets the Root Disadvantages: 1. Can be VERY slow 2. Like Bisection, need an initial interval around the root. E. T. S. I. Caminos, Canales y Puertos 32 Open Methods Roots of Equations - Open Methods Characteristics: 1. Initial estimates need not bracket the root 2. Generally converge faster 3. NOT guaranteed to converge Open Methods Considered: - One Point Iteration - Newton-Raphson Iteration - Secant Method E. T. S. I. Caminos, Canales y Puertos 33 Open Methods • An alternative method consists of separating the function into two parts. E. T. S. I. Caminos, Canales y Puertos 34 Open Methods (Fixed point method) Fixed point Method predict a value of xi+1 as a function of xi. Convert f(x) = 0 iteration steps: to x = g(x) xi+1 = g(xi ) x(new) = g(x(old) ) E. T. S. I. Caminos, Canales y Puertos 35 Open Methods (Fixed point method) Example I: 1 1 i 20 7,500 1, 000 i 1 1 i 20 ii1 1.0 7.5 Example II: sin(x) f (x) 1.0 0.0 x x = sin(x) –> xi+1 = sin(xi) OR x = arcsin(x) –> xi+1 = arcsin(xi) E. T. S. I. Caminos, Canales y Puertos 36 Open Methods (Fixed point method) Convergence: Does x move closer to real root (?) Depends on: 1. nature of the function 2. accuracy of the initial estimate Interested in: 1. Will it converge or will it diverge? 2. How fast will it converge ? (rate of convergence) E. T. S. I. Caminos, Canales y Puertos 37 Open Methods (Fixed point method) Convergence of the Fixed point Method: Root satisfies: xr = g(xr) The Taylor series for function g is: xi+1 = g(xr) + g'(x)(xi - xr) xr < x < xi Subtracting the second equation from the first yields (xr – xi+1) = g'(x) (xr – xi) or E i 1 g' ( ) Ei 1. True error for next iteration is smaller than the true error in the previous iteration if |g'(x)| < 1.0 (it will converge). 2. Because g'(x) is almost constant, the new error is directly proportional to the old error (linear rate of convergence). E. T. S. I. Caminos, Canales y Puertos 38 Open Methods (Fixed point method) Further Considerations: Convergence depends on how f(x) = 0 is converted into x = g(x) So . . . Convergence may be improved by recasting the problem. E. T. S. I. Caminos, Canales y Puertos 39 Open Methods (Fixed point method) Convergence Problem: For slowly converging functions x new x old a x new x 100% can be small, even though xnew is not close to root. Remedy: Do not completely rely on a to ensure that the problem is solved. Check to make sure |f(xnew) | < k . E. T. S. I. Caminos, Canales y Puertos 40 Open Methods (Fixed point method) E. T. S. I. Caminos, Canales y Puertos 41 Open Methods E. T. S. I. Caminos, Canales y Puertos 42 Roots of Equations Two Fundamental Approaches 1. Bracketing or Closed Methods - Bisection Method - False-position Method 2. Open Methods - One Point Iteration - Newton-Raphson Iteration - Secant Method E. T. S. I. Caminos, Canales y Puertos 43 Open Methods (Newton-Raphson Method) Newton-Raphson Method: Geometrical Derivation: Slope of tangent at xi is f (x i ) 0 f '(x i ) xi xi1 Solve for xi+1: f (xi ) xi1 xi f '(xi ) [Note that this is the same form as the generalized onepoint iteration, xi+1 = g(xi)] E. T. S. I. Caminos, Canales y Puertos 44 Open Methods (Newton-Raphson Method) Newton-Raphson Method f(x) Tangent w/slope=f '(xi ) f(x) f '(x i ) f (x i ) 0 f '(x i ) x i x i1 f(xi) f (x i ) 0 x i x i1 f(xi) f(xi+1) f(xi+1) xi+ xi x (xi) x 1 xi1 xi f (x i ) f '(x i ) xi+ 1 xi = xi+1 E. T. S. I. Caminos, Canales y Puertos 45 Open Methods (Newton-Raphson Method) First order Taylor Series Derivation: 0 = f(xr) f(xi) + f '(xi) (xr – xi) solve for xr to yield next guess xi+1: f (x i ) x r xi1 xi f '(x i ) This has the form xi+1 = g(xi) with: g '(x r ) 1 E. T. S. I. Caminos, Canales y Puertos (f 'f ' f f ") (f ') 2 46 Open Methods (Newton-Raphson Method) Newton-Raphson iteration: f (xi ) xi1 xi f '(x i ) This iteration process is repeated until: 1. f(xi+1) 0, i.e., | f(xi+1) | < k, with k = small number 2. x i1 x i a 100% s x i1 3. Maximum number of iterations is reached. E. T. S. I. Caminos, Canales y Puertos 47 Open Methods a) Inflection point in the neighboor of a root. b) Oscilation in the neighboor of a maximum or minimum. c) Jumps in functions with several roots. d) Existence of a null derivative. E. T. S. I. Caminos, Canales y Puertos 48 Open Methods (Newton-Raphson Method) Bond Example: To apply Newton-Raphson method to: 1 1 i 20 0 f (i) 7,500 1, 000 i We need the derivative of the function: 20 1, 000 1 1 i 21 20 1 i f '(i) i i E. T. S. I. Caminos, Canales y Puertos 49 Open Methods (Newton-Raphson Method) Score Sheet for Newton-Raphson Example: Method Initial Est(s). s = 2 E-2 s = 2 E-7 Bisection (0.00, 1.00) (0.05, 0.15) 9 6 26 22 False-pos. (0.00, 1.00) (0.05, 0.15) 11 3 28 14 N-R 1.0 0. 5 0.25 0.15 0.05 EXCEL E. T. S. I. Caminos, Canales y Puertos diverges diverges 2, but wrong 48 5 7 3 5 4 5 50 Open Methods (Newton-Raphson Method) Error Analysis for N-R : Recall that xi1 xi f (xi ) f '(xi ) Taylor Series gives: f (x r ) f (x i ) f '(x i )(x r x i ) where f "() (x r x i ) 2 2! xr x xi and f(xr) = 0 E. T. S. I. Caminos, Canales y Puertos 51 Open Methods (Newton-Raphson Method) Dividing through by f '(xi) yields f (x i ) f "() 0 (x r x i ) (x r x i ) 2 f '(x i ) 2f '(x i ) f ''() (x i1 x i ) (x r x i ) (x r x i ) 2 2f '(x i ) OR f ''() (x r x i ) 2 2f '(x i ) f "() Ei 2 2!f '(x i ) (x r x i1 ) Ei1 Ei+1 is proportional to Ei2 ==> quadratic rate of convergence. E. T. S. I. Caminos, Canales y Puertos 52 Open Methods (Newton-Raphson Method) Summary of Newton-Raphson Method: Advantages: 1. Can be fast Disadvantages: 1. May not converge 2. Requires a derivative E. T. S. I. Caminos, Canales y Puertos 53 Open Methods (Secant Method) Secant Method f (x i1 ) f (x i ) f '(x) x i1 x i Approx. f '(x) with backward FDD: Substitute this into the N-R equation: to obtain the iterative expression: E. T. S. I. Caminos, Canales y Puertos xi1 xi f (x i ) f '(x i ) f (x i )(x i1 x i ) x i1 xi f (x i1 ) f (x i ) 54 Open Methods (Secant Method) Secant Method f(x) f(x) f '(x i ) f(xi-1 ) - f(xi ) xi-1 - xi f '(x i ) f(xi-1 ) - f(xi ) xi-1 - xi f(xi-1) f(xi) f(xi) xi+ x xi xi-1 1 x i1 xi f (x i )(x i1 x i ) f (x i1 ) f (x i ) xi+ xi f(xi-1) x xi-1 1 xi = xi+1 E. T. S. I. Caminos, Canales y Puertos 55 Open Methods (Secant Method) 1) Requires two initial estimates: xi-1 and xi These do NOT have to bracket root ! 2) Maintains a strict sequence: x i1 xi f (x i )(x i1 x i ) f (x i1 ) f (x i ) Repeated until: a. | f(xi+1) | < k with k = small number b. x i1 x i a 100% s x i1 c. Max. number of iterations is reached. 3. If xi and xi+1 were to bracket the root, this would be the same as the False-Position Method. BUT WE DON'T! E. T. S. I. Caminos, Canales y Puertos 56 Open Methods (Secant Method) – In the secant method, the values are replaced in a strict sequence, xi+1 to xi, and this to xi-1. Thus, the new values can be on the de same sode of the root, and sometimes diverge. E. T. S. I. Caminos, Canales y Puertos 57 Open Methods (Secant Method) Score Sheet for Secant Example: Method Initial Est(s). s = 2 E-2 s = 2 E-7 Bisection (0.00, 1.00) (0.05, 0.15) 9 6 26 22 False-pos. (0.00, 1.00) (0.05, 0.15) 11 3 28 14 N-R 1.0 0.5 0.25 0.15 0.05 diverges 2, but wrong 5 3 4 diverges 48 7 5 5 Secant (0, 1) diverges diverges (0.00, 0.50) 4, but wrong (chaotic) 27 (0.05, 0.15) 3 6 E. T. S. I. Caminos, Canales y Puertos 58 Open Methods Why do open methods fail? Function may not look linear. Remedy: recast into a linear form. For example, 1 (1 i) 20 f (i) 7500 0 i Is a poorly constrained problem in that there is a large, nearly flat zone for which the derivative is near zero. Recast as: i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ] E. T. S. I. Caminos, Canales y Puertos 59 Open Methods 1 (1 i) 20 f (i) 7500 0 i Recast as: i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ] – The recast function, "i f(i) will have the same roots as f(i) plus an additional root at i = 0. – It will not have a large, flat zone, thus: h(i) = i f(i) = 7,500 i – 1000 [ 1 – (1+ i)–20] – To apply N-R we also need the first derivative: h'(i) = 7,500 - 20,000 (1+ i)-21 E. T. S. I. Caminos, Canales y Puertos 60 Open Methods Score Sheet for Open Methods: Method Initial Est(s). s = 2 E-2 s = 2 E-7 N-R 1.0 0. 5 0.25 0.15 0.05 Secant (0.00, 0.50) (0.05, 0.15) 4, but wrong 3 27 6 N-R [as i f(i)] 1.00 0.150 0.050 0.047 0.03 3 2 4 crazy results converges to i=0 4 4 5 ** ** E. T. S. I. Caminos, Canales y Puertos diverges diverges 2, but wrong 48 5 7 3 5 4 5 61 Open Methods Cases of Multiple Roots Multiple Roots: f(x) = (x – 2)2 (x – 4) 3 2 x = 2 represents two of the three roots. 1 0 -1 -2 -3 1.0 E. T. S. I. Caminos, Canales y Puertos 2.0 3.0 4.0 62 Open Methods Problems and Approaches: Cases of Multiple Roots 1.Bracketing Methods fail locating x = 2. Note that f(x) f(xr) > 0. 2. At x = 2, f(x) = f '(x) = 0. • Newton-Raphson and Secant methods may experience problems. • Rate of convergence drops to linear. • Luckily, f(x) 0 faster than f '(x) 0 3. Other remedies, recasting problem: Find x such that u(x) = 0 where: f (x) u(x) 0 f '(x) Note that u(x) and f(x) have same roots. E. T. S. I. Caminos, Canales y Puertos 63 Summary -- Rates of Convergence lim i Ei1 Ei m A0 m = 1: linear convergence m = 2: quadratic convergence Method Bisection False Position Secant, mult. root NR, mult. root Secant, single root NR, single root Accel. NR, mult. root (f(x)/f'(x)=0) E. T. S. I. Caminos, Canales y Puertos m 1 1 1 1 1.618 "super linear" 2 2 64 Multivariate (Multidimensional) Equations Solve fi(x1, ..., xn) = 0 for i = 1,...,n Let X = (x1, ..., xn)T Given intial guess Xt, try to solve df f (X) (Xi1 Xi ) 0 dX where: df dfi dX dX j Obtain X = (Xi+1 – Xi) from linear equations: df X f (Xi ) dX E. T. S. I. Caminos, Canales y Puertos 65 Alternative Stopping Criteria 1. Always limit number of iterations using an outer DO loop. The problem may not converge and could try to go on forever. 2. Absolute error criteria for "small" differences: | xt - xt-1 | < 3. Relative error criteria for "relatively small" changes | xt – xt-1 | < | xt | 4. Can combined error criteria 2 & 3 for large and small x-values: | xt – xt-1 | < + | xt | 5. Converge on zero residual | f(xt) | < k E. T. S. I. Caminos, Canales y Puertos 66 Three Performance Criteria Stopping Criteria: | xi – xi-1 | < + |xi| or | f(xi) | < k or Max. iterations Convergence Criteria: | xi – xi-1 | < + |xi| and | f(xi) | < k N-R and Secant Confirmation of Convergent Behavior: x in feasible region and | f(xi) | ≤ 0.5 | f(xi-1) | and | xi – xi-1 | ≤ 0.6 | xi-1 – xi-2 | otherwise, do Bisection for a while. E. T. S. I. Caminos, Canales y Puertos 67 Three Phase Rootfinding Strategy A real rootfinding problem can be viewed as having three phases: 1) Opening moves: One needs to find the region of the parameter space in which desired root can be found. Understanding of problem, physical insight, and common sense are valuable. 2) Middle Game: Use robust algorithm to reduce initial region of uncertainty. 3) End game: Generate a highly accurate solution in a few iterations. E. T. S. I. Caminos, Canales y Puertos 68