Transcript Document

Concurrent Managers 101 & 202
Gary Piper
UKOUG 2004 Birmingham
1 - 3 November 2004
Case 1: The Case of Too Many Managers…
• Standard and Custom Managers
• As a general rule: No more than
1.8 to 2 * the number of CPU’s
• Common misconceptions:
– More managers are better 
• 35 on 2 CPU’s ( 17.5 )
• 70 on 8 CPU’s ( 8.8 )
• 48 on 12 CPU’s ( 4 )
– Double the number of managers overnight
Resist the urge to add managers
Financial Statement Generator…
• Assume 1 FSG will consume 1 CPU
• Limit the number of FSGs that can run to less than the
number of CPUs
• Create a custom FSG manager and setting the number
of manager processes to less than the number of CPUs
• This reasoning should hold true for any resource
intensive process
Case 1: Example…
Configuration: 8 CPU & High FSG usage
Manager Name
Max Proc
Run Proc
Sleep
Cache
FNDICM
1
1
30
0
STANDARD
70
70
30
100
FNDCRM
1
1
30
0
FNDSCH
1
1
30
0
INVMGR
1
1
60
5
PODAMGR
3
3
60
0
RCVOLTM
3
3
60
0
 Advantages
 Disadvantages
 Application Impact
 Business Impact
 Corrective Action
Beware the Silver Bullet…
• Site was heavily customised
• Several custom programs have an average run time of
over 10 hours.
• Account hierarchy reports take over 14 hours to run
• The site has a seven segment chart of accounts with
over 200,000 code combinations and no indexes.
• Other Sessions:
–
–
–
–
–
103 Discoverer sessions
30 Forms sessions
1 ADI session
5 Toad sessions  In Production
1 SQL*PLUS  In Production
Case 2: Sleep and Cache…
• Configuration
– 4 CPUs
– Max 8 standard managers
•
•
•
No custom managers
Site experienced exceptional growth
In this example:
– The concurrent program runs in less than 6
seconds.
– Items are scanned at a rate of 10 per minute
Single Manager…
Managers: 1
Sleep: 60 seconds
Cache: 2
Execution rate: 2 per min
48
80%
Scan rate:
10 per min
Input rate:
600 per hour
Run rate:
120 per hour
Deficit:
480 runs per hour
Approximately 4 hours backlog per hour
Multiple Managers…
Managers: 3
Sleep: 60 seconds
Cache: 2
Execution rate: 6 per min
48
48
48
Scan rate:
10 per min
Input rate:
600 per hour
Run rate:
360 per hour
Deficit:
240 runs per hour
Approximately 40 min backlog per hour
144/180
80%
Single Manager…
Managers: 1
Sleep: 60 seconds
Cache: 10
Execution rate: 10 per min
Scan rate:
Input rate:
Run rate:
Deficit:
10 per min
600 per hour
600 per hour
0 runs per hour ( theoretical )
A More Practical Solution…
Managers: 2
Sleep: 60 Seconds
Cache: 10
Execution rate: 20 per min
Scan rate:
Input rate:
Run rate:
Head room:
10 per min
600 per hour
Up to 1200 per hour
600 per hour
Case 2: Sleep and Cache Example…
Configuration: 12 CPU & 3,500 request per day
Manager Name
Run Proc
Sleep
Cache
Hits
FNDICM
1
20
0
3
STANDARD
16
2
1
480
FNDCRM
1
5
0
12
IMMEDIATE
2
10
1
6
QUICK
2
5
1
24
SLOW
3
5
2
36
1
6
SPECIAL
3
30
23 additional managers ( INV,OE, PO, RCV) @ 5 second sleep
 Advantages




Disadvantages
Application Impact
Business Impact
Corrective Action
567
Hidden Resource Usage…
Slow Q
Slow Q
Lookup Frequency…
Manager Configuration
Manager
Name
Running
Processes
Sleep
(Seconds)
Accesses
Per Min
% of
Total
Slow_Queue
1
60
1
20
Standard
2
30
4
80
Manager Impact
Total
Managers
Total
Processes
Accesses
Per Min
Accesses
Per Hour
Accesses
Per Day
2
3
5
300
7,200
Additional I/O…
•
Per Request (5,000)
–
–
–
–
•
•
•
Insert row (7) Indexes = (35,000)
Update to running (2) indexes = (10,000)
Update to complete (3) Indexes = (15,000)
5,000 requests generate 60,000 index activities
Manager reads @100 per Min
144,000 per day plus 144,000 index lookups
User accesses ( has it finished yet? )
Conflict Resolution Manager
– Higher overhead
– fnd_crm_history
Graphs reproduced with permission of PIPER-Rx
23:00 - 23:30
22:00 - 22:30
21:00 - 21:30
20:00 - 20:30
19:00 - 19:30
18:00 - 18:30
17:00 - 17:30
16:00 - 16:30
15:00 - 15:30
14:00 - 14:30
13:00 - 13:30
12:00 - 12:30
11:00 - 11:30
10:00 - 10:30
09:00 - 09:30
08:00 - 08:30
07:00 - 07:30
06:00 - 06:30
05:00 - 05:30
04:00 - 04:30
03:00 - 03:30
02:00 - 02:30
01:00 - 01:30
00:00 - 00:30
Review Request Activity…
14 Standard Managers
04-APR-04 (Sun)
06-APR-04 (Tue)
08-APR-04 (Thu)
10-APR-04 (Sat)
180-200
12-APR-04 (Mon)
160-180
14-APR-04 (Wed)
140-160
16-APR-04 (Fri)
120-140
18-APR-04 (Sun)
100-120
20-APR-04 (Tue)
80-100
Review Running Requests…
14 Standard Managers
02-APR-04 (Fri)
04-APR-04 (Sun)
06-APR-04 (Tue)
08-APR-04 (Thu)
15-18
10-APR-04 (Sat)
12-15
12-APR-04 (Mon)
9-12
6-9
14-APR-04 (Wed)
16-APR-04 (Fri)
Graphs reproduced with permission of PIPER-Rx
23:00
22:00
21:00
20:00
19:00
18:00
17:00
16:00
15:00
14:00
13:00
12:00
11:00
10:00
9:00
8:00
7:00
6:00
5:00
4:00
3:00
2:00
1:00
00:00
18-APR-04 (Sun)
20-APR-04 (Tue)
3-6
0-3
Case 3: Specialisation Rules Gone Wrong…
• Specialisation rules can be defined at the following levels:
– By Program
– By user
– By Oracle User
– Complex
– Request Type
• Once created are hard to find
• Creating a specialization rule will:
– Rebuild the fnd_concurrent_worker_requests view
– Restart the concurrent managers
– Not a good idea during month end
• KIS principle
Spcialisation rules…
Systems Administrator
Concurrent
Manager
Define
Select the Manager
Select specialization
Preparation…
•
•
•
•
•
Step 1 – Create a concurrent manager “slow”
Step 2 – Activate the manager ( multiple steps )
Step 3 – Reduce the number of STD managers
Step 4 – Ensure the new manager is active
Step 5 – Set the specialisation rules
– “Include” the selected program in the Slow Manager
– “Exclude” the selected program from the Standard
manager
• All custom developed programs should be initially assigned to the
“slow” queue and earn the right to be moved to the “normal” queues
Example of Include Only…
Manager Name
Max Proc
Run Proc
Sleep
Cache
Pending
Internal Manager
1
1
15
Conflict_Resolution_Manager
1
1
30
Financial Statement Generator
12
12
60
General Accounting
10
10
8
128
INV Remote Procedure Manager
10
10
30
4
CRP Inquiry Manager
2
0
60
Inventory Manager
1
1
60
1
MRP Manager
1
0
60
1
Material Transactions Manager
3
3
300
PA Streamline Manager
1
0
1
Shipping Transaction Manager
1
0
60
Standard ( Disabled )
0
0
60
48
3
1
1
128
Review… Configuration 8 CPU
Manager Name
Max Proc
Run Proc
Sleep
Cache
Pending
Internal Manager
1
1
15
Conflict_Resolution_Manager
1
1
30
Financial Statement Generator
12
12
60
General Accounting
10
10
8
128
INV Remote Procedure
Manager
10
10
30
4
CRP Inquiry Manager
2
0
60
Inventory Manager
1
1
60
1
MRP Manager
1
0
60
1
Material Transactions Manager
3
3
300
PA Streamline Manager
1
0
1
Shipping Transaction Manager
1
0
60
Standard ( Disabled )
0
0
60
48
3
1
1
128
Request Types…
Request Types
Request Types…
• Change the queue a program will run in
• Cannot change the queue once the request has been
submitted
• Can prevent a program from running
– Assign a program to run in a queue that has been
deactivated - “Pending Error”
– Once activated, pending request will run
– Assign a program to a request type “Don’t Run” that has
not been assigned a manager
Request Types (cont)…
• Create a Request Type
– Concurrent > Program > Types
– No impact
• Create Specialisation Rules
– Concurrent > Manager > Define – Specialisation Rules
– Disallow & Allow
– FND_CONCURRENT_WORKER_REQUESTS view
• Assign a program a request type
– Concurrent > Program > Define
Example: Overnight / Don’t Run…
• Create a manager that runs only overnight
– (work shifts * 2)
•
•
Create a request type ”overnight”
Disallow ”overnight” from standard / allow in the
overnight manager
• Assign the program the request type ”overnight”
• Request type “don’t run” 
– Create a request type “don’t run”
– Do not assign to a manager
– Assign the program the request type “don’t run”
Questions…
Questions
or
Visit us at the Quest stand
Disclaimer: All material contained in this document is provided by the author "as is" and any
express or implied warranties, including, but not limited to, any implied warranties of
merchantability and fitness for a particular purpose are disclaimed. In no event shall the author be
liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including,
but not limited to, loss of use, data, or profits; or business interruption) however caused and on any
theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise)
arising in any way out of the use of any content or information, even if advised of the possibility of
such damage. It is always recommended that you seek independent, professional advice before
implementing any ideas or changes to ensure that they are appropriate
Concurrent Manager 101 & 202
Gary Piper
OUUG 2004 Birmingham
1 - 3 November 2004