Transcript Slide 1

WW TSS-05
InBatch Advanced Troubleshooting
Matt Mummert
Shri Mariyala
October, 2012
© 2012 Invensys. All Rights Reserved. The names, logos, and taglines identifying the products and services of Invensys are proprietary marks of
Invensys or its subsidiaries. All third party trademarks and service marks are the proprietary marks of their respective owners.
Project Start.
Best Practices.
Batch systems are often considered mission critical in the production
process; therefore, batch systems need to be designed so that the
process integrity is ensured even during unexpected external events.
A batch management solution, facilitating the batch execution on
physical process equipment, expands across multiple layers of the
system, including (but not limited to):
Control system infrastructure
Network infrastructure
Computer system infrastructure
Batch Management software
User interaction
Each one of these layers must be thoroughly analyzed and considered as
a critical part of the batch management solution. It is vital that, after the
critical variables surrounding the batch execution are identified, the
system is designed to address their needs at the right level.
(From InBatch R&D WHITE PAPER - Best Practices and Considerations for Designing Reliable InBatch Solutions)
Slide 4
Project Start.
Best Practices.
One of the most important things you can do on a
project is to make sure that the system functional
specification completely describes the desired
operation of the system and no more.
(From InBatch deployment guide)
Slide 5
Best Pratices –
What Do We Take Away?
-InBatch Deployment Guide
-InBatch R&D WHITE PAPER - Best Practices and Considerations for
Designing Reliable InBatch Solutions
Slide 6
Advanced Troubleshooting - Topics
• InBatch Architecture
• Batch Equipment
• Batch Phase Logic
• IBCli/DAServer
• Batch IBMX
• Batch Warm Restart Files
• BatchMngr Parameters
Slide 7
Advanced Troubleshooting - Topics
• BatchGuiConfig Ctrl
• Batch Redundancy
• Batch History
• Reports
• System Monitoring
• Log Flags
Slide 8
InBatch Architecture 1
Access Reports
Remotely
InBatch Runtime
Clients/Touch
Batch Server
+ DAServer
InBatch Dev
Client
Batch History
PLCs
Slide 9
Information
Server
InBatch Integrate with System Platform
Access Reports
Remotely
InBatch Runtime
Clients/Touch
Batch Server
+ DAServer
AOS
GR
InBatch Dev
Client
•AOr
PLCs
Slide 10
Batch History
Information
Server
System Platform Integration – How ?
Extremely flexible and powerful batch
management capabilities
InBatch Server
Phases modeled
within Control System
MX communication
PHASE 4
PHASE 1
PHASE 5
PHASE 2
Device Integration
PHASE 6
Phases modeled
within System
Platform
Slide 11
Device
System
Platform
PHASE 3
Extremely flexible and powerful
platform and plant model
Batch Equipment Analysis
● Batch started but no activity, why?
● What is equipment status?
● How to track and identify which equipment in use?
● What is Unit, Connection & Segment Status?
● How to update equipment status (TagView)
Slide 12
Equipment Definition - ModelEditor
Slide 13
Units/Connections Status - Batch Display
Slide 14
Segments Overview
Slide 15
Segments Linked to Connections
Slide 16
Equipment Status – TagView
Slide 17
Where is Tagview.exe?
Slide 18
Equipment Status –
What Do We Take Away?
• Unit Status values must be consistent - Process Model Vs USTAT
tags
• USTAT tags are linked to the correct addresses in PLC
• Blank or incorrect values for USTAT leads to “waiting for available
equipment” state
• Allocating a connection allocates all segments of that connection
• All connections that contain segment allocated by another
connection are ‘busy’
• Technote 25 – Using Segments to define Model Limitations in
InBatch.
Slide 19
Monitor Phase Status – Batch Display
Slide 20
Monitor Phase Status – Phase Logic Tool
Slide 21
Where is InBatch Phase Logic Tool?
Slide 22
Phase Logic Tool – Add to Env. Display
Slide 23
Phase Logic Tool-Add to Env. Display
Slide 24
Monitor Phase Status – TagView
Slide 25
Monitor Phase Status – WWClient
Note: Phase status values using WWClient
Slide 26
Phase Monitor –
What Do We Take Away?
• Phase Control/Status bits linked to the control system must be
unique and valid addresses in PLC. Duplicate items cause issues
• Phase must always be in ready state prior to the beginning of the
phase execution
• At any point in time, only status is present.
• Technotes as follows:
Slide 27
Phase Monitor –
What Do We Take Away?
• TN 555 – Active batch behavior when IBCli looses communications
to IOServer
• TN 504 - Phase control bit behavior
• TN 312 – InControl phase logic for InBatch
• TN 23 – Developing Allen Bradley Phase logic for InBatch
Slide 28
InBatch IBCli Parameters
● Response Time - Time after which an IBCli request to the
IOServer will timeout. Default 15 seconds
● Ping Time - Interval at which IBCli will ping the IOServer to
detect a connection loss. Default 2 seconds. To enable debug
for IBCli, set the Ping Time to “-999”
● Advise All - Performs an Advise All on initialization
● Disable Timestamp – Disable the end of phase timestamp
feature for this topic.
Slide 29
IBCli debug – Log Messages with
PingTime = -999
Reading a value from DAServer/IADAS:
2011/07/20 16:20:13.399/-03B4:14A4/IBCli /...(ProcessUpdate)
Update for [G2_V9072-V9074-CS-DONE] of [1] 2011/07/20
16:20:13.399/-03B4:14A4/IBCli /...(ProcessUpdate) Time Stamp for
[G2_V9072-V9074-CS-DONE] of [2011/07/20 20:20:11] Quality [1c0]
Slide 30
IBCli debug – Log Messages with
PingTime = -999
InBatch writing value to DAServer/IADAS
2011/07/20 16:20:13.899/-03B4:14A4/IBCli /(tt) wwCliPoke - request:
[22869912] tag: <G2_V9072-V9074-CS-RESET> Value:[1]
2011/07/20 16:20:13.899/-03B4:14A4/IBCli /...(tt) Adding tag to request request: [22869912] tag: <G2_V9072-V9074-CS-RESET> req type: [1]
2011/07/20 16:20:13.899/-03B4:14A4/IBCli /...(ProcessUpdate) Update for
[G2_V9072-V9074-CS-RESET] of [1] 2011/07/20 16:20:13.899/-
Slide 31
Where are IBCli Parameters?
Slide 32
InBatch 9.5 – Trace IBCli via Logger
Slide 33
InBatch - Environment Editor
• Any updates to Environment
Editor requires Update
Environment to affect changes,
which means, stop runtime
services, which means, it costs
production down time.
Slide 34
DAServer/IADAS Parameters
Slide 35
DAServer/IADAS – Log Flags
Slide 36
InBatch IBMX - Add Application in Env.
Editor
Slide 37
InBatch IBMX – Configure in TagLinker
Slide 38
InBatch IBMX – Launch Galaxy Browser
from TagLinker
Slide 39
InBatch IBMX – Galaxy Browser
Slide 40
IBMX with Verbose Mode-Log Messages
Slide 41
InBatch IBMX - ObjectViewer
Slide 42
InBatch IBMX –
What Do We Take Away?
● Platform must be deployed to enable IBMX
● IDE must be installed in order to browse Galaxy browser inside of
TagLinker
● Requires IDE and Runtime license for Platform
● Use TagView, Phase Logic, Batch Display and ObjectViewer to track
Phase Control/Status tags
● InBatch and ArchestrA AppServer Communication Recommendations
using IBMX – White Paper (Tech Article)
Slide 43
Batch Warm Restart
● Warm Restart files - If there is no warm restart folder,
batchmngr doesn’t startup
● .batchwrdebug – Creating a .batchwrdebug folder from
command line enables the debug for warm restart information.
This folder created under x:\>program
files\wonderware\InBatch\Cfg\Config_a\.batchwrdebug
Slide 44
Batch Parameters
Slide 45
BatchMngr Parameters
● Param Timeout - Time in seconds, of all attempts to read a
parameter tag value before timeout failure. Default value set
to 30 seconds. If you see any parameter read timeouts
in the logger tune this setting
● Phase Batch Status – Puts all phases on hold when batch is
on hold – Add this setting if all phases needs to be on
hold when batch is on hold
● Disconnect Clients – Disconnect all deadlock (misbehaving)
clients – Add this setting if any clients misbehaving,
otherwise it will cause negative impact to the
BatchMngr
Slide 46
InBatch ActiveX Ctrl - BatchGuiConfig
Slide 47
BatchGuiConfig – Properties
● Note: MessageBoxes has to be disabled from popup, otherwise
it will cause the view to not respond until message is acked.
Slide 48
BatchGuiConfig – Events
● Terminate/Initialize the control using Startup/Shutdown events of
Batch Server accordingly.
Slide 49
SFCCtrl and OcxBatchCtrl
● Terminate/Initialize these controls inside of view using
Shutdown/Startup events of BatchServer
Slide 50
InBatch ActiveX Controls –
What Do We Take Away?
● View locks up due to InBatch ActiveX Controls not terminated
gracefully when connection lost to BatchServer (Server
shutdown/loss network connection)
● Technote 364 – How to configure BatchGuiConfig ActiveX Ctrl for
Redundancy Client Switch-Over. This technote is good for
Standalone Server also.
● Starting with InBatch 9.5, use Technote 797 “Configuring InBatch
Runtime Client Communication Timeouts.” These timeouts are
configured in registry and technote covers details on configuration.
Slide 51
InBatch Redundancy
● Configuration Utility
Slide 52
InBatch Redundancy (cont.)
● RedCfg File
● LMHost File
● Messages in the logger
Slide 53
InBatch Redundancy (cont.)
● Contents of RedCfg file
Line 1: Remote machine name.
Line 2: (blank, not used).
Line 3: The path to the remote machine’s config_A directory.
Line 4: (blank, not used).
Line 5: The local hostname for the primary network (must not be a
machine name).
Line 6: The local hostname for the secondary/backup network, if one
exists.
Line 7: The remote hostname for the primary network (must not be a
machine name).
Line 8: The remote hostname for the secondary/backup network, if one
exists.
Note: The file content must end with a carriage return. Just press the
Enter key after the last entry and then save the file. The contents of the
RedCfg file must exactly match the entries in the Hosts or LMHosts files.
Slide 54
InBatch Redundancy (cont.)
● Contents of LMHost file
100.100.100.1 BATCHMASTER #PRE
100.100.100.2 BATCHBACKUP #PRE
100.100.100.10 INTOUCH1 #PRE
100.100.100.11 INTOUCH2 #PRE
100.100.100.12 INTOUCH3 #PRE
100.100.100.1 BATCHMAST #PRE
100.100.100.2 BATCHBACK #PRE
200.200.200.1 BACKUP1 #PRE
200.200.200.2 BACKUP2 #PRE
Slide 55
InBatch Redundancy (cont.)
● Technotes:
● 139 – Installing and configuring Redundancy Option for
InBatch
● 302 – How to create host file and redcfg file for InBatch 8.0
redundancy setup which is still valid for later versions also
● 881 – Configuring InBatch Redundant Servers to Sync in a
WorkGroup Environment
Slide 56
Batch History & Reports (cont.)
● ReportQReader
● Custom Reports – Tech Note 767
Slide 57
Log Flags for 9.5
● XES Diags – all service apps
● App Load: %
● Max Delay: s
● Trace – HistQReader
● E.g. transactions and admin thread activity
● Trace – IBMX / IBCLI
● E.g. read, write, data updates, quality, ack/nak, etc
● Replaces –d (debug) and –p-999 (PingTime) options
Slide 58
Log Flags for 9.5 (cont.)
● Trace – OcxBatch ActiveX / Batch Display / Batch Scheduler
● Init, term, heartbeat, focus
● Trace – Recipe Editor
● XML Import and Export
Slide 59
Log Flags for 9.5 (cont.)
Security and Active Directory Trace Log Options
● [HKLM\SOFTWARE\Wonderware\InBatch\ArchestraSec\Debug]
● [HKLM\SOFTWARE\Wonderware\InBatch\NativeSec\Debug]
Slide 60
Log Flags for 9.5 (cont.)
Batch WR File Log Option
● Create folder before startup. Logs to folder. Send to Tech
Support.
Slide 61
Diagnostic Tools
● Operating System Log Files - Application/System/Security
Event Log files
● ArchestrA Log files
● Windows Debugging tool
● Phase Logic Tool
● WWClient
Slide 62
Diagnostic Tools (cont.)
Microsoft WinDbg - used by Tech Support to examine DMP files.
Slide 63
Diagnostic Tools (cont.)
Technotes:
Slide 64
Questions?
THANK YOU
Slide 65