Transcript Session Title - Lenoir
Correctly Size Your Dashboards to Prevent Performance Problems
Wednesday, March 20, 2013 4:45 pm - 6:00 pm 118
Dr. Bjarne Berg COMERIT
© Copyright 2013 Wellesley Information Services, Inc.
All rights reserved.
In This Session …
• • •
We will look at how to size your BI environment, by doing a step by-step sizing effort of a dashboard project and look a five real world sizing examples Then we will conduct a structured walkthrough of compatible and required software components We will end by taking a quick peek at dashboard performance options with in-memory processing from SAP NetWeaver BW Accelerator and SAP HANA
1
What We’ll Cover …
• • • • •
Sizing environment for SAP BusinessObjects BI 4.0 and SAP BusinessObjects Dashboards Core components of SAP BusinessObjects Dashboards and SAP BusinessObjects BI Compatibility Requirements In-memory performance options Wrap-up
2
The Sizing Tool — Getting Started
SAP has provided a sizing tool for the BI environments. It is based on Flash and is actually a dashboard itself.
Output Area (Sizing Results) Download it: www.sdn.sap.com/irj/scn/index ?rid=/library/uuid/1055c550 ce45-2f10-22ad-a6050fff97f1 Input Areas (items and users) This tool can help you size your SAP BusinessObjects BI 4.0 environments with a few key assumptions and inputs
3
The Sizing Tool — Entering Users
First, you have to enter the estimated Active Concurrent Users (ACU) for the following user types:
•
Information Consumers
•
Business Users
•
Expert Users
4
The Sizing Tool — Online Help User Definitions
The tool provides online definitions of the user types and guidelines on how to determine Active Concurrent Users (ACU). This is defined as approximate 10% of the active users. Many dashboard users in large organizations may be classified as Information Consumers. They may not wait five minutes between clicks, but typically do little drilldown and filtering.
5
The Sizing Tool — Assumptions
•
The next step is to make an assumption on the size of dashboards
•
The sizing tool classifies small dashboards as having 25 rows in the result set, medium having 250, and large dashboards having 2,500 rows Assumptions: The tool was based on supporting two queries per dashboard and benchmarked for accessing two relational data sources — One with 6 dimensions with 77,000 entries and 400,000 line items, and one with 6 dimensions with 7,000 rows and 40,000 line items
6
The Sizing Tool — Output
• •
The output of the tool is measured in SAP Application Performance Standard (SAPS). 100 SAPS is defined as 2,000 fully business processed order line items per hour.
It is a measure that hardware vendors can use to decide which of their configurations can meet your performance requirements. All hardware vendors are familiar with this measure and this is what you will provide them when requesting a hardware quote.
7
The Sizing Tool — Memory Requirements
The sizing tool also provides a sizing estimate for the hardware memory required for each of the tiers. This is measured in Gigabytes
8
The Sizing Tool — Terminology
If you get stuck on the terminology used in SAP sizing and performance benchmarking, there is a link to the SAP benchmark glossary in the tool There are also performance benchmark and installation guides available on SAP Marketplace for individual software components
9
The Sizing Tool — Saving Your Sizing Example
Your BI and dashboard sizing effort can be saved or printed from the tool and you can have many scenarios
10
The Sizing Tool — Demo
11
Functionality vs. Performance — What Wins?
12
12
Types of Dashboards — Formatted Number Based
Some dashboards may have little navigation and be number- or key performance indicator (KPI)-based, similar to SAP Crystal Reports Static information Basic graphing of key numbers KPIs
13
Types of Dashboards — Graphical Dashboards
•
Dashboards for the senior management should be graphically oriented
•
Consider using logos and images instead of text for this purpose
•
Navigation should be very simple
•
For senior managers, the ability to interact with the data (what-if), see performance numbers relative to plan, budgets, and prior years are critical functionalities
14
Types of Dashboards — Linked to Web Services
•
Dashboards are most useful when shared with others
•
Power users can create great departmental dashboards that can be shared inside smaller organizational units In this dashboard, the data is merged with Google maps and external news feeds. This makes the dashboard much more interactive and interesting, but can kill any performance.
15
Dashboard Objects That Can Cause Slow Performance
These are dashboard objects that you need to consider carefully before employing
16
Excel Performance Considerations — What to Avoid
• •
The logic you build into your Excel spreadsheet is also compiled into the Flash file when you export it Since some “daisy-chain” functions are very time consuming, you should be careful not to add too many conditions in the data Lookup functions and conditioning that should be avoided include:
Lookups
Mid strings (MID)
Right and left strings (RIGHT/LEFT) Horizontal Lookups (HLOOKUP) Vertical Lookups (VLOOKUP)
Condition
General conditioning (IF) Count if a condition is true (COUNTIF) Sum if a condition is true (SUMIF)
Complex logic and nested logic create large SWF files and take a long time to open. Try to keep as much of the calculations and logic in the query instead of the spreadsheet.
17
• • • •
The Sizing Tool — Companion Guide
With the BI sizing tool, there is also a sizing companion guide written by Jason DeMelo This document explains how each tool was benchmarked and the assumptions made when building the sizing tool You can download it from: http://scn.sap.com/docs/DOC-33126 Involve your basis team in the sizing effort and also make sure that the assumptions you made are realistic from a functional standpoint (i.e., how complex and intensive are your dashboards)
18
PC Hardware Requirements for Client Side: SAP BusinessObjects BI 4.0
From a PC client perspective you need: Memory
Minimum of 2.0GB memory (really want more in practice) I recommend 4.0GB (or more if you can afford it) Processor
Minimum of 2.0 GHz core (more cores the better) Disk Space
Min of 3.5GB free space if you only install English
Min of 7.5GB free space if you install all languages Screen Size
Recommended resolution size is Make sure you build dashboards on a standardized screen resolution and size so that everyone sees the same images
19
Real-World Examples
Tool
SAP BW Dashboards Analysis WebI Hardware
Area
BW Version Named Users (#) Concurrent Users (#) Simultaneous Requests (#) Named Users (#) Concurrent Users (#) Simultaneous Requests (#) Named Users (#) Concurrent Users (#) Simultaneous Requests (#) Server Memory Server Disk PC Memory (standard) PC CPUs (standard) Other Portal version Server Operating System Flash version Database Version
Performance overall (1-10)
*Subjective
Manufacturing Company
7.3
192 7-8 5-10 45 6 4-5 16 GB 100 GB 4 GB 2.33 GHz (dual core) WebSphere AIX 11 SQL express
9 Airline
7.3
503 30-35 4-10 -
8 Pharma distributor
7.0 Enpk 1 168 26 4-20 84 22 5-15 16 GB 95 GB 2 GB 2.0 GHz SAP Win 2008 8 GB 120 GB 2 GB 2.0 GHz (dual core) SAP Win 2008 11 10 SQL express SQL express
8 Paper company
7.3
109 11-14 3-8 63 7 2-3 ~30 7 2-3 8 GB 70 GB 4 GB 2.3 GHz (dual core) SAP Win 2008 11 SQL
7
7.0 Enpk 1 309 22 5-15 46 6-8 7-10 1604 60-70 18-40 32 GB 230GB 4 GB 2.3 GHz (dual/quad core) SharePoint Win 2008 11 SQL
9 These are real examples from companies that have been using SAP BusinessObjects BI 4.0 for at least 6 months Retailer
20
What We’ll Cover …
• • • • •
Sizing environment for SAP BusinessObjects BI 4.0 and SAP BusinessObjects Dashboards Core components of SAP BusinessObjects Dashboards and SAP BusinessObjects BI Compatibility Requirements In-memory performance options Wrap-up
21
The Different Tiers in SAP BusinessObjects BI
• • •
First we have the application tier. This includes the Web Application Services, such as the Central Management Console (CMC) and the BI Launch Pad.
SAP recommends adding a Web application server for each 500 ACUs and that at least 5GB heap memory is assigned and 900 threads are configured
Next is the intelligence, or management tier, which includes the dashboard cache service, File Repository Service (FRS), and CMS Only the first File Repository input and output service pair (FRS) to register in the CMS is the active pair. If you add more FRSs, these are assumed to be passive backups for fault tolerance and failures.
Lastly, we have the processing tier. This include the Adaptive Job Service and the Processing Services for the various BI tools. Each BI tool has different memory and processor requirements
22
The SAP BusinessObjects BI Scalability
SAP BusinessObjects BI is highly scalable Tier Components for Intelligence If you have a high number of users, you can split the application, processing, and intelligence layers on separate hardware servers Tier Components for Processing You can also horizontally partition the processing and intelligence layers on several servers If you split the Web application components on multiple servers, make sure you also deploy a hardware load balancer
23
•
Dashboard Performance — Some Recommendations
You can scale the number of instances based on the Active Concurrent Users (ACUs), and SAP has made some recommendations: The CMS can handle up to 500 ACUs per instance and you can currently scale this to eight instances (will be increased in next release). You can add more CMSs if you see over 80% utilization of the CPUs.
The dashboard cache can handle up to 400 ACUs per instance and you can add as many instances as you want (no limitations), but you are unlikely to need more than one The dashboard processing is normally one per machine with no limitations (the server automatically spawns and manages child processes). If you need more, add more instances.
24
• • • •
More Key Factors That Determine Dashboard Performance
Concurrent number of users during peak load times of system Logical design of dashboards Simple, complex, and incredibly complex Number of records retrieved by the dashboards Network capacity Database speed of source data
• •
Number of instance This is used for spreading service loads on multiple nodes Number of CPUs and available memory of each server
25
What We’ll Cover …
• • • • •
Sizing environment for SAP BusinessObjects BI 4.0 and SAP BusinessObjects Dashboards Core components of SAP BusinessObjects Dashboards and SAP BusinessObjects BI Compatibility Requirements In-memory performance options Wrap-up
26
The Components of SAP BusinessObjects BI 4.0
The complete components of the BI landscape consist of a variety of software Each of these components have their own requirements We will now look at each of these components and their respective size and version requirements
27
• •
Compatibility Operating Systems: SAP BusinessObjects BI 4.0 Server Side
All server components only support 64-bit operating systems. The mobile server is currently only supported on Windows operating systems. For others, the following versions are supported: IBM Microsoft 64-bit BI Server Products
AIX 5.3 Power - TL 9 SP3 AIX 6 .1 Power - TL5 AIX 7.1 Power - TL1 SP1 Server 2003/2003 R2 and 2008 R2 x64 - SP2 Server 2008 R2 x64 - SP1
Oracle/ Sun
Solaris 10 SPARC -
Linux
Suse SLES 10 x86_64 - SP2* Suse SLES 11 x86_64 - * Red Hat EL 5 x86_64 - Update 2* Red Hat EL 6 x86_64 - Update 2*
Business Intelligence platform for OLAP Explorer Web Intelligence Crystal Reports - Enterprise Crystal Reports 2011 (*3) Dashboards (Xcelsius) Mobile Server (*4) 28
28
• •
Compatibility: SAP BusinessObjects BI 4.0 and Database Servers
Microsoft SQL Server Express 2008 is included with SAP BusinessObjects BI 4.0, but you may also use databases from SAP, Oracle, Sybase, IBM, and MySQL The database is used for storing CMS and the audit repository Vendor Oracle
Database Oracle DB Version 11g -R1 and R2 AIX 10g -R2
IBM
IBM DB2 Workgrou p Edition 9.7
SAP
IBM DB2 HANA MaxDB Sybase Adaptive Server Enterprise Sybase SQL Anywhere 9.7; 9.5 and 9.1
SP 3&4 7.7
15.7; 15.5 and 15 12.0.1
2012
Microsoft
SQL Server 2008 SP2/3 & R2 2005 SP3/SP4 SQL Server Express 2008 Linux
MySQL AB
MySQL 5.1
Solaris Windows
Using the SQL Server database that comes bundled with SAP BusinessObjects BI 4.0 may save you a lot of money
29
• •
Compatibility: SAP BusinessObjects BI 4.0 and Application Servers
The application server handles the logic of BI 4.x and can run on a variety of hardware The supported Java application servers include: Supported Java Application Servers Application Server
Tomcat
Version Java Version
6.0.20 and 7.0 Java 6 JBoss App. Server 5.0 and 5.1 WebLogic Java 5 / Java 6 Java 5 10 JRockit 5.0
Java 6 10.3
JRockit 6.0
10.3.3, 10.3.4*, 10.3.5*, 10.3.6
Java 6 JRockit 6.0
AIX Linux IBM Java Sun JDK
WebSphere SAP NW CE SAP NW JAS 6.1.0.7* 7.0.0.11
7.2 SP3 7.3
Java 5* Java 6* SAP JVM 1.6
SAP JVM 1.6
Solaris Windows Sun JDK Sun JDK Sizing the application server is based on SAP Application Performance Standard (SAPS) instead of CPU and Memory Note: WebSphere use IBM Java SDK for all operating systems and v6 is not supported for Analysis, OLAP edition
30
Hardware: Server-Side Requirements
•
From a server sizing perspective you need: Minimum CPU
4 x 2.0 GHz Core CPU Minimum Memory of Server
Min of 8.0GB memory – 16GB recommended (but more based on number of users) Minimum Disk Space
If you only install English: 11GB Windows, 13GB AIX/Solaris, and 14GB for Linux
If you install all languages: 14GB Windows, 15GB AIX/Solaris, and 16GB for Linux
31
•
Compatibility: BI 4.0 and Web Portals
The Web Portal supported by SAP BusinessObjects BI 4.0 is extensive and covers all major portal servers and their latest versions Supported Web Portal Servers Sever MS SharePoint (.NET 3.5) IBM Websphere SAP NW Enterprise Portal Oracle WebLogic Version
2010 2007 7 6.1.5
6.1
7.3
7.0
10.3.4* 10.3.2
10.3
10 * as part of 10.3.2
AIX Linux Solaris Windows Note: Microsoft’s SharePoint portal product only runs on the Windows operating system
32
Compatibility: BI 4.0 and Clients
• •
Most Windows operating systems are supported by SAP BusinessObjects BI 4.x at the client side All desktop components of SAP BusinessObjects BI 4.x are 32-bit 32-bit BI Desktop Products Mac
Mac OS X
Dashboards (Xcelsius) Web Intelligence Rich Client Crystal Reports for Enterprise Crystal Reports 2011 Crystal Reports Viewer 2011 Crystal Presentation Design BI Client Tools – Client Components Live Office Desktop
Windows XP Prof. (SP3) Windows Vista (SP2) Windows 7 (SP1)
Servers
Server 2003 & 2003 R2 (SP2) Server 2008 (SP2) & 2008 R2 (SP1)
Mac OS X is currently not supported except for the SAP Crystal Reports Viewer
33
•
Compatibility: BI 4.0 and Web Browsers
There are various browser and Java requirements for SAP BusinessObjects BI depending on the type of operating system you are using Supported Browsers and Java Versions
Safari 5.1 (Apple 1.6) Firefox 10* (JRE 1.6) Internet Explorer 7 (JRE 1.5/1.6 for WebI) Internet Explorer 8* (JRE 1.6) Internet Explorer 9* (JRE 1.6)
Mac Desktop
Mac OS X Windows XP Windows Vista Windows 7
Servers
Server 2003 & 2003 R2 Server 2008 (SP2) & 2008 R2
PS: On server side, IIS is supported as Integration option in MS SharePoint
V6 for Server 2003 V7 for Server 2008 V7.5 for Server 2008 R2 • • •
The Safari browser 5.1 is supported only by the Mac OS X operating system.
Internet Explorer 8/9 standards mode is currently not supported by BI Launch Pad. For Firefox usage, you need the Extended Support Release (ESR).
34
Other Software Requirements
• • •
To run the system correctly, there are several components needed on the client side Software and Version Windows Mac OS
Microsoft Office 2010 Microsoft Office 2007 SP2 Microsoft Office 2003 SP3 Adobe Reader 10.0
Adobe Reader 9.3.x / 9.4.x
Adobe Reader 9.0
Adobe Flash Player 11 Adobe Flash Player 10
While SAP BusinessObjects Explorer requires at least Flash player 10.1, there has been some issues with large Flash files in SAP BusinessObjects Dashboards For example, Flash v10.1 seems to be stuck in “initializing” phase when files are over 2 MB. This seems to be less of an issue for smaller files and we have seen no significant issues in version 11.
35
• •
Compatibility — Connections to SAP NetWeaver BW
There are many ways to connect the BI tools to an underlying SAP NetWeaver BW system With SAP BusinessObjects BI 4.0, the following BW versions are supported Product Connection BW 7.30 SPS 03+ BW 7.02 SPS 02+ BW 7.01 SPS 08+ BW 7.01 SPS 06+ Dashboards
Web Intelligence Crystal Reports for Enterprise
Dashboards
Analysis, edition for OLAP
Dashboards
Web Intelligence Crystal Reports Enterprise Explorer Web Intelligence Live Office BEx Web Applications Explorer Crystal Reports for Enterprise Crystal Reports 2011 BICS (transient universe) BICS (NW BI Java) BICS Multi source universe (SQL) OLAP Universe (MDX) Integrated with BI 4 BWA / HANA D2D Directly to BW
Dashboards can also be built on BW systems on SAP HANA through a relational universe if you have applied SAP HANA service pack 3 or 4
36
What We’ll Cover …
• • • • •
Sizing environment for SAP BusinessObjects BI 4.0 and SAP BusinessObjects Dashboards Core components of SAP BusinessObjects Dashboards and SAP BusinessObjects BI Compatibility Requirements In-memory performance options Wrap-up
37
Why In-Memory Processing?
CPU Memory Technology 1990 2013 Improvement 0.05
MIPS/$ 389.32
MIPS/$ 0.02
MB/$ 97.5
MB/$ 7796 x 4875x Addressable Memory 2 16 Network Speed 100 Mbps 2 64 100 Gbps 2 48 x 1000 x Focus Disk 5 Data Transfer MBPS 722 MBPS 144x
Source: 1990 numbers SAP AG, 2013 numbers, Dr. Berg Source: BI Survey of 534 BI professionals, InformationWeek,
Disk speed is growing slower than all other hardware components, while the need for speed is increasing
38
In-Memory Processing — SAP NetWeaver BW Accelerator (BWA)
•
BWA is a tool that has been available for almost six years and is being used by hundreds of companies
•
It can load data in-memory from SAP and non-SAP sources and you can build dashboards on the data using standard BEx queries and BICS connections (as covered earlier)
39
• •
An Example of an SAP HANA System We Installed Recently
The long-term idea with SAP HANA is to replace the databases under SAP NetWeaver BW and SAP ERP with in-memory processing databases, instead of traditional relational databases This means much faster query response time and a smaller database SAP HANA is an appliance that can be implemented fast, is cost effective, and can super-charge the data delivery and calculations in your dashboards!
40
Looking Inside SAP HANA — In-Memory Computing Engine
Vs.
Metadata Manager Authorization Manager Transaction Manager Disk Storage Data Volumes Log Volumes Relational Engine SQL Script
-Row Store -Column Store
Calculation Engine SQL Parser MDX Session Manager Load Controller Replication Server We can also move the whole database that has the source data for your dashboards to the in-memory platform of SAP HANA. This makes the system much faster! (SAP BusinessObjects Dashboards can consume data from SAP HANA right now)
41
What We’ll Cover …
• • • • •
Sizing environment for SAP BusinessObjects BI 4.0 and SAP BusinessObjects Dashboards Core components of SAP BusinessObjects Dashboards and SAP BusinessObjects BI Compatibility Requirements In-memory performance options Wrap-up
42
• • • •
High-Volume User Management and Access Control
Plan for a gradual rollout to a limited number of users Keep the numbers comparable if possible This will allow you to predict system loads and performance issues by stipulations from real performance data
I.e., roll out to 50 users each week
Simplified versions of high-impact dashboards may be created for casual users E.g., a dashboard with only one query and summarized data with limited navigation and passing of variables Create a hardware contingency plan and budget accordingly Only in rare cases should you use a big-bang approach. Since user patterns are hard to predict, this may cause significant performance issues.
43
Where to Find More Information
• • • • •
SAP BusinessObjects BI 4.0 Sizing Estimator
www.sdn.sap.com/irj/scn/index?rid=/library/uuid/1055c550 ce45-2f10-22ad-a6050fff97f1 Evan Delodder and Ray Li, Creating Dashboards with Xcelsius: Practical Guide (SAP PRESS, 1st Edition; 2010). David Lai and Xavier Hacking, SAP BusinessObjects Dashboards 4.0 Cookbook (Packt Publishing, 2011). Dashboard and Presentation Designer (Xcelsius) forum on SDN
http://forums.sdn.sap.com/forum.jspa?forumID=302 SAP BusinessObjects Dashboards 4.x tutorials on SDN www.sdn.sap.com/irj/boc/dashboards-elearning
44
7 Key Points to Take Home
• • • • • • •
Use the SAP Sizing tool for initial sizing estimates Size your system based on concurrent users and SAPS Use realistic data volumes, users, and dashboard complexity in your assumptions Use the SAP system guides on the SAP Service Marketplace, but plan to operate your system at maximum 70% load for “spare capacity” Keep the SAP BusinessObjects BI 4.0 environment on a separate stack from SAP NetWeaver BW Make sure the PCs have enough memory Examine the “standard” PC of the users and developers; pay attention to connectivity, screen size, and resolutions, CPUs, and all software release versions to assure compatibility
45
Your Turn!
How to contact me: Dr. Bjarne Berg [email protected]
Please remember to complete your session evaluation
46
Disclaimer
SAP, R/3, mySAP, mySAP.com, SAP NetWeaver ® , Duet ® , PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.
47