Transcript TriZetto 2014 Health Conference

1
Confidential
|
Copyright © 2014 TriZetto Corporation
Virtualization Best
Practices
Rene Lopez, VP Architecture
2
Confidential
|
Copyright © 2014 TriZetto Corporation
Introduction and History
 TriZetto’s Journey with ESX
 3.5, 4.1, 5.0, 5.5 coming soon
 Older versions created bad stigma due to Virtualization
inefficiencies
 Bad CPU scheduling
 Memory ballooning
 Immature licensing
 Immature operations
 Our disclaimers
 We have been a VMWare shop since 2.x Vmware
 We test and certify with VMWare and Hyper-V
 We believe our approach can be used for all vendors of similar
platforms and are happy to work with you on that
3
Confidential
|
Copyright © 2014 TriZetto Corporation
Where did we start
 TriZetto’s Infrastructure
 ESX 3.5
 HP DL 580 G5s (16 core x 196 GB)
 11 – 1 Gig E connections
 5 ports to create 1 5 GB connection
 Mix of DMX SAN/NetApp
 What happened
 Governance around solution was not good enough.
 Older tricks betrayed us.
 Memory was our bottleneck
4
Confidential
|
Copyright © 2014 TriZetto Corporation
Iteration 2
 TriZetto’s Infrastructure




ESX 4.1
Older 580 Architecture
HP DL 380 G7s – 12 core x 256 GB RAM
11 – 1 Gig E connections
 5 ports to create 1 5 GB connection
 Mix of DMX SAN/NetApp/Isilon
 What happened
 New approach helped in tuning but the introduction of Isilon
created major issues with IO
 Immediate move off of Isilon
5
Confidential
|
Copyright © 2014 TriZetto Corporation
Today
 TriZetto Infrastructure – deployed in Development
for 5.x, deployed in Hosting for 5.x
 Cisco UCS B230/B200 M3 (20 core by 384 GB)
 8-16 node cluster
 EMC vMAX SAN
 Cisco 10 GB Interconnects using pod design
 Improved Operations
 vSEA – Standardized deployments
 Cloning processes – templates and procedures
 Hosting is expanding platform and use
 Virtualization is one example
 Oracle RAC and Exadata, non-Websphere, Linux under Sybase
and Oracle
6
Confidential
|
Copyright © 2014 TriZetto Corporation
UCS Design
7
Confidential
|
Copyright © 2014 TriZetto Corporation
Process and
Governance
8
Confidential
|
Copyright © 2014 TriZetto Corporation
Our Process
 Platform
 TriZetto software development team codes and test on the same
platforms we use in our hosting production & non-prod
environments.
 For every iteration we review the full platform.
 Great time to let vendors lower costs.
 Look at the long term licensing of virtualization.
 Get roadmaps and understand costs.
 Implement firm platform life-cycle. Focus on the plan to upgrade.
 Build it without a clear plan and you will get stuck on
platforms.
 Create standards and rules so people know when things
would have to migrate – build around your organization.
 Define ‘acceptable use’
9
Confidential
|
Copyright © 2014 TriZetto Corporation
Reference
Architectures
with VM Specifications
10
Confidential
|
Copyright © 2014 TriZetto Corporation
Virtualization
Guidelines
11
Confidential
|
Copyright © 2014 TriZetto Corporation
Profile Workloads
 Create profile for Each workload
 Citrix Server is a 2x8 (40 Users), 2x4 (20 Users
 WAS Server NWX/WKF is a 4x8 – 4 JVMs
 New standard of Linux VMs due Q3 2014 (Tomcat/Apache)
 QNXT App Servers – run as both 4x8 and 2x8 depending on
Module
 Hub server – 1x4 and 2x8
 BizTalk –2x8 we did not get any performance gains with larger
VM’s
 Database Servers
 Linux on x86 8 core by 64 up to 24 by 512 GB
 AIX on Power 8 core by 64 up to 64 by 2 TB
 SQL on x86 8 core by 64 up to 24 by 512 GB
 Some 4 core and lower workloads approved on VMs
12
Confidential
|
Copyright © 2014 TriZetto Corporation
Solution Sizing Guidelines
13
This table represents a guideline to virtualizing TriZetto software, please
test in your environment as each deployment can contain different
variables
Confidential
|
Copyright © 2014 TriZetto Corporation
Solution Sizing Guidelines
14
This table represents a guideline to virtualizing TriZetto software, please
test in your environment as each deployment can contain different
variables
Confidential
|
Copyright © 2014 TriZetto Corporation
Best Practices
15
Confidential
|
Copyright © 2014 TriZetto Corporation
Best Practices


If you are going to run anything above 2 procs, you need to use ESX
5.0. There are performance issues in previous versions going above 2
procs.
Use SAN as the underlying disk structure for ESX.
 If you do not use SAN, use high end NAS. Do not use iSCSi

Be careful with your licensing and make sure they you understand the
costs.
 Windows is typically better licensed at the host Level.
 Software like WAS and SQL have very specific licensing that may require
dedicated ESX clusters for that software.

We do not go above core count of a single blade or ESX host based on best
practices. 20 core blade, highest CPU we will do is 16.
• Our Sizing
•
•
16
Confidential
|
2x4, 2x8, 4x8, 8x16 (SQL specialized loads Only).
Use vCenter Ops tools to tweak VM sizes based on usage.
Copyright © 2014 TriZetto Corporation
Best Practices – Over Provisioning
 Configuring a virtual machine with more vCPUs than
its workload can use might cause increased
resource usage, potentially impacting performance
on a very heavily loaded system.
 An example of this might be a single-threaded workload running
in a multiple vCPU VM or a multi-threaded workload in a VM with
more vCPUs than the workload can effectively use.
17
Confidential
|
Copyright © 2014 TriZetto Corporation
Best Practices - Over Provisioning
 Even if the guest OS doesn't use some of its vCPUs,
configuring VMs with those vCPUs still impose a
small resource requirement on ESXi that translate to
real CPU consumption on the host. For example:
 Unused vCPUs still consume timer interrupts in some operating
systems.
 Maintaining a consistent memory view among multiple vCPUs
can consume additional resources, both in the guest OS and in
ESXi. (although hardware assisted MMU virtualization helps
this).
 Most guest operating systems execute an idle loop during
periods of inactivity. Within this loop, most of these guest
operating systems halt by executing the HLT or MWAIT
instructions. Some older guest operating systems, however, use
busy-waiting within their idle loops.
18
Confidential
|
Copyright © 2014 TriZetto Corporation
Lessons Learned
 Operations changes
 Must review the mundane things!!!
 Governance
 ESX makes it easier to add capacity. Processes must be
enhanced to not overcommit resources.
 Licensing
 Software licensing was always behind Virtualization. Read your
licensing carefully
 Cost Transparency
 Finance does not always get it. So come up with a model to
show cost.. It’s a fun math problem
 Change Control
 It’s not a silver bullet
19
Confidential
|
Copyright © 2014 TriZetto Corporation
Where does this leave us?
 TriZetto Hosting has successfully virtualized about
90% of our workload types
 OLTP – not virtual
 Small DBs – virtual if approved and tested virtual
 We have designed a license and cluster model that fits our
business need. WAS Clusters to reduce costs.
 Starting to work with Licensed customers on what
we have done
 The future
20
Confidential
|
 Platform Transparency – we will let people know what we use
and test on.
 Solutions may be required to reproduce on Physical servers –
Hypervisors still have issues
 Working with product to release official documentation on what
works.
Copyright © 2014 TriZetto Corporation
Wrap Up
 Virtualization Works
 TriZetto software is virtualizable
 TriZetto Hosting does not recommend specific
hardware/platform/software for hypervisor
 Help us help you…
Questions?
21
Confidential
|
Copyright © 2014 TriZetto Corporation
Session Survey
We would like to extend you an
opportunity to provide candid feedback.
During the workshop you should’ve received an e-mail
notification for you to take an on-line survey.
If you could take a few minutes to complete at this time, we would
greatly value your feedback. For your convenience, the survey will be
available throughout the remainder of the conference should you not
be able to complete immediately.
22
Confidential
|
Copyright © 2014 TriZetto Corporation
Thank You!
23
Confidential
|
Copyright © 2014 TriZetto Corporation