Oct 18

What’s New with VMware Virtual SAN 6.5

Introducing Virtual SAN 6.5

vSAN 6.5VMware Virtual SAN 6.5 is the latest release of the market-leading, enterprise-class storage solution for hyper-converged infrastructure (HCI). Virtual SAN 6.5 builds on the existing features introduced in 6.2 by enhancing automation, further reducing total cost of ownership (TCO), and setting the stage for next-generation cloud native applications.

Virtual SAN continues to see rapid adoption with more than 5000 customers utilizing the solution for a number of use cases including mission-critical production applications and databases, test and development, management infrastructures, disaster recovery sites, virtual desktop deployments, and remote office implementations. Virtual SAN is used by 400+ Fortune-1000 organizations across every industry vertical in more than 100 countries worldwide.

Let’s take a look at the new features included with Virtual SAN 6.5…

Accelerate Responsiveness

The Virtual SAN API and vSphere PowerCLI have been updated in this release. It is now possible to automate the configuration and management of cluster settings, disk groups, fault domains, and stretched clusters. Activities such as maintenance mode and cluster shutdown can also be scripted. This video demonstrates some of the capabilities of of the Virtual SAN API and PowerCLI: Creating a Cluster and Configuring Virtual SAN PowerCLI can be used to monitor the health of a Virtual SAN cluster. Health issue remediation and re-sync activities can be automated with this latest release.

20-50% Additional TCO Savings

Now that flash devices have become the preferred choice for storage, it makes sense to adjust the Virtual SAN licensing model to account for this change in the industry. All Virtual SAN 6.5 licenses include support for both hybrid and all-flash configurations. Please note, however, that deduplication, compression, and erasure coding still require Virtual SAN Advanced or Enterprise licenses. Adding support for the use of all-flash configurations with all licensing editions provides organizations more deployment options and the ability to take advantage of increased performance while minimizing licensing costs.

vSAN 6.5Virtual SAN supports the use of network crossover cables in 2-node configurations. This is especially beneficial in use cases such as remote office and branch office (ROBO) deployments where it can be cost prohibitive to procure, deploy, and manage 10GbE networking equipment at each location. This configuration also reduces complexity and improves reliability.

While we are on the subject of ROBO deployments, it is also important to mention a related Virtual SAN licensing change. previously did not support the use of all-flash Virtual SAN cluster configurations and the corresponding space efficiency features. A new license has been added with the release of Virtual SAN 6.5 and it is called >strong>Virtual SAN for ROBO Advanced. This new license includes support for using deduplication, compression, and erasure coding. Using these features lowers the cost-per-usable-GB of flash storage, which further reduces TCO. Organizations get the best of both worlds: The extreme performance of flash at a cost that is on par with or lower than similar hybrid solutions.

Increased Flexibility

Virtual SAN 6.5 extends workload support to physical servers and clustered applications with the introduction of an iSCSI target service. Virtual SAN continues its track record of being radically simple by making it easy to access Virtual SAN storage using the iSCSI protocol with just a few vSphere Web Client mouse clicks. iSCSI targets on Virtual SAN are managed the same as other objects with Storage Policy Based Management (SPBM). Virtual SAN functionality such as deduplication, compression, mirroring, and erasure coding can be utilized with the iSCSI target service. CHAP and Mutual CHAP authentication is supported.

Enable vSAN iSCSI target service

Enable vSAN iSCSI target service

Utilizing Virtual SAN for physical server workloads and clustered applications can reduce or eliminate the dependency on legacy storage solutions while providing the benefits of Virtual SAN such as simplicity, centralized management and monitoring, and high availability.

Scale To Tomorrow

Photon OS New application architecture and development methods have emerged that are designed to run in today’s mobile-cloud era. For example,“DevOps” is a term that describes how these next-generation applications are developed and operated. “Container” technologies such as Docker and Kubernetes are a couple of the many solutions that have emerged as options for deploying and orchestrating these applications. Cloud native applications naturally require persistent storage just the same as traditional applications. Virtual SAN is an excellent choice for next-generation cloud native applications. Here are a few examples of the efforts that are underway:

vSphere Integrated Containers Engine is a container runtime for vSphere, allowing developers familiar with Docker to develop in containers and deploy them alongside traditional virtual machine workloads on vSphere clusters. vSphere Integrated Containers Engine enables these workloads to be managed through the vSphere GUI in a way familiar to vSphere admins. Availability and performance features in vSphere and Virtual SAN can be utilized by vSphere Integrated Containers Engine just the same as traditional virtual machine environments.

Docker Volume Driver for vSphere enables users to create and manage Docker container data volumes on vSphere storage technologies such as VMFS, NFS, and Virtual SAN. This driver makes it very simple to use containers with vSphere storage and provides the following key benefits:

– DevOps-friendly API for provisioning and policy configuration.
– Seamless movement of containers between vSphere hosts without moving data.
– Single platform to manage – run virtual machines and containers side-by-side

Next-Gen Hardware Support

vSphere 6.5 and Virtual SAN 6.5 also introduce support for 512e drives, which will enable larger capacities to meet the constantly growing space requirements of today’s and tomorrow’s applications. New hardware innovations such as NVMe provide dramatic performance gains for Virtual SAN with up to 150k IOPS per host. This level of performance combined with the ability to scale up to 64 hosts in a single cluster sets the stage for running any app, any scale on Virtual SAN.

Visit Virtual SAN on vmware.com and VMware StorageHub for more details on this exciting new release of Virtual SAN.

To learn more about vSphere 6.5, please see the following resources.

@jhuntervmware on twitter

Rating: 5/5


Oct 18

What’s New in vSphere 6.5: Host & Resource Management and Operations

Posted on October 18, 2016 by Charu Chaubal

vSphere 6.5 brings a number of enhancements to ESXi host lifecycle management as well as some new capabilities to our venerable resource management features, DRS and HA. There are also greatly enhanced developer and automation interfaces, which are a major focus in this release. Last but not least, there are some notable improvements to vRealize Operations, since this product is bundled with certain editions of vSphere. Let’s dig into each of these areas.

Enhanced vSphere Host Lifecycle Management Capabilities

With vSphere 6.5, administrators will find significantly easier and more powerful capabilities for patching, upgrading, and managing the configuration of VMware ESXi hosts.

VMware Update Manager (VUM) continues to be the preferred approach for keeping ESXi hosts up to date, and with vSphere 6.5 it has been fully integrated with the VCSA. This eliminates the additional VM, operating system license, and database dependencies of the previous architecture, and now benefits from the resiliency of vCenter HA for redundancy. VUM is enabled by default and ready to handle patching and upgrading tasks of all magnitudes in your datacenter.

Host Profiles has come a long way since the initial introduction way back in vSphere 4! This release offers much in the way of both management of the profiles, as well as day-to-day operations. For starters, an updated graphical editor that is part of the vSphere Web Client now has an easy-to-use search function in addition to a new ability to mark individual configuration elements as favorites for quick access.

vSphere Host Profile Editor

vSphere Host Profile Editor

Administrators now have the means to create a hierarchy of host profiles by taking advantage of the new ability to copy settings from one profile to one or many others.

Although Host Profiles provides a means of abstracting management away from individual hosts in favor of clusters, each host may still have distinct characteristics, such as a static IP address, that must be accommodated. The process of setting these per-host values is known as host customization, and with this release it is now possible to manage these settings for groups of hosts via CSV file – undoubtedly appealing to customers with larger environments.

Compliance checks are more informative as well, with a detailed side-by-side comparison of values from a profile versus the actual values on a host. And finally, the process of effecting configuration change is greatly enhanced in vSphere 6.5 thanks to DRS integration for scenarios that require maintenance mode, and speedy parallel remediation for changes that do not.

Auto Deploy – the boot-from-network deployment option for vSphere – is now easier to manage in vSphere 6.5 with the introduction of a full-featured graphical interface. Administrators no longer need to use PowerCLI to create and manage deploy rules or custom ESXi images.

Auto Deploy

Auto Deploy


New and unassigned hosts that boot from Auto Deploy will now be collected under the Discovered Hosts tab as they wait patiently for instructions, and a new interactive workflow enables provisioning without ever creating a deploy rule.

Custom integrations and other special configuration tasks are now possible with the Script Bundle feature, enabling arbitrary scripts to be run on the ESXi hosts after they boot via Auto Deploy.

Scalability has been greatly improved over previous releases and it’s easy to design an architecture with optional reverse proxy caches for very large environments needing to optimize and reduce resource utilization on the VCSA. And like VUM, Auto Deploy also benefits from native vCenter HA for quick failover in the event of an outage.

In addition to all of that, we are pleased to announce that Auto Deploy now supports UEFI hardware for those customers running the newest servers from VMware OEM partners.

It’s easy to see how vSphere 6.5 makes management of hosts easier for datacenters of all sizes!

Resource Management – HA, FT and DRS

vSphere continues to provide the best availability and resource management features for today’s most demanding applications. vSphere 6.5 continues to move the needle by adding major new features and improving existing features to make vSphere the most trusted virtual computing platform available. Here is a glimpse of the what you can expect to see when vSphere 6.5 later this year.

Proactive HA

Proactive HA will detect hardware conditions of a host and allow you to evacuate the VMs before the issue causes an outage. Working in conjunction with participating hardware vendors, vCenter will plug into the hardware monitoring solution to receive the health status of the monitored components such as fans, memory, and power supplies. vSphere can then be configured to respond according to the failure.

Once a component is labeled unhealthy by the hardware monitoring system, vSphere will classify the host as either moderately or severely degraded depending on which component failed. vSphere will place that affected host into a new state called Quarantine Mode. In this mode, DRS will not use the host for placement decisions for new VMs unless a DRS rule could not otherwise be satisfied. Additionally, DRS will attempt to evacuate the host as long as it would not cause a performance issue. Proactive HA can also be configured to place degraded hosts into Maintenance Mode which will perform a standard virtual machine evacuation.

vSphere HA Orchestrated Restart

vSphere 6.5 now allows creating dependency chains using VM-to-VM rules. These dependency rules are enforced if when vSphere HA is used to restart VMs from failed hosts. This is great for multi-tier applications that do not recover successfully unless they are restarted in a particular order. A common example to this is a database, app, and web server.

In the example below, VM4 and VM5 restart at the same time because their dependency rules are satisfied. VM7 will wait for VM5 because there is a rule between VM5 and VM7. Explicit rules must be created that define the dependency chain. If that last rule were omitted, VM7 would restart with VM5 because the rule with VM6 is already satisfied.

Orchestrator HA

Orchestrator HA


In addition to the VM dependency rules, vSphere 6.5 adds two additional restart priority levels named Highest and Lowest providing five total. This provides even greater control when planning the recovery of virtual machines managed by vSphere HA.

Simplified vSphere HA Admission Control

Several improvements have been made to vSphere HA Admission Control. Admission control is used to set aside a calculated amount of resources that are used in the event of a host failure. One of three different policies are used to enforce the amount of capacity is set aside. Starting with vSphere 6.5, this configuration just got simpler. The first major change is that the administrator simply needs to define the number of host failures to tolerate (FTT). Once the numbers of hosts are configured, vSphere HA will automatically calculate a percentage of resources to set aside by applying the “Percentage of Cluster Resources” admission control policy. As hosts are added or removed from the cluster, the percentage will be automatically recalculated. This is the new default configuration, but it is possible to override the automatic calculation or use another admission control policy.

Additionally, the vSphere Web Client will issue a warning if vSphere HA detects a host failure would cause a reduction in VM performance based on the actual resource consumption, not only based on the configured reservations. The administrator is able to configure how much of a performance loss is tolerated before a warning is issued.

Admission Control

Admission Control

Fault Tolerance (FT)

vSphere 6.5 FT has more integration with DRS which will help make better placement decisions by ranking the hosts based on the available network bandwidth as well as recommending which datastore to place the secondary vmdk files.

There has been a tremendous amount of effort to lower the network latency introduced with the new technology that powers vSphere FT. This will improve the performance to impact to certain types of applications that were sensitive to the additional latency first introduced with vSphere 6.0. This now opens the door for even a wider array of mission critical applications.

FT networks can now be configured to use multiple NICs to increase the overall bandwidth available for FT logging traffic. This is a similar configuration to Multi-NIC vMotion to provide additional channels of communication for environments that required more bandwidth than a single NIC can provide.

DRS Advanced Options

Three of the most common advanced options used in DRS clusters are now getting their own checkbox in the UI for simpler configuration.

  • VM Distribution: Enforce an even distribution of VMs. This will cause DRS to spread the count of the VMs evenly across the hosts. This is to prevent too many eggs in one basket and minimizes the impact to the environment after encountering a host failure. If DRS detects a severe imbalance to the performance, it will correct the performance issue at the expense of the count being evenly distributed.
  • Memory Metric for Load Balancing: DRS uses Active memory + 25% as its primary metric when calculating memory load on a host. The Consumed memory vs active memory will cause DRS to use the consumed memory metric rather than Active. This is beneficial when memory is not over-allocated. As a side effect, the UI show the hosts be more balanced.
  • CPU over-commitment: This is an option to enforce a maximum vCPU:pCPU ratios in the cluster. Once the cluster reaches this defined value, no additional VMs will be allowed to power on.
DRS settings

DRS settings

Network-Aware DRS

DRS now considers network utilization, in addition to the 25+ metrics already used when making migration recommendations. DRS observes the Tx and Rx rates of the connected physical uplinks and avoids placing VMs on hosts that are greater than 80% utilized. DRS will not reactively balance the hosts solely based on network utilization, rather, it will use network utilization as an additional check to determine whether the currently selected host is suitable for the VM. This additional input will improve DRS placement decisions, which results in better VM performance.

SIOC + SPBM

Storage IO Control configuration is now performed using Storage Policies and IO limits enforced using vSphere APIs for IO Filtering (VAIO). Using the Storage Based Policy Management (SPBM) framework, administrators can define different policies with different IO limits, and then assign VMs to those policies. This simplifies the ability to offer varying tiers of storage services and provides the ability to validate policy compliance.

VM Storage Policy

VM Storage Policy

Content Library

Content Library with vSphere 6.5 includes some very welcome usability improvements. Administrators can now mount an ISO directly from the Content Library, apply a Guest OS Customization during VM deployment, and update existing templates.

Performance and recoverability has also been improved. Scalability has been increased, and there is new option to control how a published library will store and sync content. When enabled, it will reduce the sync time between vCenter Servers are not using Enhanced Linked Mode.

The Content Library is now part of the vSphere 6.5 backup/restore service, and it is part of the VC HA feature set.

Developer and Automation Interfaces

The vSphere developer and automation interfaces are receiving some fantastic updates as well. Starting with the vSphere’s REST APIs, these have been extended to include VCSA and VM based management and configuration tasks. There’s also a new way to explore the available vSphere REST APIs with the API Explorer. The API Explorer is available locally on the vCenter server itself and will include information like what URL the API tas is available to be called by, what method to use, what the request body should look like, and even a “Try It Out” button to perform the call live.

API explorer

API explorer


Moving over to the CLIs, PowerCLI is now 100% module based! There’s also some key improvements to some of those modules as well. The Core module now supports cross vCenter vMotion by way of the Move-VM cmdlet. The VSAN module has been bolstered to feature 13 different cmdlets which focus on trying to automate the entire lifecycle of VSAN. The Horizon View module has been completely re-written and allows users to perform View related tasks from any system as well as the ability to interact with the View API.

The vSphere CLI (vCLI) also received some big updates. ESXCLI, which is installed as part of vCLI, now features several new storage based commands for handling VSAN core dump procedures, utilizing VSAN’s iSCSI functionality, managing NVMe devices, and other core storage commands. There’s also some additions on the network side to handle NIC based commands such as queuing, coalescing, and basic FCOE tasks. Lastly, the Datacenter CLI (DCLI), which is also installed as part of vCLI, can make use of all the new vSphere REST APIs!

Check out this example of the power of DCLI’s interactive mode with features like tab complete:

DCLI interactive

DCLI interactive

Operations Management

There’s been some exciting improvements on the vSphere with Operations Management (vSOM) side of the house as well. vRealize Operations Manager (vR Ops) has been updated to version 6.4 which include many new dashboards, dashboard improvements, and other key features to help administrators get to the root cause that much faster and more efficient. Log Insight for vCenter has been also updated, and will be on version 4.0. It contains a new user interface (UI) based on our new Clarity UI, increased API functionality around the installation process, the ability to perform automatic updates to agents, and some other general UI improvements. Also, both of these products will be compatible with vSphere 6.5 on day one.

Digging a little further into the vR Ops improvements, let’s first take a look at the three new dashboards titled: Operations Overview, Capacity Overview, and Troubleshoot a VM. The Operations dashboard will display pertinent environment based information such as an inventory summary, cluster update, overall alert volume, and some widgets containing Top-15 VMs experiencing CPU contention, memory contention, and disk latency. The Capacity dashboard contains information such as capacity totals as well as capacity in use across CPU count, RAM, and storage, reclaimable capacity, and a distributed utilization visualization. The Troubleshoot a VM dashboard is a nice central location to view individual VM based information like its alerts, relationships, and metrics based on demand, contention, parent cluster contention, and parent datastore latency.

vROPS Dashboard

vROPS Dashboard

One other improvement that isn’t a dashboard but is a new view for each object, is the new resource details page. It closely resembles the Home dashboard that was added in a prior version, but only focuses on the object selected. Some of the information displayed is any active alerts, key properties, KPI metrics, and relational based information.

vROPS details

vROPS details

Covering some of the other notable improvements, there is now the ability to display the vSphere VM folders within vR Ops. There’s also the ability to group alerts so that it’s easy to see what the most prevalent alert might be. Alert groups also enable the functionality to clear alerts in a bulk fashion. Lastly, there are now KPI metric groups available out of the box to help easily chart out and correlate properties with a single click.

To learn more about vSphere 6.5, please see the following resources.

Rating: 5/5


Oct 18

What’s new in vSphere 6.5: Security

Posted on October 17, 2016 by Mike Foley

vSphere 6.5 is a turning point in VMware infrastructure security. What was mostly an afterthought by many IT folks only a few short years ago is now one of the top drivers of innovation for vSphere. Security has become a front and center focus of this release and I think you’ll like what we’ve come up with.

Our focus on security is manageability. If security is not easy to implement and manage then the benefit it may bring is offset. Security in a virtual infrastructure must be able to be done “at scale”. Managing 100’s or 1000’s of security “snowflakes” is something no IT manager wants to do. She/He doesn’t have the resources to do that. The key to security at scale is automation and in these new features you’ll see plenty of that.

VM Encryption

Encryption of virtual machines is something that’s been on-going for years. But, in case you hadn’t noticed, it just hasn’t “taken off” because every solution has a negative operational impact. With vSphere 6.5 we are addressing that head on.

Encryption will be done in the hypervisor, “beneath” the virtual machine. As I/O comes out of the virtual disk controller in the VM it is immediately encrypted by a module in the kernel before being send to the kernel storage layer. Both VM Home files (VMX, snapshot, etc) and VMDK files are encrypted.

The advantages here are numerous.

    1. Because encryption happens at the hypervisor level and not in the VM, the Guest OS and datastore type are not a factor. Encryption of the VM is agnostic.
    2. Encryption is managed via policy. Application of the policy can be done to many VM’s, regardless of their Guest OS.
    3. Encryption is not managed “within” the VM. This is a key differentiation to every other solution in the market today! There are no encryption “snowflakes”. You don’t have to monitor whether encryption is running in the VM and the keys are not contained in the VM’s memory.
    4. Key Management is based on the industry standard, KMIP 1.1.
    In vSphere vCenter is a KMIP client and works with a large number of KMIP 1.1 key managers. This brings choice and flexibility to customers. VM Keys do not persist in vCenter.
    5. VM Encryption makes use of the latest hardware advances inherent in the CPU’s today. It leverages AES-NI for encryption.
VM Encryption

VM Encryption

vMotion Encryption

This has been an ask for a long time and with 6.5 we deliver. What’s unique about vMotion encryption is that we are not encrypting the network. There are not certificates to manage or network settings to make.

The encryption happens on a per-VM level. Enabling vMotion encryption on a VM sets things in motion. When the VM is migrated, a randomly generated, one time use 256-bit key is generated by vCenter (it does not use the key manager for this key).

In addition, a 64-bit “Nonce” (an arbitrary number used only once in a crypto operation) is also generated. The encryption key and Nonce are packaged into the migration specification sent to both hosts. At that point all the VM vMotion data is encrypted with both the key and the Nonce, ensuring that communications can’t be used to replay the data.

vMotion encryption can be set on unencrypted VM’s and is always enforced on encrypted VM’s.

Encrypted vMotion

Encrypted vMotion

Secure Boot support

For vSphere 6.5 we are introducing Secure Boot support for virtual machines and for the ESXi hypervisor.

ESXi Secure Boot

ESXi Secure Boot

ESXi SECURE BOOT – With Secure Boot enabled, the UEFI firmware validates the digital signature of the ESXi kernel against a digital certificate in the UEFI firmware. That ensures that only a properly signed kernel boots. For ESXi, we are taking Secure Boot further adding cryptographic assurance of all components of ESXi. Today, ESXi is already made up of digitally signed packages, called VIB’s. (vSphere Installation Bundle) The ESXi file system maps to the content of those packages (the packages are never broken open). By leveraging that digital certificate in the host UEFI firmware, at boot time the already validated ESXi Kernel will, in turn, validate each VIB against the firmware-based certificate. This assures a cryptographically “clean” boot.

Note: If Secure Boot is enabled then you will not be able to forcibly install un-signed code on ESXi. This ensures that when Secure Boot is enabled that ESXi will only be running VMware digitally signed code.

Dramatically Simplified Experience

VIRTUAL MACHINE SECURE BOOT

For VM’s, SecureBoot is simple to enable. Your VM must be configured to use EFI firmware and then you enable Secure Boot with a checkbox. Note that if you turn on secure boot for a virtual machine, you can load only signed drivers into that virtual machine.

Secure Boot for Virtual Machines works with Windows or Linux.

Secure Boot for Virtual Machines

Secure Boot for Virtual Machines

Enhanced Logging

vSphere logs have traditionally been focused on troubleshooting and not “security” or even “IT operations”. This changes in vSphere 6.5 with the introduction of enhanced logging. Gone are the days where you’ll make a significant change to a virtual machine and only get a log that says “VM has been reconfigured”.

We’ve enhanced the logs and made them “actionable” by now sending the complete vCenter event such as “VM Reconfigure” out via the syslog data stream. The events now contain what I like to call “actionable data”. What I mean by that rather than just getting a notice that “something” has changed you now get what changed, what it changed from and what it changed to. This is data that I can “take action” against.

In 6.5, you will get a descriptive log of the action. For example, if I add 4GB of memory to a VM that has 6GB today, I’ll see a log that tells me what the setting was and what the new setting is. In a security context, if you move a VM from the vSwitch labeled “PCI” to the vSwitch labeled “Non-PCI” you will get a clear log describing that change. See the image below for an example.

Actionable Loging

Actionable Loging

Enhanced/Actionable Logging

Solutions like VMware Log Insight will now have a lot more data to display and present but more importantly, more detailed messages mean you can create more prescriptive alerts and remediation’s. More informed solutions help make more informed critical datacenter decisions.

Automation

All of these features will have some level of automation available out of the gate. In future blog articles you’ll see PowerCLI examples for encrypting and decrypting VM’s, enabling Secure Boot for VM’s, setting Encrypted vMotion policies on a VM and a script I used to build an Enhanced Logging demo that you can tweak to show the benefits of Enhanced Logging in your own environment. All of the script example will be released on GitHub.

Wrap Up

That’s it for vSphere 6.5 security! I hope you are as excited as I am about it! More details on each will be forthcoming in blogs and whitepapers. One thing to add is the vSphere 6.5 Security Hardening Guide. This will, as always, come out within 1 quarter after the GA of 6.5. I don’t anticipate major changes to the guide. Features like VM Encryption are not something you should expect in the hardening guide. For more information on the types of information that is now in the guide please reference this blog post.

As always, I appreciate your feedback and questions. You can reach out to me via email (mfoley at vmware dot com) or on Twitter @vspheresecurity or @mikefoley.

mike

To learn more about vSphere 6.5, please see the following resources.

Rating: 5/5