jump to navigation

Linked Clones February 28, 2010

Posted by wholmes in Linked Clones.

Below is a compilation of material and my own findings, regarding linked clone technology.

Linked Clone Architecture:

“Copy on write” (COW) disks are the disk structures utilized by Linked Clone technology found in VMware Lab Manager and View Composer.The COW disk starts as an empty (16MB) file and captures all writes to the instance it represents in a sparse format. By reading from both the COW disk and the underlying base disk(s) you get the equivalent of a new disk instance. A multitude of COW disks can reference the same base disk to create many copies of that disk. COW disks can also be built off of other COW disks, so the eventual structure looks like a tree of disks, whose internal nodes are shared. Lab Manager limits the tree to a depth of 30, and there is no limit to the width of the tree. Lab Manager also ensures that all internal nodes are read-only at the application level, and that leaf nodes are the only nodes on the tree that can be “run” so that we don’t destroy dependent COW disks.  All COW operations are block-level operations.

When COW disks are used, their data structures take up ESX kernel memory (the “COW heap”). The default COW heap is 40MB in ESX 3. Most of the COW heap is taken up by “root entries” that use 4 bytes of COW heap for every 2MB of disk represented by COW disks. The result is that we can only open COW disks representing a sum of 20TB (40Mb/4b*2Mb) of base disk on each ESX 3.x server. This includes all nodes in a chain of COW disks, so if you are deploying VMs having a 10-deep chain of COW disks built off 250GB base disks, you could only open 8 of those VMs before running out of COW heap.

In ESX 4.x, the default COW heap has been increased to 192MB, allowing 96TB of disk to be represented (192Mb/4b*2Mb) by linked clones. It is still recommended to keep the chain depth 30 deep or less.


Linked clone performance varies, and at times can perform better than monolithic disks. One reason for potentially greater performance is metadata caching. On VM Startup, metadata dictating which file to access to get data is written to the ESX COW heap . When a VM does a virtual SCSI read and hits the metadata cache, each virtual read will result in a single physical read. However, if there is a ESX cache miss, there will be a virtual read in addition to multiple physical reads for a VM reading across many disk sectors.

Linked clone performance can be further boosted at a lower level in the architecture through storage array caching. This can cause commonly used COW disks to be read from storage array memory cache instead of disk. Ample storage array cache will greatly benefit an environment utilizing linked clones.


Linked Clone creation is supported through VMware Lab Manager, VMware View, and the vSphere Web Services SDK. The following document details Linked Clone creation through the vSphere Web Services


(note, this document incorrectly states a limitation of 8 VMs in a linked virtual machine group. The true limitation is based on the COW heap, noted above)

There are also a number of unsupported scripts available within the VMware community to create linked clones within ESX 3.5, such as the following created by William Lam.


A number of in guest operations can cause large amounts of disk writes to occur, increasing the size of the delta disk and negating space savings. These operations should be limited/avoided in linked clones when possible.

  • Disk-defragmentation
  • system memory-dumps
  • application generated file writes/logs

Reference: http://communities.vmware.com/thread/221114


1. William Lam - March 1, 2010

Hi Wade,

Great article about Linked Clones and thanks for the mention.

Here is also an example vSphere SDK for Perl script for creating Linked Clones using the vSphere API (which is supported by VMware) – http://engineering.ucsb.edu/~duonglt/vmware/vGhettoLinkedClone.html


2. hypervisor - March 1, 2010

As i understand, the COW heap limitation also applies for snapshoted VM ?
Great post BTW !

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: