I have migrated some data to ZFS filesystems recently and the capacity consumed has surprised me a couple times. In general, it has appeared that the data uses more capacity when stored on the ZFS filesystem. This prompted me to do a little investigating. Is ZFS using more capacity? Is it simply a reporting anomaly? Where is that space going? Does ZFS record size have a major impact? Does enabling compression have a significant impact?

In part, the extra space use is a result of ZFS reporting space utilization differently than other filesystems. When a ZFS filesystem is formatted, almost no capacity is used. A df command will show nearly the entire raw capacity. Many other filesystems take a portion of the raw capacity off the top and reserve it for metadata. This reserve will not show up in df. As data is added to the ZFS filesystem, blocks are allocated for both data and metadata. Both the data and metadata blocks will show up as used capacity. In many other filesystems, at least some of the metadata blocks will be taken from the reserve and only the data blocks will show as consumed capacity. For example, in Solaris, the du command will return the capacity used by the data blocks in a file. In ZFS, that du command returns the total space consumed by the file including metadata and compression. So the question at hand is, when storing a given set of files, does ZFS use more total space than other file systems? That one is difficult to test, given all the variables. But we can test various ZFS configuration options to determine the best settings for minimizing block use.

Read more…

In May, we published a blog entry about Sun’s “T” servers (also known as CMT or coolthreads). Those servers are terrific for some applications but are sometimes an ill-fit for others.  That blog posting was expanded into a full column for ;login: Magazine, which is available to USENIX members. Thanks to USENIX, we are allowed to republish the column to make it available to non-members. You can download the full column here:

  August 2009 Usenix ;login: column (150.9 KiB)

A client invited us to give a presentation at their internal IT conference based on the virtualization whitepaper that we published last month.  The whitepaper is available in this post. Registration is not required, but if you register, we will let you know when the next whitepaper comes out. We have a simple privacy policy and we will not fill your inbox with junk.

The talk went over well, including a lively discussion of the pros and cons of both approaches and how they would fit into the client’s infrastructure.

We are making a .pdf of the talk available today, containing much of the content of the talk.  You can download the talk here: 

  Virtualization Presentation - VMware vSphere vs. Microsoft Hyper-V (659.7 KiB)

Also, if you are in the Northeastern U.S. and are interested in hearing this talk first hand, please get in touch and perhaps we could present the talk at a lunch-and-learn event at your company.

The questions are going to continue here until Oracle officially owns Sun and perhaps beyond. Will Oracle sell the Sun hardware business? As I have said in the past, I do not think they will. I could certainly be wrong and many industry analysts think I am. Here are a few new data points to think about:

1. The rumor mill continues to churn and CNNMoney.com is suggesting that HP may want to purchase the Sun hardware business. HP has the cash, but does the investment make sense? Would Oracle sell Solaris as well? HP would be in a tough position if they bought the hardware but Oracle still owned Solaris. Interestingly, in the article, CNNMoney points Mark Hurd at HP out as the unnamed “Party B” in the Sun regulatory filings.
2. Oracle ran this front page ad in the Wall Street Journal today promoting Oracle DB on Sun SPARC. Is this just Oracle bluffing? Perhaps.
3. If Oracle wants to be in the appliance space, I believe they need to sell general purpose servers. Without the volume that comes from selling general purpose servers, the cost of the appliance platform goes through the roof. Oracle would also have a difficult time getting specialized hardware without paying a premium for a small production run of servers.
4. Oracle and Larry Ellison want to own the IT budget. The “save money on hardware and spend it on Oracle software” go to market strategy  was nothing short of brilliant. Keeping the Sun hardware business is Ellison’s opportunity to compete head to head with IBM. Oracle would have all the applications and the hardware to run it on. That would be quite a legacy for Ellison.

The US Department of Justice as approved the acquisition. Now, the European Union needs to make a decision before we will get any more answers.

We’re pleased to make available our first whitepaper. This one is a technical analysis of  vSphere 4 vs. Hyper-V R2. If you have any comments, please post them here.

The Executive Summary should give you guidance as to whether this whitepaper will be of use to you:

The battle to be your virtualization vendor is in full swing, and it has important ramifications for the vendors involved, and for your data center. The goal of this whitepaper is to analyze the technical aspects of the two major choices: VMware vSphere 4 and Microsoft Hyper-V R2 (as part of Windows Server 2008 R2). This paper considers server virtualization alone, not desktop virtualization or “presentation virtualization”. Certainly presentation virtualization will be an important aspect of the virtualization gamut, but with the entry of Microsoft into the server virtualization market, and the still-unrealized huge potential for server virtualization, this is a topic of great interest to many datacenters.

This whitepaper covers the following topics:
› A summary of virtualization technologies and terms.
› The reasons to consider virtualizing.
› The features of virtualization and the effect it has on application implementation, and datacenter facility implementation and management.
› The impact that future server technology will have in driving virtualization, based on the need of datacenters to achieve optimal resource use and optimal application performance.
› Decision criteria to use in determining when and how to virtualize a datacenter.
› A description and comparison of the features and pricing of vSphere and Hyper-V.
› An analysis of the current state of virtualization and best practices to consider when deploying virtualized infrastructure.
› Our prognosis of the future of virtualization, the expected next feature sets of virtualization, and the future of data centers management and application deployment.
› Advice on how to determine which of the virtualization offerings to consider and how to test that chosen path.
› Reference pointers and suggestions for further reading.

The whitepaper is free and available for download in .pdf format. Registration is not required, but if you register, we will let you know when the next whitepaper comes out. We have a simple privacy policy and we will not fill your inbox with junk.

  Virtualization Whitepaper - VMware vSphere vs. Microsoft Hyper-V (1.1 MiB)

Sun uses three CPUs as the basis for its products: SPARC VI and VII, SPARC T1 and T2, and x86. Choosing the best CPU, in the best system, to solve a problem is more challenging the more choices there are. Frequently, I’ll be asked to recommend a best-fit solution. Sometimes, I’ll need to debug the performance of a system to determine where its bottlenecks are and if it is the best-fit for the workload. Frequently the “T” CPUs are used in the wrong environment, causing users and sysadmins to be unhappy with the provided performance.

In this blog entry I’ll talk about how to determine whether a given workload will run well on Sun’s T servers (the servers that use the T CPUs).

The T servers have one to four sockets. Each socket holds a CPU with up to eight cores. The CPUs currently range up to 1.4GHZ in clock rate. Each core can have eight “hot” threads, in that eight threads can be making progress on the CPU without the system performing a context switch. However, there are not 8 computation engines per core. Rather, each of the eight threads is round-robin scheduled on the core. For details of the architecture of the Niagara CPUs take a look at the Sun Niagara page. An architecture diagram of a single socket of Niagara II CPU is shown here for easy reference.

These T system CPUs are more than just integer units, adding to the expectations of stellar functionality. Each chip also includes eight cryptographic accelerators and eight floating point units, in some configurations the systems also have dual 10-Gb ethernet ports. Finally, Logical Domains, or LDOMS, are an included virtualization technology that allows at the maximum a virtual machine per thread. The T systems have won many benchmarking records, including world record single socket SPEC integer and floating point benchmarks. So what could go wrong?
Read more…

Why is there such a buzz among the analyst, press, and blogging community that Oracle is going to sell of the Sun hardware business? It makes no sense to me. I shared my thoughts on the acquisition in a previous post, but I am going to elaborate a bit here. Not only do I believe Oracle will continue selling Sun hardware, I think it is the primary reason they bought Sun.

Why would Oracle spend $7.4B to buy Sun? Is it for Solaris? I don’t think so. Solaris is open source and Sun would have welcomed Oracle’s help in tuning the operating system for Oracle’s software applications. Is it for Java? That is a little more plausible, but there was no need for Oracle to control Java. As far as I know, Sun was not doing anything to make it difficult for Oracle to use Java. Oracle is buying Sun for the hardware business. The hardware (and support) business is what generates the revenue at Sun.

I would like to share a few relevant quotes. The first comes from Larry Ellison in a recent interview. He did his best to shut down the rumor mill churning on what will happen to Sun’s hardware business.

Interviewer – “Are you going to exit the hardware business?”

Read more…

As promised previously, we’ve posted the second FAQ. The Solaris System Analysis FAQ is now live. The purpose of this FAQ is to provide details on how to determine the cause of a performance, reliability, or functionality problem of a Solaris system. There is a link to the new FAQ in the menu bar on the blog front page. If any of the data is inaccurate, please email. If there is something missing, send the question along and we will take a look at it.

A few weeks ago, it looked like IBM was going to make a deal to purchase Sun. That fell through when the Sun board could not come to agreement. On April 20th, with very little rumor in the marketplace, Oracle announced they were buying Sun for $7.4B in cash. What does this mean for the new company?

• To quote a recent Oracle publication, “Oracle plans to engineer and deliver an integrated system – applications to disk – where all the pieces fit and work together, so customers do not have to do it themselves.” Sun is already shipping Infiniband switches and blades with InfiniBand on the motherboard. They have also mentioned IB is on the roadmap for the Sun 7000. Andy Bechtolsheim mentioned it at the Sun product announcement on April 14th. What about an integrated Oracle appliance running on Nehalem blades, Solaris x64,  Sun 7000 storage, and using Infiniband switches. It should not be a major technology leap to put it all together. What would this mean for the Oracle/HP appliance?
• Sun SPARC processors are at an Oracle pricing disadvantage to IBM Power processors in the current Oracle pricing model. Oracle has never been afraid to use pricing to move the market in their direction. Watch for them to use their pricing model to encourage customer to buy Sun servers.
• Solaris x64 has been intentionally neglected by Oracle. Oracle delivers patches on Solaris SPARC and Linux immediately. Then, they have historically waited up to 6 months to release that same patch for Solaris x64. In the past, this has helped Oracle push their Linux agenda in the marketplace. Given the ease of porting the Oracle patches to Solaris x64, there is no logical technical reason for this lag. Watch for Solaris x64 to become a first class citizen in the Oracle OS support matrix now that growth of Solaris x64 means growth for Oracle.

Read more…

My April 2009 column has been published in ;login: Magazine. This month it’s The Sun Virtualization Guide- making sense of and decisions about the various Sun virtualization options. LDOMs, Containers, Domains and Xen are all options worth considering, and this guide leads you through what each does and when to use them. Some ;login: contents is freely available at ;login:, but my column this month is not one of them. I’ve posted the .pdf here for those without a USENIX membership (although I strongly recommend you get one if you are interested in all things Unix).