When I first began installing SAP HANA 2.0 on SLES for SAP v15GA, the SAP-provided instructions explicitly said to utilize Btrfs (aka “Butter FS”) for the OS filesytems - that is, filesystems other than /usr/sap, /hana/{data,log,shared}. Basically, SAP repeated the SuSE party line on this topic.
I'll get right to the point: I disagree.
Of course, anyone who read the title of this article already knew that.
Partitions Are Slices, But Not Really
When one reads the SuSE-hosted installation instructions for SLES for SAP, and follows all the links, the path basically ends up at the SLES for SAP v12 documentation. Chapter 6 discusses how the root filesystem should use Btrfs.
Of course, that's not a supported filesystem for HANA 2.0. SAP wisely recommends XFS (or GPFS or NFS for shared-storage situations); ext3 will do in a pinch, they
also say.
In the HANA Scale-Up environment for which I designed my solution, there was no need to share /usr/sap or anything mounted under /hana. Beyond that, I saw no reason to involve Btrfs.
What's Wrong With Btrfs?
The main advantages offered by Btrfs (aside from the fact that it is the default for SUSE) are snap-shotting, device-spanning, and sub-volumes. However, for my environment, all of the systems are VMs/LPARs. Snapshotting and backups are handled at the enterprise storage level. And since the storage is already virtualized, the device-spanning capabilities are also not useful. Finally, sub-volumes didn't address any need that I was unable to address with Linux LVM. In short, the aspects of Btrfs that are generally called “advantages" proved to not be such in my target environment.
Furthermore, exploration of technical issues surrounding Btrfs turned up some
annoying limitations, such as difficulty determining free space, along with its high meta-data overhead (relative to other filesystems). As a CoW (copy-on-write) filesystem, which means that writing a file typically requires twice the space the file nominally occupies until the write is completed and the old copy of the file can be deleted, the general advice is to be sure to allocate plenty of extra storage to btrfs filesystems.
In my environment, Btrfs made no sense, nor did it make sense to use more than one filesystem type. So, very early in the design process, I settled on making my SLES for SAP installs 100% XFS.
What About XFS Downsides?
The single biggest limitation to XFS is that the filesystem cannot be shrunk. Of course, I'm hard pressed to remember when I've seen an enterprise database shrink. They generally just grow, and grow, and grow.....
In more than a year since Go Live, I've never been asked to reduce the size
of /hana/data or any of its friends. Nor have I seen any more reliability problems in XFS than I've seen in any other filesystem.
So my advice is to ignore SuSE and SAP - XFS Uber Alles unless you need a shared filesystem for the database.
Side Note
In the time since I developed my initial design, SAP has revised their guidance to use XFS for everything. Also, the SAP hwacct (“check tool”) no longer requires ntpd on SLES for SAP v15 (the SAP Patterns specifically install chrony). And, if you’re implementing the SLES for SAP v15 High Availability Extensions, the SAP documentation no longer defaults to recommending multicast for totem.
Amazingly, those were all…discussions…I had with the SAP-badged consulting resources our organization brought in for our migration. Now, I can’t prove that I’m the one who explained the error of their ways to SAP, but I know some of our resources had direct back-channels to SAP developers (including to the hwacct developers).
Coincidence? Maybe.