%include "default.mgp" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page %nodefault %size 7, font "standard", fore "white", center XFS for Linux %size 4 Christoph Hellwig LST e.V. hch@lst.de %font "typewriter" http://verein.lst.de/~hch/ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page What exactly is a filesystem? Organization of disk space We have a big disk and want to store small data items on them Files and directories used to organize In UNIX everything is a stream of bytes More complex file defintion in other OSes Devices are also presented in the filesystem Linux takes it to the extreme (lots of virtual filesystems) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page So where does XFS fit exactl? Three generation of UNIX filesystems: v7 / sysv / coherent / minix ffs / ext2 vxfs / jfs / xfs (/reiserfs) Features of the 3rd generation UNIX filesystems Intent logging / journaling Flexible metadata structures Dynamic inode allocations Extents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page A little bit of history Back in Stoneage (1993) Berkely FFS was state of the art IRIX had EFS (FFS + Extents) Limitations Small file sizes (2 GB) Small filesystem sizes (8 GB) Statically allocated metadata Long recovery times Very slow operation on big directories No extentended attributes Not very suitable for media streaming %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page All this is addressed by XFS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page XFS Features (1) XFS uses B+ trees extensively instead of linear structures locate free space index directory entries manage file extents keep track of the locations of file index information XFS is a fully 64-bit file system 64bit variables for global counters 64bit disk addresses 64bit inode numbers (not useable under Linux) 18 million terabytes theoretical max filesystem size %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page XFS Features (2) Partitioned into Allocation Groups each AG manages its own free space and inodes provides scalability and parallelism within the file system limits the size of the structures needed to track this information allows many internal pointers to be 32-bits AGs typically range in size from 0.5 to 4GB files and directories are not limited to a single AG. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page XFS Features (3) Sophisticated support utilities fast mkfs (make a file system) dump and restore (utilities for backup) xfsrepair to fix corrupt filesystem xfs_fsr (XFS defragmenter) xfsdb (XFS debug) xfscheck (XFS check) xfs_growfs (enlarged filesystems online) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Porting XFS to Linux (1) - Basics Porting XFS to Linux (1) - Basics Why? Linux had the same issues (in 1999) SGI wants to sell Linux Servers SGI wants to be credible in the OSS Community How? Kernel code is not portable Either rewrite or add a glue layer XFS port started with lots of glue More and more native interfaces used %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Porting XFS to Linux (2) - Glue layers Linvfs Maps Linux file/inode ops to IRIX vnode/vfs ops Nowdays very small Pagebuf Implements an IRIX-like buffercache ontop of the linux pagecache The support/ directory Implements IRIX helpers ontop of linux ones %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Porting XFS to Linux (3) - Mismatches ioctl vs fnctl XFS has many fcntl on IRIX Linux doesn't allow fs-specific fcntls Use ioctls instead Credentials IRIX passed down credentials to the fs Fs has to do access checking by itself Linux does access checks in the VFS Solution: empty struct cred %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Porting XFS to Linux (4) - Refinements Direct use of Linux data structures Passing down dentry struct statfs vs statvfs Duplicate code removal Linux does _much_ more work in common code About 2000 LOC gone Use the generic I/O code Early versions uses pagebuf-based I/O path Now uses generic Linux routines Delalloc was hard to fit into this model %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Volume Manager Integration Linux filesystems traditionally use fixed size I/O requests Makes volume managers a lot easier Too much overhead Linux 2.5 allows variable sized I/O requests Exactly what XFS needs Not properly handled by all drivers for a long time Linux 2.4 needs hacks Guess whether a Volume Manager is used The vary_io extension would help XFS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Interesting XFS Features Direct I/O Allows to perform non-cached, direct to userspace I/O Ported to Linux together with XFS Independent implementation in Linux 2.4.10 XFS ported to generic framework Still advantages over generic implementation Delayed Allocation Very important for XFS performance IRIX buffercache rewritten around it Difficult to fit into 2.4 VM 2.5 way of buffer writeout helps a lot Same concept used on 2.4 now too %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Opensourceing XFS Licensing questions Opensource or not? Community doesn't care about proprtary drivers Filesystem API changes a lot Not GPL-compatible code won't be merged into mainline Enncumbrance review Contact as much as possible original hackers Compare with other codebases (SVR4, BSD, ..) keywords search prototype comparism Very few matches found and corrected usually removal of uneeded code %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page XFS/Linux Release History 2000/03/30 Linux XFS source code officially available 2000/06/23 Usenix 2000 XFS pre-beta iso Image 2000/09/22 XFS Beta Release 2001/05/01 XFS 1.0 Release 2001/07/10 XFS 1.0.1 Release 2001/09/27 Mandrake 8.1 is available with native XFS support. 2001/11/16 XFS 1.0.2 Release 2002/04/17 XFS 1.1 Release 2002/04/18 SuSE 8.0 is available, with XFS filesystem support. 2002/09/16 XFS is merged into Linus' 2.5 development tree. 2003/02/11 XFS 1.2 Release 2003/04/28 XFS is now in Alan Cox's 2.4.21-rc1-ac3 kernel. 2003/07/?? XFS 1.3 Release %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %page Ressources XFS/Linux homepage %font "typewriter" http://oss.sgi.com/projects/xfs/ %font "standard"