Writing a GEOM GATE module, part 1

For various reasons, including the common good, I'd like to write a short-ish tutorial about writing GEOM GATE modules for FreeBSD. I hope to do this in a series of a few blog posts, dealing with the steps of writing such a module, this being the first post in the series. Finally, I'd like to do it while also creating a usable module, so this will not be an academic excercise but (hopefully) useful work.

I'd like to start with explaing what GEOM and GEOM GATE are.


GEOM

GEOM is the FreeBSD's storage subsystem, a part of the FreeBSD kernel. Anything dealing with storage - which includes (but is not limited to) plain old disk drives, SSDs, USB drives, etc. - goes trough GEOM. The actual storage device drivers make up the "bottom" part of GEOM, and the "top" part can (but doesn't have to) consist of various data transformation classes, such as RAID, encryption, network sharing, etc. GEOM classes form an acyclic graph. The users of GEOM pass data to GEOM, where it is optionally transformed through an arbitrary graph of classes, before being passed down to device drivers.

On top of GEOM are usually file systems, which access their underlying "device" without actually knowing what it is or how it is implemented, or which transformations are used. The GEOM system itself looks like a black box to both file systems on the top side and the device drivers on the bottom side (though there is a way to pass certain important attributes around, such as drive sizes, sector sizes, and storage device properties).

Storage device drivers themselves are a special, degenerate type of GEOM classes, which offer a storage device to GEOM, but don't consume other GEOM devices (as a transformation), instead talking directly to hardware.

GEOM is basically created as a message passing system, and its messages are READ, WRITE, DELETE and GETATTR. READ and WRITE messages are simple and intuitive, carrying information such as the data offset and length and the data buffer to read or write from/to the device. DELETE is a special message by which commands such as SATA TRIM are implemented, and GETATTR is a generic attribute-retrieval message.

GEOM GATE

GEOM GATE (also called ggate) is a GEOM class which offloads GEOM operations to a userland program. It enables userland applications to implement a storage device, effectively dividing the whole mechanism into two parts which are a black box to each other: the kernel GEOM side and the userland application side. Each of these boxes doesn't care what the other does, as long as it behaves properly. GEOM GATE simply exports GEOM messages to the userland and collects the results back into the kernel.

The general architecture of a GEOM GATE module is very simple, and the simplest useful module is ggatel, which resides in /usr/src/sbin/ggate/ggatel on every FreeBSD system with the sources installed.

Such a program basically does some boilerplate setup, and then executes a loop similar to the following:

forever {
gctl_cmd = G_GATE_CMD_START(); // receive message from kernel GEOM
switch (gctl_cmd) {
case BIO_READ:
do_reads();
break;
case BIO_WRITE:
do_writes();
break;
case BIO_DELETE:
do_deletes();
break;
case BIO_GETATTR:
do_getattr();
break;
default:
error = ENOTSUPP;
}
G_GATE_CMD_DONE(error); // return response to kernel GEOM
}

This loop can be found in ggatel.c, starting at the line 100. Of course, the above pseudocode is very simplified.

In the next part, I'll write about the specifics of GEOM and GEOM GATE operation, and analyze ggatel some more.


#1 Re: Writing a GEOM GATE module, part 1

Added on 2012-06-17T22:29 by Jack
Interesting and useful. Ready for the next part!

#2 Re: Writing a GEOM GATE module, part 1

Added on 2012-08-05T23:50 by hpanchas

Thanks a lot for the post.

Its very useful for a complete beginner like me.

Comments !

blogroll

social