June « 2009 « Musings

XAM Mandated Fields

In this post I look at what fields are mandated by the SNIA XAM v1.0 specification and write a small Java application to retrieve their default values using the XAM Reference VIM and EMC’s Centera XAM VIM.

What is meant by a field in XAM? According to Section 3.1.5 of the XAM specification v1.0, Part 1, a field is a piece of uniquely identifiable data that can be attached to an XSet, an XSystem, or a XAM Library.

More concretely, a field has a name, a number of attributes that describe how to interact with the object, and a value. Any XAM primary object, i.e. an XSystem, an XSet, or a XAM Library object can contain one or more fields.

Field names are case sentitive UTF-8 encoded strings with a maximum length of 512 bytes and no embedded NULL characters. To avoid namespace clashes, the field namespace is allocated between SNIA, XAM storage system vendors and XAM application vendors. The following table shows the currently reserved namespace for field names: NAMESPACEDESCRIPTION .xam.* The XAM Library-owned portion of the namespace. Fields in this namespace shall be defined in this specification and its follow-ons and shall not be extended by XAM Storage System vendors. .xsystem.*The XSystem-owned portion of the namespace. Fields in this namespace shall be defined in this specification and its follow-ons and shall not be extended by XAM Storage System vendors. .xset.* The XSet-owned portion of the namespace. Fields in this namespace shall be defined in this specification and its follow-ons and shall not be extended by XAM Storage System vendors. .vnd.<reverseDNS>.*The XAM System vendor-owned namespace within the XSystem namespace, where is the XAM Storage System vendor’s reverse DNS name org.snia.*Reserved for SNIA org.snia.xam.* Reserved for SNIA FCWG To avoid field namespace clashes between XAM storage system vendors in the remaining unreserved namespace and aleviate the need for a central XAM field name registry, the first portion of a vendor field name shall be the vendor’s domain name in reverse order, followed by the vendor-defined field name, e.g. com.emc.centera.xam.vim.version.

As mentionly previously a field can have attributes. The following four attributes are mandated by the XAM specification: ATTRIBUTE NAMEDESCRIPTION TypeThe MIME type of the value. The type attribute shall be US-ASCII encoded with a maximum length of 512 bytes. BindingA Boolean value indicating if the field is bound to the XUID of the XSet. ReadonlyA Boolean value indicating

Atahualpa Theme

Welcome to the new home of my blog. I have switched to WordPress 2.8 with the Atahualpha theme. I hope you like the clean simple layout and lines of this theme which, BTW, is garnishing great reviews from the blogging community.

I will probably leave the old blog in place at Blogger for a few months but will not be adding any new posts to it.

XAM Canonical Format

One of the key requirements for achieving long term data persistence is the ability to move data between archiving systems or, in the language of the SNIA XAM (eXtensible Access Method) specification, moving XSets between XSystems.

The XAM v1.0 specification supports this requirement by providing support for exporting and importing Xsets. It specifies the methods used to export an XSet from an XSystem, the resultant XSet canonical data interchange format (package) and the methods used to import an Xset into an Xsystem.

This post assumes that you are somewhat familiar with XAM and how to program to that specification using Java. It focuses on the format and content of the XSet canonical format package which consists of two main parts: an XML document which describes the policies, properties and streams of one or more XSets followed by the binary representation of the stream(s).

The package format conforms to the 2005 W3C XML-binary Optimized Packaging (XOP) recommendation. To quote from the recommendation:

XOP define a general purpose serialization mechanism for the XML Infoset with binary content that is not only applicable to SOAP and MIME packaging, but to any XML Infoset and any packaging mechanism.

If you are unfamiliar with XOP, and most people are, an article by Andrey Butov in the December 2005 issue of Doctor Dobb’s Journal contained a good introduction.

More than one XSet can be contained in a package. However the current XAM SDK reference implementation only supports one XSet. The XML document (AKA the XSet manifest) is a valid and well-formed XML document whose root element is xsets. It can be parsed and manipulated using XSLT and other XML tools. Annex B of the XAM Architecture document contains an XML Schema Definition (XSD) for the XSet manifest.

In order to study the package format in more detail, I wrote a small Java application called StoreHelloWorld which creates a new XSet containing two XStreams. The first Xstream contains the source code for the ubiquitous HelloWorld.java program. The second XStream contains the binary object HelloWorld.class encoded to base64 and with a MIME type of application/base64. Normally you should not encode an XStream but displaying binary files in a blog is problematic and hence the workaround.

Here is the source code for StoreHelloWorld. import java.io.BufferedOutputStream; import java.io.BufferedInputStream; import java.io.BufferedReader; import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.FileNotFoundException; import java.io.FileWriter; import java.io.IOException; import java.io.InputStreamReader; import java.io.InputStream; import

Fedora 11 New Extended File Attributes Namespace

I am experimenting with XAM (eXtensible Access Method), which is a storage standard developed by SNIA (Storage Networking Industry Association and have developed a first prototype of a VIM (Vendor Interface Module) for the ext4 file system based on adding another namespace to the current extended file attributes support. Since other Linux developers might be interested in how to add an extended attributes namespace to a file system, I decided to publish this post as a guide.

Extended file attributes (EA) are extensions to the normal attributes which are associated with inodes in a file system. They are simply name:value pairs associated with files and directories and whose purpose is to provide additional functionality which is either defined by the operating system or a user application. An EA may be defined or undefined. If an EA is defined, its value may be empty or non-empty. Most of the initial work to support EAs in Linux was done by Andreas Gruenbacher in the 2001 to 2004 timeframe and is based on work done in the SGI XFS file system.

EAs are stored directly in inodes and on additional disk blocks if necessary. Currently all EAs associated with a file must fit in an inode and one additional block. Blocks that contain the identical set of EAs may be shared among several inodes. EAs in inodes and on blocks have a different header followed by multiple entry descriptors. Entry descriptors are sorted in disk blocks but are left unsorted in inodes.

EA names are zero-terminated strings and are always specified using a fully qualified namespace.attribute e.g. system.posix_acl_access or security.selinux. Namespaces are used to define different classes of EAs. Different EA classes are required because the permissions and capabilities required for manipulating EAs in one namespace may differ from one to another. Currently Linux supports EA namespaces for SELinux (security.), system (system.), trusted (trusted.) and user (user.).

Fedora 11 comes with EA support in the Linux kernel and hence it is the kernel which must be modified in order to add another EA namespace. In my case I wanted to add an EA namespace called snia in order to support XAM XSet metadata (see below).

Here is the patch file for the Fedora 11 2.6.29.4-167.fc11 kernel which implements the snia namespace. It should work for both 32-bit and 64-bit kernels but I have only tested it on a 64-bit kernel. diff

Fedora 11 nVidia Twinview Support

Fedora 11 (Leonidas) ships with the nouveau nVidia graphics driver preloaded by default if a nVidia graphics card is detected at install time. Previous versions of Fedora used the older X.Org nv driver.

The nouveau project aims at producing Open Source 3D drivers for nVidia graphics cards. According to the nouveau project Wiki

2D-support is in fairly good shape with EXA acceleration, Xv and Randr12 (think of dual-head, rotations, etc.). Randr12 should work for all cards up to, and including, Geforce 9000 series, although some issues with Geforce 8/9 laptops may still exist, for such issues bug reports should be submitted. Randr12 is now the default. Any 3D functionality that might exist is still unsupported, do not ask for instructions to try it. Also, VT switching while X is running is considered lucky.”

Well, I certainly quickly ran into the VT switching issue! It worked but not consistently.

Unfortunately the nouveau driver currently does not support nVidia TwinView functionality and I suspect that it will be a long time before it does if ever!

To use TwinView with Fedora 11, you have to load the correct nVidia drivers from rpmfusion.org. I described how to do this in detail in a previous post so I will not repeat that information here.

You also need to modify your grub.conf file to include the nopat kernel boot option as shown below. title Fedora (2.6.29.4-167.fc11.x86_64) root (hd0,1) kernel /vmlinuz-2.6.29.4-167.fc11.x86_64 ro root=/dev/mapper/vg_ultra-lv_root rhgb quiet nopat initrd /initrd-2.6.29.4-167.fc11.x86_64.img

The nopat option is needed for this particular kernel (2.6.29.4) as it appears to still have broken PAT functionality.

For those readers who are unaware of what PAT is, here is a brief explanation. Traditionally page caching was controlled by a CPU feature called Memory Type Range Registers (MTRR). A CPU has a finite and limited set of MTRRs each of which control part of the physical address space. To overcome this limitation and provide a more flexible architecture, Intel and other x86 CPU vendors added a set of bits to page table entries to control how a CPU does page caching. These bits are called the Page Attribute Table (PAT). Incidentally, the 2.6.26 kernel was the first Linux kernel to support PATs.

Unless you rebuild your initial ramdisk (initrd), the nouveau driver will remain loaded in the kernel. I prefer not to have the nouveau driver loaded in my kernel if I am

Musings

Translate

Archives

XAM Mandated Fields

Atahualpa Theme

XAM Canonical Format

Fedora 11 New Extended File Attributes Namespace

Fedora 11 nVidia Twinview Support

Musings

Translate

Archives

XAM Mandated Fields

Atahualpa Theme

XAM Canonical Format

Fedora 11 New Extended File Attributes Namespace

Fedora 11 nVidia Twinview Support

Tags