Libferris – Virtual Filesystem

Updated

What is Libferris?

Libferris is a virtual filesystem with federated index and search.

In non technical terms libferris makes the file system and other hierarchical storage systems easier to use. For the geeks out there, libferris is a virtual file system (VFS) that runs in the user address space.

As of July 2005 libferris can mount many interesting things ranging from a filesystem from your local Linux kernel through to LDAP, Evolution, PostgreSQL, dbXML, and RDF. To get an impression of the current capabilities of libferris mounting see the plugins/context directory of the lastest release.

Other than mounting things as a filesystem, the other core concept of libferris is extraction of interesting metadata from your libferris filesystems. This means that simple things like width and height of an image file become first class metadata citizens along with a file’s size and modification time. The limits on what metadata is available extend far beyond image metadata to include XMP, EXIF, music ID tags, geospatial tags, rpm metadata, SELinux integration, partially ordered emblem categories and arbitrary personal RDF stores of metadata. Though some consider the last point of purely academic interest the end result is that you can add metadata to *all* libferris objects even those you only have read access too, for example, you can attach emblems to this website just as you would a normal file. The metadata interface gives all metadata from file size to digital signature status information equal standing. As such you can sort a directory by any metadata just as easily as you would ls -Sh to sort by file size. Sorting on multiple metadata values is also supported in libferris, you can easily sort your files by mimetype, then image width, then modification time with all three pieces of metadata contributing to the final directory ordering.

Late in 2004 extensive support for both fulltext and metadata indexing was added to libferris. This means you can supply queries against the contents or metadata of any libferris accessable object and have the results returned as a virtual filesystem. With the above mentioned metadata available for searching, finding your files can be done in many different ways instead of being forced to generate fixed directory trees using part of a file collections semantics as directory names. The metadata and virtual filesystem play together here allowing you to geospatially tag both your digital pictures, trip plans, and relevent websites and recall these objects in a single virtual directory no matter what their path or URL may be.

There is also a Samba VFS module which allows you to expose a libferris filesystem as a Samba share. Kfsmd uses the inotify kernel interface to allow libferris to watch changes made to your kernel filesystem by non libferris applications and update its indexes appropriately. Ferriscreate provides a command line and GTK+2 application for creating “new files” with libferris. With this you can create a new db4 database, dbXML database or fulltext index just as easily as you can make a regular file.

The ego filemanager is a GTK+2 interface built on top of libferris. It provides GTK treeview, gevas/edje and gecko based interfaces and makes extensive use of libferris’ clients to provide its functionality.

For the geeks out there, libferris is a virtual file system (VFS) that runs in the user address space. At the moment libferris is a shared object that each application can dynamically link to in order to see the file system through a nicer abstraction.

What can libferris do?

There are optional wrapper libraries allowing Perl, Python and OCaml to access libferris.

The core of libferris is to provide a C++ abstraction over many tree like structures and present the main content of each object in the tree as an IOStream. This core abstraction also provides STL style iterators for a begin() to end() iteration of a directory. Arbitrary metadata for a file is accessable via getStrAttr() and setStrAttr(). Some of the metadata is automatically extracted from files and presented through this same interface, some of the plugins to extract / handle such attributes are listed below.

Many other features like transparent compression of content, xml serialization, metadata indexing, full text indexing, automatic file classification through manchine learning agents are also provided to enhance the VFS.

The following sections describe what things libferris can mount, what sort of metadata is supported and what clients currently exist.

The following clients exist in the tarball, graphical versions use GTK+2:

  • ferriscp, a superset of coreutils ‘cp’ command. A graphical version of cp called gfcp which has progress displays and nice ‘cp -i’ interface.
  • ferrisls, a superset of coreutils ‘ls’ command.
  • ferrisrm, replacement for coreutils ‘rm’ command. A graphical version called gfrm.
  • ferrismv, replacement for coreutils ‘mv’ command. A graphical version called gfmv.
  • fcat, replacement for coreutils ‘cat’ command.
  • ferriscd, bash2 ‘cd’ which handles XPath resolution
  • ftouch, a superset of coreutils ‘touch’ command.
  • fmkdir, replacement for coreutils ‘mkdir command.
  • fclipcopy, fclipcut, fcliplink, fclippaste, fclipredo fclipundo: command line tools for performing file movements. (Cut a file and paste it someplace to “move” that file).
  • findexadd, feaindexadd: clients to add new items to fulltext and EA indexes.
  • findexquery, feaindexquery: clients to query to fulltext and EA indexes.
  • findexcompact, feaindexcompact: clients to compact indexes, performing costly reclaim operations. Such tools are needed to allow index evolution without loosing great deals of query performance.
  • fschema: interaction with attribute schemas
  • fmedallion: querying and setting emblem associations for files.
  • fcompress: setting up and removing transparent (un)compression for files.
  • ferris-import-desktop-file, ferris-import-desktop-hive.sh: adding new freedesktop.org “.desktop” files to the apps:// filesystem.
  • ferris-first-time-user: setting up “~/.ferris”
  • gfdl: download manager using the http and ftp filesystems (which use libcurl).
  • gfproperties: view all the lstat(2) information for a file in a GTK2 GUI. Also presents the emblem attachemnts for a file.
  • FerrisXalanTransform: like XalanTransform except some libferris functions are exported for the XSLT to use (see xsltfunctions/ for more detail).
  • ferris-capplet-auth, ferris-capplet-curl-ftp, ferris-capplet-general, ferris-capplet-index, ferris-capplet-locale, ferris-capplet-logging, ferris-capplet-version: tools for setting up libferris.

Virtual file systems in a virtual data room

A filesystem is hierarchical storage of data adhering to a specific structure. Using virtual data room software simplifies the business document organization, and allows you to customize your filesystem structure according to your needs.