playing with the idea of splitting files, mapping their tree structure to the filesystem, I came to the conclusion that the opposite of the pattern could be more than useful: aggregate filesystem.

What and Why? Just simply imagine a large project with hundreds of files, where you would have to do a huge huge huge huge refactoring that would affect more than 50% of files. (renaming for example). Opening each file is out of question. You could write some sed scripts to do the job. But why? Why you would not be able to open all the files in vi or you favourite editor, first do some analysis by searching for occurrences, then mass replace, etc, etc.

The aggregate filesystem should have some configuration file that would define what is aggregated by listing the files or defining rules (regular expressions for example, globing) about what kind of files to be included (for example all files under a directory). You would end up with a huge file that would be composed by all the files (in random order). End of files of each original file would be marked with a special line, I will call it pivot line. These pivot lines would unambiguously identify the end of file. Removing such a line would mean that the content would flow to the file that follows. The pivot line would also contain the full path of the original file.

When you open an aggregate file and start to read it the aggregate fs would open all the aggregated files (or eventually define other strategies). If you would read the file, the aggregate fs would read one by one all the aggregated files. When saving an aggregated file, the aggregate fs would save each part of the content to the original file indicated by the pivot line.

Such pivot line would also help to create commands inside your editor that would have eventually affect on the original file. For example open the original file.

Such a filesystem could be implemented both in kernel and userspace with fuse.

Well the experience with umview was interesting, but too complicated . Working with GIT_WORK_TREE or core.worktree in .git/config, solves the problem, but you are not “inside” the directory.

***** old post follows ****

Under this name I will try to write a few articles about my design and experience in using git on top of clearcase in order to achive two things:

• make the enterprise happy by keeping the sources in a central repository
• make my work flow much easier by using the power of git and avoiding dead times when waiting for the slow clearcase to do operations like view setup, align with baseline, create branch, merge etc. The goal is to do all necessary development steps fast with git (including collaboration with other developers), and let the computer do the rest of required formalism against clearcase (checkout, “edit”, merge, etc)

At the enterprise where I would like to deploy this design, one of the first handicaps I faced is that the there is no place in the filesystem hierarchy before the clearcase vob, where I could place the .git folder, to map it over the cleracase views. One of the solutions came to my mind is to aske sysadmins to install the fuse module, but my request was refused for system security reasons.

Another alternative seemed to me was to run myself a virtual machine and mount there the vobs under a subfolder of a folder that contains .git. Running whatsoever virtual machine was not a choice do to my access limitations, so only UML could come in question. Uml would mean build a kernel, build a minimal filesystem with necessary tools, etc, etc.

Accidentally however I came across a simpler solution. And that is called virtual square umview. Virtual Square’s “main goal is to create an unified environment that allows virtual machines, systems and networks to communicate and interact”. Umview is a tool that is part of the project. It traps practically any system calls issued by any program that a user runs, and transform them as instructed. For example you can configure it to transform “read /mydiskspace/project.git/clearcase/vob1/module1/src/file1.c” into “read /clearcase/vob1/module1/src/file1.c”. That is to mount virtually “/mydiskspace/project.git/clearcase”  to “/clearcase” inside the virtual environment. Once having that in place you may create the mirroring .git repository in /mydiskspace/project.git. You will feed with source data from the virtually mounted /mydiskspace/project.git/clearcase/vob1/module1/src/file1.c.

How would you concretelly issue your commands? Here are they (see comments)

#>umview $SHELL #start a shell under the virtual environment

#>#you are now in the virtual environment but you have the “same” view

#>#until you start to “manipulate” it

#>um_add_service viewfs #load the module that handles virtual mounts

#>mount -t viewfs /clearcase/vob1/ /mydiskspace/project/git/clearcase

#>cd /mydiskspace/project.git

#>git init  #if this is the first git action

#>git add . #fill the repository with your files from the clearcase view

It is obvious that in the example above you are in the context of a view. You might be able to create wrapper commands around git, that will execute always in this “manipulated” environment view.

In following articles I will describe how I feed the git repo with history from clearcase. To start to work it is not strictly necessary to have the history. It is a nice convenience if you would want to access the history from git.

Note:

• It is mentioned on the umview website that the viewfs module is in early development phase, so it might be a bit buggy. If that does not work for you, try umfuse module
• The umview version that comes with Ubuntu Jaunty does not work well on my test machine, so I had to build one from the sources

However it is stated on the homepage of wikipediafs that the project is not maintained, it works! The only one change one has to make is:

user.py:56:        self.login_page = “%s?title=Special:Userlogin” % self.basename

into

user.py:56:        self.login_page = “%s?title=Special:UserLogin” % self.basename

(note the change from Userlogin to UserLogin)…

and now it is straightforward and easy to edit the wiki with vim… I will write probably a plugin to jump to links automatically. I wish one day there will be fuse fore wince and I will be able to edit wikis with vim on mobile

Please note that wikipediafs is a userspace filesystem and is mounted with help of http://fuse.sourceforge.net.

I did not check if by default the fuse is built in the openmoko kernel, but I came today to the conclusion that I missed a lot of oportunities by not using fuse. I think on an embedded device this could create some interesting opportunities. Even more I was investigation dbus performance, and came to the conclusion that a lot was dbus does, could be done by fuse in a more efficient way…

Something like http://www.runtimeaccess.com/projects/rta/index.html could help programs to publish there data in a filesystem… etc…

Anyway the idea to map databases you use in everyday life to the filesystem, and do things like find/grep index etc on them, is really powerful!!!

I will do all this from now on…

even more… I will mount even things like wikipedia… dictionaries etc.

I hope it will not mean security issues:)

Anyway, the most exciting are:

• Database filesystems ( mysqlfs )
• http://httpfs.sourceforge.net/

Fuse is a way the prolongation of the ideas like reiserfs…