google gold linker

Ian Lance Taylor from google announced already one year ago a new linker which is a complete rewrite. I gave it a try on one of the projects I am working an which contains more than 2 milion lines of C++ code + lot of libraries. It links in 14 seconds, compared to 10 minutes the default linker from binutils-2.17.50.0.6 needs. There is however a patch arond for the default linker that reduces link time for the same executable from 10 minutes to 40 seconds. As I link often, it makes a difference between 40 seconds and 14 seconds…

upstart openmoko

Due to the complex dependencies in the linux boot process the safest way to boot is to run the different initialization scripts sequentially. The result is a long boot process, subsystems are initialized earlier than needed etc.

There are different approaches like upstart, initng, runit, einit that try to improve the boot process on linux, each approach concentrating on several aspects, like running initialization scripts in parallel, handling device generated events in a more flexible way etc. Ubuntu for example comes now with upstart as default init, but it failes to use even at minimum the strength of the ideas behind upstart.

With the intent to speed up the linux process both for desktop and for embedded devices like ubuntu, I started to write some scripts that generates native upstart scripts based on the information from the /etc/init.d/* scripts. As first stage it would be enough to create the native upstart script as wrapper scripts around the /etc/init.d/* scripts.

One could speed up the boot process if initialization scripts could be run in parallel. As we know certain scripts need to have other scripts already started before. To simplify my description I will use the terms target as synonim for the script and dependencies for scripts that need to be started. Having the dependency relation ship we identify two categories of scripts: dependants and dependees. Of course the dependee can be dependant and so on… It is a general requirement that there should be no cyclic dependencies in sysVinit scripts.

The most trivial approach is to create upstart scripts for the dependees containing following

start on starting …. #list those targets that require the current one

pre-start exec /etc/init.d/$MYSCRIPT start

pre-stop exec /etc/init.d/$MYSCRIPT stop

the dependants (targets) would contain sthg. like:

stop on stopping … #list those dependencies that if stopped, the current one should be stopped

pre-start exec /etc/init.d/$MYSCRIPT start

pre-stop exec /etc/init.d/$MYSCRIPT stop

the two above would be merged like:

start on starting …. #list those targets that require the current one

stop on stopping … #list those dependencies that if stopped, the current one should be stopped

pre-start exec /etc/init.d/$MYSCRIPT start

pre-stop exec /etc/init.d/$MYSCRIPT stop

the first version of a python script that would generate such wrappers would be sthg. like… (GPL!!!!)

def createfile(file, startingon):
f = open(”/etc/event.d/upgen/” + file, ‘w’)
f.write(’#file automatically generated by upgen \n\n’)
for dep in startingon:
f.write(’start on starting ‘ + dep + ‘\n\n’)
f.write(’\n\n’)
f.write(’pre-start exec /etc/init.d/’ + file + ‘ start \n\n’)
f.write(’pre-stop exec /etc/init.d/’ + file + ‘ stop\n\n’)
f.close()

def createfiles(startingonmap):
for file in startingonmap:
createfile(file, startingonmap[file])

depfile=’/etc/init.d/.depend.boot’

startingonmap={}

for line in open(depfile, ‘r’).readlines():
#skip lines with
if (line.find(’TARGETS =’) != -1) or (line.find(’INTERACTIVE =’) != -1):
pass
else:
t = line.split(’:')
dependant = t[0]
dependies = t[1].split()
for dependy in dependies:
inversemap = {}
try:
inversemap = startingonmap[dependy]
except:
pass
inversemap[dependant] = 1
startingonmap[dependy]=inversemap

createfiles(startingonmap)

fuse everywhere, even on openmoko

I did not check if by default the fuse is built in the openmoko kernel, but I came today to the conclusion that I missed a lot of oportunities by not using fuse. I think on an embedded device this could create some interesting opportunities. Even more I was investigation dbus performance, and came to the conclusion that a lot was dbus does, could be done by fuse in a more efficient way…

Something like http://www.runtimeaccess.com/projects/rta/index.html could help programs to publish there data in a filesystem… etc…

Anyway the idea to map databases you use in everyday life to the filesystem, and do things like find/grep index etc on them, is really powerful!!!

I will do all this from now on…

even more… I will mount even things like wikipedia… dictionaries etc. :)

I hope it will not mean security issues:)

Anyway, the most exciting are:

Fuse is a way the prolongation of the ideas like reiserfs…