Relational database access is a key component of most of the software applications we use, and this is particularly true for web based applications. Unfortunately as C++ lost a lot of popularity and as Java and scripting languages gained the most of the web applications field, we cannot say that there are lot of open-source C++ frameworks for web development out there. And a good framework for web development means clever sub framework/library for database access.

For one of my projects I was desperately looking for such a framework/library. For the presentation layer I will probably go for Wt, but for database I am a little confused. I will try to compare some of the libraries I found; maybe I will get some valuable feedback from the readers. By writing this it helps me to even better clarify my criteria.

The libraries I found interesting to deeper look at would be:

• Dtl (http://dtemplatelib.sourceforge.net/)
• Otl (http://otl.sourceforge.net/)
• Mysql++
• soci
  

Relational database access is a key component of most of the software applications we use, and this is particularly true for web based applications. Unfortunately as C++ lost a lot of popularity and as Java and scripting languages gained the most of the web applications field, we cannot say that there are lot of open-source C++ frameworks for web development out there. And a good framework for web development means clever sub framework/library for database access.

For one of my projects I was desperately looking for such a framework/library. For the presentation layer I will probably go for Wt, but for database I am a little confused. I will try to compare some of the libraries I found; maybe I will get some valuable feedback from the readers. By writing this it helps me to even better clarify my criteria.

The libraries I found interesting to deeper look at would be:

• Dtl (http://dtemplatelib.sourceforge.net/)
• Otl (http://otl.sourceforge.net/)
• Mysql++

… to be continued…

the only needed libraries for wt are:

• libboost_thread
• libboost_program_options
• libboost_signals
• libboost_date_time
• libboost_filesystem-gcc42
• libboost_regex
• libboost_system

Therefore one should run the configure script for boost as

../boost/configure –without-icu \

–with-libraries=thread,program_options,signals,date_time,filesystem,regex,system \

–prefix=`pwd`/deploy

If you try to build soci-3.0.0 on ubuntu 8.10 you might get some annoying error. I list below the errors and their fixes:

common.h:24: error: ‘std::tm’ has not been declared

fix: add to common.h:

#include <ctime>

errors in file session.cpp:

session.cpp:134: error: ‘strtol’ is not a member of ‘std’

error: ‘INT_MAX’ was not declared in this scope

fix: add to session.cpp

#include <cstdlib>

#include <climits>

in file standard-into-type.cpp:

standard-into-type.cpp:84: error: ‘strncpy’ is not a member of ‘std’

fix:add to standard-into-type.cpp

#include <cstring>

Links related to the soci library:

• http://soci.sourceforge.net/
• http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Relational_Database_Access
• http://www.ddj.com/cpp/188700800?pgno=3
• http://erdani.org/publications/traits.html

Unfortunately c++ did not make it in the world of web applications. There are however experiences to fill the gap:

• http://www.appwebserver.org/
• http://webtoolkit.eu

The first one seems to fit better on low cpu and memory systems, however the second provides much reacher functionality.

I would like to propose a little architectural improvement for the tbbmalloc. (lack of time does not allow me to double-check if what I post here is true:) )

As I described in some of my previous posts on my blog, integrating the tbbmalloc is a little problematic, as tbb relies on malloc. This means that a complete replacement of the malloc with tbbmalloc is impossible.

Custom malloc implementations try to link with the host application before libc, thus implementing the symbols of the malloc family. This linking (in case of shared libraries) can happen at runtime as well.

An elegant combination of malloc with tbbmalloc would be by extending the signature of the tbbmalloc functions by adding an extra formal parameter a pointer to the default malloc family function. Thus all calls in the custom malloc definition could be redirected to the tbbmalloc tegether with a pointer to the pointer the libc (wtih getsymbol etc.). tbb could decide if that allocation his job (small block), if not,redirect the call to the passed function. Eventually if the pointer is null, call malloc itself, allowing thus more flexibility.

so the malloc would look:

void * malloc(size_t size)

{

if(!libc_malloc) //check if libc_malloc symbol found

{

libc_malloc=find_libc_malloc();\

}

return tbb_malloc(libc_malloc,size);

}

Scenario:

legacy application (server) that uses 1Gb of memory.

Using the tbb allocator:

memory usage dropped to 1/3, that is 300MB!

this is not true… the measurement was wrong… however there is a gain of ~17%.

but:

it is impossible to replace completely the malloc, as tbb calls malloc to allocate large blocks used for the small blocks.

(partial) solution:

override the c++ new operator, and redirect it to the tbb allocator.

Recommendation for tbb:

extend the definition of the allocator function with a further formal parameter pointer to the default malloc function.

I was made again aware about Intel’s thread building blocks . The default allocator on hp-ux (pa-risc) allocates 16 byte of memory even for blocks smaller than 8 bytes. If you have a system that allocates hundreds of megabytes of small blocks of usefull memory smaller than 8 byte, this would mean a serious overhead! I tryed hoard (version 2.1.2a) the only one that compiles on hp-ux pa risc, but even hoard allocates 16 bytes.

TBB seems to allocate 8 bytes, but there is a little bottleneck that has to be solved. During initialisation of o multithreaded application the first library to be initialised is libpthread, probably due to some global variable. This requires memory, and if you indirect malloc to tbb allocator, this enters a deadlock, that can be seen on the following callstack:

#0 0xc020ba90 in __ksleep+0x10 () from /lib/libc.2

#1 0xc006023c in __spin_lock+0x10c () from /lib/libpthread.1

#2 0xc0057dc0 in pthread_key_create+0x34 () from /lib/libpthread.1

#3 0xc504cc40 in _Z17initMemoryManagerv () at ../../src/tbbmalloc/MemoryAllocator.cpp:1217

#4 0xc504cda8 in _Z19checkInitializationv () at ../../src/tbbmalloc/MemoryAllocator.cpp:1238

#5 0xc504d25c in scalable_malloc () at ../../src/tbbmalloc/MemoryAllocator.cpp:1343

#6 0xc07cf3b8 in malloc+0x58 () from custom_malloc.cpp

#7 0xc005845c in __pthread_specific_startup+0x20 () from /lib/libpthread.1

#8 0xc00597ec in __pthread_startup+0x1a8 () from /lib/libpthread.1

#9 0xc0027e58 in finish_dld_main+0x1520 () from /usr/lib/dld.sl

#10 0xc00154b8 in _dld_main+0x1c8 () from /usr/lib/dld.sl

#11 0x23b8e08 in __map_dld+0x650 ()

#12 0x23b82dc in $START$+0xd4 ()

#13 0xc006023c in __spin_lock+0x10c () from /lib/libpthread.1

Another ingredient for fast programs is JSON. The JSON format is a concurrent of XML and YAML. The later stays closer to JSON. I found so far two libraries for C++ that can parse and write JSON structures:

• http://jsoncpp.sourceforge.net/
• http://blog.beef.de/projects/tinyjson

… there are even further few listed on the homepage json.org

… There is even a query language jaql