I am using vim to edit/view large files (> 600MB). Unfortunately, it looks like opening such large files even on a Pentium 3.4Ghz with PATA disk takes quite some time (17sec)(ubuntu, kernel 2.6.22-14). I know that vim
creates some internal tree representation of the file, but I wonder if something could be tuned, so that such loading times decrease, even if memory usage would increase.
I tried to open a 767.187.800 bytes file (I created this file by cat-ing all vim src files 100 times ) :
time vim -u ./.vimrc –noplugin -c”:q” ../all3.c
the result is:
real 0m17.670s
user 0m15.457s
sys 0m1.428s
the .vimrc contains only
set noswapfile
I profiled vim, and opening the same file I got the results below….
So the question would be, is it possible to tune sthg. to open faster
these files??
here is the gprof head
XXXXX$ head -40 no_swap
Flat profile:
Each sample counts as 0.01 seconds.
% cumulative self self total
time seconds seconds calls s/call s/call name
46.12 1.96 1.96 1 1.96 4.22 readfile
23.53 2.96 1.00 31025600 0.00 0.00 ml_append_int
12.94 3.51 0.55 31025601 0.00 0.00 ml_updatechunk
8.24 3.86 0.35 31361959 0.00 0.00 ml_find_line
4.94 4.07 0.21 31025600 0.00 0.00 ml_append
0.47 4.09 0.02 1876672 0.00 0.00 mf_put
0.47 4.11 0.02 1655954 0.00 0.00 mf_find_hash
0.47 4.13 0.02 1655954 0.00 0.00 mf_get
opening the same file with grep or with the script below takes on the same machine around 0.25 seconds
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char ** argv)
{
if(argc < 1) {
printf(”file name not provided..\n”);
return -1;
}
int fd = open(argv[1], O_RDONLY);
if(fd < 0) {
printf(”cannot open file %s”, argv[1]);
return -1;
}
int chunk_size = 1000;
if(argc > 2) {
chunk_size = atoi(argv[2]);
}
char *buffer = (char * )malloc(chunk_size);
int res, count = 0;
while( (res = read(fd, buffer, chunk_size) ) > 0) {
count ++;
}
printf(”total read %d*%d=%d”, count, chunk_size, count*chunk_size);
}