c - Why is realloc eating tons of memory? -

this question bit long due source code, tried simplify as possible. please bear me , reading along.

i have application loop runs potentially millions of times. instead of several thousands millions of malloc/free calls within loop, 1 malloc front , several thousands millions of realloc calls.

but i'm running problem application consumes several gb of memory , kills itself, when using realloc. if use malloc, memory usage fine.

if run on smaller test data sets valgrind's memtest, reports no memory leaks either malloc or realloc.

i have verified matching every malloc-ed (and realloc-ed) object corresponding free.

so, in theory, not leaking memory, using realloc seems consume of available ram, , i'd know why , can fix this.

what have this, uses malloc , works properly:

malloc code

void () {     {         b();     } while (someconditionthatistrueformillioninstances); }  void b () {     char *firststring = null;     char *secondstring = null;     char *someotherstring;      /* populate someotherstring data stream, example */      c((const char *)someotherstring, &firststring, &secondstring);      fprintf(stderr, "first: [%s] | second: [%s]\n", firststring, secondstring);      if (firststring)         free(firststring);     if (secondstring)         free(secondstring); }  void c (const char *someotherstring, char **firststring, char **secondstring) {     char firstbuffer[buflength];     char secondbuffer[buflength];      /* populate buffers data tokenizing someotherstring in special way */      *firststring = malloc(strlen(firstbuffer)+1);     strncpy(*firststring, firstbuffer, strlen(firstbuffer)+1);      *secondstring = malloc(strlen(secondbuffer)+1);     strncpy(*secondstring, secondbuffer, strlen(secondbuffer)+1); } 

this works fine. want faster.

now test realloc arrangement, malloc-s once:

realloc code

void () {     char *firststring = null;     char *secondstring = null;      {         b(&firststring, &secondstring);     } while (someconditionthatistrueformillioninstances);      if (firststring)         free(firststring);     if (secondstring)         free(secondstring); }  void b (char **firststring, char **secondstring) {     char *someotherstring;      /* populate someotherstring data stream, example */      c((const char *)someotherstring, &(*firststring), &(*secondstring));      fprintf(stderr, "first: [%s] | second: [%s]\n", *firststring, *secondstring); }  void c (const char *someotherstring, char **firststring, char **secondstring) {     char firstbuffer[buflength];     char secondbuffer[buflength];      /* populate buffers data tokenizing someotherstring in special way */      /* realloc should act malloc on first pass through */      *firststring = realloc(*firststring, strlen(firstbuffer)+1);     strncpy(*firststring, firstbuffer, strlen(firstbuffer)+1);      *secondstring = realloc(*secondstring, strlen(secondbuffer)+1);     strncpy(*secondstring, secondbuffer, strlen(secondbuffer)+1); } 

if @ output of free -m on command-line while run realloc-based test large data set causes million-loop condition, memory goes 4 gb down 0 , app crashes.

what missing using realloc causing this? sorry if dumb question, , in advance advice.

realloc has copy contents old buffer new buffer if resizing operation cannot done in place. malloc/free pair can better realloc if don't need keep around original memory.

that's why realloc can temporarily require more memory malloc/free pair. encouraging fragmentation continuously interleaving reallocs. i.e., doing:

malloc(a); malloc(b);  while (...) {     malloc(a_temp);     free(a);     a= a_temp;     malloc(b_temp);     free(b);     b= b_temp; } 

whereas original code does:

while (...) {     malloc(a);     malloc(b);     free(a);     free(b); } 

at end of each of second loop have cleaned memory used; that's more return global memory heap clean state interleaving memory allocations without freeing of them.