Potential memory leak in wrapper/API

Oct 4, 2009 at 11:26 PM

I have modified the Xam wrapper as mentioned in this issue http://xamsdk.codeplex.com/WorkItem/View.aspx?WorkItemId=5587, and it has decreased the memory usage dramatically, however as part of load testing I am still experiencing "Out of memory" and general memory consumption issues when inserting many objects. It appears that this problem is further down the stack than the Xam wrapper though.

I have viewed the following situation when using the Xam sdk in 2 different scenarios - once when running load testing using load runner, with the Xam api being called by a web front end to insert files of differing size continually, and again when using a console application inserting files continually.

I have been monitoring the w3wp process (i.e. the IIS/Asp.net process) and the console application using Performance Monitor, looking at the counters "Private Bytes" (unmanaged, or native memory usage), and "# Bytes in all Heaps" (i.e. all memory consumed by the .Net side of the process). In both instances, when a file is stored to the Centera, Private Bytes increases and then decreases, but it does not decrease by the same amount as it increased, and therefore it is increasing over time.

Given that it is the Private Bytes counter that is increasing and not # Bytes in all Heaps, this suggests that the issue lies within the C portion of the API.

I have taken memory dumps of both processes once the memory has increased substantially. I have then carried out diagnostics on both of the memory dumps. In each case, the diagnostic has shown that the heap associated with "fpos32!fp_newTmpfile..." is the largest heap by both committed and reserved memory. It would appear that some memory is being allocated as part of the storing of the file to the Centera and then not being released.

This issue is exhibiting as an "Out of Memory" exception in the load test after approximately half an hour, and when approximately 25000 inserts have taken place of files ranging from 40k to 1MB in size.

Is this a known issue? Does this theory sound correct? Is there any way to resolve this problem?

Note that this problem appears to be most prevalent when dealing with files of a size > 100k - I am working with test files of 10k, 40k, 100k, 400k, 600k, 800k and 1MB, and am finding that memory consumption increases to a point then plateaus with files of size 10k and 100k, but anything larger and memory usage continually increases.

Developer
Oct 7, 2009 at 7:27 PM

Hi,

Could you please go into the file-system properties and report the version number for the following files:  xam.dll, fpos32.dll?

Thanks.

 

Oct 8, 2009 at 1:00 AM

xam.dll        =>    1.0.83.0
fpos32.dll    =>    3.2.281.0

Developer
Oct 8, 2009 at 1:24 AM

Thanks very much for the requested version info.  As is, this is not an issue we are familiar with.  I want to try to reproduce this heap growth in our lab.  Could you either supply the code to the simplest possible console app which exhibits the leak behavior, or describe pseudocode of the process which would exhibit the leak behavior.

Thanks.