out of memory errors are a bug on Linux, not a setting issue

[devsk]

Ok, before you jump on me stating that this has been covered a thousand times, I can assure you that its a bug. I have set large (1536MB) initial and max VM size and it doesn't help. I have 12GB of RAM and 12GB of swap and the error is still hit. I have overcommit enabled (default for Linux), so programs should be able to ask for very large amounts of memory with this system.

And then, I looked at the debug. The issue is that the PS3MS code is asking for free memory, finding that it can't fulfill the transcode buffer request and cribs. Linux doesn't like to leave much RAM free, because the philosophy is to use free RAM for pagecache. We should make the check for total free (free+cache+buffers-unevictable-mlocked-dirty) not just free, and that too only after a malloc() has failed (which won't happen on my system, unless PS3MS wants to malloc more than 24GB). I think the code is doing it the other way around. Its checking for free (and only free) first and halving it if it can accommodate half, cribbing if not. And then doing the malloc.

I think the right sequence should be:

Code: Select all

malloc(bufsize); // equivalent like ByteBuffer.allocate() I guess, whatever the code is doing now
if NULL
{
  total_free = free+cache+buffers-unevictable-mlocked-dirty (gotten from /proc/meminfo)
  if (bufsize/2 < total_free)
    malloc(bufsize/2);
    if NULL
      CRIB;
  else
    CRIB;
}

[devsk]

Ok, I looked at the code and I was wrong about the way memory is being handled. But the following code is buggy:

Code: Select all

                                                } catch (OutOfMemoryError ooe) {
                                                        PMS.minimal("FATAL ERROR: OutOfMemory / dumping stats");
                                                        PMS.debug("freeMemory: " + Runtime.getRuntime().freeMemory());
                                                        PMS.debug("totalMemory: " + Runtime.getRuntime().totalMemory());
                                                        PMS.debug("maxMemory: " + Runtime.getRuntime().maxMemory());
                                                        //System.exit(1);
                                                        PMS.minimal("Not enough memory to allocate " + maxMemorySize + " bytes... Using half of it");
                                                        maxMemorySize = maxMemorySize/2;
                                                        byte[] copy = new byte[maxMemorySize];
                                               try {
                                                   System.arraycopy(buffer, 0, copy, 0, Math.min(buffer!=null?buffer.length:0, maxMemorySize));
                                                   buffer = copy;
                                               } catch (NullPointerException npe) {
                                                   return;
                                               }

I hit the npe exception because it couldn't allocate the half size memory either.

Why would I hit the OutOfMemory above? The heap size is 1568MB (both initial and max) and the java process has 1.6G of RES size. Is there an object leak here?