Tips to use PMS in a low memory scenario (<40Mb).

Discuss issues related to PS3 Media Server development (only for programmers)

Tips to use PMS in a low memory scenario (<40Mb).

Postby jochem » Sun Sep 18, 2011 8:11 pm

Because this is my first post on this forum: Thanks to Shagrath and other devs for this terrific program! (And please be nice to this n00b :roll: ).

My use case: "low memory" usage of PMS for my Intel Atom N270/1Gb server (with other tasks to do in the 1Gb), to serve videos from the plugin UitzendingGemist to my media player (WD TV live).

Result: Everything works fine with less than 40M total memory used for Java.

But: My version needed a bit of hacking on the source code. I did not file anything as bug/issue, because on the issue list and forum the devs are generally not working to optimize PMS for low memory type usage. That is a fine choice, but I wanted to use PMS for this type of usage anyhow ;). I post my result on the help/support forum, so other people who are interested in "low memory" usage of PMS can use/enhance my notes.

Warning: only tested with UitzendingGemist.jar and a few output profiles. Uitzending Gemist contains broadcast shows with low resolution WMV video/audio. I did test with vlc,mencoder (transcoding) and mplayer (just dump), all work fine. I did not test stuff like sending local 1080p uncompressed video, but I am (almost) sure that will not work in this setup....

How?

Actually PMS is not very demanding of memory to start with. With mentioned plugin loaded, but without the videobuffer (more on that later), PMS typically uses around 4M on the heap without and 8M with the GUI. Besides the (dynamic) heap, PMS uses around 20M 'PermGen' memory. Java uses this type of memory to load the classes/jvm and static data. This is somewhat high, (presumably) because of the usage of a lot of 3rd party libs.

So if we can keep the buffer low, memory usage is low. Now you can actually configure PMS in the Gui on the Trancoding tab, or in PMS.conf to set a maximum buffer size. I set it to 4M an monitored the program with jvisualm and TRACE logging. No succes yet, after setting the max buffer the actually assigned buffer started with 50M (the hardcoded initial buffer size) and shortly after that raised to 100M.

This is because 100M is the hardcoded lower limit for the maximum buffer, so we need to remove this barrier. In OutputParams.java, remove these lines:
Code: Select all
if (maxBufferSize < 100) {
   maxBufferSize = 100;
}

This is not enough for Mplayer, which looks at a different configuration key for the max buffer, so in MplayerWebVideoDump.java remove these lines:
Code: Select all
   params.minBufferSize = params.minFileSize;
   params.secondread_minsize = 100000;
   params.waitbeforestart = 6000;
   params.maxBufferSize = PMS.getConfiguration().getMaxAudioBuffer();

Add the following settings (on linux in ~/.config/PMS/PMS.conf) for a 4MB buffer:
Code: Select all
minvideobuffer = 1
maxvideobuffer = 4
key_videotranscode_start_delay = 1

And you are done: a lower memory usage for PMS! I use two more tweaks. As said before, the permgen mem is 20MB, which can be lowered a bit (by 5Mb or so) if you are starting the server in headless/console mode anyway. To do this you need to remove from PMS.java:
Code: Select all
import java.awt.GraphicsEnvironment;
import java.awt.Toolkit;

and the places this libraries are used. These are:
Code: Select all
try {
   Toolkit.getDefaultToolkit();
   if (GraphicsEnvironment.isHeadless()) {
      if (System.getProperty(NOCONSOLE) == null) {
      System.setProperty(CONSOLE, Boolean.toString(true));
      }
   } else {
      headless = false;
   }
} catch (Throwable t) {
   System.err.println("Toolkit error: " + t.getMessage());
   if (System.getProperty(NOCONSOLE) == null) {
      System.setProperty(CONSOLE, Boolean.toString(true));
   }
}

and
Code: Select all
if (!headless && displayProfileChooser) {
   ProfileChooser.display();
}

and replace
Code: Select all
if (System.getProperty(CONSOLE) == null) {
      frame = new LooksFrame(autoUpdater, configuration);
   } else {
      System.out.println("GUI environment not available"); //$NON-NLS-1$
      System.out.println("Switching to console mode"); //$NON-NLS-1$
      frame = new DummyFrame();
   }
}

with
Code: Select all
frame = new DummyFrame();

and recompile. Now you have a console only PMS build.

The last thing I did is I hardcode a memory limit for Java, because I dislike the kernel killing other (more critical) processes on my server. This is not needed per se: Java gives back unused memory to the OS anyway. And the downside of these limit is that the streams can crash and that you cannot serve more than one stream simultaneously (with this limit). Of course you can change these values to something a little bit more sane and/or just lower the default memory size (-Xms).

In PMS.sh (on linux) add/replace after line with exec "$JAVA" $JAVA_OPTS
Code: Select all
-Xmx20M -Xss1024k -XX:MaxPermSize=20M

Now you have a PMS server which never uses more than 40Mb memory. Tips or better options are of course welcome...
Last edited by jochem on Mon Sep 19, 2011 12:09 pm, edited 1 time in total.
jochem
 
Posts: 5
Joined: Sun Sep 18, 2011 8:03 pm

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby chocolateboy » Sun Sep 18, 2011 8:55 pm

Sounds like an opportunity for a custom build. :-)
chocolateboy
Project Member
 
Posts: 2580
Joined: Wed Sep 16, 2009 10:05 am

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby jochem » Mon Sep 19, 2011 8:48 am

chocolateboy wrote:Sounds like an opportunity for a custom build. :-)

If there are enough testers for a wide range of scenario's, who knows... But even then I prefer to solve it in the main branch (with configuration flags) because that should absolutely be possible.
jochem
 
Posts: 5
Joined: Sun Sep 18, 2011 8:03 pm

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby Coltaine79 » Mon Sep 19, 2011 3:27 pm

I tend to agree. It would be great if this could be added to the main branch - but with total isolation performed using config flags and such. Can it be done? :)
Coltaine79
 
Posts: 175
Joined: Wed Aug 25, 2010 9:00 pm

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby renszarv » Mon Sep 19, 2011 5:43 pm

Coltaine79 wrote:I tend to agree. It would be great if this could be added to the main branch - but with total isolation performed using config flags and such. Can it be done? :)


I think it's feasible.
renszarv
Project Member
 
Posts: 105
Joined: Sun Aug 21, 2011 7:37 pm

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby jochem » Mon Sep 19, 2011 8:04 pm

Coltaine79 wrote:I tend to agree. It would be great if this could be added to the main branch - but with total isolation performed using config flags and such. Can it be done? :)

The only real necessary fix is removing the hardcoded minimum of the 100M buffer in OutputParams.java. I could maybe file an issue for this? Secondary is the fix for mplayer. (And tertiary the high hardcoded initial buffer size). These patches make theoretical sense for the main branch. However these code is probably added as a pragmatic quick fix for a real world problem (as a lot of issues + patches in the list are). Changing these code would probably give other people their problems back, even if the original quick fixes were not that good in the first place.

As a n00b on this forum, I hesitate a little bit to bring this under the attention of the devs. Unless I can find out why these limits are there in the first place, because then I can test if I don't break any currently working functionality. Has anyone with more experience with PMS an advice for this?
jochem
 
Posts: 5
Joined: Sun Sep 18, 2011 8:03 pm

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby Raptor399 » Tue Sep 20, 2011 5:17 pm

jochem wrote:[...]
Now you have a PMS server which never uses more than 40Mb memory. Tips or better options are of course welcome...


Nice!

For even less memory usage: use UnbufferedOutputFile instead of BufferedOutputFile, as promoted in the network throughput thread.

jochem wrote:
Coltaine79 wrote:As a n00b on this forum, I hesitate a little bit to bring this under the attention of the devs. Unless I can find out why these limits are there in the first place, because then I can test if I don't break any currently working functionality. Has anyone with more experience with PMS an advice for this?

I've studied the code and it's history long and hard, and I am convinced they are merely magic numbers that work well. And since they work well, nobody has ever challenged them or tried to make them configurable.

As is written above, it is fairly easy to retain the defaults, yet make the hardcoded numbers configurable.
It would be a shame to have yet another build only because some hardcoded numbers need to be tweaked (disclaimer: pure conjecture here, I didn't really read/try the stuff above myself and just read the summary, but it seems to revolve about only a few key changes. ;-)).
Raptor399
Project Member
 
Posts: 1916
Joined: Thu Mar 10, 2011 12:06 am

Re: Tips to use PMS in a low memory scenario (<40Mb).

Postby chocolateboy » Wed Jul 25, 2012 11:37 pm

This has come up again (e.g. here).

Raptor399 wrote:It would be a shame to have yet another build


Custom builds are/were a way to test new changes and to get feedback on their utility and stability, in most cases as a precursor to including them in the official build. Also custom builds exist, whereas the chatter about how easily this can be "added to the main branch" has come to naught.

I'm happy to test this if it's in an easy-to-test form. A custom build or a pull request are a good way to get the ball rolling.
chocolateboy
Project Member
 
Posts: 2580
Joined: Wed Sep 16, 2009 10:05 am


Return to Developers

Who is online

Users browsing this forum: No registered users and 4 guests