PS3 Media Server

by **tsp** » Sun Apr 18, 2010 11:32 pm

Hi I did some work the last week on fixing some issues I had with PS3Mediaserver. It's mainly geared towards the linux/win32 version and has only been tested with a BRAVIA W5500 TV. All patches are against r409/r410

1) BRAVIA doesn't support AVC-video with height<288. The following patch forces transcoding instead of transmuxing for video with height<288. It should be used with the option: mencoder_decode = -vf expand=:288:::1:16\/9 in PMS.conf to ensure that the video is resized if the height is less than 288.

Code: Select all: --- net/pms/dlna/DLNAResource.java +++ net/pms/dlna/DLNAResource.java @@ -709,7 +709,7 @@ // do we have some mpegts to offer ? boolean mpegTsMux = TSMuxerVideo.ID.equals(player.id()) || VideoLanVideoStreaming.ID.equals(player.id()); if (!mpegTsMux) { // maybe, like the ps3, mencoder can launch tsmuxer if this a compatible H264 video - mpegTsMux = MEncoderVideo.ID.equals(player.id()) && ((media_subtitle == null && media != null && media.dvdtrack == 0 && media.muxable + mpegTsMux = MEncoderVideo.ID.equals(player.id()) && ((media_subtitle == null && media != null && media.dvdtrack == 0 && !(mediaRenderer.isBRAVIA() && media.height < 288) && PMS.getConfiguration().isMencoderMuxWhenCompatible() && mediaRenderer.isMuxH264MpegTS()) || mediaRenderer.isTranscodeToMPEGTSAC3()); } --- net/pms/encoders/MEncoderVideo.java +++ net/pms/encoders/MEncoderVideo.java @@ -1011,7 +1011,7 @@ dvd = true; ovccopy = false; - if (params.sid == null && !dvd && !avisynth() && media != null && (media.isVideoPS3Compatible(newInput) || !params.mediaRenderer.isH264Level41Limited()) && configuration.isMencoderMuxWhenCompatible() && params.mediaRenderer.isMuxH264MpegTS()) { + if (!(params.mediaRenderer.isBRAVIA()&&media.height<288) && params.sid == null && !dvd && !avisynth() && media != null && (media.isVideoPS3Compatible(newInput) || !params.mediaRenderer.isH264Level41Limited()) && configuration.isMencoderMuxWhenCompatible() && params.mediaRenderer.isMuxH264MpegTS()) { String sArgs [] = getSpecificCodecOptions(PMS.getConfiguration().getCodecSpecificConfig(), media, params, fileName, subString, PMS.getConfiguration().isMencoderIntelligentSync(), false); boolean nomux = false; for(String s:sArgs) {

2) AVC files that can be transmuxed is registered as MPEG2 files due to media.muxable not being initialized (It's initialized by DLNAResource.isVideoPS3Compatible), removing media.muxable fixes this error. A better solution would be to set media.muxable in mediainfo depending on the renderers capability:

Code: Select all: --- net/pms/dlna/DLNAResource.java +++ net/pms/dlna/DLNAResource.java @@ -709,12 +709,12 @@ // do we have some mpegts to offer ? boolean mpegTsMux = TSMuxerVideo.ID.equals(player.id()) || VideoLanVideoStreaming.ID.equals(player.id()); if (!mpegTsMux) { // maybe, like the ps3, mencoder can launch tsmuxer if this a compatible H264 video - mpegTsMux = MEncoderVideo.ID.equals(player.id()) && ((media_subtitle == null && media != null && media.dvdtrack == 0 && media.muxable + mpegTsMux = MEncoderVideo.ID.equals(player.id()) && ((media_subtitle == null && media != null && media.dvdtrack == 0 && PMS.getConfiguration().isMencoderMuxWhenCompatible() && mediaRenderer.isMuxH264MpegTS()) || mediaRenderer.isTranscodeToMPEGTSAC3()); } if (mpegTsMux) - dlnaspec = media.isH264()&&!VideoLanVideoStreaming.ID.equals(player.id())&&media.muxable?"DLNA.ORG_PN=AVC_TS_HD_24_AC3_ISO":"DLNA.ORG_PN=" + getMPEG_TS_SD_EU_ISOLocalizedValue(c); + dlnaspec = media.isH264()&&!VideoLanVideoStreaming.ID.equals(player.id())?"DLNA.ORG_PN=AVC_TS_HD_24_AC3_ISO":"DLNA.ORG_PN=" + getMPEG_TS_SD_EU_ISOLocalizedValue(c); else dlnaspec = "DLNA.ORG_PN=" + getMPEG_PS_PALLocalizedValue(c); } else if(media != null){

3) TSMuxer does support mp4 containers and mencoder can extract the video and audio stream anyway:

Code: Select all: --- net/pms/encoders/TSMuxerVideo.java +++ net/pms/encoders/TSMuxerVideo.java @@ -60,7 +60,7 @@ public boolean excludeFormat(Format extension) { String m = extension.getMatchedId(); - return m != null && !m.equals("mkv") && !m.equals("ts") && !m.equals("tp") && !m.equals("m2ts") && !m.equals("m2t") && !m.equals("mpg") && !m.equals("evo") && !m.equals("mpeg") //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ //$NON-NLS-6$ //$NON-NLS-7$ + return m != null && !m.equals("mp4") && !m.equals("mkv") && !m.equals("ts") && !m.equals("tp") && !m.equals("m2ts") && !m.equals("m2t") && !m.equals("mpg") && !m.equals("evo") && !m.equals("mpeg") //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ //$NON-NLS-5$ //$NON-NLS-6$ //$NON-NLS-7$ && !m.equals("vob") && !m.equals("m2v") && !m.equals("mts"); //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ }

4) VLC in windows does not work with named pipes without extra backslashes. Streams pointing to a playlist instead of the videosource cause VLC to close the handle to the named pipe and then reopens it. This requires the forced reconnect option to function. Using the .asx or .aspx extension is a crude way to detect playlists.

Code: Select all: --- net/pms/encoders/VideoLanVideoStreaming.java +++ net/pms/encoders/VideoLanVideoStreaming.java @@ -90,7 +90,7 @@ public ProcessWrapper launchTranscode(String fileName, DLNAMediaInfo media, OutputParams params) throws IOException { - PipeProcess tsPipe = new PipeProcess("VLC" + System.currentTimeMillis()); //$NON-NLS-1$ + PipeProcess tsPipe = new PipeProcess("VLC" + System.currentTimeMillis(),Platform.isWindows()&&(fileName.contains(".aspx")||fileName.contains(".asx"))?"reconnect":""); params.input_pipes[0] = tsPipe; @@ -101,7 +101,7 @@ cmdArray[0] = executable(); cmdArray[1] = "-I"; //$NON-NLS-1$ cmdArray[2] = "dummy"; //$NON-NLS-1$ - String trans = "#transcode{" + getEncodingArgs() + "}:duplicate{dst=std{access=file,mux=" + getMux() + ",dst=\"" +tsPipe.getInputPipe() + "\"}}"; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ + String trans = "#transcode{" + getEncodingArgs() + "}:duplicate{dst=std{access=file,mux=" + getMux() + (Platform.isWindows()?",dst=\"\\\\\\":",dst=\"") +tsPipe.getInputPipe() + "\"}}"; //$NON-NLS-1$ //$NON-NLS-2$ //$NON-NLS-3$ //$NON-NLS-4$ if (Platform.isWindows()) { cmdArray[3] = "--dummy-quiet"; //$NON-NLS-1$ cmdArray[4] = fileName;

5) YouTube has yet again changed the format. The following patch should be able to produce a working URL. By default it returns the highest resolution available(1080p,720p or 480p). Should work with the default MEncoder(at least it did for me).

Code: Select all: --- net/pms/dlna/WebVideoStream.java +++ net/pms/dlna/WebVideoStream.java @@ -32,6 +32,7 @@ @Override public InputStream getInputStream(long low, long high, double timeseek, RendererConfiguration mediaRenderer) throws IOException { if (URL.toLowerCase().indexOf("youtube") > -1 && URL.toLowerCase().indexOf("?") > -1) { + //YouTube parsing: http://unlockforus.blogspot.com/2008/09/directly-download-youtube-videos-in.html try { InputStream is = downloadAndSend(URL, false); ByteArrayOutputStream bout = new ByteArrayOutputStream(); @@ -42,42 +43,37 @@ } is.close(); String content = new String(bout.toByteArray()); - int fs = content.indexOf("swfArgs"); - int hd = content.indexOf("isHDAvailable = true"); + int fs = content.indexOf("swfHTML"); String newURL = "http://www.youtube.com/get_video%3F"; if (fs > -1) { - String seq = content.substring(fs+18, content.indexOf("}", fs)); + String seq = content.substring(content.indexOf("<param name=\\\"flashvars\\\"", fs)); + seq = seq.substring(0, seq.indexOf(">")); seq = seq.trim(); - StringTokenizer st = new StringTokenizer(seq, ","); + StringTokenizer st = new StringTokenizer(seq, "&"); while (st.hasMoreTokens()) { String elt = st.nextToken(); - if (elt.startsWith(" \"video_id\"")) + if (elt.startsWith("video_id=")) { newURL += "&video_id%3D"; - newURL += elt.substring(14, elt.length()-1); - if (hd>-1)newURL += "&fmt=22"; - else newURL += "&fmt=18"; + newURL += elt.substring(9); + //if (hd>-1)newURL += "&fmt=22"; + //else newURL += "&fmt=18"; } - else if (elt.startsWith(" \"l\"")) - { - newURL += "&l="; - newURL += elt.substring(6, elt.length()); - - } - else if (elt.startsWith(" \"sk\"")) - { - newURL += "&sk="; - newURL += elt.substring(8, elt.length()-1); - - } - else if (elt.startsWith(" \"t\"")) + else if (elt.startsWith("t=")) { newURL += "&t="; - newURL += elt.substring(7, elt.length()-1); + newURL += elt.substring(2); } newURL = newURL.replace("=", "%3D"); } - URL = newURL; + String[] fmts = {"37","22","18"}; + for(String fmt : fmts){ + URL = newURL+"&fmt%3D"+fmt; + java.net.URL url = new java.net.URL(URL); + int response = ((java.net.HttpURLConnection) url.openConnection()).getResponseCode(); + if(response!=404) + break; + } } } catch (IOException e) { PMS.error(null, e);

by **tsp** » Mon Apr 19, 2010 11:57 pm

6) Sometimes webstreams takes a long time to initiate. Currently the only way to avoid timeouts is to increase the key_videotranscode_start_delay but this affects all players. Another way to fix the problem is to block the main tread until the first bytes are delivered to the outputBufferConsumer or until a timeout occurs. The following patch implements this(UPDATE: there are a subtle bug in the below code on linux with players that closes and then reconnect the output pipe(read VLC). This causes the mutex to be released prematurely in OutputBufferConsumer.run leading to a timeout)( Update2: Bug fixed in the patch). :

Code: Select all: --- net/pms/dlna/DLNAResource.java +++ net/pms/dlna/DLNAResource.java @@ -971,6 +971,7 @@ if (externalProcess == null || externalProcess.isDestroyed()) { PMS.minimal("Starting transcode/remux of " + getName()); externalProcess = player.launchTranscode(getSystemName(), media, params); + externalProcess.waitForOutput(60000); try { Thread.sleep(params.waitbeforestart); } catch (InterruptedException e) { --- net/pms/io/InternalJavaProcessImpl.java +++ net/pms/io/InternalJavaProcessImpl.java @@ -23,6 +23,11 @@ } @Override + public boolean waitForOutput(long timeout) { + return true; + } + + @Override public boolean isDestroyed() { return true; } --- net/pms/io/OutputBufferConsumer.java +++ net/pms/io/OutputBufferConsumer.java @@ -21,23 +21,56 @@ import java.io.IOException; import java.io.InputStream; import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.locks.ReentrantLock; import net.pms.PMS; public class OutputBufferConsumer extends OutputConsumer { private BufferedOutputFile outputBuffer; + private ProcessWrapperImpl pw=null; + private ReentrantLock mutex=new ReentrantLock(); public OutputBufferConsumer(InputStream inputStream, OutputParams params) { super(inputStream); outputBuffer = new BufferedOutputFile(params); } + public OutputBufferConsumer(InputStream inputStream, ProcessWrapperImpl pw, OutputParams params) { + super(inputStream); + this.pw=pw; + outputBuffer = new BufferedOutputFile(params); + + } + + public boolean inputStreamReady(long timeout){ + try{ + if(mutex.tryLock(timeout, TimeUnit.MILLISECONDS)){ + mutex.unlock(); + return true; + } + } catch (InterruptedException e){} + return false; + } + public void run() { try { //PMS.debug("Starting read from pipe"); byte buf [] = new byte [500000]; int n = 0; + mutex.lock(); + if(pw!=null) + pw.setOcReady(); + while(( n=inputStream.read()) < 0){ + try { + sleep(500); + } catch (InterruptedException e) { + } + } + outputBuffer.write(n); + mutex.unlock(); + while ( (n=inputStream.read(buf)) > 0) { //PMS.debug("Fetched " + n + " from pipe"); outputBuffer.write(buf, 0, n); --- net/pms/io/OutputConsumer.java +++ net/pms/io/OutputConsumer.java @@ -40,6 +40,8 @@ } } + public abstract boolean inputStreamReady(long timeout); + public abstract BufferedOutputFile getBuffer(); public abstract List<String> getResults(); --- net/pms/io/OutputTextConsumer.java +++ net/pms/io/OutputTextConsumer.java @@ -39,6 +39,18 @@ this.log = log; } + public OutputTextConsumer(InputStream inputStream,ProcessWrapperImpl pw, boolean log){ + super(inputStream); + linesLock = new Object(); + this.log = log; + if(pw!=null) + pw.setOcReady(); + } + + public boolean inputStreamReady(long timeout){ + return true; + } + public void run() { BufferedReader br = null; try { --- net/pms/io/PipeIPCProcess.java +++ net/pms/io/PipeIPCProcess.java @@ -49,6 +49,11 @@ mkout = new PipeProcess(pipeNameOut, "out", forcereconnect2?"reconnect":"dummy"); } + @Override + public boolean waitForOutput(long timeout) { + return true; + } + public void run() { byte b [] = new byte [512*1024]; int n = -1; --- net/pms/io/ProcessWrapper.java +++ net/pms/io/ProcessWrapper.java @@ -28,5 +28,5 @@ public List<String> getResults(); public boolean isDestroyed(); public void runInNewThread(); - + public boolean waitForOutput(long timeout); } --- net/pms/io/ProcessWrapperImpl.java +++ net/pms/io/ProcessWrapperImpl.java @@ -50,6 +50,7 @@ private boolean nullable; private ArrayList<ProcessWrapper> attachedProcesses; private BufferedOutputFile bo = null; + private volatile boolean ocReady = false; public ProcessWrapperImpl(String cmdArray [], OutputParams params) { super(cmdArray[0]); @@ -72,6 +73,25 @@ attachedProcesses.add(process); } + public boolean waitForOutput(long timeout){ + synchronized(this){ + if(ocReady == false) + try{ + this.wait(timeout); + } catch (InterruptedException e) {} + if(outConsumer!=null) + return outConsumer.inputStreamReady(timeout); + return true; + } + } + + public void setOcReady(){ + synchronized(this){ + ocReady = true; + this.notifyAll(); + } + } + public void run() { ProcessBuilder pb = new ProcessBuilder(cmdArray); try { @@ -87,22 +107,22 @@ outConsumer = null; if (params.outputFile != null) { PMS.info("Writing in " + params.outputFile.getAbsolutePath()); - outConsumer = new OutputTextConsumer(process.getInputStream(), false); + outConsumer = new OutputTextConsumer(process.getInputStream(),this, false); } else if (params.input_pipes[0] != null) { PMS.info("Reading pipe: " + params.input_pipes[0].getInputPipe()); //Thread.sleep(150); bo = params.input_pipes[0].getDirectBuffer(); if (bo == null || params.losslessaudio || params.lossyaudio||params.no_videoencode) { InputStream is = params.input_pipes[0].getInputStream(); - outConsumer = new OutputBufferConsumer((params.avidemux)?new AviDemuxerInputStream(is, params, attachedProcesses):is, params); + outConsumer = new OutputBufferConsumer((params.avidemux)?new AviDemuxerInputStream(is, params, attachedProcesses):is,this, params); bo = (BufferedOutputFile) outConsumer.getBuffer(); } bo.attachThread(this); new OutputTextConsumer(process.getInputStream(), true).start(); } else if (params.log) { - outConsumer = new OutputTextConsumer(process.getInputStream(), true); + outConsumer = new OutputTextConsumer(process.getInputStream(),this, true); } else { - outConsumer = new OutputBufferConsumer(process.getInputStream(), params); + outConsumer = new OutputBufferConsumer(process.getInputStream(),this, params); bo = (BufferedOutputFile) outConsumer.getBuffer(); bo.attachThread(this); } @@ -110,8 +130,11 @@ params.stdin.push(process.getOutputStream()); } stderrConsumer.start(); - if (outConsumer != null) + if (outConsumer != null) outConsumer.start(); + else + setOcReady(); + process.waitFor(); try { @@ -121,6 +144,7 @@ if (bo != null) bo.close(); } catch (Exception e) { + setOcReady(); PMS.error("Fatal error in process starting: ", e); stopProcess(); } finally { --- net/pms/io/ProcessWrapperLiteImpl.java +++ net/pms/io/ProcessWrapperLiteImpl.java @@ -49,6 +49,11 @@ // TODO Auto-generated method stub return false; } + + @Override + public boolean waitForOutput(long timeout) { + return true; + } @Override public void runInNewThread() { --- net/pms/io/WindowsNamedPipe.java +++ net/pms/io/WindowsNamedPipe.java @@ -267,6 +267,11 @@ public ArrayList<String> getResults() { return null; } + + @Override + public boolean waitForOutput(long timeout) { + return true; + } @Override public boolean isDestroyed() {

7) Upgrading to version 1.2 of the H2 database should reduce the memory overhead. It does not seem to affect stability.

by **shagrath** » Tue Apr 20, 2010 11:48 pm

wow, many thanks, there's a lot of interesting things in here

The VLC asx workaround is very clever !

Also, even the regular version suffers from this webstream startup delay.

EDIT: I think I misread it... you just fixed an issue with the asx playlist ?

by **tsp** » Wed Apr 21, 2010 10:40 pm

shagrath wrote:wow, many thanks, there's a lot of interesting things in here
The VLC asx workaround is very clever !

Also, even the regular version suffers from this webstream startup delay.

EDIT: I think I misread it... you just fixed an issue with the asx playlist ?

I fixed the playlist bug mentioned in [url=http://ps3mediaserver.org/forum/viewtopic.php?f=6&t=969&hilit=vlc+webstream}this thread[/url].
And also fixes the problem with VLC version >0.98. The bug fix is mentioned in this thread. I'm not sure how this bugfix affects VLC versions <= 0.98.
VLC is still rather unstable. I think I will try to increase the buffer used by VLC to avoid stalling in the middle of the stream. It happens a lot with YouTube clips.
The web-stream time-out bug should be fixed/improved with patch #6. I also managed to fix the pipe reconnect bug in linux that the first version I posted suffered from. Maybe a similar fix can be used for the windows pipe class like this pseudo-code:

Code: Select all: ConnectNamedPipe(pipename) do{ retval = readfile(pipe) if(!retval){ err = GetLastError() if(err = ERROR_BROKEN_PIPE) { DisconnectNamedPipe(pipe) ConnectNamedPipe(pipename) //deadlock if no reconnection. Another thread will have to manage this } else {fatal error clean up} } }while(!retval) do usual stuff

The while loop is necessary to support more than one reconnection from the client.

by **mastercko** » Fri Aug 13, 2010 10:02 am

Can we please get these patches checked into the trunk? Honestly, that asx playlist one is a killer for me and while I used to be a coder, it's so much work just to get my computer re-setup to be able to patch it myself. Any help would be much appreciated.

by **mastercko** » Wed Aug 18, 2010 7:58 am

ok, set it up anyway (using r410), and started trying to patch, but every tool I've been attempting to use in order to apply the patches throws an error because the white space in these code blocks do not match the white space in the SVN source files themselves. The patch files have spaces and the SVN's files have tabs (which I would usually expect).

Maybe my SVN tool (TortoiseSVN) changed the whitespace around? Hmm, or maybe it's Chrome? Either of these could be the case because when I browse the Google code repository directly, it's also spaces and not tabs. Could you attach the actual patch files you have?

by **maccroche** » Wed Aug 18, 2010 10:38 am

mastercko wrote:ok, set it up anyway (using r410), and started trying to patch, but every tool I've been attempting to use in order to apply the patches throws ....

I tought there was already a build 410 see: viewtopic.php?f=14&t=919&p=31959&hilit=410#p31959 ?????

How to obtain the release with the fixes? I use a Bravia KDL V5500 it those fixes are great!
Thanks

by **chocolateboy** » Wed Aug 18, 2010 6:35 pm

mastercko wrote:ok, set it up anyway (using r410), and started trying to patch, but every tool I've been attempting to use in order to apply the patches throws an error because the white space in these code blocks do not match the white space in the SVN source files themselves.

It's the line endings, not spaces/tabs (see patch's --ignore-whitespace option). The repository is misconfigured - it needs svn:eol-style=native setting on all text files e.g.:

Code: Select all: ack -af | xargs -n 1 -I '{}' svn propset svn:eol-style native '{}'

(Then configure this to be set automatically.)

http://code.google.com/p/ps3mediaserver ... ail?id=856

The workaround is to manually convert the patch(es) and Java files to your native style. In Unix use fromdos and todos (from tofrodos in Ubuntu/Debian) or dos2unix in Cygwin/Windows.

PS3 Media Server

Misc. bug-fixes

Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Re: Misc. bug-fixes

Who is online