Black screen with Mencoder -vc coreavcwindows

[JKross]

Hello,

I am trying to transcode and stream to my PS3 using mencoder with CoreAVC. Sound plays on the TV but the picture is black. I like to think I've done my due diligence research but I'm getting pretty frustrated at this point. Any help would be great. I understand that CoreAVC will only help with the decoding of the original video but at least that task is offloaded to the GPU.

Troubleshooting info
Specs: AMD Athlon 64 X2 4800+, 4gigs ram, NVidia GTX560, Windows 7 x64 Ultimate
Render Device: PS3
Network: (wired) PC -> gigabit switch -> PS3
Transcoder: Mencoder
Software:
CoreAVC 3.0.1
AVISynth 2.6a3
PMS 1.52.1 (build 4c7410179)
NVIDIA 296.10
Java 1.7.0_03

Background:
When I try to play an appropriate video with Windows Media Player 12, my CoreAVC is activated and kicks into CUDA mode (icon turns green). CPU usage is low and the video plays fine. So I'm sure that it is installed correctly for windows at least. I even checked it loading in GraphStudio.

Problem:
I want to stream to my PS3 using the Mencoder engine with the "-vc coreavcwindows" option. I'll use this for subtitled videos and use tsMuxeR for others. I have created a custom codec specific rule in the Mencoder options ( vcodec == h264 :: -vc coreavcwindows ) to watch for h264 content and insert that flag. I have also copied the CoreAVCDecoder.ax file into the PS3 Media Server\win32\codecs directory.

I can see that working as I can see it in the debug.log file:

Code: Select all

DEBUG 2012-04-12 17:31:35.751 [mencoder.exe-3] Starting "C:\Program Files (x86)\PS3 Media Server\win32\mencoder.exe" -ss 0 -quiet "a.h264.1080.mkv" -quiet -quiet -nosound -nosound -of rawvideo -quiet -quiet -mpegopts format=mpeg2:muxrate=500000:vbuf_size=1194:abuf_size=64 -ovc lavc -channels 6 -lavdopts debug=0:threads=2 -lavcopts autoaspect=1:vcodec=mpeg2video:acodec=ac3:abitrate=640:threads=2:keyint=5:vqscale=1:vqmin=1:vrc_maxrate=254000:vrc_buf_size=7000 -quiet -quiet -subdelay 20000 -quiet -quiet -ofps 24000/1001 -quiet -quiet -lavdopts fast -vc coreavcwindows -mc 0 -noskip -o \\.\pipe\1334244695744ffmpegvideo

if I take that and run it in the command line (after changing the -o location) I get the following:

Code: Select all

MEncoder SB29 (C) 2000-2012 MPlayer Team
Custom build by SubJunk, http://www.spirton.com

success: format: 0  data: 0x0 - 0x17db4b5d
libavformat version 54.3.100 (internal)
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (dca), -aid 0, -alang eng
[lavf] stream 2: subtitle (text), -sid 0, -slang eng
VIDEO:  [H264]  1920x1080  0bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1920x1080  fps:23.976  ftime:=0.0417
libavcodec version 54.12.100 (internal)
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
==========================================================================
Forced video codec: coreavcwindows
Opening video decoder: [dshow] DirectShow video codecs
Decoder supports the following YUV formats: YUY2 UYVY YV12 I420
Decoder is capable of YUV output (flags 0x2b)
[PP] Using codec's postprocessing, max q = 4.
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
videocodec: libavcodec (1920x1080 fourcc=3267706d [mpg2])
[VE_LAVC] Using constant qscale = 1.000000 (VBR).
VideoDecoder::SetExtAttr: registry failure
Selected video codec: [coreavcwindows] vfm: dshow (CoreAVC H.264 for x86)
==========================================================================

At this point the CoreAVC icon does show up but it stays blue (ie no CUDA). Mencoder runs and the video file produced also just has a black picture. I'm using a file without subtitles for testing here so I don't think there is any VSFilter issue (although Mencoder doesn't use VSFilter itself?).

The situation is similar when I try to play from the PS3. I select Mencoder in the #Transcode# folder. The video begins playing with sound but only a black screen. If I go back to my PC I can see the CoreAVC Icon but it is blue/non-CUDA.

I have attached the Trace and debug.log output to the post.

I don't want to use Avisynth/Mencoder as I think it is impossible to pass through DTS and other audio in the resulting stream. I have an AV Receiver linked to the PS3 that it passes off the audio decoding to via HDMI. Avisynth also produces video stutter (I've tried 2.5.7 (won't play), 2.5.8(stutters), and 2.6a3(stutters) ) so it's not really an option for me.

What am I missing or making assumptions about?

I have also tried placing the dsnative.dll from another mplayer distribution in the mencoder.exe folder out of pure optimism ... but that didn't seem to make a difference.

Any help would be great.

Thank you

[ExSport]

Never thought that SubJunk incorporated "DirectShow Native Wrapper" (from Sherpya) in his custom SB builds Subjunk?
http://oss.netfarm.it/dsnative/
Try your command with Sherpya build (and yes, you need dsnative.dll)

CoreAVC Video Decoder 1.9.5 dsncoreavc CoreAVCDecoder.ax Working Needs to be purchased

Be sure to use dsncoreavc instead of coreavcwindows!
viewtopic.php?f=2&t=10324&p=52409#p52409

[JKross]

I can confirm that using MEncoder Sherpya-SVN-r34401-4.6.2 with "-vc dsncoreavc" from the command line produces a legitimate video file. CoreAVC pops up and CUDA is activated unlike with SubJunk build. The command line output using the same command as in the original post is:

Code: Select all

MEncoder Sherpya-SVN-r34401-4.6.2 (C) 2000-2011 MPlayer Team
SSE supported but disabled
SSE2 supported but disabled
181 audio & 389 video codecs
success: format: 0  data: 0x0 - 0xfe284a2a
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (dca), -aid 0
[lavf] stream 2: subtitle (text), -sid 0, -slang eng
[lavf] stream 3: subtitle (text), -sid 1, -slang eng
VIDEO:  [H264]  1920x1080  0bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1920x1080  fps:23.976  ftime:=0.0417
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
==========================================================================
Forced video codec: dsncoreavc
Opening video decoder: [dsnative] DirectShow video codecs (native win32)
Movie-Aspect is 1.78:1 - prescaling to correct movie aspect.
videocodec: libavcodec (1920x1080 fourcc=3267706d [mpg2])
[VE_LAVC] Using constant qscale = 1.000000 (VBR).
Selected video codec: [dsncoreavc] vfm: dsnative (CoreAVC Video Decoder)
==========================================================================

I then replaced all the mencoder files in my PMS installation directory with Sherpyas version and played it from the PS3. The video plays as expected with sound and audio. When I go back to my PC to look, the CoreAVC icon is there and is green showing CUDA is in use.

I also noticed that the Sherpya version includes the codecs.conf file in the mplayer subdirectory unlike the SubJunk build. This is the file in which the various -vc targets are defined ( coreavcwindows, dsncoreavc, etc. ). Is this file built into the SubJunk mencoder?

I remember seeing a post by SubJunk saying that he might include support for the dsnative.dll ( Aug 16, 2011 http://www.ps3mediaserver.org/forum/viewtopic.php?f=14&t=8883&start=1290#p55484 ) which is why I tried to just put it into the PS3 Media Server\win32folder initially.

I can now see from the Sherpya codecs.conf file that there needs to be a specific driver setting to enable dsnative for mencoder:

Code: Select all

;=============================================================================
;                   DSNATIVE VIDEO CODECS
;=============================================================================

videocodec dsncoreavc
  info "CoreAVC Video Decoder"
  status working
  fourcc H264,h264
  fourcc X264,x264
  fourcc avc1 AVC1,AVC1
  fourcc davc,DAVC
  fourcc VSSH
  format 0x10000005
  driver dsnative <----------*********
  dll "CoreAVCDecoder.ax"
  guid 0x09571a4b, 0xf1fe, 0x4c60, 0x97, 0x60, 0xde, 0x6d, 0x31, 0x0c, 0x7c, 0x31
  out YV12,IYUV,I420,YUY2

I reset back to the original PMS mencoder files, and then included the dsnative.dll as well as Sherpyas codecs.conf file in the PS3 Media Server\win32\mplayer directory with the -vc dsncoreavc flag but it tells me that the dsnative support had to be enabled at compile time:

Code: Select all

MEncoder SB29 (C) 2000-2012 MPlayer Team
Custom build by SubJunk, http://www.spirton.com

181 audio & 389 video codecs
success: format: 0  data: 0x0 - 0x17db4b5d
libavformat version 54.3.100 (internal)
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (dca), -aid 0, -alang eng
[lavf] stream 2: subtitle (text), -sid 0, -slang eng
VIDEO:  [H264]  1920x1080  0bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:44  fourcc:0x34363248  size:1920x1080  fps:23.976  ftime:=0.0417
libavcodec version 54.12.100 (internal)
Opening video filter: [expand osd=1]
Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
==========================================================================
Forced video codec: dsncoreavc
Requested video codec family [dsncoreavc] (vfm=dsnative) not available.
Enable it at compilation.
Cannot find codec matching selected -vo and video format 0x34363248.
==========================================================================

So I guess the dsnative support never made it in?

End result: I can now see that I could replace SubJunks custom mencoder build with Sherpyas in order to use CoreAVC with mencoder in PMS. Do I need to do this or have I still missed something?

I'd obviously strongly prefer to use just the PMS package as that makes maintaining it much easier when new versions come out. I also see from browsing back through the forums that PMS used to use Sherpya before SubJunk began packaging a custom build. Again, I don't want to involve AviSynth as that causes issues with video stutter and loss of audio pass through.

Am I missing something to make CoreAVC work using the original SubJunk package?

[ExSport]

Ask Subjunk and request support for it!
Codecs.conf is compiled into MEncoder.exe file in both SubJunk and Sherpya versions. If file also exists in codecs folder, it has the precedece.
I suppose Sherpya wrapper is opensource? so it should not be so hard for SubJunk to incorporate it.
I tested DiAVC/CoreAVC/FFDShow CPU/GPU decoding in past and the best one was DiAVC(fastest) also it uses CPU only
Unfortunately it had some picture bugs and reported them also to developer but never tested new versions if it was ever fixed

[JKross]

I have messaged SubJunk about support for dsnative or other means to get CoreAVC working with his build through the command line flags and will see what comes of that.

I read your testing/comparison of the various decoders but CoreAVC wins for me because: It offloads work onto the GPU (it doesn't help me if DiAVC is fastest in software if it takes up CPU cycles that are needed for encoding ... the poor CPU struggles enough)

[ExSport]

You are wrong.
That decoding is done by GPU is not relevant.
If benchmark say that final encoding is faster with DiAVC, it means it is also faster than CoreAVC with CPU or GPU decoding.
What I remember when I tested CoreAVC with GPU, it was slower than with CoreAVC CPU decoding. Yes, in this case with CPU decoding you will see 100% CPU used and when GPU used, only 80% used but in final it is slower so using CPU was faster.
It seemed like piping decoded frame from GPU to MEncoder had bigger overhead than when CPU decoding was used. So also when lower CPU usage was visible with GPU, final encoding time was slower
I will wait for SubJunk dsnative support and will try to re-test it again with latest DiAVC/CoreAVC/FFDSHOW versions.

[JKross]

I understand your point that a pure software solution that loads the CPU at 100% can have faster end to end decoding/re-encoding than involving the GPU. I understand that the CPU loading may only be at 80% using CoreAVC because Mencoder is spending time waiting for the decoded frames to become available from the GPU.

However, I do think using the GPU is relevant to my situation. The PC will not just be used for media viewing and having it loaded at possibly only 80% frees it to do other things while a video may be transcoding. And this is only going to happen when I'm using Mencoder transcoding to watch subtitled videos. At other times I'll stick with the much less intensive TsMuxeR option.

Having said that, that is just a feeling I have right now. I will have to test to see how responsive or not the machine actually becomes in that situation. I do not know how cleverly the OS might schedule other processes on the CPU while transcoding is happening and whether the performance will be acceptable at those times.

It's not just about pure speed, although that's always a plus. I'm trying to make best uses of the resources that the machine has. Of course I can't even really make any actual experience comparisons until dsnative support becomes available in SubJunk... although I can investigate using Sherpya in the mean time.


I would be interested in seeing new comparison results of the various options when that is possible using SubJunks builds. Until then Sherpya is always an option.

[ExSport]

I agree. If also another work is done on PC, not using 100% of CPU is benefit. For others who have problems with stuttering or streaming to more renderers, using special filteres(upscaling, deinterlacing,...), sometimes it is a must to use fastest option

[ExSport]

SB31 uploaded which supports DSNATIVE library
Windows only:

dsncoreavc............dsnative............working............CoreAVC Video Decoder [CoreAVCDecoder.ax]
dsndivxh264...........dsnative............working............DivX H.264 Decoder [DivXDecH264.ax]
dsnffdshow............dsnative............working............ffdshow Video Decoder [ffdshow.ax]
dsnffdshowdxva......dsnative............problems............ffdshow DXVA Video Decoder [ffdshow.ax]
dsndiavc...............dsnative............working............DiAVC H.264 Decoder (DiAVC) [diavc.ax]
dsncineformhd.......dsnative............working............CineForm HD [CFDecode2.ax]

Example:
... -vc dsndiavc, ... added in PMS GUI or in .conf files (CustomMencoderOptions set in renderer.conf etc...)
Don't miss COMMA! It says if decoder is not available(on system or for decoded video codec) it continues to load next one in row (MEncoder default one) so decoding will not fail

[ExSport]

Tested CoreAVC 3.0.1 with new mencoder. Unfortunatelly I tested on NTB which uses DXVA2 which is disabled when MEncoder loads driver. Can't test it on NVIDIA with CUDA right now.
What I spotted is that:

• "Force VMR AR correction enabled" will hang MEncoder
• "Low Latency enabled" produces video without any skipped frame but it was little bit slower
• "Low Latency disabled" produces video with some skipped frames(always different ones when tested x times) at faster speed...have to say I didn't spotted any picture difference from test above or A/V sync problems
• when dsnative wasn't used (default MEncoder ffmpeg decoder), output was similar with "low latency enabled" => no skipped frames, but speed was similar with "low latency disabled" testing

Sample was very short - 60sec (720p) but first one took about 20s encoding time, second one 18s(10% faster) and MEncoder default decoder took about 18-19s (all with SW decoder as mentioned above)
Need to test bigger sample if SW decoding with CoreAVC/DiAVC is faster than included decoder in MEncoder or not