Web content will not stream

[infidel]

It worked fine before, but as of this past week youtube videos fail to load:

Code: Select all

2010-12-11 17:19:35 [INFO|Engine|New I/O server worker #1-1]: invoking matcher for: http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata
2010-12-11 17:19:35 [INFO|Config|New I/O server worker #1-1]: matching URI: http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata
2010-12-11 17:19:35 [INFO|Profile|New I/O server worker #1-1]: matched YouTube
2010-12-11 17:19:35 [INFO|Action|New I/O server worker #1-1]: setting $video_id to dMH0bHeiRNg
2010-12-11 17:19:35 [INFO|Action|New I/O server worker #1-1]: set $video_id to dMH0bHeiRNg
2010-12-11 17:19:35 [INFO|Action|New I/O server worker #1-1]: getting http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata
2010-12-11 17:19:35 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: GET http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 200; found handler: com.chocolatey.pmsencoder.HTTPClient$_get_closure1_closure3@5d11c3f0
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Parsing response as: text/plain
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Parsed data to instance of: class java.io.InputStreamReader
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: matching content of http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata against &t=(?<t>[^&]+)
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: success
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: setting $t to vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: set $t to vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: trying fmt 37: http://www.youtube.com/get_video?fmt=37&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=37&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@58c3d9ac
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: trying fmt 22: http://www.youtube.com/get_video?fmt=22&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=22&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@2207d8bb
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: trying fmt 35: http://www.youtube.com/get_video?fmt=35&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=35&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@3125ee71
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: trying fmt 34: http://www.youtube.com/get_video?fmt=34&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=34&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@314955ec
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:36 [INFO|Action|New I/O server worker #1-1]: trying fmt 18: http://www.youtube.com/get_video?fmt=18&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:36 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=18&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:37 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@45a8123b
2010-12-11 17:19:37 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:37 [INFO|Action|New I/O server worker #1-1]: trying fmt 5: http://www.youtube.com/get_video?fmt=5&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:37 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: HEAD http://www.youtube.com/get_video?fmt=5&video_id=dMH0bHeiRNg&t=vjVQa1PpcFMLO2bBF5nbFI4NfgaiE0yrlbapqxWLodg%3D&eurl=&el=&ps=&asv=
2010-12-11 17:19:37 [DEBUG|HTTPBuilder|New I/O server worker #1-1]: Response code: 404; found handler: com.chocolatey.pmsencoder.HTTPClient$_head_closure2_closure6@1d450337
2010-12-11 17:19:37 [INFO|Action|New I/O server worker #1-1]: failure
2010-12-11 17:19:37 [FATAL|Action|New I/O server worker #1-1]: can't retrieve stream URI for http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata
2010-12-11 17:19:37 [INFO|Engine|New I/O server worker #1-1]: 1 match for: http://www.youtube.com/watch?v=dMH0bHeiRNg&feature=youtube_gdata

What gives?

I haven't made any recent changes to my setup:

• PS3ms 1.20.409
• MEncoder SVN-r1.0~rc3+svn20090426-4.4.3
• Java 1.6.0_20
• Linux amd64 2.6.32-26-generic [ubuntu lucid]

tried to attach a zip containing PMS.conf+pmsencoder.log+debug.log, but got the following error: "Sorry, the board attachment quota has been reached." Let me know if you want me to post the relevant excerpts.

Thanks in advance.

[chocolateboy]

It worked fine before, but as of this past week youtube videos fail to load

Thanks for reporting this. There's no quick fix, I'm afraid, so you'll need to wait for the next version of PMSEncoder.

[infidel]

Go for it, pmsencoder is indispensable.

[Osiris X]

@chocolateboy

Thanks for your consideration of making the pmsencoder.log file location configurable - it will allow us to manage logrotation of the file on Debian through the .deb package.

I have been experimenting with configuration of pmsencoder and I wanted to know, how did pmsencoder handle URLs with URL escape codes in?
e.g.
http%3A%2F%2Fwww.video.com%2Fvideos%2Fflv%2Ff%2F1%2F0%2Ftest_1.flv%3Fserial%3D1292326988%26h%3D1d106fa95b169a073bef1d626e25
which should evaluate to:
http://www.video.com/videos/flv/f/1/0/t ... ef1d626e25

Does pmsencoder natively handle the conversion?
Is there a command that can be issued in pmsencoder.groovy configuration file to evaluate/convert a URL containing escape codes?

Thanks in advance.

Regards

Os

[chocolateboy]

how did pmsencoder handle URLs with URL escape codes in?

There's no generic conversion because I've never seen escaped URIs in (Media) RSS feeds. They can be handled on a case-by-case basis or in a catch-all profile with (currently):

Code: Select all

profile ('Unescape', replaces: 'Default') { // The 'Default' profile is run first
    pattern {
        match $URI: '^https?%3A%2F%2F'
    }

    action {
        $URI = URLDecoder.decode($URI)
    }
}


Note that the field name (currently $URI) is subject to change. And by "currently", I'm referring to the latest code on GitHub, not to the latest released version (1.1.0).

[infidel]

It worked fine before, but as of this past week youtube videos fail to load

Thanks for reporting this. There's no quick fix, I'm afraid, so you'll need to wait for the next version of PMSEncoder.

I compiled 1.2.0 from git and youtube is working again . Thanks!
Further back in this thread you mention support for external downloaders like get_flash_videos in this version; is this currently implemented and if so how would I go about trying it out?

[chocolateboy]

how would I go about trying it out?

What platform?

[infidel]

Linux (ubuntu lucid), I have get_flash_videos installed.

[chocolateboy]

I haven't tested get_flash_videos yet, but here's what I'm currently using for youtube-dl:

Code: Select all

config {
    def YOUTUBE_DL = '/path/to/youtube-dl'
    def PYTHON = '/usr/bin/python'

    // replace the built-in YouTube profile with one that works for all YouTube-DL sites
    profile ('YouTube-DL', replaces: 'YouTube') {
        pattern {
            match 'YouTube-DL Compatible' // built-in profile
        }

        action {
            $DOWNLOADER = "$PYTHON $YOUTUBE_DL --max-quality 37 -o $DOWNLOADER_OUT ${$URI}".tokenize()
        }
    }
}


Basically, you need to set $DOWNLOADER to a list of strings representing the (full) downloader command line, and you need to send the downloader's output to the $DOWNLOADER_OUT file. You can write the list out explicitly or do it the lazy way with tokenize(), which splits the string into a list of space-separated strings.

Watch out for $URI. When embedded in a string it needs to be" ${$URI}" (rather than "$URI" or "${URI}").

For get_flash_videos, start with something like this (untested) and tweak/correct it:

Code: Select all

config {
    def GET_FLASH_VIDEOS = '/path/to/get_flash_videos'
    def PERL = '/usr/bin/perl'

    profile ('Get Flash Videos') {
        pattern {
            domain $URI, [
                'vimeo.com',
                'megavideo.com' // &c.
            ]
        }

        action {
            $DOWNLOADER = "$PERL $GET_FLASH_VIDEOS --quality high --filename $DOWNLOADER_OUT ${$URI}".tokenize()
        }
    }
}


I'll probably add a 'Get-Flash-Videos Compatible' profile at some stage.

If you need any help, let me know. And if you get it working, please post your profile and I'll add it as an example and/or test to the PMSEncoder tree.

As usual, all of this is subject to change and will probably break in the released version.

[infidel]

Thanks! Will report back.