Channel plugin (Updated 12-06-23)

[brotherjohn]

Hi SharkHunter,
that looks pretty cool right now with the ARD-Mediathek. I can browse through the folders and see the broadcasts on the XMB.
But there are 2 issues:
1. Every entry appears 4 times on the XMB...so it's really a lot to scroll
2. I tried to play one broadcast and it doesn't work at the moment... but I'm sure you will get it. The debug.log has this entry:

Code: Select all

[Thread-328] INFO  23:47:09.004 HTTP: get/0$8$0$0$2$65/Erlebnis+Erde%3A+Das+Wunder+Leben+%281%29+-+Der+Kampf+ums+Dasein / 0-0
[Thread-328] DEBUG 23:47:09.004 Wrote on socket: HTTP/1.1 200 OK
[Thread-328] DEBUG 23:47:09.004 Searching for objectId: 0$8$0$0$2$65 with children option: false
[Thread-328] DEBUG 23:47:09.004 Wrote on socket: TransferMode.DLNA.ORG: Streaming
[Thread-328] DEBUG 23:47:09.004 Asked stream chunk [0-0] timeseek: 0.0 of Erlebnis Erde: Das Wunder Leben (1) - Der Kampf ums Dasein and player PMSEncoder
[Thread-328] TRACE 23:47:09.004 Starting transcode/remux of Erlebnis Erde: Das Wunder Leben (1) - Der Kampf ums Dasein
[Thread-328] INFO  23:47:09.004 Creating pipe \\.\pipe\pmsencoder_transcoder_out_1294872429004_485
[Thread-329] INFO  23:47:09.006 Waiting for pipe connection \\.\pipe\pmsencoder_transcoder_out_1294872429004_485
[cmd.exe] INFO  23:47:09.234 Starting cmd.exe /C c:\rtmpdump.exe -o - -r rtmp://vod.daserste.de/ardfs/mp4:videoportal/mediathek/Reportage+Dokumentation/c_160000/163884/format178787.f4v?sen=Reportage+%2F+Dokumentation&for=Web-S&clip=Erlebnis+Erde+Das+Wunder+Leben+%281%29+-+Der+Kampf+ums+Dasein | win32\mencoder.exe -prefer-ipv4 -oac lavc -of lavf -lavfopts format=dvd -ovc lavc -lavcopts vcodec=mpeg2video:vbitrate=4096:threads=2:acodec=ac3:abitrate=128 -ofps 25 -cache 16384 -vf harddup -o \\.\pipe\pmsencoder_transcoder_out_1294872429004_485 -
[cmd.exe] INFO  23:47:09.244 Reading pipe: \\.\pipe\pmsencoder_transcoder_out_1294872429004_485
[cmd.exe] INFO  23:47:09.344 Attaching thread: cmd.exe
[Timer-3] DEBUG 23:47:09.344 Buffered Space: 0 bytes / inputs: 0
[Thread-330] DEBUG 23:47:09.346 RTMPDump v2.3
[Thread-330] DEBUG 23:47:09.346 (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
[Thread-330] DEBUG 23:47:09.346 Connecting ...
[Thread-330] DEBUG 23:47:10.327 WARNING: HandShake: client signature does not match!
[Thread-330] DEBUG 23:47:10.327 INFO: Connected...
[Thread-330] DEBUG 23:47:10.329 Der Befehl "amp" ist entweder falsch geschrieben oder
[Thread-330] DEBUG 23:47:10.329 konnte nicht gefunden werden.
[Thread-330] DEBUG 23:47:10.334 Der Befehl "amp" ist entweder falsch geschrieben oder
[Thread-330] DEBUG 23:47:10.334 konnte nicht gefunden werden.
[cmd.exe] DEBUG 23:47:11.334 EOF
[cmd.exe] TRACE 23:47:11.334 Process cmd.exe has a return code of 255! Maybe an error occured... check the log file
[Timer-3] DEBUG 23:47:11.344 Buffered Space: 0 bytes / inputs: 0
[Timer-3] DEBUG 23:47:13.344 Buffered Space: 0 bytes / inputs: 0
[Timer-3] DEBUG 23:47:15.344 Buffered Space: 0 bytes / inputs: 0
[Thread-328] DEBUG 23:47:15.434 Wrote on socket: Content-Type: video/mpeg
[Thread-328] DEBUG 23:47:15.434 Wrote on socket: ContentFeatures.DLNA.ORG: DLNA.ORG_PN=MPEG_PS_NTSC;DLNA.ORG_OP=01;DLNA.ORG_CI=0;DLNA.ORG_FLAGS=01700000000000000000000000000000
[Thread-328] DEBUG 23:47:15.434 Wrote on socket: Connection: keep-alive
[Thread-328] DEBUG 23:47:15.434 Wrote on socket: Server: Windows_7-x86-6.1, UPnP/1.0, PMS/1.20.414-SB11
[Thread-328] DEBUG 23:47:15.434 Wrote on socket:
[Thread-328] DEBUG 23:47:15.434 Sending stream: 0 bytes of get/0$8$0$0$2$65/Erlebnis+Erde%3A+Das+Wunder+Leben+%281%29+-+Der+Kampf+ums+Dasein
[Thread-328] DEBUG 23:47:15.434 Ready to Stop: true
[Thread-328] DEBUG 23:47:15.434 Close Connection

The reason why 'cmd' has got an error is the following

Code: Select all

[Thread-330] DEBUG 23:47:10.329 Der Befehl "amp" ist entweder falsch geschrieben oder
[Thread-330] DEBUG 23:47:10.329 konnte nicht gefunden werden.

and that means "Either the command "amp" is written in a wrong way or it couldn't be found".

[chocolateboy]

and that means "Either the command "amp" is written in a wrong way or it couldn't be found".

The URI ($URI) in the Groovy scripts needs to be quoted (on Windows):

rtmp.groovy

Code: Select all

script {
    profile ('RTMP(E)') {
        pattern {
           protocol ([ 'rtmp', 'rtmpe' ])
        }

        action {
            $DOWNLOADER = "C:\\rtmpdump.exe -o $DOWNLOADER_OUT -r \"${$URI}\""
        }
    }
}


pplive.groovy

Code: Select all

script {
    profile ('PPLive') {
        pattern {
           protocol 'synacast'
        }

        action {
            $HOOK = "C:\\Program Files (x86)\\PPLive\\PPTV\\PPLive.exe \"${$URI}\""
            $URI = 'http://127.0.0.1:8888'
        }
    }
}


Also, to avoid headaches/surprises, it's a good idea to install DOS commands to a directory without spaces e.g.

Code: Select all

C:\Apps\Downloaders\RTMPDump\rtmpdump.exe
C:\Apps\Downloaders\PPLive\PPTV\PPLive.exe


[SharkHunter]

Hi,

brotherjohn you sucessfully found an RTMP link which currently does not work on ARD. So even if you quote the uri etc. this will not work (not now a least). ARD has many (I've seen up to 5) diffrent media links so thats why you see 4 of the same (they are of different quality etc.). This has to be experimented to see which should be kept.

chocolateboy - I need some groovy script expertise here. The problem with the ARD URI is that rtmpdump needs to run it with the -y option as well as the -r option. So what I want to do is split the URI in two (based on a magic token). How to you write that in the groovy script?
The java code for what i want to do:

Code: Select all

uri="\""+uri+"\"";
String[] x=uri.split("magic_token");
if(x.length>1)
  uri=x[0]+"\" -y  \""+x[1];
$DOWNLOADER = "C:\\rtmpdump.exe -o $DOWNLOADER_OUT -r uri


[chocolateboy]

chocolateboy - I need some groovy script expertise here. The problem with the ARD URI is that rtmpdump needs to run it with the -y option as well as the -r option. So what I want to do is split the URI in two (based on a magic token). How to you write that in the groovy script?
The java code for what i want to do:

Code: Select all

uri="\""+uri+"\"";
String[] x=uri.split("magic_token");
if(x.length>1)
  uri=x[0]+"\" -y  \""+x[1];
$DOWNLOADER = "C:\\rtmpdump.exe -o $DOWNLOADER_OUT -r uri


There's no limitation (apart from good taste ) in what can be done in the Groovy scripts, and Groovy is mostly a superset of Java, so the same thing will work:

Code: Select all

action {
    if (whatever) {
        $URI = ...
    } else {
        $URI = ...
    }
}


Also: you might need to declare your String array in Groovy as:

Code: Select all

def stringArray = ... as String[]


- or:

Code: Select all

String[] stringArray = ...


As a general rule, I prefer to distinguish different actions in the pattern block (this will theoretically make it easier to port the DSL to other languages such as JavaScript/Ruby &c.), but that's not required. And I can suggest a "cleaner" version if you get it working with if statements.

[SharkHunter]

Ok I'll try and make a working version and post it here and let you clean up the mess then
Feel free to include any weird scripts into PMSencoder if you like....

[chocolateboy]

Feel free to include any weird scripts into PMSencoder if you like....

I will include your new protocols (in fact, I've added them already (not yet committed)), but I don't want to add too many scripts. They're supposed to be examples, not a standard library. The whole point is for them to be like Greasemonkey userscripts and for anyone/everyone to create their own. In fact, I will probably start removing scripts from that example directory on GitHub once I finish the script documentation (and maybe simplify things with a GUI)...

[SharkHunter]

Here is a quick fix for the ARD rtmp problem. I found a bug in the plugin as which I'll fix tonight (thats what commented out in the groovy script). For now the magic token will be "mp4:" and will hope that no other channel has that within their urls.

Code: Select all

script {
    profile ('RTMP(E)') {
        pattern {
           protocol ([ 'rtmp', 'rtmpe' ])
        }

        action {
            $URI = "\""+$URI+"\""
      //def $X=$URI.split("@@@pms_ch_dash_y@@@") as String[]
      // Temporary fix, split on "mp4:", and then add it back
      def $X=$URI.split("mp4:") as String[]
      if($X.length>1)
         $URI=$X[0]+"\""+" -y \"mp4:"+$X[1]
      $DOWNLOADER = "C:\\rtmpdump.exe -o $DOWNLOADER_OUT -r \"${$URI}\""
        }
    }
}


[SharkHunter]

So now I fixed all the details I needed to fix. I even got a ZDF channel for you germans.
Both ZDF and ARD channels should be updated with more folders I think. But I leave that.

Edit. Fetch from Github https://github.com/downloads/SharkHunte ... v_plug.jar

[brotherjohn]

AWESOME! Really, you've managed it. GREAT, GREAT JOB. Thanks a lot. It works fantastic.
Tack så mycket.

[SharkHunter]

I've made a document on how to create .ch files. It is long and of course to complicated but I hope some one can use it to produce new .ch files.
If you do please publish them either here or send me an PM. It will also be included in the plugin release from today. If ever get it up on github

Code: Select all

This document will try and describe how to construct channel files for the channel plugin
to PMS.
Bundled with the plugin there are some channels already but the .ch files are perhaps not that
human friendly to read so a quick guide to the structure of the files will be given here.

General note!
--------------

* The parser is extremely forgiving for unknown and misspelled keywords. At best it will inform
  you that there is something wrong by logging it, but most of the time it will probably just
  ignore the faulty word and continue.
* The files are line based. It is not possible to split long configurations on more lines then
  one since the parser reads one line at a time.
* If a line starts (after removing leading whitespace chars like space and tab) is #
  that line is removed. This is used for comments. NOTE! That you can only have complete lines
  as comments. You can't have the # at the end or in the middle of a line, if a # is placed
  anywhere else then first on the line it is considered to be part of the line.
 * Empty lines are simply ignored
 
 Textual conventions
 --------------------
 
The following textual conventions are used in this document.
* <xxx> is an indication that some user defined text should be placed ther. The <> should not
  be part of the string (unless you want them to be part of it).
* [xxx] is an indication that the text is optional, again the [] should be removed.
*[<xxx>] is thus an indication an optional user defined text.
* xxx|yyy means that either xxx or yyy is allowed, other values are ignored.
* Text without <> are keywords and must be written extactly as in this document.
* Keywords are not case sensitive (all other text is).
* The words keyword and tag will be used in this document to represent the same thing.
 
 High level file structure
 --------------------------
 
The channels that the plugin will display are placed in "channel files" (or .ch files).
The plugin is configured with a path where it will read (and parse) all .ch files it finds in
that directory. This allows you do separate the channels in to smaller files which are easier
to debug and read. There is however nothing that prevents you from have more than one channel
per .ch file.
The .ch file should follow the following struture:

[version=<ver tag>]
channel <channelname> {
   <channel block>
}
macrodef <macroname> {
   <macro block>
}

At the top level there are three (3) configuration elements (version,macrodef,channel). These
elements does not need to come in any specific order and there are no limit to how many times
they can be present within a file. However, a channel and a maro can only be defined one
time (that is the channelname and maroname can only be used once) and if it is defined agian
it will update the old definition.

Version tag
-------------
Each .ch file can contain a version tag. The version tag is a string that is used by the plugin
to be displayed in the PMS log when the .ch file is parsed. It is not used by the plugin in any
other way and should be used as a debug aid for the user to see that it is the correct version
of the file that has been loaded. A good practice is to set the version at the top of the file.

Channel definition
-------------------
A channel is the top most entity you can configure that will be seen on the XMB (under the
"Channels" folder). The channel definition structure:

channel <name> {
   [format=audio|video]
   [img=<url to image>]
   <block def, see below>
}

The channel name MUST be unique. If the same channel is configured more than once the "last"
configuration will be the one that is the one that is used.

The channel has two (2) properties that can be set.
* format is the format of the data that this channel contains. Can be "audio" or "video",
  the default value is video.
* The img tag is an URL to a nice image that represents the channel. This will be distributed
  by the PMS as thumbnail for the channel.
* There must be at least on block definition (or macro usage) in the channel.

Macro definition
-----------------
Macros are configuration that you probably need to use more than once in your file and you don't
want to copy it to much. A maro is just there to make the .ch file become more readable. A macro
could be defined anywhere in the file and used anywhere in the file. The macros are however bound
to one .ch file. This means that as soon as on .ch file has been completely processed all macros
that were defined in it will be forgotten. Thus, if a mcro should be used for more then one channel
the channels and the macro must be placed in the same .ch file.
The macro definition structure:

macrodef <name> {
   <block def, see below>
}

Block definition
-----------------
The block definition defines the folder structure for the channel. That is how the cahnnel
is divided in folders and where it finds the media etc. All blocks follow the same skeleton
structure that can be stated as:
block keyword {
   block parameters
   more blocks
}

The block parameters don't have to come first in the block but it is a good practice to place them
first. There is no order between the parameters.

There are three type of blocks defined:
folder,item and media.
 
Folder definition
------------------
A folder is well a folder. It contains other folders and/or media. The folder definition:
folder {
   [name=<name string>]
   [matcher=<match expression>]
   [url=<url to fetch page from>]
   [type=atz|empty|normal]
   [order=<see order def, below>]
   [prop=<properties>]
   [macro=<name of macro>]   
   [folder {}]
   [item {}]
   [media {}]
}

The parameters for a folder are:
* name - a descriptive name of the folder. Not needed for normal folder since the name should
  come from the maching.
* matcher - a regular expression matching out entries from the channel. See the matching section
  below for details.
* order - the order of how things are matched. See the macthing section.
* url - the base url of the folder.
* type - type of folder. Defaults to normal.
       * ATZ gives folders named A-Z and then sorts all matches on the page according to starting
         letter.
       * empty - the folder is just a placeholder for retriving the next url.
       * normal - a normal folder.
* prop - special properties for the folder. See the properties section.
* macro - use the macro with the specified name. This means that the macro definition
  in reality is pasted in place of the macro line.

Item definition
-----------------
Items are objects which holds media. Normally they are simple palceholders but they can
hold substantial data as well.
The item definition:
item {
   [name=<name string>]
   [matcher=<match expression>]
   [url=<url to fetch page from>]
   [order=<see order def, below>]
   [prop=<properties>]
   [macro=<name of macro>]   
   [media {}]
}

NOTE! That an item can only hold media and not other items or folder.
The parameters of items have the same meaning as those of folders.

Media definition
------------------
Media is the end result of the channel. This is the actual media stream that will be played.
The media definition
media {
   [name=<name string>]
   matcher=<match expression>
   url=<url to fetch page from>
   [order=<see order def, below>]
   [prop=<properties>]
   [macro=<name of macro>]   
   [type=asx|normal]
}

The parameters of media has the same meaning as those of item and folder. The type parameter can
be set to asx to indicate that the media stream found is an asx file which needs some special
treatment by the plugin.

Matching
---------
The whole purpose of the plugin is to present the web based channels (like SVTPlay) and allow
you browse the progrmas and finally play the media. This is done via some matching on the various
web pages that the play channel provides. This matching is done using regexps. If you need
to brush up your regexp skills go to google and do some searching.
The matcher parameter contains the regexp used to match out the relevant data for a particular
object (folder,item,media). The matching can produce three types of information to the plugin:
url,name and thumbnail. At a minimum a url must be produced, the name and the thumbnail picture
can be excluded. A name is normally also needed since it is hard to find the show you want
if all have no name. This production of data is used regexp "groups" (="()"). Which group in
a match expression that is the url,name or thumb is controlled by the order parameter.

The order is a comma separated list with the names name,url,thumb in the correct order and
combination. A keyword can occur more than once and if that is the case it will be concatenated
with the rest of the same sort.

The matched url is then concatenated with the url parameter to produce a url which corresponds
to the next page where new matches will be made.

Example:
matcher=<a href=\"([^\"]+)\">([^<]+)</a>
order=url,name
url=http://mysite.com/here

This is a very normal matcher. It matches expressions of form <a href="xxxx">yyyy</a>. It
produces two results the first (=xxxx) will be the url part and the second (=yyyy) is the
name. The matched url (=xxxx) will then be appended to the url parameter and the resulting url
(=http://mysite.com/here/xxxx) will be used for the next object as start page. A folder
(or item or media) will be visible on the XMB with the name yyyy.

If the last object in the order list ends with a "+" sign (for example name+) it means that all
the remaining matched groups are of that type.

Example:
matcher=<a href=\"([^\"]+)\">([^<]+)</a>\<\"([^\"]+)\"
order=url,name+
url=http://mysite.com/here

If the url parameter is left out the matched url part (=xxxx in the example) is considered to
be a full url.

Properties
-----------
Each object (=folder,item or media) can have some extra properties set to them. The properties
are added (just like order) as a comma separated list. Some properties have a value which
is then specified as <proerty name>=<property value>. The value part is left out for many
properties. The list of properties:

* url_separator
  name_separator
  thumb_separator:
  These thre poerties all take a value (that is =<value>) and this controls what should be used
  when concatenating multiple matched parts of theat type. For example name_separator=: means
  that if we have two matched groups for name, say a and b, the resulting name will be a:b.
 
* auto_media: The item equivalent of an empty folder. This means that the item will autamtically
  fetch the next page which the should contain the media.
 
* prepend_name: If a name parameter is given the value will be prepended to the matched name.
* append_name: If a name parameter is given the value will be appended to the matched name.
* ignore_name: If a name parameter is given it will be used in stead of the matched one. This
  will also be the case when no name match is found.
* use_conf_thumb: If a thumb url is configured it will be used instead of a matched one.

Magic token
-------------
Some rtmp streams needs to be played by rtmpdump by the use of the -y option. Tó solve this issue
a special url_separator should be used. If the url_separator is !!!pms_ch_dash_y!!! it will
be replace by -y just before given to rtmpdump.

Sample file
-----------
1. version=0.1
2. channel SVTPlay {
3.   img=http://svtplay.se/img/brand/svt-play.png
4.   folder {
5.      name=A-Z
6.      type=ATZ
7.      url=http://svtplay.se/alfabetisk
8.      folder {
9.          matcher=<a href=\"(/t/.*)\">(.*)</a>
10.          order=url,name
11.          url=http://svtplay.se
12.           item {
13.            url=http://svtplay.se
14.              matcher=<a href=\"(/v/.*)\?.*\" title=\"(.*)\" .*?>[^<]*<img class=\"thumbnail\" src=\"([^\"]+)\"
15.              order=url,name,thumb
16.              prop=auto_media
17.            media {
18.                 matcher=url:(rtmp.*),bitrate:2400
19.            }
20.         }
21.      }
22.    }
23. }
The number on the left hand side should not be there and are just ther to make it simple to
walktrough the file in this explanation.
Line 1. A simple version line, the version is 0.1 here.
Line 2. We start defining a channel with the name SVTPlat.
Line 3. We set an img URL which shows some nice picture.
(Note that no format is set so it will be video by default)
Line 4. Start defineing a folder, this will be on the top level.
Line 5. The folder name is A-Z
Line 6. This is an A to Z folder and the plugin will insert the A-Z folders and sort
the matched dat in the correct folder.
Line 7. The URL where this folder should fetch it's pag from is set here.
Line 8-20. Defines a subfolder.
Line 9-10. We matches an url and a name. If this is found on the page:
 <a href="/t/102532/a-ekonomi">A-ekonomi</a>
 Then the matched url is /t/102532/a-ekonomi and the matched name is A-ekonomi.
Line 11. The url part for the match is set here. When combined with the matched url the
 resulting URL will be http://svtplay.se/t/102532/a-ekonomi.
Line 13-20. Define an item.
Line 14. A matcher that produces an url a name and a thumbnail. The matched url will be of
 form /v/xxxx and a resulting URL will be http://svtplay.se/v/xxxx.
Line 16. The proerty auto_media is used to indicate that this item is just placeholder were
 the real media stream url is found.
Line 17-19. Defines a media.
Line 18. This will produce a full rtmp:// url. Since SVTPly has more then one rtmp-stream
 per program we add some matching to only select on (the one with the best bitrate).
Line 19-23. Just end {

Hopefully you can now make your own .ch files (or modify the existing once). If you do there
are most likely other people that wants to view that channel as well so please publish your
channel files on the PMS forum.