Thumbnail support for photos (RealFile.java)

Discuss issues related to PS3 Media Server development (only for programmers)

Thumbnail support for photos (RealFile.java)

Postby patters » Tue Aug 24, 2010 10:21 am

I have a Bravia 5500 and 2010 series Sony Bluray player (BDP-S370). I code VBscript in my work, including some stuff that modifies XML so I was quite curious to learn more about PS3MS. So, I've read the Java primer over at Java.com and I've rolled my sleeves up and dived in...

I noticed that Windows Media Player is able to display thumbnails on the BDP-S370 for photos. PS3MS does not show thumbnails for anything on this renderer. I compared wireshark traces of this to PS3MS and I see that Windows Media offers up several sizes of image: JPEG_LRG JPEG_MED JPEG_SM JPEG_TN (example here). It advertises the resolution of each and presumably the server scales the image by reading the parameters POSTed to the URL (identical in each case). Since WMP seems to have excellent compatibility, emulating its behaviour as far as possible would seem like a good idea. I gather that some renders may choose to use the JPEG_MED or JPEG_SM images as thumbnails for instance.

I spent some time looking at the source and I discovered the hack for the condition isBravia()=true in net/pms/dnla/DNLAResource.java. I can see in the source comments that Shagrath was aiming to remove these hacks so I created a new boolean value in the renderer conf called ForceJPGThumbnails (modifying net/pms/configuration/RendererConfiguration.java and net/pms/dlna/DLNAResource.java). Setting this to true gets thumbnails to work for Web image feeds, though they don't fill the available icon size like WMP's thumbnails do.

I carried on, hoping to find some thumbnail size code. Then I discovered that in net/pms/dlna/FeedItem.java the RSS feed's own image thumbnail is used, hence there is no specified size.

Finally I noticed a comment in the code that image thumbnail support is disabled for physical photo files (not yet coded - see line 423 of net/pms/dlna/RealFile.java).

So my question is, before I potentially spend a substantial amount of time trying to implement an image scaler and learning Java (this method seems a likely fit) is there some very good reason nobody implemented this yet? By my logic image thumbnails are much more important than, say, video thumbnails. The filename of a video file is often very descriptive and the thumbnail has almost no chance of being representative, whereas for a photo it's much more useful.
Last edited by patters on Wed Sep 08, 2010 9:16 am, edited 1 time in total.
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby tomeko » Sun Sep 05, 2010 7:50 pm

There is a little patch (code from http://www.rayinblu.com/ps3mediaserver). convert.exe from http://www.imagemagick.org/script/binary-releases.php is required (must be located inside PS3 Media Server\win32 folder - if windows).
Code: Select all
--- net/pms/configuration/ConfigurationProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ConfigurationProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -14,6 +14,7 @@
    private static final String KEY_FLAC_PATH = "flac_path";
    private static final String KEY_MENCODERMT_PATH = "mencodermt_path";
    private static final String KEY_DCRAW = "dcraw_path";
+   private static final String KEY_IMCONVERT_PATH = "imconvert_path";
    
    private final Configuration configuration;
    private final ProgramPaths defaults;
@@ -71,6 +72,11 @@
    @Override
    public String getDCRaw() {
       return stringFromConfigFile(KEY_DCRAW, defaults.getDCRaw());
+   }
+   
+   @Override
+   public String getIMConvertPath() {
+      return stringFromConfigFile(KEY_IMCONVERT_PATH, defaults.getIMConvertPath());
    }
 
 }
--- net/pms/configuration/LinuxDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/LinuxDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,9 @@
    public String getDCRaw() {
       return "dcraw";
    }
-
+   
+   @Override
+   public String getIMConvertPath() {
+      return "convert";
+   }
 }
--- net/pms/configuration/MacDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/MacDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,10 @@
    public String getDCRaw() {
       return "osx/dcrawU";
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return "osx/convert";
+   }
 
 }
--- net/pms/configuration/PmsConfiguration.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/PmsConfiguration.java   Mon Jan 19 23:26:36 1970
@@ -1118,4 +1118,8 @@
    public void setAudioResample(boolean value) {
       configuration.setProperty(KEY_AUDIO_RESAMPLE, value);
    }
+   
+   public String getIMConvertPath() {
+      return programPaths.getIMConvertPath();
+   }
 }
--- net/pms/configuration/ProgramPathDisabler.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ProgramPathDisabler.java   Mon Jan 19 23:26:36 1970
@@ -75,4 +75,9 @@
    public String getDCRaw() {
       return disableDCraw ? null : ifEnabled.getDCRaw();
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return disableMplayer ? null : ifEnabled.getIMConvertPath();
+   }
 }
--- net/pms/configuration/ProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -10,4 +10,5 @@
    String getFlacPath();
    String getEac3toPath();
    String getDCRaw();
+   String getIMConvertPath();
 }
--- net/pms/configuration/WindowsDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/WindowsDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,9 @@
    public String getDCRaw() {
       return "win32/dcrawMS.exe";
    }
-
+   
+   @Override
+   public String getIMConvertPath() {
+      return "win32/convert.exe";
+   }
 }
--- net/pms/configuration/WindowsRegistryProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/WindowsRegistryProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -66,5 +66,10 @@
    public String getDCRaw() {
       return defaults.getDCRaw();
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return defaults.getIMConvertPath();
+   }
 
 }
--- net/pms/dlna/DLNAMediaInfo.java   Mon Jan 19 23:26:36 1970
+++ net/pms/dlna/DLNAMediaInfo.java   Mon Jan 19 23:26:36 1970
@@ -250,6 +250,50 @@
       parsing = false;
       return pw;
    }
+   
+   private ProcessWrapperImpl getImageMagickThumbnail(InputFile media) throws IOException {
+   // convert -size 320x180  hatching_orig.jpg  -auto-orient -thumbnail 160x90   -unsharp 0x.5  thumbnail.gif
+      String args [] = new String[10];
+      args[0] = PMS.getConfiguration().getIMConvertPath();
+      args[1] = "-size";
+      args[2] = "320x180";
+      if (media.file != null)
+         args[3] = ProcessUtil.getShortFileNameIfWideChars(media.file.getAbsolutePath());
+      else
+         args[3] = "-";
+      args[4] = "-auto-orient";
+      args[5] = "-thumbnail";
+      args[6] = "160x90";
+      args[7] = "-unsharp";
+      args[8] = "-0x.5";
+      args[9] = PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + media.file.getName() + ".jpg";
+      OutputParams params = new OutputParams(PMS.getConfiguration());
+      params.workDir = new File(PMS.getConfiguration().getTempFolder().getAbsolutePath() + "/imagemagick_thumbs/");
+      if (!params.workDir.exists())
+         params.workDir.mkdirs();
+      params.maxBufferSize = 1;
+      params.stdin = media.push;
+      params.log = true;
+      params.noexitcheck = true; // not serious if anything happens during the thumbnailer
+      final ProcessWrapperImpl pw = new ProcessWrapperImpl(args, params);
+         // FAILSAFE
+      parsing = true;
+      Runnable r = new Runnable() {
+         public void run() {
+            try {
+               Thread.sleep(7000);
+               ffmpeg_failure = true;
+            } catch (InterruptedException e) {}
+            pw.stopProcess();
+            parsing = false;
+         }
+      };
+      Thread failsafe = new Thread(r);
+      failsafe.start();
+      pw.run();
+      parsing = false;
+      return pw;
+   }
 
    private String getFfmpegPath() {
       String value = PMS.getConfiguration().getFfmpegPath();
@@ -390,6 +434,21 @@
                   } else if (formatName.startsWith("TIF")) {
                      codecV = "tiff";
                   }
+                  getImageMagickThumbnail(f);
+                  String frameName = PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + f.file.getName() + ".jpg";
+                  File jpg = new File(frameName);
+                  if (jpg.exists()) {
+                     InputStream is = new FileInputStream(jpg);
+                     int sz = is.available();
+                     if (sz > 0) {
+                        thumb = new byte [sz];
+                        is.read(thumb);
+                     }
+                     is.close();
+                     if (!jpg.delete())
+                        jpg.deleteOnExit();
+                  }
+
                   container = codecV;
                } catch (Throwable e) {
                   //ffmpeg_parsing = true;
--- net/pms/dlna/RealFile.java   Mon Jan 19 23:26:36 1970
+++ net/pms/dlna/RealFile.java   Mon Jan 19 23:26:36 1970
@@ -420,8 +420,8 @@
 
    @Override
    protected String getThumbnailURL() {
-      if (getType() == Format.IMAGE) // no thumbnail support for now for real based disk images
-         return null;
+//      if (getType() == Format.IMAGE) // no thumbnail support for now for real based disk images
+//         return null;
       StringBuffer sb = new StringBuffer();
       sb.append(PMS.get().getServer().getURL());
       sb.append("/");
tomeko
Project Member
 
Posts: 154
Joined: Sat Jun 06, 2009 11:02 am

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Wed Sep 08, 2010 1:07 am

Thanks tomeko. I've applied that by hand but it doesn't work at the moment. In Windows I think the command line fed to convert.exe is probably wrong because no thumbnail is produced. If I look at debug.log and manually create the thumbnail by running the command line taken from there (quoting long filenames) it does produce it. But then if I browse the folder on either my Sony Bluray or Sony Bravia renderers nothing is displayed, despite the corresponding JPG thumbnail being present in the temp imagemagick_thumbs folder.

I should be able to fix it from here though, because I captured some nice example XML of how Windows Media Player offers up the resources. I think Sony devices might be more fussy about the exact res of the thumbs too.
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Fri Sep 10, 2010 8:27 pm

Had a look this morning and I don't really understand - even though I have mplayer thumbs enabled I don't see any being created either on the PC in %TEMP%\javaps3media\mplayer_thumbs during successful thumbnail display on a Bravia renderer. Are they deleted the moment they are requested?
Last edited by patters on Sat Sep 11, 2010 1:09 pm, edited 1 time in total.
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby tomeko » Fri Sep 10, 2010 10:13 pm

thumbs are created in imagemagick_thumbs folder. remark
Code: Select all
// if (!jpg.delete())
//     jpg.deleteOnExit();
tomeko
Project Member
 
Posts: 154
Joined: Sat Jun 06, 2009 11:02 am

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Sat Sep 11, 2010 2:19 am

Yes, I saw that path. I was comparing the mplayer_thumbs behaviour since I know those work for the Bravia TV renderer. But I never see those being created, even if I browse a folder up on the TV, and at the same time refresh the temporary mplayer_thumbs or imagemagick_thumbs folders. So are you saying I should comment out those lines for my tests?
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Sat Sep 11, 2010 3:26 pm

I've done some more troubleshooting, commenting those two lines you highlighted. It's definitely creating the thumbnail files in imagemagick_thumbs correctly, and I have verified that the SOAP XML is correct - for example:

<res xmlns:dlna="urn:schemas-dlna-org:metadata-1-0/" protocolInfo="http-get:*:image/jpeg:DLNA.ORG_PN=JPEG_TN">http://192.168.1.65:5001/get/0$0$4$14/thumbnail0000ISS.jpg</res>

Where in the code is this http URL translated into a real file location?
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Mon Sep 13, 2010 12:55 am

I'm totally stuck. I've put in a debug line at the point the thumb is read and it shows up in the log, but the image is not displayed:
Code: Select all

            getImageMagickThumbnail(f);
            String frameName = PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + f.file.getName() + ".jpg";
                  PMS.debug("imagemagick_thumb: " + PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + f.file.getName() + ".jpg");     
            File jpg = new File(frameName);
            if (jpg.exists()) {
               InputStream is = new FileInputStream(jpg);
               int sz = is.available();
               if (sz > 0) {
                  thumb = new byte [sz];
                  is.read(thumb);
                  PMS.debug("imagemagick thumbnail read!");
               }
               is.close();
            //   if (!jpg.delete())
            //      jpg.deleteOnExit();

I can see that the imagemagick thumbnail code is basically the same as the mplayer thumb code, but mplayer thumbs work for me. I also tried changing the size of the thumbnail to "160x120!" which forces it to disregard aspect ratio (since Windows Media Player sizes them at 160x120) but this still does not work for me. Any ideas why not? On both a Bravia TV and a Sony Bluray player I see the icon showing that the thumbnail is being read for a second, then I get the generic blank thumbnail.
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Re: Thumbnail support for photos (RealFile.java)

Postby tomeko » Mon Sep 13, 2010 7:46 pm

Updated patch: (two more lines must be // in RealFile.java):
Code: Select all
--- net/pms/configuration/ConfigurationProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ConfigurationProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -14,6 +14,7 @@
    private static final String KEY_FLAC_PATH = "flac_path";
    private static final String KEY_MENCODERMT_PATH = "mencodermt_path";
    private static final String KEY_DCRAW = "dcraw_path";
+   private static final String KEY_IMCONVERT_PATH = "imconvert_path";
    
    private final Configuration configuration;
    private final ProgramPaths defaults;
@@ -71,6 +72,11 @@
    @Override
    public String getDCRaw() {
       return stringFromConfigFile(KEY_DCRAW, defaults.getDCRaw());
+   }
+   
+   @Override
+   public String getIMConvertPath() {
+      return stringFromConfigFile(KEY_IMCONVERT_PATH, defaults.getIMConvertPath());
    }
 
 }
--- net/pms/configuration/LinuxDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/LinuxDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,9 @@
    public String getDCRaw() {
       return "dcraw";
    }
-
+   
+   @Override
+   public String getIMConvertPath() {
+      return "convert";
+   }
 }
--- net/pms/configuration/MacDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/MacDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,10 @@
    public String getDCRaw() {
       return "osx/dcrawU";
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return "osx/convert";
+   }
 
 }
--- net/pms/configuration/PmsConfiguration.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/PmsConfiguration.java   Mon Jan 19 23:26:36 1970
@@ -1118,4 +1118,8 @@
    public void setAudioResample(boolean value) {
       configuration.setProperty(KEY_AUDIO_RESAMPLE, value);
    }
+   
+   public String getIMConvertPath() {
+      return programPaths.getIMConvertPath();
+   }
 }
--- net/pms/configuration/ProgramPathDisabler.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ProgramPathDisabler.java   Mon Jan 19 23:26:36 1970
@@ -75,4 +75,9 @@
    public String getDCRaw() {
       return disableDCraw ? null : ifEnabled.getDCRaw();
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return disableMplayer ? null : ifEnabled.getIMConvertPath();
+   }
 }
--- net/pms/configuration/ProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/ProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -10,4 +10,5 @@
    String getFlacPath();
    String getEac3toPath();
    String getDCRaw();
+   String getIMConvertPath();
 }
--- net/pms/configuration/WindowsDefaultPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/WindowsDefaultPaths.java   Mon Jan 19 23:26:36 1970
@@ -46,5 +46,9 @@
    public String getDCRaw() {
       return "win32/dcrawMS.exe";
    }
-
+   
+   @Override
+   public String getIMConvertPath() {
+      return "win32/convert.exe";
+   }
 }
--- net/pms/configuration/WindowsRegistryProgramPaths.java   Mon Jan 19 23:26:36 1970
+++ net/pms/configuration/WindowsRegistryProgramPaths.java   Mon Jan 19 23:26:36 1970
@@ -66,5 +66,10 @@
    public String getDCRaw() {
       return defaults.getDCRaw();
    }
+   
+   @Override
+   public String getIMConvertPath() {
+      return defaults.getIMConvertPath();
+   }
 
 }
--- net/pms/dlna/DLNAMediaInfo.java   Mon Jan 19 23:26:36 1970
+++ net/pms/dlna/DLNAMediaInfo.java   Mon Jan 19 23:26:36 1970
@@ -250,6 +250,50 @@
       parsing = false;
       return pw;
    }
+   
+   private ProcessWrapperImpl getImageMagickThumbnail(InputFile media) throws IOException {
+   // convert -size 320x180  hatching_orig.jpg  -auto-orient -thumbnail 160x90   -unsharp 0x.5  thumbnail.gif
+      String args [] = new String[10];
+      args[0] = PMS.getConfiguration().getIMConvertPath();
+      args[1] = "-size";
+      args[2] = "320x180";
+      if (media.file != null)
+         args[3] = ProcessUtil.getShortFileNameIfWideChars(media.file.getAbsolutePath());
+      else
+         args[3] = "-";
+      args[4] = "-auto-orient";
+      args[5] = "-thumbnail";
+      args[6] = "160x90";
+      args[7] = "-unsharp";
+      args[8] = "-0x.5";
+      args[9] = PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + media.file.getName() + ".jpg";
+      OutputParams params = new OutputParams(PMS.getConfiguration());
+      params.workDir = new File(PMS.getConfiguration().getTempFolder().getAbsolutePath() + "/imagemagick_thumbs/");
+      if (!params.workDir.exists())
+         params.workDir.mkdirs();
+      params.maxBufferSize = 1;
+      params.stdin = media.push;
+      params.log = true;
+      params.noexitcheck = true; // not serious if anything happens during the thumbnailer
+      final ProcessWrapperImpl pw = new ProcessWrapperImpl(args, params);
+         // FAILSAFE
+      parsing = true;
+      Runnable r = new Runnable() {
+         public void run() {
+            try {
+               Thread.sleep(7000);
+               ffmpeg_failure = true;
+            } catch (InterruptedException e) {}
+            pw.stopProcess();
+            parsing = false;
+         }
+      };
+      Thread failsafe = new Thread(r);
+      failsafe.start();
+      pw.run();
+      parsing = false;
+      return pw;
+   }
 
    private String getFfmpegPath() {
       String value = PMS.getConfiguration().getFfmpegPath();
@@ -390,6 +434,22 @@
                   } else if (formatName.startsWith("TIF")) {
                      codecV = "tiff";
                   }
+                  getImageMagickThumbnail(f);
+                  String frameName = PMS.getConfiguration().getTempFolder() + "/imagemagick_thumbs/" + f.file.getName() + ".jpg";
+                  PMS.debug(frameName);
+                  File jpg = new File(frameName);
+                  if (jpg.exists()) {
+                     InputStream is = new FileInputStream(jpg);
+                     int sz = is.available();
+                     if (sz > 0) {
+                        thumb = new byte [sz];
+                        is.read(thumb);
+                     }
+                     is.close();
+                     if (!jpg.delete())
+                        jpg.deleteOnExit();
+                  }
+
                   container = codecV;
                } catch (Throwable e) {
                   //ffmpeg_parsing = true;
--- net/pms/dlna/RealFile.java   Mon Jan 19 23:26:36 1970
+++ net/pms/dlna/RealFile.java   Mon Jan 19 23:26:36 1970
@@ -376,8 +376,8 @@
          File thumbFolder = null;
          boolean alternativeCheck = false;
          while (cachedThumbnail == null) {
-            if (thumbFolder == null)
-               thumbFolder = file.getParentFile();
+//            if (thumbFolder == null)
+//               thumbFolder = file.getParentFile();
             cachedThumbnail = FileUtil.getFileNameWitNewExtension(thumbFolder, file, "jpg");
             if (cachedThumbnail == null)
                cachedThumbnail = FileUtil.getFileNameWitNewExtension(thumbFolder, file, "png");
@@ -420,8 +420,8 @@
 
    @Override
    protected String getThumbnailURL() {
-      if (getType() == Format.IMAGE) // no thumbnail support for now for real based disk images
-         return null;
+//      if (getType() == Format.IMAGE) // no thumbnail support for now for real based disk images
+//         return null;
       StringBuffer sb = new StringBuffer();
       sb.append(PMS.get().getServer().getURL());
       sb.append("/");
tomeko
Project Member
 
Posts: 154
Joined: Sat Jun 06, 2009 11:02 am

Re: Thumbnail support for photos (RealFile.java)

Postby patters » Mon Sep 13, 2010 11:49 pm

I'm afraid it still doesn't work for me. Which Bravia do you have tomeko? And is this patch against r409 or r410? It seemed when I applied it by hand that the line numbers were out for one of the files (with r410).
patters
 
Posts: 97
Joined: Sun Apr 11, 2010 12:06 pm

Next

Return to Developers

Who is online

Users browsing this forum: No registered users and 0 guests