I am having the same problem although, i do not have nvidia, so i dont think that is my problem.
Dell Studio 17 64-bit
Windows 7 Ultimate 64-bit
Intel Core 2 Duo CPU T6400 2.00GHz
RAM: 4.00 GB
Streaming to a PS3
PC -> Direct Connect -> Ubee Cable Modem/Wifi Router -> Wifi-G -> PS3
I believe it is using MEncoder since that engine is in bold under my Transcoder Settings
When i browse my directories on my PS3 as soon as I select a folder that has media content; Java crashes on my laptop with the "Java Platform SE binary has stopped working" error message also crashing the PMS software
PMS used to work just fine on this laptop/PS3 setup, the only difference is I have a new Insignia 40" LCD TV.