As far as why the difference, I am not quite sure. I don't quite fully understand all the workings of Mencoder or tsMuxer, or even PS3MS totally. That might be a better question for meskibob or shagrath.
I just saw that you have pretty much the same network layout as I have and thought I could suggest what works for me. When it comes to scriting with aviSynth or stuff like that
So you can get these files to work with tsMuxer if needed? Keep trying your various files and let me know.