System is a 2011 2.5GHz 4GB Mac Mini running Lion 10.7.3. PS3 Media Server is version 1.52.1.
I have both a PS3 and a 360 wired to the same 1GHz switch as the Mini. I'm using PS3MS out of the box, I haven't messed with any binaries or modified the configuration other than forcing binding to the en0 interface as part of troubleshooting this.
It's odd because the PS3 works fine but the Xbox doesn't. I have a Windows 7 VM on a VMware ESXi host elsewhere on the network also running 1.52.1. The Xbox DOES work with the Win7 PS3MS, just not the Lion one.
When starting the "Video Player" app on the Xbox, the Win7 installation appears almost immediately in the list of detected servers. The Lion server takes 10-20 seconds to appear but eventually does. When I try to use the one on Lion, the connection seems to time out and the Xbox tells me it can't connect and offers to troubleshoot my network connection via the same "test my network connection" you can fire up to check your Xbox Live connection. According to the Status and Traces tabs in PS3MS it "sees" both renderers.
To summarize:
PS3 -> Win7 works
Xbox -> Win7 works
PS3 -> Lion works
Xbox -> Lion does NOT work
It's just annoying because I'm trying to migrate away from my Windows system. Of course, I can use the PS3 as needed but I'd really like to see this fixed because I have a second Xbox in another part of the house and it would be great to be able to watch videos there too.
