Ah! You fiddled around with the PS3.conf!
Yes, then the test is supposed to bomb out as a warning to unsuspecting developers that just broke backwards compatibility which we hold dear.
I tried to put in as straight forward a message as possible: the test is really about testing Playstation compatibility with MKV. There's a lot of logic involved there, from the content of the "PS3.conf" via parsing the .conf files to the Format and MediaInfo code.
Any of those things can break the logic chain that should conclude PS3 is incompatible with MKV. The test itself has no idea which thing broke the chain, but it does know for sure that the chain was broken.
Hence the more general message: "PS3 is reported to be incompatible with MKV". Because that is all the test was trying to prove.
Feel free to improve test cases or add more unit tests!
It will help us a lot in keeping the behavior of PMS stable. And personally I find that writing the tests helps me in thoroughly understanding the behavior of some pieces of the existing code.