Interestingly enough after days of trying to figure this one out i got the xbox to play back a 1080 MKV without lag,
so i figured id share it with the community.
After noticing that even on HQ settings my buffer was not emptying out and my network was no where near maxed.
I figured it must be some limitation of the 360 tried tweaking the 360 config, no dice.
So i set PMS to scale video to 1280x720 and volia it works with no stuttering
To be fair this isn't really a fix since its no longer 1080p but if your like me you just want to watch some mkv's with embedded subs from the comfort of your couch instead of a desk.