PMS can send files to your player as they are (= "streaming"), or it can change the files to a different format and send the resulting data (= "transcoding").
Streaming is best; it doesn't require much work on your computer's side. Transcoding costs computing power and if it is done wrong it can mangle the data transcoded from your files, much like you report. So PMS tries to determine whether or not your player can handle a file. If it can, PMS will stream the contents of the file. If it cannot, PMS will transcode the file and send the transcoded contents.
From what you describe, it sounds like PMS 1.72 was streaming the files, whereas 1.80 appears to be mangling them and therefore must be transcoding.
To me it is interesting to learn what causes that difference.