Last night, I looked into why Adafruit's VS1053 only works with Teensy 3.1 at 24 MHz, but not 48, 72 or 96 MHz.
Turns out, the library depends SD.begin() to reconfiguring SPI. It also runs data transfer code from both main program & interrupt context (causing havoc if the interrupt occurs at the wrong moment). Pretty amazing it's worked on AVR for so long, but apparently it does crash sometimes. Faster processors increase the opportunity for the problem to strike.
Hopefully my edits from last night will fix these problems for good.