Enable Use High-Res textures and Use High-Res Full Alpha Channel - note that I personally had to toggle these a few times after quitting Retroarch but they eventually worked.
While the ROM is loaded, select the Quick Menu and go the Options tab for the core.
Start Retroarch by either using the NSP provided by Retroarch, The NSP forwarding method, or by using selecting it from the Homebrew Launcher while holding the R trigger (in SXOS) - this allows the program to use all available memory.
A) Enable the Setting to Allow Texture Replacement in Mupen64Plus:.