That's a great idea Galibert. I have a similar suspicion, and have been comparing the sysex data format with some discrete FM chips. No matches in the 2operator category with capabilities similar with the PSS480 though. The data format will definitely be a reference once I identify code that changes parameters when loaded over midi. Currently it's face down on the bench so the front panel isn't really accessible
I've largely been quiet as I had been experimenting with different ways to study the machine's behavior and get clues. Last weekend I had an epiphany and decided to start the built-in demo song knowing it sends midi data out while playing. I ran a capture on my analyzer just getting bus transactions from 0x0200 to 0x1fff to avoid stack, zero page, and known ram. One of the highest locations of traffic was 0x328 and 0x0321. 328's outbound data in a second specific capture looked -very- convincingly MIDI, so that's been marked as the midi out port. 321 so far seems to be a status register for the uart.
In the next few days I'll experiment with sending data back in with similar analysis tricks to find the data input port.
Knowing the (likely) data out and uart status ports gave me things to search for in Ghidra, and some behaviors I had been seeing are now making sense. I've identified code that ran a checksum on the firmware rom already, but very recently found some utility code for the overall sysex dump routine. Since this is copyrighted code I'm hesitant to post a ghidra screenshot. I'm not looking to fully re-source the machine anyway so that may be moot.
Getting the MIDI UART understood is pretty critical as it will provide a way to communicate with the PSS-480 using custom code. Using some controller change messages to encapsulate ascii traffic could even do it without violating the MIDI standard and allow me to use existing hardware and API's to make a simple terminal and install a machine language debugger on the keyboard itself. That's the overall next step for me, so that I can work my way around the code and more directly experiment with hardware registers to get them mapped out.
I guess I could include a screenshot of my analysis rig showing the midi sequencer dump and the analyzer's recording of 0x0328.
Pardon the odd image size; the analyzers' control unit runs dual 1024x768 displays at the moment and I do the work mostly via a remote desktop solution.
For now, no new exciting pictures. Just more info being gathered and good paths to follow. Thanks to everyone for their interest