So after a lot of digging, the issue is the Video Mode. I have installed MAME on my Qnap server, and this is done through an online apps site, requiring a special .qpkg file. This version, on the Qnap, works best in opengl Video Mode, but the bullets will not show for these games. D3D is the desirable mode, but for this install of MAME, D3D is not an option in the menu (I assume because this is not a PC and is not Direct X compatible...not sure). Anyway, I was able to get it to work better by setting the Video Mode to "Soft". The sound became incredibly choppy, so I set the Frameskip to "auto", which seems to even it out. The bullets are not great, but visible. I also cranked up the gamma and contrast and set the beam size high, and a key seemed to be setting the Beam Min to a little over 2. Beam Max was set to 4, but I guess the Beam Min may be more important to make sure it doesn't go small.
A frustration I have is that I cannot get these settings to load from an INI file, and I cannot understand this. I am changing them all onscreen by clicking "~", so I have to set them each time. I created a game INI file and put it in an INI folder and made sure MAME knows that directory, but I still have to go to the onscreen menu to change when I fire up the game to set contrast, gamma, Beam Min/Max each time. I suspect "autosave" could store these settings, but I am staying away from it for now at least. I was using autosave for high scores, but had tweaked something wrong in MAME, and it would not load, so had to delete it and reinstall. When I did this I set up "autosave" from the get go, and when I later loaded Starcastle, the sound was there, but weird. The only way to fix was to remove the autosave feature. Have no idea why...there is only one sample in my samples file for Starcastle, but it sounded very different until I did this.
I do know the game INI files can work, because I set up Star Wars with an INI file to crop the artwork, so the game would be larger onscreen, and that works fine. Cannot understand why the INI file for these screen settings does not work. Named it "starcas.ini", which should be the correct file name.
Anyway...hope this helps someone struggling with this issue...I had to dig a lot to figure this out. If anyone with MAME on QNAP figures out a better way, post it here or PM me! And any help on this INI file issue would also be appreciated. I am a newbie to MAME and Qnap, but pretty diligent with Google searches and doing my own troubleshooting.