Returning to CsoundUnity and updating old project

[metaphysician]

hi folks! it has been a number of years since i was on this forum. i’ve been looking into finally finally getting the production version of my 3D VR sequencer done. i ran a prototype version of this on an original Vive back in 2017 with a much older version of CsoundUnity. Rory might remember since he got me started with a test project.

i recently got a brand new Quest 3 and enjoying it, and i want to build the production version of this sequencer on this platform. so i fired up an older version of the project which worked fine until i ran it on my Silicon Mac M1 in Unity 2021 and 2022.

so i decided to use a blank project. installed latest CsoundUnity from the package as well as the sequencer demo, and tested that all of that works. then i exported all the files i needed as a package to this clean project, leaving out all of the CsoundUnity scripts to avoid conflicts.

after a bit of finagling and finding out ‘setChannel’ was now ‘SetChannel’ and other similar case-related things, i’ve cleared all the compilation errors, and it pretty much works although the audio is pretty glitchy and there seems to be issues even when hiding my web browser which is the biggest memory hog. on the Intel machine it ran pretty much flawlessly and i even have more RAM on my M1. wondering if maybe changing buffer settings will improve things…

anyway i think there seems to be a problem in my csd script with how it’s loading fluidsynth and/or the SF2 files. the sequencer does pretty well doing layers with the internal synths (sawtooth and sine) but when it hits a layer that controls fluidsynth, it instantly crashes.

here’s my CSD file. The SF2 files are residing in StreamingAssets as i think that should be the default path.

Sequencer-sf2player.csd (9.5 KB)

any help appreciated!

Edit - definitely setting to best performance worked. audio is much smoother. program isn’t crashing but no sound from the layers that have SF2 instruments assigned to them, so i’m pretty sure that script is not intializing them correctly. i’m in Android mode so maybe that’s an issue, but it performed the same on Standalone as well… hmm…

[giovannibedetti]

Hi @metaphysician
and welcome back to CsoundUnity!
This is a very interesting project, I’m currently working on a very similar one! I will publish it on GitHub soon

Regarding the issues, two things:

• to be able to find the .sf2 files and avoid the crashes, you will have to let CsoundUnity know where to look for those soundFont files. From version 3.2 we added the Environment Vars. You have to add the folders where you want to look for them (for each platform), something like this:
  

  

• There are issues with soundFont files on Csound for Android, and it’s not something directly related with CsoundUnity, see: this thread. I will open an issue on Csound GitHub after I will be able to test them on Csound Android (yesterday I tried and I wasn’t able to build the project )

EDIT: I just remembered that the StreamingAssets folder won’t work on Android, since it’s still a compressed folder, hence Csound cannot access it directly (usually you would use a WebRequest to load files from it on Unity Android). That’s why to let Csound load files from the SFDIR we have to rely on the PersistentDataPath folder instead. This means that you will have to copy the files there before Csound is started.
Have a look at the EnvironmentVars sample to find a way to copy files there. To make it short: place your .sf2 files in a Resources folder, rename them as .bytes, load them with Resources.Load then read and write the bytes at the destination path (dir below is a path inside the PersistentDataPath)

                var destinationPath = Path.Combine(dir, sfName + ".sf2");
                if (!File.Exists(destinationPath))
                {
                    var sf = Resources.Load<TextAsset>(sfName);
                    Debug.Log($"Writing sf file at path: {destinationPath}");
                    Stream s = new MemoryStream(sf.bytes);
                    BinaryReader br = new BinaryReader(s);
                    using (BinaryWriter bw = new BinaryWriter(File.Open(destinationPath, FileMode.OpenOrCreate)))
                    {
                        bw.Write(br.ReadBytes(sf.bytes.Length));
                    }
                }

Then activate CsoundUnity after the copy is completed!

so the Environment Settings in the above image should be this instead:

[metaphysician]

@giovannibedetti - apologies for the late response on my part but THANKS for the immediate response on your end - one of the frustrating things about working in ChucK and Chunity is that the response on Discord was often delayed quite a bit so i really value your quick response, and i should have at least replied earlier.

the MacOS streaming side worked just fine in the Editor even though the build target is Android. i will try to get to testing the Android/VR side later

and just to whet your curiosity if you didn’t read my threads from super long ago, the sequencer is going to look a lot more like this:

only imagine the cube as 16 x 16 x 16 (sixteen 16x16 layers or 4096 total objects) and layers can be assigned to make sounds, process sounds, or modulate parameters globally or on layers. it’s fairly ambitious but there it is…

[giovannibedetti]

Yes it looks ambitious but also very interesting and potentially rewarding. Keep up the hard work and let us know if you need some help. The Cabbage community is great!

Unfortunately using sound fonts won’t be that easy for now, given the issue we’re having on Android, but I’m sure it will be fixed soon. I might have a look at some point, but I’m no expert in C.
For now you can use custom instruments to recreate the sounds you need, and then switch back to sound fonts when they will work.