Cabbage Logo
Back to Cabbage Site

CsoundUnity 3.0, "sfload: cannot open SoundFont file"

#1

First of all, want to say how very excited I am about CsoundUnity 3.0, and so many thanks for this release.

I’m just starting out testing, and have one question up front with an error code "sfload: cannot open SoundFont file sf_GMbank.sf2

This had been working well in previous versions of CsoundUnity where the Resources/StreamingAssets directory was used. I was able to use the FluidSynth sf opcodes: sfload, sfilist, and sfinstr by manually copying the SoundFont file into StreamingAssets.

Is there a similar workaround that you can suggest in CsoundUnity 3.0?

PS: I was able to successfully run CsoundUnity 3.0 on Windows with another .csd file that did not use FluidSynth/SoundFont

Thanks again, I’m so excited about this new release

Saving .matrixB with .csd
#2

This sounds like a path issue, @giovannibedetti, have you any ideas?

#3

I’m currently away from a pc, but I’d say it should be enough to set the SFDIR path to the desired folder with SetEnv, now we’re not setting it anymore.
Let me know!

#4

Thanks very much for getting back. I will experiment with solutions (probably over the next week or so) and get back to you. Some things I’m thinking about.

  • which directory to put the sf2 file in that would be available at runtime for at least Android and PC builds. In the previous CsoundUnity version, PC and Mac standalone builds didn’t find StreamingAssets at runtime, so I had to do workarounds (copying files by hand). (Hector’s Android version didn’t have this problem).

  • setting env variables and execution sequencing: The csd file is being compiled in CsoundUnity.Awake(), and, as you know, one can’t control the order of Unity Awake() invocations, so the environment variable would need to be set before Unity execution starts. If this is the solution, perhaps it might be better to have a way to set the PATH in code (as a CsoundUnity public variable?) rather than having to surround the Unity program with some script that sets environment variables?

Thanks again, and if this triggers any thoughts on your end, please let me know

#5

Sorry for the late reply, I’m on holiday with poor connection.
I think you’re right. You can try to uncomment those lines in the CsoundUnityBridge constructor:

I don’t know if you’ll need to copy things on Android. Just be aware that the csd is saved as soon as you drag it in the inspector so there is no need to have it in the persistent folder.

Happy that you are experimenting with CsoundUnity again!
Any feedback is super welcome, let’s try and build something that is useful to many!

#6

Thanks very much for getting back with this suggestion. I will try that over the next few days.

#7

Hello again. Thanks very much for your reply back in June, and really sorry that it took so long to return to this, but I am focusing on this now.

Your uncommenting out suggestion worked well for Windows/Editor. Basically, I created a StreamingAssets/CsoundFiles folder and copied the sf2 file in there. It worked first time.

I wanted to give a quick update on where I’m at with Quest/Android, which I don’t yet have working, in case there’s anything different you would suggest.

I tried the uncommenting the corresponding lines for Android, and I was confused for a while as to why it didn’t work. The assets folder in the apk file was correctly populated, but sfload() wasn’t finding it at runtime with the same error message. After some head-scratching I went to look at how it was working in Hector’s Android-only version, and I noticed two things:

  • The persistent directory on the sd card (sdcard/Android/data//files) was being populated with the csound files, but only after the app had been run at least once. This was where sfload() was finding it
  • There is code in CsoundUnity.Awake() that explicitly copies files from the apk onto persistent storage (in this case it was intended to be sound files rather than soundfont files) :

#elif UNITY_ANDROID
// other stuff …
foreach (var item in filesObj.fileNames)
{
if (!item.EndsWith(".json") && !item.EndsWith(".meta") && !item.EndsWith(".csd") && !item.EndsWith(".orc"))
{
csoundFileTmp = “jar:file://” + Application.dataPath + “!/assets/CsoundFiles/” + item;
webrequest = UnityWebRequest.Get(csoundFileTmp);
webrequest.SendWebRequest();
while (!webrequest.isDone) { }
getCsoundAudioFile(webrequest.downloadHandler.data, item);
}
}
#endif

I’m relatively certain this addresses the issue because if I manually copy the .sf2 file intoQuest sd storage (using SideQuest) it works.

I will first try adding some of Hector’s code into my version of CsoundUnity.cs, and if I get that working, I will publish it here, and we can discuss if it makes any sense to include, or whether I should take a look at externalizing it into a helper class (that would have to be configured in execution order to run ahead of CsoundUnity.Awake, which wouldn’t be that great).

If you have any feedback or suggestions let me know, in any case I will report back in the next couple of days (this time:)

Thanks again for this excellent new release

#8

I’m not at a pc atm but it should be a matter of setting the SSDIR folder correctly before sfload tries to load the files. After a bit of thinking I now understand why the StreamingAssets folder is not working: it is because the file inside there on Android must be accessed like from a zip file, or a Web link, so using a WebRequest. This is why sfload cannot find them.

I’d try setting the SSDIR to Application.persistentDataFolder and copy the files there manually, and see if this works.
But yes I agree, a better handling of how we setup those folders before everything starts is needed.

Maybe it can be useful adding a way to run again the Csound instance initialization (or wait before initializing) after the directories are set as desired.
Or we can simply assume that the right folder for any external audio file is always the persistent folder.
Btw, I’d leave the copy of the files separated from the CsoundUnity repo.

Ok, try to pull this new branch always using the Unity Package Manager and copy the sf files in the persistent data folder manually before running the app (or you could have a scene that copies the files there if needed, and then loads the CsoundUnity scene):

https://github.com/rorywalsh/CsoundUnity.git#feature/custom_SSDIR

Here I’m simply setting the SFDIR, SADIR, and SSDIR to the persistent data folder (+ CsoundFiles folder) for all platforms.

image
image977×81 11.2 KB

#9

Hi, Thanks so much for your suggestions. Super happy to say that this is now working across all platforms!

  • Windows Editor, Windows standalone build,
  • Android build,
  • Mac Editor (intel) and Mac standalone build (intel).

What I have added here is probably not the best implementation, it would be nice not to have to force the execution order externally in the Project Settings, and the helper class could certainly be more sophisticated, but I wanted to get this in front of you as a work-in-progress solution. If you’d like me to post more of a working project I’d be happy to do that as well.

Code Changes/Additions

In CsoundUnityBridge.cs for the environment variables SSDIR, SFDIR, SADIR

  • both Win and Mac still need to use Application.streamingAssetsPath + “/CsoundFiles” rather than persistentDataPath which on Windows points outside the project completely into $HOME/AppData/LocalLow
  • Android uses Application.persistentDataPath

This environment-setting code currently needs to be somewhere within the Package Runtime because the Csound6.NativeMethods.csoundSetGlobalEnv() function is scoped as internal and can’t be accessed from the helper class. Alternatively, I suppose these functions could be made part of the public API.

Helper class CsoundUnityPreSetup.cs

This component can be anywhere in the project. It externalizes the Android-only-copy-to-sdcard functionality (from Hector’s code) into a “helper” class called CsoundUnityPreSetup which has one public variable for auxilliaryFileName, and currently using Unity’s Script Execution Order to force it to run ahead of CsoundUnity and it’s Awake function

Screenshot 2022-02-01 205146
Screenshot 2022-02-01 205146717×426 37.1 KB

Again, I recognize this is probably not elegant enough for publishing, so please consider it a debugging work-in-progress.

Thanks again. I’m really enjoying this new CsoundUnity version!!

CsoundUnityBridge.cs (36.6 KB)
CsoundUnityPreSetup.cs (2.3 KB)

1 Like
#10

Great!!!
Yes it makes sense to use the streaming assets folder on desktop platforms.
And of course we can expose the Csound6.NativeMethods.csoundSetGlobalEnv() function in the CsoundUnity class.
Let’s see how we can add those changes in the package :wink:

#11

I exposed the missing SetGlobalEnv method in CsoundUnity (for now on the branch feature/custom_SSDIR)
I’m not sure if this would be really useful, since the CsoundUnityBridge is created in the CsoundUnity Awake, so if you call this method before you will get a nullref.

So I also set the SADIR, SFDIR, SSDIR to the StreamingAssets path on desktop, and to the PersistentDataPath on Android in the CsoundUnityBridge constructor.
If you want to change this at runtime you would need to stop the CsoundUnity instance, and restart it, which is not easily doable with the current implementation.

Now we need to decide how to copy the needed SF files BEFORE CsoundUnity tries to load them.
I’ll try and list some approaches to achieve it:

  • One is changing the script execution order as you are suggesting, but I’m a bit worried about what could happen if there are a lot of files to copy, CsoundUnity could start before all the files have been copied. Also, I’m not sure how we could distribute this .cs file that performs the copy with the CsoundUnity package. Maybe adding a folder where we start collecting some useful utilities? For example there is a “Remap” method inside CsoundUnity which could be moved in a “Utilities” folder.

  • The other is having another starting scene where the copying takes place (if the files are not already in the destination directory of course). This method could be easier to distribute as a sample (but it would still require to modify the BuildSettings for the second scene to load)

  • Another solution is to change how CsoundUnity initializes itself. Now everything happens in the Awake function, and this means that starting with the CsoundUnity component disabled doesn’t avoid the initialization to happen. So there’s no option like: start with CsoundUnity disabled, do whatever I need to do first, then enable it when everything is ready so that it can start initializing.
    Of course the entire CsoundUnity object could be instantiated at runtime from a prefab, but it doesn’t look very user-friendly.
    So this third solution would require some rewrite to make the initialization more flexible.

  • Other options?

What do you think? Which would be the cleanest and easiest approach for a CsoundUnity user?
@rorywalsh your opinion is highly welcome here :wink:

#12

I’m not sure to be honest. Can’t we simply use some kind of copy callback to start the SF instrument ONLY when copying has complete? I had to do something like this recently for an Android app where I needed to copy all the sound assets to tables first. This was happening while Csound was running, but I didn’t start my table reading instrument until all the copying was complete.

I’m not sure if there is a good generic way to do this. I’ve a feeling that it might be best handled on a case by case basis. :woman_shrugging:

1 Like
#13

Ah yes sure, delaying the SF instrument is the fourth option, and probably the easiest one user-wise.
I agree that there is no generic way of doing this, and probably we can only create samples that show some of the options. Then the user can choose which one suits best its needs.
For now I think that the important bit is pointing at the right directories. I will add a section about this in the docs!

#14

Maybe this is what Rory already suggested, but how about exposing a Unity Event that gets triggered during Awake()

It would of course be optional, but if used it would be guaranteed to run at the right time (before csd compilation)

Btw, I definitely agree that changing the execution order in Project settings is not ideal

#15

We have already an “OnCsoundInitialized” event, I can add another event before the compilation (but after creating the CsoundUnityBridge, so after line 94 in CsoundUnityBridge), but to me it looks confusing, since the compilation and csoundstart happen very quickly, and this event could lead to false assumptions (so being sure to perform something BEFORE Csound is compiled and started).

#16

I think I understand better the sequencing issues after looking at your comments and the code.

I would agree that the setting of the environment variables is probably simplest, best, and least invasive right where you had it commented out, in the CsoundUnityBridge constructor (as long as you add the same lines from WIN into the #elif UNITY_EDITOR_OSX || UNITY_STANDALONE_OSX). …and I now see that it probably doesn’t work to have an public/external function to set the environment variables, since csound won’t have been instantiated yet.

In this particular case (and I wouldn’t be surprised if it’s a similar requirement for embedded audio files or csound includes) the ANDROID code to copy from the sd or “jar:file://” needs to happen before csound initialization, and doesn’t depend on CsoundUnityBridge or CsoundUnity. So potentially a CsoundUnity event that’s called something like “beforeCsoundInitialization” could work, and some sample code for Android might be useful to others as well.

If this all seems too single-purpose, kludgy or error-prone, I know how to solve my problem by editing the library code, adding the helper class and manually changing the execution order. Perhaps at a minimum I could suggest that you to add back in the environment variable setting code to CsoundUnityBridge?

Thanks again…in any case I’m very happy that there is an identified solution :slight_smile:

#17

So thinking a bit more about it, I think it could make sense to add an event “OnBeforeCsoundInitialized”, right after setting the environment vars, so that you could override the vars if needed, and perform specific tasks like copying files.
This because C# events are blocking, so until all the registered event handlers have been executed the thread execution will be blocked, so the Csound compilation/start won’t happen.

#18

Makes sense to me; that would be great for my purposes, and I could use a vanilla Csound build.

As I mentioned in my earlier comments, I wouldn’t be surprised if other Android developers might also need to do something like this for includes or sound files.

Thanks very much for all your help on this!!

#19

So I’m trying to implement the above, but I’m having some doubts / difficulties.
I need to add the “OnBeforeCsoundInitialized” event inside CsoundUnityBridge.cs constructor.
But this means that I cannot subscribe to it from CsoundUnity.cs before the CsoundUnityBridge constructor is executed, because CsoundUnityBridge is null before.
A solution to this would be making the above event static, but this means that for every instance of CsoundUnityBridge the event would be executed, resulting in multiple calls.
Also adding an accessor for the CsoundUnityBridge (let’s call it CsoundUnity.GetBridge()) won’t work, because the event would have to be subscribed to before the constructor is executed.
So I’m thinking to expose the environment folders strings in the CsoundUnity settings, so that they can be modified from the inspector as desired (maybe with selectors like StreamingAssets folder / PersistentDataFolder + “subfolder”).
Any better idea?

btw @ceberman I’m also trying to complete my first CsoundUnity VR sample, I will setup a repo hopefully today.
Would it be better to be a @rorywalsh repo, so that it goes along with the official CsoundUnity repo?

#20

It might be nice to have this example as part of the main CsoundUnity stuff? But we can always make your repo a submodule of the main CsoundUnity one if you prefer.