Cabbage Logo
Back to Cabbage Site

Microtonal synthesizer in Unity/FMOD, is Csound an option?

#1

Hi Everyone!

First of all I want to say thank you for this amazing project and I’m very exited to explore it further. I have had a blast so far (mostly playing around with the examples).

I’m planning to incorporate a microtonal synthesizer into a unity project that will make use of FMOD. Right now I have a binary that calculates microtonal sequences at runtime and I start that process as an external process from within Unity. The way how the inter process communication works and what format the note information is in is not carved in stone, it’s probably just going to be simple JSONs with Frequency/Duration, received in callbacks through a stdio stream.

My first idea was to build my synth as a plugin instrument for FMOD and then update that plugin instrument with the new notes from within unity. While looking for solutions for the synthesizer-programming part, I discovered Cabbage. I successfully built and then imported several example projects into FMOD, but my big problem is: I can’t seem to get any notes into those plugins from within Unity (my first attempt was MIDI). Am I missing some obvious way to “play” a FMOD plugin instrument? Exposing VCOs and VCAs as knobs for all oscillators seems a bit tedious and I don’t really want to go in that direction…

The other option I was thinking about now, was to to generate the audio in Unity and stream it into FMOD but the downside of that is that it adds additional time-lag into the project.

Do you think the Cabbage and FMOD plugin instrument approach is not viable? And if not, do you think Csound directly in Unity and then streaming into FMOD is a viable idea? And if not, do you have any alternative ideas that involve Cabbage, because I would love to use it :slight_smile:

#2

Hi @RaphaelMaschinsen, and welcome to the forum :+1:

The simplest way to do this is to add a parameter to your FMOD plugin that you can send the note values to. You will need to create an FMOD.Studio.ParameterInstance in your Unity code for this. Then pass each note value to that parameter. Csound will pick up the notes on that channel. I think that should work fine.

Btw, unless you plan to use FMOD for lots of other things too, I would just do all of this using CsoundUnity and bypass FMOD altogether.

#3

Thanks a lot for the help @rorywalsh!

I’ll check that out as soon as possible. I’ll be using FMOD to create adaptive music with recorded audio in conjunction with the synth so I think I’ll stick with FMOD, but I already thought about trying out the plugin in another Unity project. I always wanted to learn cSound and I’m happy that its finally happening.

#4

I haven’t used FMOD in a long time, but if you get stuck shout out. I’m glad to see people using Csound with it. it’s a powerful combination. However, I’m not sure anyone has ever packaged the CsoundFMOD plugin into a standalone build though. This is something you might want to check out, if you are thinking of distributing the game on a large scale.

#5

At the moment the focus is on learning/exploring cool new tools for interactive music and not on large scale distribution. However building the project is important since I also want to incorporate online functionality and therefore need to be able to build instances in some way.

I just tried out building a standalone version of the project and I successfully embedded the white noise example in a standalone build using Windows 10, Unity 2019.4.4f1 and FMOD Unity integration 2.01.05.

At first I got an FMOD exception in the built version which prevented the FMOD Banks from successfully loading. I figured that Unity only copies the .dll files but not the .csd files from Assets\Plugins\FMOD\lib\win\x86_64 into the Build folder at Build\Project_Data\Plugins\x86_64. After manually copying the .csd files into the Build folder it appears to be working perfectly. Very cool! I think I’ll write a post-building script to automate this at some point.

Now I’m planning to try out building with the CsoundFMOD plugin.

#6

Yes, but the CsoundFMOD plugin is using the csound64.dll that is installed on the system. When it comes to distributing the game you will need to think about how best to include Csound with it.

This should be pretty straight forward. Simple make a copy of the fmod_noise VS project located in FMOD Studio API Windows\api\core\examples\plugins. Rename it csound_fmod or something like that. Then open it with VS2019, remove the noise source and add one of the CsoundFMOD source files. Then add a path to the Csound include directory C:/ProgramFiles/Csound6_x64/include and add the Csound lib, which is located at C:/ProgramFiles/Csound6_x64/lib

It should build after that.

#7

Ok, I see the problem. It didn’t work on the target computer even after installing cSound/Cabbage, rebuilding the plugin on that target machine and then replacing the dll/csd files of the build with the newly rebuilt ones. I wasn’t able to bundle cSound library with the plugin so far. I think in order to properly deliver cSound with the plugin I would have to recompile the plugin from scratch with different complier settings, which doesn’t seem to be possible in an easy way. But maybe I don’t have the best overview yet… I’ll keep on it for now and try again later. Btw, I couldn’t find any CsoundFMOD source files in C++ that I could have used to replace code in the FMOD core/examples with. Does cabbage generate such Code as an intermediate step when exporting an FMOD plugin?

#8

I assumed you had already been to the github repo:

If Csound is installed on the target machine, everything should run fine. But you probably don’t want to have to install Csound on each machine. A better approach would be to bundle it yourself. On Windows you can include csound64.dll and put it somewhere the game can find it1. You may need to add it to the system path. On OSX you can simple relink the csoundFmod plugin to the Csound framework that you bundle with your game.

1 This is fine for basic setups, but if other users have a different version of Csound installed there might be conflicts. When I need to distribute a Cabbage/Csound plugin on Windows, I build a version of Csound with a unique name, and then link to that when building Cabbage. I then distribute this custom Csound version with my plugin. This doesn’t violate the terms of the LGPL so long as Csound is still in the csound library name.

#9

Thanks for the help. I tried a few different things all without success so far. The short version is: Installing cSound on the target machine doesn’t seem to do trick.

While everything is working fine on my windows machine, the same build does not work on two different target 64bit windows machines, even after installing cSound. What I tried:

  • I installed cSound on both target machines through the official cSound installer and restarted.

  • After that I uninstalled cSound and then installed the same Cabbage version I have on my Computer (where everything is working fine) on the target Computers (where FMOD can’t load the plugins) and installed the included cSound version that comes with the Cabbage installer. I then tested on both machines if the install worked by running the white noise example. The build still fails to load the FMOD plugins, that was working on my machine. FMOD still fails to load the plugin.

  • Then I rebuilt the dll on both target machines with Cabbage and replaced the dll in the build with the newly built ones. This also didn’t work. FMOD fails to load the plugin.

  • I also tried to put the csound64.dll in all possible folders in the build and this also doesn’t do trick, with or without system wide installed cSound.

  • Then I stared to doubt if the official Unity FMOD Plugin even works on my machine with custom built “fmod plugin instruments”. So I built the FMOD Studio API Windows\api\core\examples\vs2019\fmod_noise.vcxproj on the same machine I built the Cabbage plugin with and embedded it into my FMOD project. I distributed to the same 2 target machines that weren’t working with the cSound/Cabbage plugins and the fmod_noise.dll works on both target machines.

The same Unity error message I get on those target machines in all of the above not working cases is:

[Exception]: SystemNotInitializedException: [FMOD] Initialization failed : Loading plugin 'csound_noise' from 'C:/Project/Assets/Plugins/FMOD/lib/win/X86_64/csound_noise.dll' : ERR_FILE_NOTFOUND : File not found.

This is the same error I get on my personal machine (where everything works and from where I build the Unity project) if I delete csound_noise.dll completely. The weird part is, this dll (and the .csd file) is on those machines in those folders.

#10

That’s very strange. There is no reason the dlls shoudn’t work on a target machine so long as Csound is installed. Can you send me your dll? You won’t be able to attach it, but maybe you can post a link and I can try it here.

#11

It really is strange. I did some further testing:

  • Cabbage is capable of running the examples on all machines (pressing the play button, I hear Audio, the knobs appear to be working).

  • On both (a bit older) target machines the dll’s built with Cabbage fail to load in FMOD Studio 12. Actually any dll’s built with Cabbage fail to load on those 2 machines (including the ones I built with my own machine, where everything is working fine)

  • the non-cSound dll I built with Windows\api\core\examples\vs2019\fmod_noise.vcxproj works well on all 3 machines.

The real mystery is, I don’t know what’s fundamentally different on my machine. Maybe the windows version that’s not the newest on the two target machines? The two target machines are intel based an my machine is AMD based? Some other dependencies that are missing? The hardware difference really shouldn’t be a problem I think though.

I made a zip with a dll built on my machine (FractalNoise.dll & FractalNoise.csd), a dll built on a target machine (FractalNoiseFromTarget.dll & FractalNoiseFromTarget.csd) and the dll built out of the FMOD API example without Cabbage (fmod_noise64.dll).

But if your machine generally builds working FMOD plugins with Cabbage then all 3 dll’s are expected to work as intended I think and you probably wouldn’t even be able to reproduce the problem…

FmodCSoundProblem.7z (45.0 KB)

#12

Could be your FMOD version. What version of the CsoundFMOD plugins are you using? Or are you building them yourself now against the API you have?

#13

I am using the newest Cabbage (2.5.0) with the “Export as FMOD Studio Plugin” function. I’m saving the files directly into the plugins folder of a fresh FMOD project made with the newest unity verified version (2.01.05, 64-bit). Those versions are identical on all 3 machines.

Did I miss to additionally install a CsoundFMOD plugin on those 2 target machines? The plugins built (from all 3 machines) work all on my own machine, but none of them work on the 2 other machines.

#14

Try the Cabbage installer in this drop link:
https://dev.azure.com/rorywalsh/cabbage/_build/results?buildId=1005&view=artifacts&pathAsName=false&type=publishedArtifacts
It gives you export options for FMOD 2.0. At least we will be on the same wavelength then.

#15

Thank you a lot for the link to new Cabbage installer. First tests are very promising now. After installing this new Cabbage version on all 3 Machines the dlls from all 3 Machines appear to also be working on all 3 Machines. I also rebuilt the Unity project with the newly built FMOD plugins and the built Unity project is now working with the embedded plugin on the 2 target computers (they have this new Cabbage version installed atm)! :partying_face:

Again, thank you so much for the help and this awesome project! I’m going to do some more tests tomorrow and going to try the easiest ways on how to bundle the cSound dll with the build.

#16

Great. Let us know how you get on :+1:

#17

It’s still working really well and as you mentioned above, distributing the build with the csound64.dll is easy.

All you have to do is to provide the csound64.dll next with the UnityGameName.exe. It only works when the .dll is in the root folder next to the .exe and it can’t be in and sub/data folders of the build. I tried it with a system wide installed version of cSound on the target machine too and there were no conflicts but I think that was to be expected, since the cSound versions are identical. I’ll try it out at some point what happens when the cSound versions aren’t identical. But those tests do not have priority atm.

Now I’m finally ready to dive into learning cSound and thinking about how to get data into the plugin.

1 Like