Cabbage Logo
Back to Cabbage Site

Returning to CsoundUnity and updating old project

#21

Hey @metaphysician I just published my sequencer experiment here:

Feel free to ask for any clarification, or to suggest what you would change / improve.
It is a proof of concept I developed in a week, there are plenty of things I’d like to add/modify/improve:

  • it needs a custom editor to write the sequences, like this it is very tedious, since it’s based on arrays
  • make it more generic to be able to support any number of tracks (currently you have 4+4 tracks, each track has 2 variants you can switch using the mouse LR)
  • improve performance when sending a set with long sequences
  • check if the duration calculations are correct
  • add ability for each track to select the instrument that it will trigger (currently hardcoded)
  • it would be interesting to be able to send / create instruments on the fly (using CompileOrc)

Let me know if it helps!

#22

thanks a bunch @giovannibedetti! i’ll take a look at this and will get back to you with questions. in the meantime i’ll see if ChatGPT can get me started on some basic scripting.

1 Like
#23

okay, doing a bit more thinking on Item #1 and i realized that having a metronome with 16 clock outputs is too literal and not necessary. the reason being that all i need is a message output for each of the beat multiples (which will be from 0.125 up to 4). then each layer can select which beat multiple it wants to subscribe to and we’re good.

i’ve been getting what seems to be some pretty usable scripts out of ChatGPT for this:

<CsoundSynthesizer>
<CsOptions>
-odac
</CsOptions>
<CsInstruments>

sr = 44100
ksmps = 32
nchnls = 2

0dbfs = 1

; Initialize BPM channel
chnset 120, "BPM"

instr 1
    ; Read BPM value from channel
    bpm chnget "BPM"
    
    ; Calculate the duration of a beat in seconds
    beat_duration = 60 / bpm
    
    ; Initialize variables for each division
    divisions[] init 0.125, 0.25, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0
    
    ; Send timing messages for each division
    kcount = 0
    while kcount < 8 do
        ; Calculate the time to send the next message
        ktime = kcount * beat_duration
        
        ; Send the timing message for the current division
        event_i sprintf("div_%d", kcount), ktime, 0, divisions[kcount]
        kcount = kcount + 1
    od
endin

</CsInstruments>
<CsScore>

; Infinite loop for playing the score
i 1 0 z

</CsScore>
</CsoundSynthesizer>

since ChatGPT is keyword based and i used the term ‘subscribe’ it set me up with a delegate/Event script example as far as getting a specific division from it in Unity. is this usable as it is?

#24

That code won’t run. I can spot quite a few errors. But I think this is to be expected with ChatGPT…

#25

yeah the BPM is definitely done wrong - it did it better before in an earlier script. and now that i look at it more i don’t think it’s getting the divisions right because the divisions[] isn’t being calculated before sending out the event. what else is wrong? i’m trying to get it to generate timing messages at all of the divisions under the division[] array, and be able to subscribe to any of them from Unity.

#26

It’s not even declaring variables correctly. I thought it would have done a better job tbh.

#27

so basically just throw this out and start over from scratch?

#28

I think the best thing would be throw it out and build yourself from the ground up, it’s the only way you’ll get your head around Csound. This is a pretty complex project. I’d park Unity for the time being and see if you can build a series of small .csd files that each handle some of the logic involved (not unlike the one you asked ChatGPT about). Then as you get to grips with the language you can start piecing them together.

Here is an example of some code that will trigger another instrument to play according to the beat timing you’ve listed But I can’t say for sure it’s what you are looking for?

<Cabbage>
form size(500, 300), caption("Untitled")
</Cabbage>

<CsoundSynthesizer>
<CsOptions>
-odac
</CsOptions>
<CsInstruments>

sr = 44100
ksmps = 32
nchnls = 2

0dbfs = 1

; Initialize BPM channel
chnset 120, "BPM"

instr 1
    ; Read BPM value from channel
    kbpm chnget "BPM"
    
    ; Calculate the duration of a beat in seconds
    kbeat_duration = 60 / kbpm
    
    ; Initialize variables for each division
    kdivisions[] fillarray 0.125, 0.25, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0
    
    if metro(1/(kbeat_duration*4)) == 1 then
        event "i", 2, kdivisions[0]*kbeat_duration, 1
        event "i", 2, kdivisions[1]*kbeat_duration, 1
        event "i", 2, kdivisions[2]*kbeat_duration, 1
        event "i", 2, kdivisions[3]*kbeat_duration, 1
        event "i", 2, kdivisions[4]*kbeat_duration, 1
        event "i", 2, kdivisions[5]*kbeat_duration, 1
        event "i", 2, kdivisions[6]*kbeat_duration, 1
        event "i", 2, kdivisions[7]*kbeat_duration, 1
    endif
    
endin


instr 2
    kexp expseg 0.01, 0.01, 1, p3, 0.01
    aout oscili kexp, 200
    outs aout, aout
endin
</CsInstruments>
<CsScore>
i 1 0 z
</CsScore>
</CsoundSynthesizer>
#29

actually this is pretty close - but what i’m thinking about IS actually simplifying what i had before. what i need as an output to Unity is to move the playhead at a certain rate on a layer, timing it with a message ‘tick’ - i’m removing the need to keep sequences or row definitions like the older script did.

so if i just renamed all of the 'event “i” to unique names could i get that information out to Unity? that’s actually what i’m looking for. i’m leaving aside the audio at the moment and just seeing if i can get accurately timed messages/bangs/ints into Unity. on the Unity side i want a layer to be able to select which timing stream it wants. that’s all for this half.

the other half is to format data from Unity to control synths and samplers based on values kept entirely in Unity and then sent to Csound as MIDI chords to a selectable instrument in the manner of a live performance. that will be coming later, though. i just want to see if i can get accurately timed messages into Unity from Csound.

#30

If you only need a list of the times then you can just pass them to a table in Unity (we can’t yet pass arrays via channels). Instead of using an array you just need to create a function table:

giTimings ftgen 101, 0, 8, -2, 0.125, 0.25, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0

Then in Unity call the CsoundUnity.GetTable() method.

Of are you hoping to send the events out in realtime and have them picked up in Unity?

#31

yes i want them to be generated in realtime and pick them up on the fly. that’s exactly it.

#32

The problem here is that Unity could miss 100s of events between each frame update. So accurately timed messages from Csound to Unity is not really possible. The visuals will only ever be a rough representation of what’s happening on the audio side. But with that in mind, it’s still possible to create something that looks very tightly synced. If you swap out the code in instrument 2 for a chnset, you can add a CsoundUnity.GetChannel() to your Unity script. In this instance I would send a random value each time so that I can test in Unity if the channel has been updated. You could also use a string, but string generation in Csound isn’t the most efficient. Just keep in mind that some events could be lost. All Unity can do in this case is wait for the next one.

#33

hmmm - okay. understood and thanks for the tip and info!

i’m not necessarily looking for super tight timing at the millisecond level. i was just trying to think of a design that didn’t involve quite as much sequencer marshaling overhead as the previous design did. the previous design was capable of pretty tight timing audio wise.

this redesign relies on the fact that in Unity i can update the physics events efficiently enough (probably under FixedUpdate) that when they go BACK into Csound the result isn’t a complete timing mess. i fully acknowledge that this may not be possible - i’m okay with some slippage of fast events if that happens. but if this design idea works and the timing is good enough to feel solid, it might be worth it.

if not it’s back to the drawing board and handling everything in one or more complex CSD scripts. that may be the case anyway.

i was thinking of running the timing messages in the single CsoundUnity instance, then the instruments as CsoundUnityChild objects using the switch off example you posted, so that only one instrument would be active/loaded at a time in each layer. but i don’t know the resource usage issues of CsoundChild or whether its useful to use it in this specific case.

ideally because i want to spatialize the audio output in Unity using Steam Audio, each layer would have to have its own audio output. as i recall the audio output of a child has to go back to the master script CsoundUnity ‘parent’ - is that right?

#34

The overhead of each CsoundUnityChild is the time it takes to update its AudioSource output with the channel(s) grabbed from the Csound instance it is hooked with.

Your idea would suggest the use of some sort of callback, but they haven’t been implemented yet in CsoundUnity (well there’s a branch where they are), also, the Csound 7 API will probably get rid of callbacks. I always thought that some sort of callback that can be called anywhere from a Csound instrument would be super handy. The thing that makes this difficult is that in Unity it would be triggered from the audio thread, meaning that it wouldn’t have access to resources that live on the main thread. :thinking:

#35

thanks! so here’s my thinking. since we’re not actually playing the instrument live from a MIDI keyboard the latency should not matter as much. the only issue is consistency. we don’t hear any output from Csound until it comes back out of Unity as a MIDI chord or MIDI value. ideally i’d like it synced to the visuals, but i might be able to delay the visuals to compensate if the latency is excessive.

the main issue here is reliable timing out to Unity and back from Unity. i just want to see if it’s possible. if not then i’ll go back to the older method doing it all in Csound.

#36

okay - managed to get something working in Unity with a slightly altered version of Rory’s example - a clock signal of 32nd notes at 60 BPM which was triggering an AudioSource.

and… the results are good but not great in terms of consistent timing. it doesn’t immediately start either and takes a while to ‘catch’ the clock. anyway, i get the best results with Best Latency and its decent but it’s still definitely NOT stable timing. interesting that playing in the Editor on my M1 Mac the timing results are virtually identical in the build, which is pretty useful that it behaves the same way.

also that script, as basic as it was, consumed 7% on the DSP meter at Best Latency.

but due to this audible timing jitter, it would likely be compounded by having the clock triggering a message telling a game object to move and then detect a change on Trigger Enter to the MeshRenderer, triggering an instrument to play a cluster of notes.

One thing I was a bit curious about was how stable timing like this might be in a DOTS/ECS setting. i’m definitely not as familiar with how to do that though.

so i guess it is back to the drawing board unfortunately.

#37

So you’re sending a clock signal from Csound to trigger an audio source in Unity? Why not simply trigger the audio in Csound?

#38

Unity has always been pretty slow when calling play on an AudioSource, especially on mobile platforms. Are you testing on the Quest already?
You can have a feeling of the delay of it if you try to play the AudioSource using a UI button.
Then with another button instead try to send a score to Csound to play an instrument. It should be way more responsive.
As Rory says, it’s probably better to leave everything audio related to Csound, instead of mixing the two worlds.
About the 7% DSP usage, it could be expected. Be aware that for every CsoundUnity / CsoundUnityChild in the scene it will have to fill the related AudioSource content using OnAudioFilterRead.
About DOTS/ECS, I don’t think you would have any improvement on the DSP side of things in terms of performance, since the operations to perform would be the same. But I could be wrong as I have no experience in using entities. It would be nice to have a look at some point.

Another thing to think about to improve performance would be implementing the audio part using a native plugin. You wouldn’t have the overhead of OnAudioFilterRead, for a start. Be prepared though, it is a lot of work!
but probably for complex scenarios it is something worth evaluating.

#39

yeah - i realize it was probably impractical to decouple the clocking from the actual playback of the notes but it was tempting to not have Csound tracking all the note info. so we’re back to having to have two complete systems again - one audio and the other physical/visual.

#40

I think for the system you have in mind you’re always going to be up against it in terms of timing problems. In this kind of model Unity will only be sampling the current state at fixed frame intervals. I still think it’s possible though :slight_smile: