Cabbage Logo
Back to Cabbage Site

CsoundUnity Package (UPM) development

I just thought of a much simpler way to do this…

… Without the need for callbacks…

I hope so, I’m kind of stuck at passing the custom data, always get an empty pointer. I have to study a bit the generic types since I’ve always used them passively!

In this case you know the type. But is this generic?

Hi guys, looks like a lot of progress is being made, great work! Let me know if you’d like me to test anything out.
I tried dev2 out on Sunday and it was saving all my playmode changes, woohoo! The audio on OscBankSynth was back to sounding weird though, I assume that’s because the fixes were made on a different branch?

Thanks for pointing that out.
We will be very careful when we’re going to merge everything :sweat_smile:
It would be great if you do some tests on the dev2 branch! We’re adding new methods so the rest should work. You can post your findings and your thoughts here.
Now the changes you made on your Csound editor are reflected in the CsoundUnity inspector and as you noticed the channels are being saved.

But in fact the weird behaviour on OscBankSynth wasn’t expected…

I guess, it’s a void pointer, so it can point to any type of data structure.

I did a few more experiments and I’m not sure this will work. The ‘simple’ way I mentioned is to simply fill each node’s buffer in the CsoundUnity ProcessBlock method and play it back in the node component’s OnAudioReadFilter() method. So Csound fills each childs buffer. But unless the child uses the exact same sample index as the main CsoundUnity object we get clicks. The problem is I can’t access the ksmpsIndex value while the OnAudioFilterRead method is playing. Nor do I think this would prove to be a good design!

Instead of writing directly to the buffer, maybe I can try to dynamically write to a clip and fill that instead. I have a feeling with will result in the same thing, but it’s worth a shot.

I just took a look now and I can’t see how that would result in anything different. Arrggghhhh. This is really frustrating! There has to be a way. For what it’s worth, the other functions we looked at wouldn’t have worked either.

I think we should try the senseEvent one and see if we can get it working. Right now I’m doing something like this:

if ((ksmpsIndex >= ksmps) && (ksmps > 0))
{
    PerformKsmps();
    foreach(CsoundUnityNode node in csoundUnityNodes)
    {
        foreach(string channelName in node.GetChannels())
        {
            node.FillNodeBuffer(GetAudioChannel(channelName));
        }
    }
    ksmpsIndex = 0;
}

if (processClipAudio)
{
    SetInputSample((int)ksmpsIndex, (int)channel, samples[i + channel] * zerdbfs);
}

samples[i + channel] = (float)GetOutputSample((int)ksmpsIndex, (int)channel) / zerdbfs;

But the problem is FillNodeBuffer only gets called once the OnAudioFilterRead()/ProcessBlock() method is finished. So instead of picking up a new buffer of samples every ksmps samples, I’m only getting them ever 2048 samples. Hence the crappy audio. If I can call GetAudioChannel() outside the OnAudioFilterRead() method, but at a k-boundary, then I think we have a chance. What do you think?

Yes indeed, it is similar to the SetYieldCallback. What I don’t understand is how to link the user data with the callback IntPtr.
I tried with a typed Action with a generic type, the callback is called, but then I have a crash. But again, in this code:

/// <summary>
/// Registers a callback proxy (below) that transforms csound callbacks into .net events.
/// </summary>
/// <param name="callback"></param>
/// <returns></returns>
internal void SetSenseEventCallback<T>(Action<T> callback, T userData) where T : class
{
   Csound6.SenseEventCallbackProxy cb = new Csound6.SenseEventCallbackProxy((csd, ptr) =>
    {
        callback?.Invoke(userData);
    });

    GCHandle gch = FreezeCallbackInHeap(callback);
    Csound6.NativeMethods.csoundRegisterSenseEventCallback(csound, cb);
 }

the csd is automatically referenced, ptr is (of course) always 0x0.

I’m calling this code from CsoundUnity like this:

public void SetSenseEventCallback<T>(Action<T> action, T type) where T : class
{
    csound.SetSenseEventCallback(action, type);
}

and from a test script like this:

public class MyClass
{
    public string name;
}

void Start(){
    var c = new MyClass() { name = "test" };
    Action<MyClass> action = new Action<MyClass>((cc) =>
    {
        Debug.Log("MyClass name: " + cc.name);
    });
    csound.SetSenseEventCallback(action, c);
}

Richard instead implemented it like this:

    public event Csound6SenseEventCallbackHandler SenseEventsCallback
    {
        add
        {
            Csound6SenseEventCallbackHandler handler = m_callbackHandlers[_inputChannelEventKey] as Csound6SenseEventCallbackHandler;
            if (handler == null) SetSenseEventCallback(RawSenseEventsCallback);
            m_callbackHandlers.AddHandler(_senseEventKey, value);
        }
        remove
        {
            m_callbackHandlers.RemoveHandler(_senseEventKey, value);
        }
    }

    /// <summary>
    /// Registers a callback proxy (below) that transforms csound callbacks into .net events.
    /// </summary>
    /// <param name="callback"></param>
    /// <returns></returns>
    internal GCHandle SetSenseEventCallback(SenseEventCallbackProxy callback)
    {
        GCHandle gch = FreezeCallbackInHeap(callback);
        NativeMethods.csoundRegisterSenseEventCallback(Engine, callback);
        return gch;
    }

    public void RawSenseEventsCallback(IntPtr csound, IntPtr userdata) {  }

    public class Csound6SenseEventsArgs : EventArgs
    {
        public object UserData;
    }

    public delegate void Csound6SenseEventCallbackHandler(object sender, Csound6SenseEventsArgs e);

so there’s this object UserData that should hold the data set by the user.

I think I should call it like this from a test script:

    public void Test() {
        SenseEventsCallback += Csound6NetRealtime_SenseEventsCallback;
    }
    MyClass data;
    private void Csound6NetRealtime_SenseEventsCallback(object sender, Csound6SenseEventsArgs e)
    {
        data = (MyClass) e.UserData;
    }

I’m trying this now, but I am not sure of this last line!

EDIT:

I forgot to add how m_callbackHandlers are used in Richard code, this is the part I don’t understand clearly:

    private void RawInputChannelEventCallback(IntPtr csound, string name, IntPtr pValue, IntPtr pChannelType)
    {
        var cstype = (CS_TYPE)Marshal.PtrToStructure(pChannelType, typeof(CS_TYPE));
        Csound6ChannelEventArgs args = null;
        ChannelDirection dir = (ChannelDirection)((cstype.argtype != 0) ? cstype.argtype : 3);
        switch (cstype.varTypeName[0])
        {
            case 'k':
                args = new Csound6ChannelEventArgs(name, ChannelType.Control, dir, pValue);
                args.Value = (double)Marshal.PtrToStructure(pValue, typeof(double));
                break;
            case 'S':
                args = new Csound6ChannelEventArgs(name, ChannelType.String, dir, pValue);
                args.Value = Marshal.PtrToStringAnsi(pValue);
                break;
            case 'a': //audio ksmps buffer: not supported by csound input callbacks
            case 'p': //pvs: not supported by csound input callbacks
            case 'v': //var??? no csound code supports this yet
            default:
                //only S and k should be sending output channel callbacks.  Ignore for now and implement as csound adds
                break;
        }
        Csound6ChannelEventHandler handler = m_callbackHandlers[_inputChannelEventKey] as Csound6ChannelEventHandler;
        if ((args != null) && (handler != null))
        {
            handler(this, args);
        }
    }

    /// <summary>
    /// 
    /// </summary>
    /// <param name="csound"></param>
    /// <param name="name"></param>
    /// <param name="pValue"></param>
    /// <param name="pChannelType"></param>
    private void RawOutputChannelEventCallback(IntPtr csound, string name, IntPtr pValue, IntPtr pChannelType)
    {
        var cstype = (CS_TYPE)Marshal.PtrToStructure(pChannelType, typeof(CS_TYPE));
        Csound6ChannelEventArgs args = null;
        ChannelDirection dir = (ChannelDirection) ((cstype.argtype != 0) ? cstype.argtype : 3);
        switch (cstype.varTypeName[0])
        {
            case 'k':
                args = new Csound6ChannelEventArgs(name, ChannelType.Control, dir);
               args.Value = (double)Marshal.PtrToStructure(pValue, typeof(double));
                break;
            case 'S':
                args = new Csound6ChannelEventArgs(name, ChannelType.String, dir);
                args.Value = Marshal.PtrToStringAnsi(pValue); 
                break;
            case 'a': //audio ksmps buffer: not supported by csound output callbacks
            case 'p': //pvs: not supported by csound output callbacks
            case 'v': //var??? no csound code supports this yet
            default:
                //only S and k should be sending output channel callbacks.  Ignore: someday csound might be supporting some of these
                break;
        }
        Csound6ChannelEventHandler handler = m_callbackHandlers[_outputChannelEventKey] as Csound6ChannelEventHandler;
        if ((args != null) && (handler != null))
        {
            handler(this, args);
        }
    }

So the call of the handler should be this:
handler(this, args);
But now I have to set the _inputChannelEventKey and I don’t know where this comes from.
Sorry this post is too long. I’m trying to clarify for myself :exploding_head:

EDIT: _inputChannelEventKey is the string channelName in node.GetChannels()?

But the

[quote="rorywalsh, post:128, topic:1916"]
if ((ksmpsIndex >= ksmps) && (ksmps > 0)) { 
    PerformKsmps();
    ...
[/quote]

should happen every 32 samples, it was what ksmpsIndex was for, or?

That’s fine. But I can’t seem to call public functions from other game objects in the onAudio…read function.

Yes from the docs:

Also note that OnAudioFilterRead is called on a different thread from the main thread (namely the audio thread) so calling into many Unity functions from this function is not allowed (if you try, a warning shows up at run time).

So the SenseEventCallback could help, but we need to track all the channels in use, and add some update routine?

The yield callback would work fine if the actual callback function didn’t need to be static?

Is it static? Now it’s not! Are you saying of making it static?

I am still stuck.
With the action approach I have the call but no data, and then very soon I have a crash.
With Richard’s approach I have no crash but also no call at all.

Btw, what about pinging Richard for some help? Who better than him can give us an advice on this?
He already did most of the work!

Oh, let me check, I thought the function passed to SetYield… was static? I’ll take a look when I get a chance. I can ping Richard, he was most helpful when I started this project. But let’s just wait until we really get stuck first :joy:

Yes, but I just can’t wait to have everything working!! we’re so close!
Btw you should definitely add him to the contributors in the github page :bowing_man:

For sure!

Sorry, perhaps static is not the right term, but I can’t access any of our members in it, for example try:

event csoundcsharp.Csound6.NativeMethods.YieldCallback YieldCallback = new csoundcsharp.Csound6.NativeMethods.YieldCallback((csd) =>
{
    print(ksmps);
    return 1;
});

It gives the following error:

Packages\CsoundUnityPackage\Runtime\CsoundUnity.cs(493,15): error CS0236: A field initializer cannot reference the non-static field, method, or property 'CsoundUnity.ksmps'

???

try this outside CsoundUnity:

void Start()
{
    csound = GetComponent<CsoundUnity>();
    csound.SetYieldCallback(() =>
    {
        Debug.Log("ksmps: " + csound.GetKsmps());
    });

    csound.SetYieldCallback(MethodToCall);
}

private void MethodToCall()
{
    Debug.Log("ksmps: " + csound.GetKsmps());
}

or this inside CsoundUnity:

SetYieldCallback(() => { Debug.Log("ksmps: "+ksmps); });