Returning to CsoundUnity and updating old project

metaphysician · 5 November 2023 11:58

hi folks! it has been a number of years since i was on this forum. i’ve been looking into finally finally getting the production version of my 3D VR sequencer done. i ran a prototype version of this on an original Vive back in 2017 with a much older version of CsoundUnity. Rory might remember since he got me started with a test project.

i recently got a brand new Quest 3 and enjoying it, and i want to build the production version of this sequencer on this platform. so i fired up an older version of the project which worked fine until i ran it on my Silicon Mac M1 in Unity 2021 and 2022.

so i decided to use a blank project. installed latest CsoundUnity from the package as well as the sequencer demo, and tested that all of that works. then i exported all the files i needed as a package to this clean project, leaving out all of the CsoundUnity scripts to avoid conflicts.

after a bit of finagling and finding out ‘setChannel’ was now ‘SetChannel’ and other similar case-related things, i’ve cleared all the compilation errors, and it pretty much works although the audio is pretty glitchy and there seems to be issues even when hiding my web browser which is the biggest memory hog. on the Intel machine it ran pretty much flawlessly and i even have more RAM on my M1. wondering if maybe changing buffer settings will improve things…

anyway i think there seems to be a problem in my csd script with how it’s loading fluidsynth and/or the SF2 files. the sequencer does pretty well doing layers with the internal synths (sawtooth and sine) but when it hits a layer that controls fluidsynth, it instantly crashes.

here’s my CSD file. The SF2 files are residing in StreamingAssets as i think that should be the default path.

Sequencer-sf2player.csd (9.5 KB)

any help appreciated!

Edit - definitely setting to best performance worked. audio is much smoother. program isn’t crashing but no sound from the layers that have SF2 instruments assigned to them, so i’m pretty sure that script is not intializing them correctly. i’m in Android mode so maybe that’s an issue, but it performed the same on Standalone as well… hmm…

giovannibedetti · 5 November 2023 17:09

Hi @metaphysician
and welcome back to CsoundUnity!
This is a very interesting project, I’m currently working on a very similar one! I will publish it on GitHub soon

Regarding the issues, two things:

to be able to find the .sf2 files and avoid the crashes, you will have to let CsoundUnity know where to look for those soundFont files. From version 3.2 we added the Environment Vars. You have to add the folders where you want to look for them (for each platform), something like this:

image835×625 45 KB
There are issues with soundFont files on Csound for Android, and it’s not something directly related with CsoundUnity, see: this thread. I will open an issue on Csound GitHub after I will be able to test them on Csound Android (yesterday I tried and I wasn’t able to build the project )

EDIT: I just remembered that the StreamingAssets folder won’t work on Android, since it’s still a compressed folder, hence Csound cannot access it directly (usually you would use a WebRequest to load files from it on Unity Android). That’s why to let Csound load files from the SFDIR we have to rely on the PersistentDataPath folder instead. This means that you will have to copy the files there before Csound is started.
Have a look at the EnvironmentVars sample to find a way to copy files there. To make it short: place your .sf2 files in a Resources folder, rename them as .bytes, load them with Resources.Load then read and write the bytes at the destination path (dir below is a path inside the PersistentDataPath)

                var destinationPath = Path.Combine(dir, sfName + ".sf2");
                if (!File.Exists(destinationPath))
                {
                    var sf = Resources.Load<TextAsset>(sfName);
                    Debug.Log($"Writing sf file at path: {destinationPath}");
                    Stream s = new MemoryStream(sf.bytes);
                    BinaryReader br = new BinaryReader(s);
                    using (BinaryWriter bw = new BinaryWriter(File.Open(destinationPath, FileMode.OpenOrCreate)))
                    {
                        bw.Write(br.ReadBytes(sf.bytes.Length));
                    }
                }

Then activate CsoundUnity after the copy is completed!

so the Environment Settings in the above image should be this instead:

metaphysician · 12 November 2023 00:47

@giovannibedetti - apologies for the late response on my part but THANKS for the immediate response on your end - one of the frustrating things about working in ChucK and Chunity is that the response on Discord was often delayed quite a bit so i really value your quick response, and i should have at least replied earlier.

the MacOS streaming side worked just fine in the Editor even though the build target is Android. i will try to get to testing the Android/VR side later

and just to whet your curiosity if you didn’t read my threads from super long ago, the sequencer is going to look a lot more like this:

only imagine the cube as 16 x 16 x 16 (sixteen 16x16 layers or 4096 total objects) and layers can be assigned to make sounds, process sounds, or modulate parameters globally or on layers. it’s fairly ambitious but there it is…

giovannibedetti · 12 November 2023 15:16

Yes it looks ambitious but also very interesting and potentially rewarding. Keep up the hard work and let us know if you need some help. The Cabbage community is great!

Unfortunately using sound fonts won’t be that easy for now, given the issue we’re having on Android, but I’m sure it will be fixed soon. I might have a look at some point, but I’m no expert in C.
For now you can use custom instruments to recreate the sounds you need, and then switch back to sound fonts when they will work.

giovannibedetti · 20 January 2024 18:35

Sound fonts issues have been fixed in CsoundUnity release 3.4.2, it includes Csound 6.19 beta.

Let us know how it goes!

metaphysician · 4 February 2024 09:45

hi there Giovanni! apologies for the long delay responding. i had a lot of work just getting the visual mechanics to work for the project. now that i’ve got that more dialed in i’ve just barely started including the Csound script to get sound out of it. have NOT tried in Android just yet but that’s going to be coming.

so i need some advice on Csound scripting optimization from you or possibly @rorywalsh because i am now running into the issue that multiple copies of that script are causing DSP to spike and visual speed to tank accordingly. I had a feeling that would be the case going up to using 16 layers. each layer runs an edited copy of that script i sent you - it’s been slightly altered to handle 16 steps and not 8. with 4 layers active it seems to do okay but getting it to 8 slows it down and with 16 it’s just crawling playing 16th notes at 60 BPM - actually that’s just running and NOT playing notes, since i’m not triggering them just yet.

so this is my thinking. i’m thinking that the way it’s written now, each script loads all 8 instruments into memory and only one of those instruments is ever going to be needed per layer. i don’t know if the script uses resources on instruments that haven’t been chosen or selected but i’m thinking it is doing this.

so, i need some suggestions on how to selectively manage instrument resources so that instruments not selected by that layer do NOT use resources. about the most obvious thing i could imagine is placing all of the synths and sound font player instruments in separate scripts and then instance them as i need them. since i actually duplicated them up there’s only 3 unique instruments - sine player, sawtooth-ish synth, and sound font player(actually there’s 2 distinct sound fonts, so maybe that requires two). then the remaining script would only handle the sequence note data, beat length and tempo.

anyway, any suggestions would be appreciated! thanks a bunch!

rorywalsh · 5 February 2024 12:24

Is there no way you can use just one instance/script an have it manage everything on the Csound side of thing?

metaphysician · 5 February 2024 19:34

possibly, but i’m not knowledgable enough in Csound to approach how to do this. the instrument definitions and sequencer settings could be split apart. i am planning on adding in effects as well. should i toss this over to the Csound section of the forum?

rorywalsh · 5 February 2024 19:54

Feel free to ask here. Each time you create a new CsoundUnity instance, you spawn another instance of Csound. The more you spawn the more resources you end up using, and the more you risk things moving out of sync. I’d always recommend, where possible, using a single orchestra. You can just use a channel to enable/disable processing.

instr 10
    if chnget:k("goodToGo") then
    //all your audio processing
    endif
endin

If goodToGo is 0, no resources will be used. you can also start and stop instrument from a master instrument. There are lots of options.

metaphysician · 5 February 2024 20:14

okay, thanks a bunch. so the main issue as i see it is twofold.

first - i would need to spawn 16 instances of the sequencer or define a sequencer with 16 layers, each layer being 16 x16. that would need to run continuously all the time.

second - the instrument collection. i’ve got those defined but i want to be able to instance one selected instrument of these per layer on request from the layer control in Unity.

third is to have a series of effects and mix routing, but that’s down the road.

i really appreciate the selective shutoff code - that could definitely come in handy, though. but the main issue is instancing something 16 times (layers) where i can address each instance, and instancing on demand to assign one instrument to a specific layers sequence data. thanks very much for your help!

rorywalsh · 5 February 2024 22:40

When I’ve had to deal with this level of complexity in the past I’ve built my layers up via the Csound score section. I’d define an instrument like this:

instr 1
    kGainChan sprintf "layer%dGain", p4 
    kGain chnget kGainChan
    kSendChan sprintf "layer%Send", p4
    kSend chnget kSendChan
    //etc...
endin

My score would then look like this:

i1 0 z 1
i1 0 z 2
i1 0 z 3
i1 0 z 4
i1 0 z 5
i1 0 z 6
etc

This way I only have to design one layer. Each instance is unique and easy to access using numbered channel names. Just an idea. This pattern has work nicely for me in the past.

metaphysician · 5 February 2024 23:34

okay - being a newbie to Csound i’m not sure if i’m getting all this. it seems to me like your score instances the instruments? if so then i think it might not work for me because i need to change things on the fly, and ideally i’d like the user to be able to instance an instrument on request on the fly. can you change score values on the fly?

i’m confused about what i1 does at the moment. i see no reference to ‘i1’ anywhere except the score. so, is the score acting like a switch to any of the instruments for the single layer you mentioned? i don’t know what the three columns represent. i’m guessing 0 in the first column might be enabled/disabled? no idea what the z represents either. but if the last column represents the instrument assigned and one is able to edit the score to enable or disable that instrument, then i make 16 copies of that (i1 through i16) and i should be good for instrument assignments.

each layer will have its own MIDI channel / note data and audio output, but i would like it if multiple layers could use instances of the same instruments. let’s say i wanted 4 layers of the sine synth for example. the key here is dynamic allocation of instruments.

in terms of the layer sequencer that can be set up with a static setting, each layer having 16 notes and 16 events carrying midi notes to the instrument and routing audio from the selected instrument.

rorywalsh · 6 February 2024 11:26

Correct.

You can change anything to do with these instances on the fly, by sending channel data from Unity to the instance. The score is only used to instantiate the instruments. You could start new instances of the instrument from your game, but it might be hard to keep them in sync.

The Csound score section is discussed here. In this case we telling instrument 1 to start straight away and run forever. We pass a unique ID as the fourth parameter - which can be picked up in instrument 1 as p4.

From my experiments in the past, i’ve found that triggering new instances ot start based on in game event will always end up sounding off because our audio is running at 44100+ frames a second, whereas the graphics are running at circa 70 frames a second. Starting all your instrument at the same time, even though they might not be processing and audio, is the best way to ensure that all layers sync up to each other. The audio clock is king in this instance.

I’d start with something extremely simple!

metaphysician · 6 February 2024 12:31

thanks for the detailed reply, Rory! things are considerably clearer now.

allright i know the old script worked in terms of a single layer. i’ve since edited mine to cover 16 x 16 in a layer. so now it seems to me i should define/wrap the relevant parts of THAT in the script into an object and then instance that 16 times. so nested objects - possible/common? i seem to remember that this was one of Csound’s strengths.

as for the sync, sure - all of the layers should start at one time based on a clock. in the previous incarnation, the sync was per layer but stayed very locked. that was on Windows using a Vive though.

anyway, i think i need to learn more about instancing an array of objects composed of sequence layers/tracks. i’ll have to look at some examples.

metaphysician · 8 February 2024 10:44

hey there @rorywalsh - i did try to get started on this but it would help if there were some resources. what i would like to do is find out how to create an array of objects/instruments.

it appears that what i did was to define a single row sequencer, based on the script you gave me - jeez - 6 years ago or so - time flies! i’ve altered it to have 16 events in the row and 16 rows from the original 8:
instr ROW_SEQUENCER

;each Row will take one of these notes and use it when a beat is enabled
;notes - modifiable by Layer control script (LayerController)
;C major scale by default ()
kNoteValues[] fillarray 48, 50, 52, 53, 55, 57, 59, 60, 62, 64, 65, 67, 69, 71, 72, 74
;amps - 0 by default. changed by cube state
kNotesAmps[] fillarray 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

;run through note amps array and trigger note if amp is set to 1/on
kBeat init 0
kTempo chnget "tempo"
kInst chnget "inst"
kSeqLength chnget "seqLength"
kDur chnget "duration"
kOffset chnget "noteOffset"
kNoteList chnget "noteRowList"

after this comes a big section governing which synth/sampler the sequencer instrument will play. so i’m skipping that for now. then we get this:

;get row number and create unique channel name
SRowCubeIndex sprintf "rowCubeIndex%d", p4
SRowCubeState sprintf "rowCubeState%d", p4
SrowCubeNote sprintf "rowCubeNote%d",p4
kCubeIndex chnget SRowCubeIndex
kCubeState chnget SRowCubeState
kCubeNote chnget SrowCubeNote

;if user has enabled a note, update note amp array
kChanged changed kCubeState, kCubeIndex
if changed(kChanged) > 0 then
	kNotesAmps[kCubeIndex] = kCubeState
	kNoteValues[kCubeIndex] = kCubeNote
	printks "Updating row %d - index: %d - value: %d - note: %d - inst: %d",0, p4, kCubeIndex,kNotesAmps[kCubeIndex],kNoteValues[kCubeIndex],kInst
endif

all of that handles the note triggering and reading to and from Unity.

then it appears that the score section is what instances the row 16 times in order to make the single layer of 16 x 16.

<CsScore>
f100 0 8 2 0 10 0 0 8 0 0 0
i"ROW_SEQUENCER" 0 [3600*12] 1
i"ROW_SEQUENCER" 0 [3600*12] 2
i"ROW_SEQUENCER" 0 [3600*12] 3
i"ROW_SEQUENCER" 0 [3600*12] 4
i"ROW_SEQUENCER" 0 [3600*12] 5
i"ROW_SEQUENCER" 0 [3600*12] 6
i"ROW_SEQUENCER" 0 [3600*12] 7
i"ROW_SEQUENCER" 0 [3600*12] 8
i"ROW_SEQUENCER" 0 [3600*12] 9
i"ROW_SEQUENCER" 0 [3600*12] 10
i"ROW_SEQUENCER" 0 [3600*12] 11
i"ROW_SEQUENCER" 0 [3600*12] 12
i"ROW_SEQUENCER" 0 [3600*12] 13
i"ROW_SEQUENCER" 0 [3600*12] 14
i"ROW_SEQUENCER" 0 [3600*12] 15
i"ROW_SEQUENCER" 0 [3600*12] 16
</CsScore>
</CsoundSynthesizer>

so the salient question here is - can this instrument and score itself be encapsulated into another instrument, and then have THAT instrument be instanced by another score 16 times?

basically can one define a 16 x 16 sequencer object and then create an array of those objects? is there much documentation or examples of creating arrays of abstracted/nested objects?

also i’m curious what that top row of values represented in this case. i updated anything that indicated 8 values to 16 but that top line in the score section has not been updated and i see a few '8’s there. so, just curious…

as always, any help appreciated!

giovannibedetti · 8 February 2024 11:38

Sorry @metaphysician I’m late to the party (busy days)
I have a project which I think could be useful, I will create a repo and share it here. Hopefully I will have some time this afternoon.
In it you can send sequences from Unity as strings, Csound does the parsing and creates arrays to be used in the sequencer. The sequences can be updated at runtime.
Currently 4 tracks but can be surely extended and most of all made more generic to support any number of tracks. Ideally it would be nice to be able to add/remove tracks directly from Unity.

Side note: you will soon notice that array handling in Csound is not as in C# (more details in the csd I will share)

EDIT: Some preview info here: https://forum.csound.com/t/arrays-handling/1458

rorywalsh · 8 February 2024 11:43

I’d have to see the full code, but I’ve a feeling thi is not used as the instrument uses arrays instead.

Yes but it can all be done from the same csd file. As you will now be using 16 row sequencers times 16, you should probably trigger the instrument to start in a loop in your orchestra. You’ll also need to pass another unique ID to the row_sequencer instrument so you know what parent it belongs to. The following instrument will set up your sequencer instances. p4 in this instance is the sequencer ID, and p5 is now the row id:

instr 1
    iRowIndex init 0
    iSequencerIndex init 0
    while iSequencerIndex < 16 do
        while iRowIndex < 16 do
            SEvent sprintf {{i"ROW_SEQUENCER" 0 [3600*12] %d %d}}, iSequencerIndex+1, iRowIndex+1
            scoreline_i SEvent
            iRowIndex += 1
        od
        iRowIndex = 0
        iSequencerIndex += 1
    od
endin

The short code above produces the following score, but saves you a lot of typing

i"ROW_SEQUENCER" 0 [3600*12] 1 1
i"ROW_SEQUENCER" 0 [3600*12] 1 2
i"ROW_SEQUENCER" 0 [3600*12] 1 3
i"ROW_SEQUENCER" 0 [3600*12] 1 4
i"ROW_SEQUENCER" 0 [3600*12] 1 5
i"ROW_SEQUENCER" 0 [3600*12] 1 6
i"ROW_SEQUENCER" 0 [3600*12] 1 7
i"ROW_SEQUENCER" 0 [3600*12] 1 8
i"ROW_SEQUENCER" 0 [3600*12] 1 9
i"ROW_SEQUENCER" 0 [3600*12] 1 10
i"ROW_SEQUENCER" 0 [3600*12] 1 11
i"ROW_SEQUENCER" 0 [3600*12] 1 12
i"ROW_SEQUENCER" 0 [3600*12] 1 13
i"ROW_SEQUENCER" 0 [3600*12] 1 14
i"ROW_SEQUENCER" 0 [3600*12] 1 15
i"ROW_SEQUENCER" 0 [3600*12] 1 16
i"ROW_SEQUENCER" 0 [3600*12] 2 1
i"ROW_SEQUENCER" 0 [3600*12] 2 2
i"ROW_SEQUENCER" 0 [3600*12] 2 3
i"ROW_SEQUENCER" 0 [3600*12] 2 4
i"ROW_SEQUENCER" 0 [3600*12] 2 5
i"ROW_SEQUENCER" 0 [3600*12] 2 6
i"ROW_SEQUENCER" 0 [3600*12] 2 7
i"ROW_SEQUENCER" 0 [3600*12] 2 8
i"ROW_SEQUENCER" 0 [3600*12] 2 9
i"ROW_SEQUENCER" 0 [3600*12] 2 10
i"ROW_SEQUENCER" 0 [3600*12] 2 11
i"ROW_SEQUENCER" 0 [3600*12] 2 12
i"ROW_SEQUENCER" 0 [3600*12] 2 13
i"ROW_SEQUENCER" 0 [3600*12] 2 14
i"ROW_SEQUENCER" 0 [3600*12] 2 15
i"ROW_SEQUENCER" 0 [3600*12] 2 16
i"ROW_SEQUENCER" 0 [3600*12] 3 1
i"ROW_SEQUENCER" 0 [3600*12] 3 2
i"ROW_SEQUENCER" 0 [3600*12] 3 3
i"ROW_SEQUENCER" 0 [3600*12] 3 4
i"ROW_SEQUENCER" 0 [3600*12] 3 5
i"ROW_SEQUENCER" 0 [3600*12] 3 6
i"ROW_SEQUENCER" 0 [3600*12] 3 7
i"ROW_SEQUENCER" 0 [3600*12] 3 8
i"ROW_SEQUENCER" 0 [3600*12] 3 9
i"ROW_SEQUENCER" 0 [3600*12] 3 10
i"ROW_SEQUENCER" 0 [3600*12] 3 11
i"ROW_SEQUENCER" 0 [3600*12] 3 12
i"ROW_SEQUENCER" 0 [3600*12] 3 13
i"ROW_SEQUENCER" 0 [3600*12] 3 14
i"ROW_SEQUENCER" 0 [3600*12] 3 15
i"ROW_SEQUENCER" 0 [3600*12] 3 16
i"ROW_SEQUENCER" 0 [3600*12] 4 1
i"ROW_SEQUENCER" 0 [3600*12] 4 2
i"ROW_SEQUENCER" 0 [3600*12] 4 3
i"ROW_SEQUENCER" 0 [3600*12] 4 4
i"ROW_SEQUENCER" 0 [3600*12] 4 5
i"ROW_SEQUENCER" 0 [3600*12] 4 6
i"ROW_SEQUENCER" 0 [3600*12] 4 7
i"ROW_SEQUENCER" 0 [3600*12] 4 8
i"ROW_SEQUENCER" 0 [3600*12] 4 9
i"ROW_SEQUENCER" 0 [3600*12] 4 10
i"ROW_SEQUENCER" 0 [3600*12] 4 11
i"ROW_SEQUENCER" 0 [3600*12] 4 12
i"ROW_SEQUENCER" 0 [3600*12] 4 13
i"ROW_SEQUENCER" 0 [3600*12] 4 14
i"ROW_SEQUENCER" 0 [3600*12] 4 15
i"ROW_SEQUENCER" 0 [3600*12] 4 16
i"ROW_SEQUENCER" 0 [3600*12] 5 1
i"ROW_SEQUENCER" 0 [3600*12] 5 2
i"ROW_SEQUENCER" 0 [3600*12] 5 3
i"ROW_SEQUENCER" 0 [3600*12] 5 4
i"ROW_SEQUENCER" 0 [3600*12] 5 5
i"ROW_SEQUENCER" 0 [3600*12] 5 6
i"ROW_SEQUENCER" 0 [3600*12] 5 7
i"ROW_SEQUENCER" 0 [3600*12] 5 8
i"ROW_SEQUENCER" 0 [3600*12] 5 9
i"ROW_SEQUENCER" 0 [3600*12] 5 10
i"ROW_SEQUENCER" 0 [3600*12] 5 11
i"ROW_SEQUENCER" 0 [3600*12] 5 12
i"ROW_SEQUENCER" 0 [3600*12] 5 13
i"ROW_SEQUENCER" 0 [3600*12] 5 14
i"ROW_SEQUENCER" 0 [3600*12] 5 15
i"ROW_SEQUENCER" 0 [3600*12] 5 16
i"ROW_SEQUENCER" 0 [3600*12] 6 1
i"ROW_SEQUENCER" 0 [3600*12] 6 2
i"ROW_SEQUENCER" 0 [3600*12] 6 3
i"ROW_SEQUENCER" 0 [3600*12] 6 4
i"ROW_SEQUENCER" 0 [3600*12] 6 5
i"ROW_SEQUENCER" 0 [3600*12] 6 6
i"ROW_SEQUENCER" 0 [3600*12] 6 7
i"ROW_SEQUENCER" 0 [3600*12] 6 8
i"ROW_SEQUENCER" 0 [3600*12] 6 9
i"ROW_SEQUENCER" 0 [3600*12] 6 10
i"ROW_SEQUENCER" 0 [3600*12] 6 11
i"ROW_SEQUENCER" 0 [3600*12] 6 12
i"ROW_SEQUENCER" 0 [3600*12] 6 13
i"ROW_SEQUENCER" 0 [3600*12] 6 14
i"ROW_SEQUENCER" 0 [3600*12] 6 15
i"ROW_SEQUENCER" 0 [3600*12] 6 16
i"ROW_SEQUENCER" 0 [3600*12] 7 1
i"ROW_SEQUENCER" 0 [3600*12] 7 2
i"ROW_SEQUENCER" 0 [3600*12] 7 3
i"ROW_SEQUENCER" 0 [3600*12] 7 4
i"ROW_SEQUENCER" 0 [3600*12] 7 5
i"ROW_SEQUENCER" 0 [3600*12] 7 6
i"ROW_SEQUENCER" 0 [3600*12] 7 7
i"ROW_SEQUENCER" 0 [3600*12] 7 8
i"ROW_SEQUENCER" 0 [3600*12] 7 9
i"ROW_SEQUENCER" 0 [3600*12] 7 10
i"ROW_SEQUENCER" 0 [3600*12] 7 11
i"ROW_SEQUENCER" 0 [3600*12] 7 12
i"ROW_SEQUENCER" 0 [3600*12] 7 13
i"ROW_SEQUENCER" 0 [3600*12] 7 14
i"ROW_SEQUENCER" 0 [3600*12] 7 15
i"ROW_SEQUENCER" 0 [3600*12] 7 16
i"ROW_SEQUENCER" 0 [3600*12] 8 1
i"ROW_SEQUENCER" 0 [3600*12] 8 2
i"ROW_SEQUENCER" 0 [3600*12] 8 3
i"ROW_SEQUENCER" 0 [3600*12] 8 4
i"ROW_SEQUENCER" 0 [3600*12] 8 5
i"ROW_SEQUENCER" 0 [3600*12] 8 6
i"ROW_SEQUENCER" 0 [3600*12] 8 7
i"ROW_SEQUENCER" 0 [3600*12] 8 8
i"ROW_SEQUENCER" 0 [3600*12] 8 9
i"ROW_SEQUENCER" 0 [3600*12] 8 10
i"ROW_SEQUENCER" 0 [3600*12] 8 11
i"ROW_SEQUENCER" 0 [3600*12] 8 12
i"ROW_SEQUENCER" 0 [3600*12] 8 13
i"ROW_SEQUENCER" 0 [3600*12] 8 14
i"ROW_SEQUENCER" 0 [3600*12] 8 15
i"ROW_SEQUENCER" 0 [3600*12] 8 16
i"ROW_SEQUENCER" 0 [3600*12] 9 1
i"ROW_SEQUENCER" 0 [3600*12] 9 2
i"ROW_SEQUENCER" 0 [3600*12] 9 3
i"ROW_SEQUENCER" 0 [3600*12] 9 4
i"ROW_SEQUENCER" 0 [3600*12] 9 5
i"ROW_SEQUENCER" 0 [3600*12] 9 6
i"ROW_SEQUENCER" 0 [3600*12] 9 7
i"ROW_SEQUENCER" 0 [3600*12] 9 8
i"ROW_SEQUENCER" 0 [3600*12] 9 9
i"ROW_SEQUENCER" 0 [3600*12] 9 10
i"ROW_SEQUENCER" 0 [3600*12] 9 11
i"ROW_SEQUENCER" 0 [3600*12] 9 12
i"ROW_SEQUENCER" 0 [3600*12] 9 13
i"ROW_SEQUENCER" 0 [3600*12] 9 14
i"ROW_SEQUENCER" 0 [3600*12] 9 15
i"ROW_SEQUENCER" 0 [3600*12] 9 16
i"ROW_SEQUENCER" 0 [3600*12] 10 1
i"ROW_SEQUENCER" 0 [3600*12] 10 2
i"ROW_SEQUENCER" 0 [3600*12] 10 3
i"ROW_SEQUENCER" 0 [3600*12] 10 4
i"ROW_SEQUENCER" 0 [3600*12] 10 5
i"ROW_SEQUENCER" 0 [3600*12] 10 6
i"ROW_SEQUENCER" 0 [3600*12] 10 7
i"ROW_SEQUENCER" 0 [3600*12] 10 8
i"ROW_SEQUENCER" 0 [3600*12] 10 9
i"ROW_SEQUENCER" 0 [3600*12] 10 10
i"ROW_SEQUENCER" 0 [3600*12] 10 11
i"ROW_SEQUENCER" 0 [3600*12] 10 12
i"ROW_SEQUENCER" 0 [3600*12] 10 13
i"ROW_SEQUENCER" 0 [3600*12] 10 14
i"ROW_SEQUENCER" 0 [3600*12] 10 15
i"ROW_SEQUENCER" 0 [3600*12] 10 16
i"ROW_SEQUENCER" 0 [3600*12] 11 1
i"ROW_SEQUENCER" 0 [3600*12] 11 2
i"ROW_SEQUENCER" 0 [3600*12] 11 3
i"ROW_SEQUENCER" 0 [3600*12] 11 4
i"ROW_SEQUENCER" 0 [3600*12] 11 5
i"ROW_SEQUENCER" 0 [3600*12] 11 6
i"ROW_SEQUENCER" 0 [3600*12] 11 7
i"ROW_SEQUENCER" 0 [3600*12] 11 8
i"ROW_SEQUENCER" 0 [3600*12] 11 9
i"ROW_SEQUENCER" 0 [3600*12] 11 10
i"ROW_SEQUENCER" 0 [3600*12] 11 11
i"ROW_SEQUENCER" 0 [3600*12] 11 12
i"ROW_SEQUENCER" 0 [3600*12] 11 13
i"ROW_SEQUENCER" 0 [3600*12] 11 14
i"ROW_SEQUENCER" 0 [3600*12] 11 15
i"ROW_SEQUENCER" 0 [3600*12] 11 16
i"ROW_SEQUENCER" 0 [3600*12] 12 1
i"ROW_SEQUENCER" 0 [3600*12] 12 2
i"ROW_SEQUENCER" 0 [3600*12] 12 3
i"ROW_SEQUENCER" 0 [3600*12] 12 4
i"ROW_SEQUENCER" 0 [3600*12] 12 5
i"ROW_SEQUENCER" 0 [3600*12] 12 6
i"ROW_SEQUENCER" 0 [3600*12] 12 7
i"ROW_SEQUENCER" 0 [3600*12] 12 8
i"ROW_SEQUENCER" 0 [3600*12] 12 9
i"ROW_SEQUENCER" 0 [3600*12] 12 10
i"ROW_SEQUENCER" 0 [3600*12] 12 11
i"ROW_SEQUENCER" 0 [3600*12] 12 12
i"ROW_SEQUENCER" 0 [3600*12] 12 13
i"ROW_SEQUENCER" 0 [3600*12] 12 14
i"ROW_SEQUENCER" 0 [3600*12] 12 15
i"ROW_SEQUENCER" 0 [3600*12] 12 16
i"ROW_SEQUENCER" 0 [3600*12] 13 1
i"ROW_SEQUENCER" 0 [3600*12] 13 2
i"ROW_SEQUENCER" 0 [3600*12] 13 3
i"ROW_SEQUENCER" 0 [3600*12] 13 4
i"ROW_SEQUENCER" 0 [3600*12] 13 5
i"ROW_SEQUENCER" 0 [3600*12] 13 6
i"ROW_SEQUENCER" 0 [3600*12] 13 7
i"ROW_SEQUENCER" 0 [3600*12] 13 8
i"ROW_SEQUENCER" 0 [3600*12] 13 9
i"ROW_SEQUENCER" 0 [3600*12] 13 10
i"ROW_SEQUENCER" 0 [3600*12] 13 11
i"ROW_SEQUENCER" 0 [3600*12] 13 12
i"ROW_SEQUENCER" 0 [3600*12] 13 13
i"ROW_SEQUENCER" 0 [3600*12] 13 14
i"ROW_SEQUENCER" 0 [3600*12] 13 15
i"ROW_SEQUENCER" 0 [3600*12] 13 16
i"ROW_SEQUENCER" 0 [3600*12] 14 1
i"ROW_SEQUENCER" 0 [3600*12] 14 2
i"ROW_SEQUENCER" 0 [3600*12] 14 3
i"ROW_SEQUENCER" 0 [3600*12] 14 4
i"ROW_SEQUENCER" 0 [3600*12] 14 5
i"ROW_SEQUENCER" 0 [3600*12] 14 6
i"ROW_SEQUENCER" 0 [3600*12] 14 7
i"ROW_SEQUENCER" 0 [3600*12] 14 8
i"ROW_SEQUENCER" 0 [3600*12] 14 9
i"ROW_SEQUENCER" 0 [3600*12] 14 10
i"ROW_SEQUENCER" 0 [3600*12] 14 11
i"ROW_SEQUENCER" 0 [3600*12] 14 12
i"ROW_SEQUENCER" 0 [3600*12] 14 13
i"ROW_SEQUENCER" 0 [3600*12] 14 14
i"ROW_SEQUENCER" 0 [3600*12] 14 15
i"ROW_SEQUENCER" 0 [3600*12] 14 16
i"ROW_SEQUENCER" 0 [3600*12] 15 1
i"ROW_SEQUENCER" 0 [3600*12] 15 2
i"ROW_SEQUENCER" 0 [3600*12] 15 3
i"ROW_SEQUENCER" 0 [3600*12] 15 4
i"ROW_SEQUENCER" 0 [3600*12] 15 5
i"ROW_SEQUENCER" 0 [3600*12] 15 6
i"ROW_SEQUENCER" 0 [3600*12] 15 7
i"ROW_SEQUENCER" 0 [3600*12] 15 8
i"ROW_SEQUENCER" 0 [3600*12] 15 9
i"ROW_SEQUENCER" 0 [3600*12] 15 10
i"ROW_SEQUENCER" 0 [3600*12] 15 11
i"ROW_SEQUENCER" 0 [3600*12] 15 12
i"ROW_SEQUENCER" 0 [3600*12] 15 13
i"ROW_SEQUENCER" 0 [3600*12] 15 14
i"ROW_SEQUENCER" 0 [3600*12] 15 15
i"ROW_SEQUENCER" 0 [3600*12] 15 16
i"ROW_SEQUENCER" 0 [3600*12] 16 1
i"ROW_SEQUENCER" 0 [3600*12] 16 2
i"ROW_SEQUENCER" 0 [3600*12] 16 3
i"ROW_SEQUENCER" 0 [3600*12] 16 4
i"ROW_SEQUENCER" 0 [3600*12] 16 5
i"ROW_SEQUENCER" 0 [3600*12] 16 6
i"ROW_SEQUENCER" 0 [3600*12] 16 7
i"ROW_SEQUENCER" 0 [3600*12] 16 8
i"ROW_SEQUENCER" 0 [3600*12] 16 9
i"ROW_SEQUENCER" 0 [3600*12] 16 10
i"ROW_SEQUENCER" 0 [3600*12] 16 11
i"ROW_SEQUENCER" 0 [3600*12] 16 12
i"ROW_SEQUENCER" 0 [3600*12] 16 13
i"ROW_SEQUENCER" 0 [3600*12] 16 14
i"ROW_SEQUENCER" 0 [3600*12] 16 15
i"ROW_SEQUENCER" 0 [3600*12] 16 16

metaphysician · 10 February 2024 02:31

okay, that’s a lot to chew on, but very helpful for sure! so your loop code speeds up the process of bruteforcing the row array definitions and the fourth value from the left represents the layer object. that would mean that ‘instr 1’ is the whole thing - all 16 layers.

hmmm - there are some considerations since layers need several variables besides notes whereas rows just need position info, and note on/off and velocity. most of the layer variables don’t get updated as frequently so i don’t think they have to be included in a note event for example. but it does mean that it might be a better idea if the layer could be defined as an object/instrument instead of everything set up as rows. i’m not against this method though, as long as the other layer specific data can be exchanged.

BUT another possibility might be to just set the basic framework and rather than keeping the note data in the script as an array, we just pass the notes to an instrument per layer as a live performance setup.

trying to muse on this design approach in terms of realtime performance and just thinking ONLY about notes for the time being, that would mean that the master tempo clock and any divisions/multiplications of that clock come from the script (with play and stop commands coming from Unity). then the clock info goes out to Unity in 16 separate rhythmic settings (like a tick) derived from master clock, one for each layer. those ticks then drive the playhead for each layer which increments based on seq length, figures out which rows are active with notes and then sends the array of notes back to the script. the script then assigns those notes to the instrument that’s been selected for that layer and sends the audio on its own stream to Unity.

this way the script doesn’t have to keep track of the position or even to remember what notes happen. it also doesn’t have to keep track of rows as long as an array of up to 16 notes sent to an instrument can result in a chord. all it has to do is read the assignment of the synth/SF2 player from the layer, route that to the appropriate instrument and route the audio back to a separate channel to an AudioSource. that seems like an improvement of the design to me, and possibly with much less DSP overhead. @giovannibedetti is that somewhat similar to your design aesthetic since you’re using Unity to handle sequence data?

the only other issue i would see is how to dynamically instance a synth/sf2 player so that multiple copies of a synth could be selected if desired. so if the user wanted layers 1-4 to have the same instrument timbre, it would dynamically create 4 copies of the instrument and then be able to assign each of these to each layer. i think that loop method you showed could do the trick since the variables in the loop could change.

so, let’s say we’re forgetting rows completely and instead we want:

a 16 channel clock array dependent on a master clock where each channel with a separate tempo division or multiplication goes out to Unity. i don’t know how the master tempo is actually set in the current CSD script though or in fact if or how the script handles tempo divisions. i suppose it might be possible that the output from the script might be 16th notes and then Unity handles multiplying those values, or counting ticks to get larger beat divisions which is not a bad approach.
a way to send note arrays to the script to a Csound instrument from Unity, interpreted as MIDI notes

do you see any issue with this redesign concept? latency/sync shouldn’t be a problem because nothing makes an audible sound until Csound generates the notes from the instrument.

let me know your thoughts, and how i could get started with this new approach. thanks! i’m enclosing the previous script for reference - this one was modified to allow 16 steps in the sequence.Sequencer16-sf2player.csd (9.8 KB)

giovannibedetti · 10 February 2024 08:16

This is a super interesting and useful thinking!
Yes hopefully my experiment sending sequences from Unity could be of some help. It’s a proof of concept so it can definitely be improved, but its a start.
The limitation I have noticed with that is when you have long sequences with a very small tick subdivision. If you send lots of them you will have a big overload on Csound and it will produce clicks or worse (tested on a Quest 3).
I am super busy these days but I have a simplified/clean project (with a 2D test scene with mouse and keyboard) that I will upload to GitHub hopefully this evening. I will be afk all day

metaphysician · 10 February 2024 08:40

thanks for the reply @giovannibedetti! so far the length of my sequences will always be 16 as that is the width of an individual row inside a layer, but there will be multiple layers - ostensibly that’s also 16, BUT the eventual plan is that some layers can be sound producing, while others can be effects modules or a layer that modifies the settings of another layer. also the sequence length of a layer can be set longer than 16 but that will just result in the layer waiting the remaining steps before looping back again - so a length of 21 would be the 16 notes followed by a 5 note rest.

the idea of my design idea is that you have between say 8 and 12 layers dedicated to generating pitches or samples, while the rest can be set to modify other layers or change effects parameters on those layers. however these extended concepts are not yet designed or worked out and may change later. for now the idea is to just get the notes happening in a way that hopefully doesn’t tax the system for multiple layers.

i’m not as familiar with making and using scriptable objects, though - i’ve watched some tutorials but i’ve yet to design something where a scriptable object is the solution. that’s generally how i best learn. i’m usually more comfortable with structs. but i will be interested to see what approaches you’re using!