Cabbage Logo
Back to Cabbage Site

Unity 3D music sequencer project - advice on options

#41

let me see if i can clarify point #3 in the meantime. in the case of layers i want them both to be driven by the same base clock, but i will want the clock to be divisible or multipliable per layer. let assume for the moment though that both are being driven at the same tempo and same divisor. however layer 0 cells should play with one timbre and layer 1’s cells with another. in other words, i think the cell needs properties other than simply being turned on when the playhead arrives. initially i’d like it to be aware of what layer, or instrument, it plays. additionally i’d like to add a velocity/intensity component to the cell rather than just be 100% on or off.

looking at the script it seems that it should be hooked up to the kBeat array as an additional property, so something like kInsts[kBeat] and then query that value for a number. and i think the intensity/velocity/volume change per beat could be rewritten as if kNotesAmps[kBeat] > 0. i’ll try and tinker with the Instrument idea first.

#42

ok - some progress! i proceeded down my line of reasoning, reedited the script, and i basically now have two almost independent layers happening. the ‘almost’ in this case pertains to the fact that i cannot have the same exact cell in a layer playing two instruments at the same time. i can activate cells in one layer and it will play one instrument and if i activate different cells in the second layer that are not the same i’ll get the second instrument. but if i activate a cell which is at the same position that has already been activated in the first layer, the setting for the second layer will override that of the first.

what i would like to have is 8 independent functioning layers of 8X8 points. the original example is row and cell based, to work with a single 8X8 grid. now i need 8 instances of that functionality so that each instance can be assigned the following separately - tempo, instrument, notes, volume/amp(via velocity), and panning.

i am suspecting that the unique string creation for a cell does not currently include an identifier for layer number which means that Layer 0, Row2, pos2 will leave out the layer/instrument number and looks like ‘row2 pos2’. each layer needs to be unique to itself, so that ‘layer0 row2 pos2’ is unique from ‘layer1 row2 pos2’. i think it may be how the data is formatted at the Unity end under EnableCubeToPlaySound in the MainController script. i’m maybe just a little confused about how to parse the layer out once you get it in.

anyway latest is here:Sequencer-sf2player.csd (2.4 KB)

#43

It’s all rather confusing to follow but these things usually are, especially when one combines several interfaces together. It seem the biggest issue is not being able to play two timbres at the same time? Can you not simply trigger two different sounds using two event calls as I mentioned in my previous post?

It might be time for a screencast with some voice-over explaining what’s going on? You can PM me the link if you wish to keep your work out of the public domain.

#44

okay, i’ll see what i can get together on this front. the independence of each layer is pretty crucial to how it operates. earlier in the thread i posted the data that each layer stores currently:

“LayerObject0”:{“InstNumber”:0,“SeqLength”:21,“BeatDivision”:2,“NoteList”:[12,17,19,24,26,31,34,36],“PosIntensity”:[12,17,19,24,26,31,34,36]}

so there would be eight settings like this where data is stored, one for each layer. All layers are started at one time but Seq Length and Beat Division will cause them to play their sequences at different rates. i’m just experiementing with two layers at the moment to test the concept of independence

as for different interfaces, yes it’s needed i believe, as the entire cube rotates around its center, repositioning notes within the layers. each layers settings are adjustable and remembered via an interface, which means we at least have to be able to get them to Unity and from Csound. right now the interface works fine with retrieving from/writing to JSON, so to integrate Csound i just take the values and copy each layer’s setting in Unity that i’ve retrieved from JSON file to the CSD so it can be used. interface looks something like this:

http://forum.cabbageaudio.com/uploads/default/original/1X/e075ee7a15086c23027dde80fe6c36d627dfbc88.png

here’s a youtube movie of the prototype working with PD - same one that was on Dropbox:

let me know what other information you might need on this. in the meantime i’ll see what i can come up with in terms of an explanation/screencast.

#45

okay hopefully this screencast will help explain a bit more what’s happening behind the scenes:https://dl.dropboxusercontent.com/u/14608044/ScreencapForRory-Constellation.mp4

basically it’s 8 layers of independently configurable 8X8 grid sequencers. they are slaved to a master tempo but due to the ability to change sequence length (basically to wait past 8 events to the length specified and start again) as well as tempo division the results can be considerably offset between layers. i would like to assign any instrument to any layer, and i’ll need to be able to mute the events or audio of each layer as well.

can an instr definition be itself put in an array and defined as another instr? because that’s pretty much what i need. you’ve set up one layer and it works great. if there could be 8 instances of that instr definition with a pannable output that could be addressable per instance, i think that would work fine. if not, maybe there’s another way to tackle this. i’m open to possibilities.

#46

Now I understand. Thanks for your patience. Have you tried running multiples instances of the sequencer script? CsoundUnity can run as many .csd files as you like, simply attach a new CsoundUnity component to another GameObject, then you should have another layer no? Note, there might be a slight issue with timing this way, but it migh tbe worth a try.

Alternatively you could just create 64 instances of ROW_SEQUENCER. They each use a unique array to hold notes and states, you can create as many of them as a you wish. It might also be a good idea to pass the layer number to each instance using a pfield, although it may not be absolutely necessary. So your score will look like this.

<CsScore>
;layer 1
i"ROW_SEQUENCER" 0 [3600*12] 1 1
i"ROW_SEQUENCER" 0 [3600*12] 2 1
i"ROW_SEQUENCER" 0 [3600*12] 3 1
i"ROW_SEQUENCER" 0 [3600*12] 4 1
i"ROW_SEQUENCER" 0 [3600*12] 5 1
i"ROW_SEQUENCER" 0 [3600*12] 6 1
i"ROW_SEQUENCER" 0 [3600*12] 7 1
i"ROW_SEQUENCER" 0 [3600*12] 8 1

;layer 2
i"ROW_SEQUENCER" 0 [3600*12] 1 2
i"ROW_SEQUENCER" 0 [3600*12] 2 2
i"ROW_SEQUENCER" 0 [3600*12] 3 2
i"ROW_SEQUENCER" 0 [3600*12] 4 2
i"ROW_SEQUENCER" 0 [3600*12] 5 2
i"ROW_SEQUENCER" 0 [3600*12] 6 2
i"ROW_SEQUENCER" 0 [3600*12] 7 2
i"ROW_SEQUENCER" 0 [3600*12] 8 2

;layer 3
i"ROW_SEQUENCER" 0 [3600*12] 1 3
i"ROW_SEQUENCER" 0 [3600*12] 2 3
i"ROW_SEQUENCER" 0 [3600*12] 3 3
i"ROW_SEQUENCER" 0 [3600*12] 4 3
i"ROW_SEQUENCER" 0 [3600*12] 5 3
i"ROW_SEQUENCER" 0 [3600*12] 6 3
i"ROW_SEQUENCER" 0 [3600*12] 7 3
i"ROW_SEQUENCER" 0 [3600*12] 8 3
(...)

And if you need to know which layer an instrument belongs to you can add this to the ROW_SEQUENCER instrument def:

iLayerNumber = p5

Changing the sound of each layer should also be quite straightforward. You are already do this:

kInst chnget "inst"

So can’t you simply use that to determine which sounds is played back? If you pass the kInst variable to the sf instrument you should be able to change the sound:

(...)
      kInst chnget "inst"
      if metro(kTempo) == 1 then
            if kNotesAmps[kBeat] == 1 && kNotesInsts[kBeat] == 0 then
      			event "i", "SYNTH", 0, 3, kNoteValues[p4-1]
      	elseif kNotesAmps[kBeat] == 1 && kNotesInsts[kBeat] == 1 then
      			event "i", "sf2inst", 0, 3, kNoteValues[p4-1],75, kInst
(...)

Then in your sf instrument you do something like this:

(...)
iInstr = p6 
a1,a2	sfinstr	ivel, inum, kamp*ivel, kfreq, iInstr, gisf
	outs	a1, a2
(...)

Panning and what not can also be passed to the event call, in much the same way as we just added kInstr.

I hope I am finally understanding this…

#47

well, i think we are on the right track. i’m going to pursue the one CSD script per layer idea.

honestly, ideally, since you’re using Audio Sources to output the audio it’d be really convenient to use its panning and spatial capabilities to regulate panning and routing to the mixer. at that point i could run busses and put in effects without having to involve routing it all within Csound, which would be preferable at least until i determine if the timing slop is too bad to keep layers separate. but i think that may be a revision requiring access to Unity to test in order to determine its effectiveness. the last i tried panning had no effect on a CSD file, but the demo you watched actually had hard panning per instrument in PD, so if that doesn’t appear i can get by of course.

okay so i’ll be starting work on this today. so if each script instance is a 8X8 layer then we’re looking at this line as a modified option:

if kNotesAmps[kBeat] == 1 && kNotesInsts[kBeat] == 0 then
      			event "i", "SYNTH", 0, 3, kNoteValues[p4-1]

so i imagine i can probably have a big if/then earlier to set the instrument with a string called kInstrument or something where i assign the instrument name or number and then use the event call in a more generic way using kInstrument instead of the quoted “SYNTH”. and with the sf2player i have MIDI velocity as well, and i assume i can use velocity 0 as a noteoff event.

one thing i definitely do need though is a duration parameter. i’ve found that the layer with the sf2player plays notes as if the sustain pedal is down, so i need a gate time to be added. is that just another event “i” statement?. i’m guessing the synth you made already had an envelope defined, so there was no issue as it already had a built-in decay. i can add an event containing a velocity of 0 but i’d need to be able to make it adjustable in the script with a chnget. all fairly doable i should think, just curious how you indicate duration in terms of a MIDI note on.

anyway, took a break on this Sat but i’ll start tackling the multiple script instance approach with the two layers and see how far i get before i hit a wall.

#48

You won’t need a big if/else, just pass the instrument number directly to the event call instead of 3 as I explained in my earlier post. Although I suggested trying different scripts on different objects, I think the timing will be off. You may well be better off just using one script. That way the timing will always be spot on. Take a look a the docs for the event opcode. I’m not sure you fully get how useful it is :wink: With it you can pass any amount of parameters to an instrument when you start it, including panning, duration, pitch, amp, date of birth, social security number, favorite colour, tomorrow’s winning lottery numbers…

#49

well, i started down the road of this method of one script per layer and the results so far are incredibly impressive. maybe my sequencer visual environment/ textures/particles etc. will add huge amounts of overhead to this, but it is indeed very accurate so far. here’s a quick video demonstration with 8 separate layers. i’m running layers 4 and 8 at 8Hz:

https://dl.dropboxusercontent.com/u/14608044/SequencerTest-8layers-Csound.mp4

fixed the duration - you’re right - it was easy as it’s built into the event configuration. i also have sequence length wired in, tweaked the resize function to be more accurate (transform.Find rather than GameObject.Find), and can load multiple sf2 files. so i’m pretty far along actually!

so, one thing i’m wondering is if it’s possible to pan the output of the layer, in other words the whole script, rather than just the instrument. since i am heading down this path of multiple script instances, it means each script instance will be loading no more than one instrument. maybe i should be running a pan variable to every instrument definition? or maybe create a global pan instrument and route final output to the global pan.

there are currently two issues i have with the setup:

one is that if you set the length of the sequence greater than 8 it will repeat the last note. i’d like it to just wait. i tried a couple of changes to modify the amplitude if kBeat was over 7, but it may be that Csound is actually only seeing 0-7 and doesn’t receive numbers higher than that.

second, i created a Random() function in CubeController() when i press the R key to randomly enable every cube based on a dice roll. it’s called as a BroadcastMessage from the Layer object’s MainController. the results are weird and very inconsistent, and it doesn’t appear to be a performance issue - it happens on slow moving tracks as well as fast moving ones. i wonder if it might be a message input speed limit into Csound? every cell that changes color shows up as state being true as far as i’ve found, yet only a very small amount of cubes play this way. for testing i disabled all but one layer and the method still results in Csound not picking up the change far more often than getting it right. i turned on logging and it seems to be utterly confused as to which cells are being enabled and which aren’t. however, clicking on the cube (or double clicking an already enabled one) results in proper working. all i can think is that it can’t deal with multiple messages arriving at once, or close in time. here’s a screen cap on that behavior:

https://dl.dropboxusercontent.com/u/14608044/ScreenCap-RandomMethodIssue.mp4

wondering if the EnableCubeToPlaySound function is formatting correctly when sending to Csound. i’ll include the latest script version and latest Unity scripts too.
Sequencer-sf2player.csd (5.3 KB)
UnityScripts.zip (3.4 KB)

anyway, as always any suggestions welcome. i’m definitely making progress thanks to you!

#50

I’m glad you’re making progress. Right now I’m in a remote location with no laptop, but I don’t think you can send too many messages to Csound. So I think you can rule out the message input hypothesis. I’ll check in over the next few days, but aunt be able to contribute much I’m afraid!

#51

okey doke - dang your vacationing tendencies! i’ll obviously be banging my head on this in the meantime. i’m going to try send the message in order by row so i can slow it down. pretty sure there’s something not working with the message formatting sent from Unity or processed by Csound but one click at a time it doesn’t show up for some reason.

#52

Enable Csound logging and use any of the print opcodes to print messages to Unity’s console. Between what you print from Unity and what you print from Csound you sit be able to get a clearer picture of what’s going on.

Usually my vacations wouldn’t be an issue, but this time i brought a beat up old thinkpad instead of something a little more powerful. :roll_eyes:

#53

no worries. yes i turned logging on. that’s how i found out that Csound seems really confused about which cubes are on. it’s pretty much like night and day. i also put a Debug Log statement in the EnableCubeToPlaySound function and i think it looks similarly confusing. so it might be malformed before getting into Csound. just wish i knew why clicking seems fine but calling the exact same function from a routine is a problem.

one thing i noticed in EnableCubeToPlaySound when sending to Csound is that you defined a string called channel and then reassigned different variables to the channel variable. i thought that might be a culprit:

`	public void EnableCubeToPlaySound(int row, int index, bool state)
	{	
		//Debug.Log("Cell Instrument is:"+instrument);	
		row = row +1;
		string channel = "rowCubeIndex"+row;
		csound.setChannel(channel, index);
		channel = "rowCubeNote"+row;
		csound.setChannel(channel,noteRowList[index]);
		channel = "rowCubeState"+row;
		csound.setChannel(channel, state == true ? 1 : 0);

		Debug.Log("Playing"+row+","+index+","+state);
	}`

i changed the script a bit to get the note from the Layer object but removing that bit doesn’t improve the accuracy.

#54

I don’t see an issue there as channel is changed each time it’s sent to Csound. Something must be set in the on-click event that is not happening when programmatically calling the EnableCubeToPlay method? Do you use the debugger at all? If you’re on Windows you can step through the code with Visual Studio. Not sure about Mono and OSX…

#55

i don’t think there’s any difference. when i first started having the issue i changed the code so that both the OnMouse Down event and the Randomize event call EnableCubeToPlaySound. the difference between them is essentially that one is called via Broadcast when R key is released, and the other is called when physically clicked on. there’s no extra code in the OnMouseDown event anymore, or rather i have OnMouseDown and the Randomize function call the same method. i can’t test right now as i’m waiting for another project to finish baking lightmaps, but i’ll keep hammering away at it.

#56

If you post the full code again and scripts I can take a look later.

#57

hey Rory - i posted the current state of all scripts in the earlier post (about 7 posts back) with the sequencer video clips. i have the CSD file and zipped up the Unity scripts as file called ‘UnityScripts.zip’ attached.

once the light map for this other project finishes baking, my plan of attack was to create a way to iterate in a layer in order by row, looking for objects that were enabled and then send messages to Csound that way. i was going to use a Coroutine so i could also slow the process down and see if transmission speed improved the results or consistency.

#58

okay - lightmapping utterly failed on the other project and bolloxed up my machine for an hour or so to boot. anyway i finally got around to implementing the fix by going through the objects in a layer in order, and lo and behold, it works! :slight_smile: so i’m getting even closer at this point to something i can re-integrate into my original project.

sooo, the last issue i have is the hanging on the last note of the sequence if i set the sequence to a longer length than 8 notes. i’m pretty sure this is controlled from the Csound side of things. so just to describe it plays through all the notes of the sequence but if i have say a sequence length of 12 it will simply repeat the last index value (pos7’s value) until the sequence repeats. i would like it to rest and be silent past the 8th index. i haven’t altered the CSD script from the post before. here’s the part i tried to alter:

if metro(kTempo) != 0 then
		;if amplitude/vel is greater than 0, set event
		if kBeat > 7 then
			kNotesAmps[kBeat]=0
		endif
		if kNotesAmps[kBeat] != 0 then
			if kInst == 0 then
				event "i", "inst0", 0, kDur, kNoteValues[p4-1]+kOffset		
			elseif kInst == 1 then
				event "i", "inst1", 0, kDur, kNoteValues[p4-1]+kOffset
			elseif kInst == 2 then
				event "i", "inst2", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 3 then
				event "i", "inst3", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 4 then
				event "i", "inst4", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 5 then
				event "i", "inst5", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 6 then
				event "i", "inst6", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 7 then
				event "i", "inst7", 0, kDur, kNoteValues[p4-1]+kOffset,80
			endif
			
		endif
		;set next beat if less than seqLength
		chnset kBeat, "beat"
		;advance kBeat if less than seqLength - otherwise reset count
		kBeat = (kBeat<kSeqLength-1 ? kBeat+1 : 0)

	endif

i think it may have to do with the score information you have posted at the end as well. but so far i haven’t been able to figure how to set amps for any beat past index 7 position to be 0 and then return to normal behavior at index 0. will keep trying of course.

#59

Where do you set kSeqLength? I guess you’ve checked it is the right value? How about instead of
kNotesAmp[kBeat] = 0

do

if kBeat < 7 then
		if kNotesAmps[kBeat] != 0 then
			if kInst == 0 then
				event "i", "inst0", 0, kDur, kNoteValues[p4-1]+kOffset		
			elseif kInst == 1 then
				event "i", "inst1", 0, kDur, kNoteValues[p4-1]+kOffset
			elseif kInst == 2 then
				event "i", "inst2", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 3 then
				event "i", "inst3", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 4 then
				event "i", "inst4", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 5 then
				event "i", "inst5", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 6 then
				event "i", "inst6", 0, kDur, kNoteValues[p4-1]+kOffset,80
			elseif kInst == 7 then
				event "i", "inst7", 0, kDur, kNoteValues[p4-1]+kOffset,80
			endif
			
		endif
     endif

That should stop any events from taking place at all?

#60

bingo! that did the trick. excellent! i think that mostly addresses the issues and i can go forward from here. i did want to clarify something here. a few exchanges ago i was talking about resetting the sequencer and you quoted this:

i was curious about -10 to 10 random number as the trigger. any reason?