Cabbage Logo
Back to Cabbage Site

Ideas for saving generic data in DAW sessions

Following from this discussion, I’ve started working on a system for saving generic data whenever a plugin is saved as part of a DAW session. We declare as JSON widget:

<Cabbage>
json channel("jsonData")
form ...
</Cabbage>

That channel’s data will be written to the plugin state memory whenever a session is saved. To make it easy to write and write to that string I’ve started writing a few utility opcodes. Here is an example of how they work.

instr WriteJSON
    //overwrite existing data
    iRes writeJSONToChannel 0, "jsonData", "{ \"happy\": true, \"pi\": 3.141}"

    if iRes == -1 then
        prints "Couldn't write JSON data"
    endif
endin

instr UpdateJSON
    //updated existing data
    iRes writeJSONToChannel 1, "jsonData", {{
        { 
        "happy": true, "pi": 1.141, "test": 12
        }
    }}

    if iRes == -1 then
        prints "Couldn't write JSON data"
    endif

endin

instr ShowJSON
    prints chnget:S("jsonData")
endin

instr GetJSONFloatValue
    kVal getJSONFloatValue "jsonData", "pi"
    printk2 kVal
endin

I just wonder if the opcodes names are a little long? Note these opcodes will only be accessible in Cabbage. They’ll be baked into the code-base. I will add string, and array getters too. I’m just putting it out there now so we can decide on opcodes names and other use case I may have overlooked. I prefer to use of camelCase naming because a) it differs from the majority of Csound opcodes, which helps make it clear that these don’t belong to Csound, and b) I find them easier to read
:wink:

Let me know what you think. .

Let me introduce a new set of Cabbage opcodes that will let users save generic data across DAW sessions.


Basic read/write opcodes
These opcodes perform basic reading and writing of ‘flat’ JSON object.


iRes writeStateData iMode, SJSONData

Will write JSON data to the plugins internal state if it saved in a DAW session. It lets you overwrite the existing data, or update it.

SData readStateData
Returns a string that containing all current state data


Value reading/writing
These opcodes let you read write and access values from the JSON data structure. While writeStateData (see above) lets you write chunks of JSON data in one go, it also requires you to construct your JSON data in Csound which is a little cumbersome. These opcodes are probably better for filling the data structure.


iRes setStateValue SKeyName, SData
kRes setStateValue SKeyName, kData
iRes setStateValue SKeyName, iData
iRes setStateValue SKeyName, SData[]
kRes setStateValue SKeyName, kData[]

iRes will return -1 one if there were any errors.


iVal getStateValue SKeyName
kVale getStateValue SKeyName
SVal getStateValue SKeyName
iVal[] getStateValue SKeyName
kVal[] getStateValue SKeyName
SVal[] getStateValue SKeyName


There are available in the latest beta build. Note they are only useful if saving session data within a DAW. I’ve also added a simple JSONOpcodes.csd file to the Misc examples folder.

Hey Rory,
It’s really cool that storing data within the DAW session is now possible. It’s actually necessary for the sampler-ish plugin I’m programming.
But I’m coming across some issues with recalling string values. Actually I need to recall a whole array of strings containing file paths of the individual samples loaded into the plugin.

First there is the problem that in order for the string to be stored/recalled (not sure where the problem occurs) the string has to be set into double quotes. So I need to use something like sprintfk “”%s"", String to make it happen and that I can’t do with an array at once. So I need to store them individually as I load them.
I would probably do it like that anyway, but an issue occurs when I try to recall the data singly back into an/the array in a while loop. When ever I left a blank “sample slot” the getStateValue returns nothing and the String from the last successful read gets assigned again to the next “sample slot/s” which are actually supposed to be empty or “”. I hope I’m making sense here!

I’ve attached a sample csd file of the issue and a video of what’s happening with it in the DAW.
You can see that the unsuccessful reads are reported in the cSound output. It would be good to somehow get this returned from the opcode to handle “empty slots” in the code.

Another bug seems to be that the very first string always get cut short, but only the first time it’s read. Not sure if I’m using the right terminology here, but it’s also shown in the video.
GetSetState.csd (3.5 KB)


btw: How do you get the codes displayed so nicely in this forum? When I use the </> button and copy paste from my code it gets randomly formatted and not formatted.

I guess I could do a workaround and create a boolean array which just holds information about IF a index was loaded with a sample, store that and then recall it before I recall any samples to only read the ones that were actually in the session.
Still, do you think it could be possible to get a “nil” return or something, if the json data isn’t available?
The first-read-cut-short-issue I work around by just reading the data twice with a bit of a delay. That actually works.

That sounds like something that might get extremely annoying over time! I think I should sort that!

Thanks, I’ll take a look at get back to you when I have an update.

Let me know when you fix this. I worked my way around it by first adding the quotes for storing and then removing them after recall.
Programming the recalling procedure was extremely annoying until I found out what works with iVars and not with kVars and the other way around. But [drumroll] this works:

 instr restorePaths
     icnt        init 0
     SArray[]    init 26
     while icnt < 26 do
         SKey            sprintfk "filepath(%d)",icnt
         SArray[icnt]    sprintfk "%s",getStateValue:S(SKey) ;temporary array just to be able to read json data
         icnt            = icnt + 1
     od
     kcnt        init 0
     while kcnt < 26 do
         if(gkSampleSize[kcnt] != 0)then                     ;can only be checked with k-vars
             kLength         strlenk     SArray[kcnt]
             gSArray[kcnt]   strsubk     SArray[kcnt], 1, kLength-1  ;assigning temp. array to path-array and cut quotes
         endif
         kcnt            = kcnt + 1
     od 
 ;printarray  gSArray
 endin`

I have an array of the sizes of my samples that I store now in order to select the paths for recall. Have to do this to avoid duplication of paths when the getStateValue opcode doesn’t return anything and the prior read would get assigned again. Have to do it in two loops though because it seems that i can’t check my k-var array for the sample sizes in an i-var loop. I’m using a lot of k-rate stuff to work with the sample size throughout my plugin, so i need it to be k-rate. Anyway, blabla, works this way. Just let me know when you change the double quote issue.

I’m not seeing any issues with double strings? Can you test this simplified .csd file?

CabbagePlugin.csd (1.4 KB)

I found the reason for this. It was working for me because all of my strings were the same length. The same thing started happening when I tried strings of various lengths. It’s fixed now in git. And note that I’ve also updated the name of these opcode so they start with a ‘cabbage’. It just makes it easier to see they are unique to Cabbage. I don’t want to confuse general Csound users!

Hi Rory,
The cutting short of the strings doesn’t happen anymore.

But now, my DAW doesn’t recall any of the data. Everything’s NULL

In case you need a script:

ksmps = 32
nchnls = 2
0dbfs = 1

massign	0,0

opcode UpdateDisplay, 0, kS                                         // opcode to change the displayed
    SIdentNumber    init "number"
    SIdentDisplay   init "filedisplay"
    
    kNumber, SPath  xin
    
    String          sprintfk    "text(\"%d\")",kNumber              // index
    cabbageSet      1,    SIdentNumber, String
    String          sprintfk    "text(\"%s\")",SPath                // and string / filepath
    cabbageSet      1,    SIdentDisplay, String
endop


instr 1

    kNumber         init 0                                                                                      // index for string array
    SPath[]         fillarray "n/a","n/a","n/a","n/a","n/a","n/a","n/a","n/a","n/a","n/a"
    kExists[]       init 10                                                                                     // array to flag that a string has been stored
     
    SFileDropped        chnget "LAST_FILE_DROPPED"
    if(changed(SFileDropped) == 1)then                                                                          // load filepath to string via drag & drop
        SPath[kNumber]  sprintfk "%s",SFileDropped
        UpdateDisplay   kNumber, SPath[kNumber]                                                                 // update GUI
        SInstrCall      sprintfk " i \"StoreData\" 0 0 \"%s\" \"path(%d)\"",SFileDropped,kNumber                // call instrument to store string
        scoreline       SInstrCall, 1                                                                           // via SetStateValue opcodes
        kExists[kNumber] = 1                                                                                    // set a flag for a path being loaded to this index
        cabbageSetStateValue "exists", kExists                                                                  // and store the whole array
        kNumber         limit kNumber + 1, 0 , 9
    endif
  
    kNumber = (changed(chnget:k("next")) == 1 && kNumber < 9)? kNumber + 1 : kNumber        // change active index 
    kNumber = (changed(chnget:k("prev")) == 1 && kNumber > 0)? kNumber - 1 : kNumber        // with buttons
   
    if( changed(kNumber) == 1 ) then                                                        // on index change 
        UpdateDisplay   kNumber, SPath[kNumber]                                             // change displayed index number and filepath
    endif
    // print stored data
    if( changed(chnget:k("print")) == 1) then                                               // call instrument to print the stored data
        prints "\n"
        event "i","printStoredData",0,0.001
    endif
    if( changed(chnget:k("print2")) == 1) then                                               // call instrument to print the stored data
        prints "\n"
        event "i","printStoredData2",0,0.001
    endif

endin


instr StoreData
    SData           = p4 
    SKey            = p5
    SDataInQuotes   sprintfk "\"%s\"", SData                    // String has to be in Quotes for setStateValue to work in version 2.5.29
    cabbageSetStateValue SKey, SData;InQuotes           // This is not the case in 2.5.31 - 2.5.34,
endin                                                           // but in those versions nothing is recalled in the DAW


instr   printStoredData                                         // Instrument to print stored data
    StoreTmp[]      init 10
    Store[]         init 10
    kExists[]       init 10                                    // can't pass StateValue to i-var
    iCnt            init 0
    
    kExists         cabbageGetStateValue "exists"
    
    while (iCnt<=9) do
        SKey        sprintfk "path(%d)",iCnt
        StoreTmp[iCnt] sprintfk "%s", cabbageGetStateValue:S(SKey)  
        prints "------\n%s\n------\n", StoreTmp[iCnt]; getStateValue:S(SKey)       
        iCnt = iCnt +1                                         
    od
    
    kCnt            init 0                                      // so I need another k-var loop to use the array for determining
    while (kCnt<=9) do
        if(kExists[kCnt] == 1)then                              // which indexes where loaded in the string array
            Store[kCnt] sprintfk "%s", StoreTmp[kCnt]     
        endif      
        kCnt = kCnt +1                                   
    od
    prints "\n\n_____________________________________________________________________________________\n"
    printarray Store
    prints "\n\n_____________________________________________________________________________________\n"
    printarray StoreTmp
    
    
endin

instr   printStoredData2
    prints "\n\nPrinting using readStateData\n"
    prints cabbageReadStateData:S()
endin

Out of the frying pan and into the fire :rofl: I’ll take a look. The last time I checked the example in the Misc folder it was working Ok.

I took the example file and changed some lines:

instr 2

    kSlider chnget "sliderValue"

    ;if changed2(kSlider) == 1 then
    ;    event "i", "SetFloatValue", 0, 0
    ;endif
    if changed:k(chnget:k("storeData")) == 1 then
        event "i", "SetFloatValue", 0, 0
    endif
    
    if changed:k(chnget:k("showData")) == 1 then
        event "i", "ShowData", 0, 0
    endif

endin

I took out that changing the slider would immediately store the data, because that would happen also on init run with the stored position of the slider, which could lead to the false assumption that the data was restored correctly while it was written on init run.

So I added a button for saving that “pi” value, and a cSound Output:

form caption("Internal State Opcodes") size(400, 400), pluginId("def1")
rslider bounds(14, 14, 100, 100), channel("sliderValue"), range(0, 1, 0.5, 1, 0.01), text("SliderValue"), trackerColour(0, 255, 0, 255), outlineColour(0, 0, 0, 50), textColour(0, 0, 0, 255)
button bounds(122, 16, 120, 40), channel("showData"), text("Print State Data")
button bounds(250, 16, 120, 40), channel("storeData"), text("Store State Data")
csoundoutput bounds(0, 150, 400, 250)

With this it still looks like my DAW isn’t recalling any of the data.
Sorry for pointing to another annoying blib. Thanks for taking care of it!

Can you upload your full .csd, it will save me from having to perform surgery on my one!

of course:
GetSetStateXmple.csd (2.5 KB)

1 Like

Thanks Stefan. I’ve fixed this now. Should be available in Azure shortly. :+1:

Hi Rory,
I’m still experiencing issues with the cabbageGet(/Set)StateValue opcodes. It drove me crazy because some things were working and some not, but I might have an idea about the issue now. It’s actually not connected to the cabbageGet(/Set)StateValue opcodes, but to the cabbageGetValue opcodes. So in my plugin I store scalar data arrays because the different “sample-slots” enjoy different settings. Whenever I change a setting I store the whole array.
Reading the Statedata before closing and reopening the session was always correct, but afterwards everything seemed lost.
At first the issue revealed to be as simple as this:

When I use cabbageGetValue, querying the change gets initialised and the State Data gets overwritten with the array that’s been initialised and therefore empty and my state data is lost.

When I use chnget, initialisation seems to not run what’s inside the change query and the data keeps intact.

So I made a sample script with both methods and then both of them worked. Which was even more confusing. Then I changed it back to only using cabbageGetValue and the problem reappeared.
So at least for my sample script it looks like if only oone of the channels gets queried by chnget the other cabbageGetValue also doesn’t get a “changed” flag on init, but if you only use the new opcodes that happens. Maybe it’s even more complex. I’ll just go back to chnget for my sliders for now and hope that’s gonna solve the problem with my plugin for now. I just wanted to bring your attention to this. Attached the script I used to experiment with this.GetSetStateOpcodes.csd (2.5 KB)

OK. swapping back to chnget also doesn’t solve my problem. Seems like my problem is more complicated/something else. Don’t bother for now!
I’ll let you know when I get more clarity…

Hi Stefan. First of all, thanks for persisting with this. I think for now you are the only person using these opcodes, so your patience is most welcome. I first tested your example here with a VST2 export and Reaper and it worked fine. Then I made some quick edits, and it no longer works, so yeah, something is not quite right. I can reproduce the problem here so let me work it out.

Ok, I think I found the problem, and it has nothing to do with chnget or cabbageGetValue. It’s here:

if( changed(chnget:k("print")) == 1) then                                               // call instrument to print the stored data
            event "i","printStoredData",0,0
            gkchngetArray cabbageGetStateValue "chnget"
            gkcabbageGetArray cabbageGetStateValue "cabbageGet"
endif

Each time you trigger this it replaces the global arrays with whatever is currently set internally. Don’t do this and everything works fine. Here is the example I used, based largely on yours but I had to change the variable names because I found all those kchnget’s to be quite confusing :rofl:
GetSetStateOpcodes.csd (2.4 KB)

I’m just about to use them! :thinking: Wish me luck!

Great. The more the better. I love bug reports :stuck_out_tongue_winking_eye: