Csound wrapper/plugin for Unreal Engine

[Andreas_Steinbrecher]

Hey there,
mayby you can help me with finding informations about the ue4 wrapper. my goal for now is to control a self made synth with csound in unreal engine. the new audioengine is running fine, i just have no clue how to setup my own csound synth in UE4 and sadly i cant find any usefull infos about that.

Any tips welcome

[rorywalsh]

Do you have an C/C++ knowledge? You will need it to use the Csound host API with UE4. My hands are tied with other projects at the moment, but I’m happy to direct you. I wonder is @virtualHC has anything he can share?

[Andreas_Steinbrecher]

yeah, i know a bit c/c++, im no pro though ^^ i am in touch with VirtualHC. thx

[rorywalsh]

Great. Hope it works out.

[Syl_Morrison]

Performing actual necromancy here (sorry if that’s frowned upon), but this is pretty much exactly what I’m trying to figure out at the moment,

@virtualHC

and buffer copy to the UE4 output buffer

was wondering @virtualHC or @rorywalsh if you could point me in the direction of Unreal’s interface to that, their docs are melting me
Hope everyone’s keeping well!

[rorywalsh]

Do you have any simple examples from thei docs? I could probably let you know what needs to be done. I haven’t looked at their API since they fixed it and made it usable. I think that was bout 3 years ago?

[Syl_Morrison]

So they’re talking about it in this, particularly the Generating sound source bit, and then here are their API docs re sound, I feel like all the stuff there overcomplicates it for me, where I basically just want to get the spout buffer, accumulate until theyre the same size and then copy that to their (very well hidden) output buffer!

[rorywalsh]

Actually, I was hoping for some sample code? Is there there the equivalent of hello world somewhere I can look at?

[Syl_Morrison]

Of Unreal in general or of the audio side? I’ve been looking for examples of audio all day but can’t find anything helpful or simple on it
(Oh wait theres some stuff here which might be of use) https://forums.unrealengine.com/development-discussion/c-gameplay-programming/18425-procedural-audio

[rorywalsh]

That thread discusses the old API, which was a crock of crap. The new one is much better, but I’m wondering what they did with the bloody examples? Have you seen Aron McLeran’s talk on YouTube, he goes through some examples there. It’s the one where he first shows the new system. I think it’s from GDC? There are code samples there. In fact, i think they are the only code samples I’ve seen. Are there really no sample plugins available with the API?

[Syl_Morrison]

I’ll check them out while I have lunch for sure, cheers for the tip! I mean you’d think were would be wouldn’t there, all the audio tutorials and stuff I’ve seen have either been blueprint based or literally like unity’s PlayOneShot method, nothing to do with accessing the output buffer

[rorywalsh]

If you have an example of a PlayOneShot in C++ that would be enough I’d say…

[Syl_Morrison]

So this is the default player pawn that gets generated in a project, has a fireShot() function with a sound associated if thats any good

CsoundUnrealPawn.cpp and 1 more file

[rorywalsh]

Can you use make FireSound a USoundWave instead of the USoundBase. Then you can access a pointer to its PCM data and write to it accordingly? I don’t have UE4 set up here at the minute so I’m can’t test, But I’ll have my gaming PC out again tomorrow, I will install it there.

[Syl_Morrison]

I’ll give this a go in a sec and send on the results, basically before that I’d also wanna trigger a score event on an instrument before doing that in the fire shot method right?

[rorywalsh]

Actually, I think this is a better bet:
https://docs.unrealengine.com/en-US/API/Runtime/Engine/Sound/USoundWaveProcedural/index.html
I have to head out for a bit now, but I can have a go at this in the morning for you if you haven’t already figured it out

[Syl_Morrison]

So I’m using that and have it no longer erroring or crashing (but still no noise), here’s some code I wrote to interface:

void AudioProcessor::processNextBlock()
{
auto csoundData = csoundWrapper.getOutputBuffer();
uint8 buffer[instanceData.ksmps];
for(auto i = 0; i < instanceData.ksmps; i++){
auto sample = *csoundData;
sample *= 128;
sample += 128;
buffer[i] = sample;
csoundData++;
}
procedural->QueueAudio(buffer, instanceData.ksmps);
}

and to schedule that:

void AudioProcessor::processBlockScheduler(std::future exitStatus)
{
auto waitTime = instanceData.ksmps / instanceData.sr;
waitTime *= 1000;
while(exitStatus.wait_for(std::chrono::milliseconds(1)) == std::future_status::timeout)
{
UE_LOG(LogTemp, Warning, TEXT(“Processing buffer…”));
std::this_thread::sleep_for(std::chrono::milliseconds(waitTime - 1));
processNextBlock();
}
}

which gets called on an std::thread, and the value of exitStatus gets set in the destructor of the actor that launches that thread, does that look like I’m in the right ballpark or am I overcomplicating this?
Edit, think I should be getting a pointer to SPOUT instead of outputBuffer

[rorywalsh]

Is your getOutputBuffer() method accessing csoundGetOutputBuffer()? It should be accessing the audio buffers using csoundGetSpin() and csoundGetSpout(). But generally I’d say this look like a valid approach…

[Syl_Morrison]

Yeah, just clocked that, with SPOUT, say I’m holding a pointer to that, can I just increment that pointer every time I want a new sample and then dereference it to get the sample’s value?
here’s the CsoundWrapper class:

CsoundWrapper::CsoundWrapper()
{
}

CsoundWrapper::~CsoundWrapper()
{

}

bool CsoundWrapper::loadCsd(std::string csdPath)
{
std::lock_guard< std::mutex > guard(csIns._mutex);
csIns.cs = new Csound();

csIns.cs->SetHostImplementedAudioIO(true, 1024);
if(csIns.cs->CompileCsd(csdPath.c_str()) == 0)
{
    csIns.cs->Start();
    csIns.csPerfThread = new CsoundPerformanceThread(csIns.cs);
    csIns.csPerfThread->Play();
    return true;
}
else{
    return false;
}

}

void CsoundWrapper::passParameter(std::string channel, MYFLT value)
{
std::lock_guard< std::mutex > guard(csIns._mutex);
csIns.cs->SetChannel(channel.c_str(), value);
}

void CsoundWrapper::passParameter(std::string channel, std::string value)
{
std::lock_guard< std::mutex > guard(csIns._mutex);
char *cVal = new char[value.size() + 1];
std::copy(value.begin(), value.end(), cVal);
cVal[value.size()] = ‘\0’;
csIns.cs->SetStringChannel(channel.c_str(), cVal);
delete[] cVal;
}

void CsoundWrapper::sendScoreEvent(std::vector p)
{
std::lock_guard< std::mutex > guard(csIns._mutex);
MYFLT pFields[p.size()];
std::copy(p.begin(), p.end(), pFields);
csIns.csPerfThread->ScoreEvent(‘i’, 0, p.size(), pFields);
std::vector().swap§;
}

MYFLT* CsoundWrapper::getOutputBuffer()
{
return csIns.cs->GetOutputBuffer();
}

MYFLT* CsoundWrapper::getSpoutBuffer()
{
return csIns.cs->GetSpout();
}

InstanceData CsoundWrapper::getInstanceData(){
std::lock_guard< std::mutex > guard(csIns._mutex);
auto sr = csIns.cs->GetSr();
auto ksmps = csIns.cs->GetKsmps();
auto nchnls = csIns.cs->GetNchnls();
return InstanceData(ksmps, sr, nchnls);
}

No idea whats going on with the formatting there hahaha, csIns is just a struct with a Csound*, a CsoundPerformanceThread* and a mutex
Edit: Swapping to SPOUT seems promising, promising in that I got a lovely new crash, saying my call to QueueAudio wasn’t in the game thread (Surely it shouldn’t be lol)

[rorywalsh]

Oh hold on. You shouldn’t be using your own performance thread for Csound. That’s a no-no, or at least I can’t think any any good reason to do so. Unless you and pain have are on extremely good terms, I would stay clear of that approach. You need to tap into Unreal’s audio processing methods. Surely there is a simple synth example out there somewhere? I mean a recent one? Btw, did you find that youtube talk? Im’ curious because I couldn’t find it earlier when I looked…