CSound Unity IOS build failure

rorywalsh · 2 December 2022 15:12

Trying to work that out, volume appears to be up full, and I can get audio from browsers, but as of yet no audio from any Unity scene…

rorywalsh · 2 December 2022 15:55

This is driving me mad I’m assuming that because I’m not getting any crashes, and because I’m not able to get audio out of even the most basic iOS scenes in an emulator that everything is actually working fine. I’m careful to make sure that the minimum iOS target is above 13. Once that is set it all looks good. I’ll see if I can get an iOS app at some point next week.

giovannibedetti · 2 December 2022 16:11

I can try later this evening on an iPad. Any other changes other than setting the iOS target to 13?

rorywalsh · 2 December 2022 16:12

So it turns out that I need to have an audio input in order for the emulator to work. My Mac Mini doesn’t have one, but I was able to use my Windows card. Now I get audio from Unity, but nothing from Csound Darn it. I was sure it was working Ok…

giovannibedetti · 2 December 2022 16:14

Setting nchnls_i maybe also?
Definitely it asked me to set the recording description in the PlayerSettings

rorywalsh · 2 December 2022 16:22

It’s strange that I get the normal output from Csound. If performKsmps wasn’t running I wouldn’t be seeing any messages. The message output I get matches exactly what I see from the desktop version. I’m going round in circles here. I can’t think what is causing the lack of audio output. Csound is running, and it is generating audio. Btw, I haven’t got a single crash with the latest hot-fix branch. Hmm, I’ll have to leave this now for a bit as I won’t have access to a Mac over the weekend.

FWIW, I’m using an M1 Mac, Unity 2021.3.15LTS, and Xcode 13.2.1.

giovannibedetti · 2 December 2022 16:24

Ok it’s not bad news though
I will give it a go later and report here my results

rorywalsh · 2 December 2022 16:33

One last experiment here. I added some print info to the instrument block. It displays in the editor but not in the XCode debug console. So perhaps performKsmps() is not be called

[edit] debugging this is a bit of a nightmare. Can I step through Unity’s C# code in Xcode? I’ve no idea where to start in terms of actual debugging here…

giovannibedetti · 2 December 2022 16:45

Be sure to enable Csound Log Output in the CsoundUnity settings.
To debug C# code you should use Visual Studio for Mac with Unity tools installed.
There you should be able to attach to the running instance.
I think you should create a development build and also tick the “Wait for Managed Debugger” box.
I’ll try later!

David_Harris · 2 December 2022 17:07

I pulled the fix/ios_builds from github, and I built the basic sample. I still get the same error as far as I can tell on my ipad - in the output window in XCode I get this

CSoundIOS(2057,0x10127b880) malloc: can't allocate region
:*** mach_vm_map(size=3882893312, flags: 100) failed (error code=3)
CSoundIOS(2057,0x10127b880) malloc: *** set a breakpoint in malloc_error_break to debug
Csound zerdbfs: 0
CsoundUnity:Awake()

CsoundUnity done init, compiledOk? True
CsoundUnity:Awake()

With the UnityFramework`csoundGetSpoutSample: Thread 33: EXC_BAD_ACCESS (code=1, address=0x0)

I also had a go at running it in the simulator, and it ran succesfully but muted due to some issue with my audio drivers - it tries to use the driver for a usb audio card which isn’t pluged in and refuses to use the alternative device for the output.

giovannibedetti · 2 December 2022 17:50

Thanks @David_Harris for your test.
That is exactly what is happening, for some reason the memory allocation fails, but I have no clue on why that is happening.
I will try with different Unity version.
And then I will open a Unity issue if I am not able to fix this on Sunday

giovannibedetti · 2 December 2022 20:13

Just discovered that iOS audio runs at 24000Hz

giovannibedetti · 2 December 2022 20:37

And here is the issue:
setting the params apparently fails.

CsoundUnity Awake

AudioSettings.bufferSize: 1024 numBuffers: 4

Csound created and started.
AudioSettings.outputSampleRate: 24000
GetSr: 5.86304302748164E-315
GetKr: 5.86304302748164E-315
Get0dbfs: 5.26354424712089E-315
GetKsmps: 1

giovannibedetti · 2 December 2022 20:48

The crash is gone, but still I have no audio.
Same branch

giovannibedetti · 2 December 2022 22:12

Some logs after disabling SetParams:

CsoundUnity Awake

AudioSettings.bufferSize: 1024 numBuffers: 4

Csound created and started.

AudioSettings.outputSampleRate: 44100
GetSr: 5.86304302748164E-315
GetKr: 5.65581687602019E-315
Get0dbfs: 5.26354424712089E-315
GetKsmps: 32

CsoundUnityBridge:.ctor(String, List`1)

–Csound version 6.18 (float samples) Nov 30 2022
[commit: 2901a4a1f2961ac11cb92dacce87962d6f54002d]

Csound zerdbfs: 0
CsoundUnity done init, compiledOk? True

libsndfile-1.0.29pre1

displays suppressed

sr = 24000.0,
kr = 750.000,
ksmps = 32
0dBFS level = 1.0,
A4 tuning = 440.0

orch now loaded

audio buffered in 1024 sample-frame blocks

SECTION 1:

ftable 1:

new alloc for instr 1:

So all looks correct to me, maybe to be able to read and set sr it needs at least a control cycle.
Here is displaying correctly the sr I set in the csd.
But still no audio output.

EDIT!
So the issue is that we receive 0 from GetOutputSample in ProcessBlock.
I’m opening an issue on Csound github.

Here it is:

rorywalsh · 2 December 2022 22:20

Nice detective work @giovannibedetti

giovannibedetti · 3 December 2022 11:10

And we have sound!
I will clean the branch tomorrow and merge into master, but it’s good to go.

giovannibedetti · 3 December 2022 12:15

So the issue was inside SetParams, the struct had some double values there that were causing the sr and kr and 0dbfs to be 0 on iOS.
Strange that this doesn’t have the same effect on Android.

github.com

rorywalsh/CsoundUnity/blob/9f1edde5f1326502e5a4cd912590de5a9054b241/Runtime/CsoundUnityBridge.cs#L823


}


[Flags]
public enum ChannelDirection
{
    Input = 1,
    Output = 2
}


[StructLayout(LayoutKind.Sequential)]
public class CSOUND_PARAMS
{
    public int debug_mode;     /* debug mode, 0 or 1 */
    public int buffer_frames;  /* number of frames in in/out buffers */
    public int hardware_buffer_frames; /* ibid. hardware */
    public int displays;       /* graph displays, 0 or 1 */
    public int ascii_graphs;   /* use ASCII graphs, 0 or 1 */
    public int postscript_graphs; /* use postscript graphs, 0 or 1 */
    public int message_level;     /* message printout control */
    public int tempo;             /* tempo (sets Beatmode)  */
    public int ring_bell;         /* bell, 0 or 1 */

I tried some samples, the Env Variables sample doesn’t work. I will investigate later.
Merging to master in some minutes.

rorywalsh · 3 December 2022 13:40

Well playing @giovannibedetti Great work

David_Harris · 5 December 2022 11:30

Great work @giovannibedetti thank you so much for fixing this.

I’ve just tested and that does solve the crashing I was having and now the test projects in the samples folder are all running with sound on my iPad Air 2.

I think the crash was only caused by the float / double issue for me though.

I tried removing the sample rate fix (as I wanted to be able to output higher than 12kHz audio) in the source and setting it to 48000Hz in Unity and the csd files and it worked fine on my iPad. So I don’t think you need that to be hardcoded, I’m not sure how you can confirm what the possible sample rates for different IOS hardware are though. According to the Apple docs “The available range is device dependent and is typically from 8000 through 48000 hertz” but with no other advice!
https://developer.apple.com/documentation/avfaudio/avaudiosession/1616523-setpreferredsamplerate

I read an anecdotal reply in a forum that mentioned newer iPhone X and above era run at 48000 by default whereas previous to that they ran at 44100, but can’t find clarification in Apple docs.

I was wondering if maybe just outputting a warning before setting the values would be enough?

#if !UNITY_IOS
        if( AudioSettings.outputSampleRate != 24000f)
        {
            Debug.Log("Csound setting sample_rate_override to user defined value, some values maybe incompatible with older hardware.");
        }
#endif

Happy to send a pull request with that in if it helps?