CSound Unity IOS build failure

David_Harris · 28 November 2022 10:35

Hi all, hoping someone might be able to point me in the right direction for building CSound Unity on IOS.
I’ve been looking for a sound generation platform (synth, drums & engine noises) for a Unity project and wanted to test out CSound performance on an iPad.

I’ve taken the Theremin sample and attempted to compile and run (its all working fine in editor on a Mac). The project gets created fine in Unity and then baulks in XCode.

I tried swapping out the libsndfile.a and libcsound.a from Csound 6.18.0 to 6.17.0 but that gave the same error, had a (very) quick dig into the csound repo but my c++ skills are woeful

The final error output is this: csound-ios-compile-issue.txt.zip (19.8 KB)
(full text is over a meg)

Undefined symbols for architecture arm64:
  "_vDSP_fft_zrip", referenced from:
  _csoundRealFFT2 in libcsound.a(fftlib.o)
  _vDSP_DCT_execute in libcsound.a(fftlib.o)
  "_vDSP_create_fftsetup", referenced from:
  _csoundRealFFT2Setup in libcsound.a(fftlib.o)
  "_vDSP_destroy_fftsetup", referenced from:
  _setupDispose in libcsound.a(fftlib.o)
  "_csoundLoadPlugins", referenced from:
  _CsoundUnity_LoadPlugins_m13BAF490EE306782405D86636896CE744656D8E2 in com.csound.unity.o
  _CsoundUnityBridge_LoadPlugins_mAAF7A94D9172D9A13C86F6D261CFE8893D5CE025 in com.csound.unity.o
  _NativeMethods_csoundLoadPlugins_m19D2E988F8523A80D55C0356E98EAFBE41390D11 in com.csound.unity.o
  _U3CU3Ec__DisplayClass3_0_U3CSetEnvironmentSettingsU3Eb__0_mE4F05EB5F4219EAA61114ADA9969042CD936FEFD in com.csound.unity.o
 (maybe you meant: _NativeMethods_csoundLoadPlugins_m19D2E988F8523A80D55C0356E98EAFBE41390D11)
  "_csoundSetOpcodedir", referenced from:
  _CsoundUnityBridge__ctor_mE785F06E63D4DBF87CED070292CB7F0FD2914CD2 in com.csound.unity.o
  _NativeMethods_csoundSetOpcodedir_m5336AAF9ED7C0536A7B300879F16E19553CC694A in com.csound.unity.o
 (maybe you meant: _NativeMethods_csoundSetOpcodedir_m5336AAF9ED7C0536A7B300879F16E19553CC694A)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

rorywalsh · 28 November 2022 10:52

Where exactly did you find the libsndfile.a and libcsound.a files? Also, can you run ‘file’ on them from the command line to see if they contains an arm64 slice? Perhaps there are x64?

David_Harris · 28 November 2022 11:00

Hi Rory, thanks for the super quick reply (and all your amazing work on here!)

The plugin files in my Unity project are in the package cache (I installed CsoundUnity from git in the Unity package manager)
/Library/PackageCache/com.csound.csoundunity@9f1edde5f1/Runtime/iOS/

so they’re from here

the ones I tried swapping out for were from the csound (ios) releases on https://github.com/csound/csound/releases. I md5’d the ones in the CsoundUnity folder and noticed they were the same as the ones in the 6.18.0 csound release (but swapping them was probably overly hopeful).

Running file on them gives me this
david$ file libcsound.a
libcsound.a: Mach-O universal binary with 5 architectures: [arm_v7:current ar archive random librarycurrent ar archive random library] [arm_v7s] [i386] [x86_64] [arm64]
libcsound.a (for architecture armv7): current ar archive random library
libcsound.a (for architecture armv7s): current ar archive random library
libcsound.a (for architecture i386): current ar archive random library
libcsound.a (for architecture x86_64): current ar archive random library
libcsound.a (for architecture arm64): current ar archive random library
david$ file libsndfile.a
libsndfile.a: Mach-O universal binary with 5 architectures: [arm_v7:current ar archivecurrent ar archive] [arm_v7s] [i386] [x86_64] [arm64]
libsndfile.a (for architecture armv7): current ar archive
libsndfile.a (for architecture armv7s): current ar archive
libsndfile.a (for architecture i386): current ar archive
libsndfile.a (for architecture x86_64): current ar archive
libsndfile.a (for architecture arm64): current ar archive

David_Harris · 28 November 2022 11:15

Incase its helpful I’m on
XCode Version 13.4.1 (13F100)
macOS Monterey 12.5

rorywalsh · 28 November 2022 12:01

It looks like they are the right architecture. I’m going to ping @giovannibedetti as I think he has successfully built sample scenes for iOS

giovannibedetti · 28 November 2022 12:25

Hi @David_Harris,
It looks like the error is related with some Environment Variable.
What settings do you have there? (Under CsoundUnity Settings)

David_Harris · 28 November 2022 12:57

Hi @giovannibedetti,
Not sure where to look for CsoundUnity Settings. In Xcode the Environemnt Variables are empty, in the Unity Player settings, its using IL2CPP for compiling to XCode, .Net Standard 2.1, Minimal Stripping Level.

I did try building the Csound library from source (https://github.com/csound/csound) running the build.sh script in the IOS folder, it built libcsound.a and I tried using that in the XCode project (incase some stripping via Unity had happened) which did remove a couple of the errors but XCode still complains of:

Undefined symbols for architecture arm64:
“_vDSP_fft_zrip”, referenced from:
_csoundRealFFT2 in libcsound.a(fftlib.o)
_vDSP_DCT_execute in libcsound.a(fftlib.o)
“_vDSP_create_fftsetup”, referenced from:
_csoundRealFFT2Setup in libcsound.a(fftlib.o)
“_vDSP_destroy_fftsetup”, referenced from:
_setupDispose in libcsound.a(fftlib.o)
ld: symbol(s) not found for architecture arm64

I’m way out of my depth (so this is probably nonsense) but in the csound code I can see a lot of defines wrapping the various funtion calls

github.com

csound/csound/blob/master/OOps/fftlib.c#L3501-L3523


void csoundRealFFT2(CSOUND *csound,
                     void *p, MYFLT *sig){
  CSOUND_FFT_SETUP *setup =
        (CSOUND_FFT_SETUP *) p;
  switch(setup->lib) {
#if defined(__MACH__)
  case VDSP_LIB:
    vDSP_execute(setup,sig);
    break;
#endif
  case PFFT_LIB:
    pffft_execute(setup,sig);
    break;
  default:
    (setup->d == FFT_FWD ?
      csoundRealFFT(csound,
                     sig,setup->N) :
      csoundInverseRealFFT(csound,
                     sig,setup->N));
    setup->lib = 0;

This file has been truncated. show original

I’m wondering if there could be a missing definition wrapping the referenced functions

David_Harris · 28 November 2022 13:08

Ah okay, I think the issue is I’m missing the accelerate framework in XCode. “_vDSP_fft_zrip()” is part of that library. I will beaver away and see how I sort that out and report back.

giovannibedetti · 28 November 2022 13:11

I built CsoundUnity for iOS long time ago, unfortunately I have no access anymore to a developer account so I’m a bit blindfolded atm.
But I’m happy that someone is using CsoundUnity on iOS
I’m sure we will understand what’s wrong.
Can you try building the official Csound iOS project and see if it works? https://github.com/csound/csound/releases/download/6.18.0/Csound-iOS-6.18.0.zip
We can start from there.

Btw the environment settings are there (and indeed they are part of the recent updates):

Have a look here: http://rorywalsh.github.io/CsoundUnity/#/environment_variables

giovannibedetti · 28 November 2022 13:26

I’m wondering why those two methods are not available for arm64:

csoundLoadPlugins
csoundSetOpcodedir

FYI we are not using them on iOS when Csound is created.

David_Harris · 28 November 2022 15:36

The project compiled once I added in the Accelerate framework

It has to be in the General > Framework and Libraries (not the Build Phase)

Those functions _vDSP_fft_zrip, _vDSP_create_fftsetup, _vDSP_destroy_fftsetup can now be found and it compiles

David_Harris · 28 November 2022 15:43

Unfortunately I’m now getting some nasty errors in the console and the app crashes, still some progress

Unloading 5 Unused Serialized files (Serialized files now loaded: 0)
UnloadTime: 7.019167 ms
CSoundPerf(1495,0x1015af880) malloc: can't allocate region
:*** mach_vm_map(size=3614457856, flags: 100) failed (error code=3)
CSoundPerf(1495,0x1015af880) malloc: *** set a breakpoint in malloc_error_break to debug
--Csound version 6.18 (float samples) Oct 20 2022
[commit: f0bea4d87b49da2be7e9f1fc77826a7327cbc391]
<Logging>d__160:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
CsoundUnity:Awake()

Csound zerdbfs: 1
CsoundUnity:Awake()

CsoundUnity done init, compiledOk? True
CsoundUnity:Awake()

(lldb)

XCode shows me these compiled lines as the source of the crash

// Native function invocation
	float returnValue = reinterpret_cast<PInvokeFunc>(csoundGetSpoutSample)(___csound0, ___frame1, ___channel2);

and if I print out the ___csound0 and ___channel2 variables it tells me “the register x0 is not available”

I’ve not much experience of debugging in xcode but if there’s anything that I can look into let me know, I’m in way over my head now for sure

giovannibedetti · 28 November 2022 16:36

Good find, this must be in the docs then!
Ideally we should add a PostProcessBuild action to add the framework to the exported XCode project.

About the error, we experienced something similar on Android. I’ll have to find a developer account to be able to test this on my side, but it could be simply related with a missing setting on the csd.
Can you check that those lines exist in the section?

sr = 44100 (or whatever sampling rate you have)
ksmps = 32 (or whatever sr/kr ratio you have - as integer, try different values here because not all ratios could be supported, 1 is a good bet)
nchnls = 2

Edit: Developer account found, will test this later this evening and get back to you

David_Harris · 28 November 2022 17:00

Great. I’m using Unity 2021.3.13f.
I managed to run without crashing. I think the issue was the sampling rate being set to “0” in the project settings, I haven’t modified that so I assume it must be like that for new projects (I created a fresh project then imported CsoundUnity into it for testing).

Also I updated the sampling rate in the csd file via the Unity inspector, I think that also caused some issue as when I opened the file up in my IDE it had some non-printable character at the end, which I thne deleted.

I think you’re onto the root of the issue, modifying the sr and ksmps does seem to stop the crashing from happening, however I haven’t found a value that gets me audio generating (I haven’t tried “1” yet)

I did try creating a build post process, but it adds the framework into the “Unity-iPhone” target rather than the “UnityFramework” target, I’m attaching it incase its useful (it needs to go into an “Editor” folder). Gotta sign off today but will have another crack at it at somepoint this week.

ios_build_accelerate_framework.cs (2.1 KB)

rorywalsh · 29 November 2022 14:29

Hi @David_Harris, so it’s compiling, the scene runs, but you are getting no audio?

David_Harris · 29 November 2022 17:36

Once the Accelerate framework is added in XCode it compiles and is uploaded to my iPad. From there it seems hit and miss to either run without any audio or just crash. It does seem to depend on the settings in the csd, although the build that didn’t crash yestarday crashed today. I haven’t managed to pin down what the cause is, the crashes are always at the same point, the same as I noted above.

The only other useful bit of info is the call stack of the crash, this is the IL2CPP c++ compiled code but I think the method names are preserved from the C#

#0	0x0000000103511bd4 in csoundGetSpoutSample ()
#1	0x0000000103ba5f2c in NativeMethods_csoundGetSpoutSample_mAF972518DA7168A8264F15DE902EFD24B952AFBF [inlined] at /Volumes/samone/Projects/Unity/CSound/Builds/IOSE/Classes/Native/com.csound.unity.cpp:17174
#2	0x0000000103ba5f28 in CsoundUnityBridge_GetSpoutSample_m413A42DC0A5C6B9A7BAEBBFCCE8A7F1CDB05DF06 [inlined] at /Volumes/samone/Projects/Unity/CSound/Builds/IOSE/Classes/Native/com.csound.unity.cpp:14064
#3	0x0000000103ba5f24 in CsoundUnity_GetOutputSample_m2B3796E4F6C067CD65FC11C4F9578A0B20A3EB25 [inlined] at /Volumes/samone/Projects/Unity/CSound/Builds/IOSE/Classes/Native/com.csound.unity.cpp:6701
#4	0x0000000103ba5f24 in ::CsoundUnity_ProcessBlock_m3166840E2ADDD2197D09E05894E74F4542E3962C(CsoundUnity_t640C9ECC15E6B894414145782A4E47A1B300119F , SingleU5BU5D_t89DEFE97BCEDB5857010E79ECE0F52CF6E93B87C , int32_t, const RuntimeMethod *) at /Volumes/samone/Projects/Unity/CSound/Builds/IOSE/Classes/Native/com.csound.unity.cpp:10498
#5	0x00000001032beeec in il2cpp::vm::Runtime::InvokeWithThrow(MethodInfo const, void, void**) at /Users/bokken/build/output/unity/unity/External/il2cpp/builds/libil2cpp/vm/Runtime.cpp:567
#6	0x00000001032bed4c in il2cpp::vm::Runtime::Invoke(MethodInfo const, void, void, Il2CppException) at /Users/bokken/build/output/unity/unity/External/il2cpp/builds/libil2cpp/vm/Runtime.cpp:553
#7	0x0000000102a848f8 in ::scripting_method_invoke() at /Users/bokken/build/output/unity/unity/Runtime/ScriptingBackend/Il2Cpp/ScriptingApi_Il2Cpp.cpp:292
#8	0x0000000102a90438 in ::Invoke() at /Users/bokken/build/output/unity/unity/Runtime/Scripting/ScriptingInvocation.cpp:298
#9	0x0000000102bdf554 in Invoke [inlined] at /Users/bokken/build/output/unity/unity/Runtime/Scripting/ScriptingInvocation.h:72
#10	0x0000000102bdf544 in ::readCallback() at /Users/bokken/build/output/unity/unity/Modules/Audio/Public/AudioCustomFilter.cpp:367

David_Harris · 29 November 2022 17:37

Currently I’m testing using the “Basic Test” scene from the samples, I was using the Thermin sample but I thought there maybe some issue in the Awake setup, but both crash at exactly the same point.

giovannibedetti · 29 November 2022 17:53

I had the same results yesterday unfortunately.
I tried adding sr and different ksmps but no luck yet.
Will try again in some minutes.

giovannibedetti · 29 November 2022 23:42

So I tried building Csound for iOS and using the built libcsound.a file.
I tried calling PerformKsmps after Csound compile, but it crashes there too. So it’s not strictly related with csoundGetSpoutSample.
It is very similar to what we are experiencing on Android on 32bits, where even if we load 32bit libraries correctly it crashes on the same methods as above.
Not sure yet if it makes sense to open a Unity issue, but considering they have been so helpful with the macOS builds, maybe it’s worth a try!
The only good news I have is that I implemented a working PostProcessBuild script, it adds the Accelerate framework correctly and also sets a signing option for old XCode versions (like the one that runs on my old Macbook :D, 12.4).

It is on this branch:

https://github.com/rorywalsh/CsoundUnity.git#fix/ios_builds

David_Harris · 30 November 2022 11:25

I had a few attempts at compiling on different versions of Unity,
2019.2 with CsoundUnity 3.2 - same crashing error
2022.2 (b13) with CsoundUnity 3.4 - same crashing error
I also tried with 2018 but had issues with the UPM / package setup, but while trying to copy in the files directly into the Assets folder I noticed you can add the Accelerate framework from within the plugin setup in Unity, you don’t need the build post process, I can’t alter that as its a package in my builds but I guess you can modify in your working copies before pushing to git.