Tips for buggy iOS CI automation

Running systematic CI for physical iOS devices has plenty of challenges including the dependence on Mac hardware, the new versions of Xcode every year, the corresponding MacOS version requirements, the difficulties of downloading Xcode releases with Apple ID credentials, the provisioning profiles, the keychain permissions, preventing phones from auto-updating, figuring out the xcodebuild invocations…

…which is to say nothing of the outright bugs, which are what end up occupying most of my time. Today I want to mention three of them and how to work around them.

Nowadays the standard way to list the USB-connected devices is xcrun xctrace list devices. Sometime during the era of Xcode 13 this became horrifically unreliable. If I run this command on a machine with two iPhones, on each invocation I might get 0, 1 or 2 of them. My solution: try it up to 10 times with short sleeps inbetween and see if I manage to find all of them. This problem only seems to affect xctrace. Once I get the UDID it reliably works later with xcodebuild.

Then there are the OS bugs. CoreBluetooth can eventually get into a state where it doesn’t work any more, or your tests might fail with an error of the form operation never finished bootstrapping - no restart will be attempted. The solution for me is to reboot the affected iPhone. The good news is that you can do this programmatically within your CI script before you start your tests; it doesn’t even take that long, only 30 seconds or so.

To prepare:

brew install libimobiledevice

In CI:

idevicediagnostics restart -u ${UDID}

If you want to save some time you can do the reboot in parallel with some other build work.

idevicediagnostics restart ...
sleep 35 &
# Run some other build command here while the phone does its thing
wait

Finally, my least favourite error of all: Failed to terminate com.mycompany.App:0, while trying to run UI automation managing permissions dialog boxes. This pernicious failure comes up more consistently on newer devices. This can be fixed by making sure the app is not installed from a previous CI run and starting from a blank slate. This can also be achieved with some libimobiledevice-based magic.

To prepare:

brew install ideviceinstaller

In CI:

# List apps installed
ideviceinstaller -u ${UDID} -l

# Uninstall one of them
ideviceinstaller -u ${UDID} -U com.mycompany.App

Happy testing!