I'm trying to connect an iPod Touch with a macOS Catalina guest and Ubuntu MATE 20.04 guest but it's not working. Seems the host snatches access back for itself. Sometimes I get the "device is in use by another application" error. VirtualBox also had the same sort of issue.

While I've hardly tested on a wide selection of devices, others get redirected fine, including an Android smartphone and my mouse (whoops!). So it could be an Apple-specific problem.

Lost where I took it from, but adding a hostdev block in the XML with the iPod's vendor and product ID didn't fix it, nor did disabling usbmuxd or removing libimobiledevice-utils (the other answer on that page kinda worked, but it being in recovery mode wasn't much help and I had to use idevicerestore to fix it). There's also potentially some use in this comment.

I'm fairly out of ideas at this point and have broken a fair amount along the way. But there must be someone else out there who is trying to develop iOS apps from a Ubuntu machine and run them on real mobile hardware, right?

Update:
Doing lsusb then sudo fuser /dev/bus/usb/[bus]/[device] reveals that gvfsd-gphoto2 is using the device. I can kill this and begin connecting the iPod to the VM, but it still disconnects.

Update 2:
Found some more obscure links today. This question shows it might be an issue only with newer Apple devices and this one gave me the idea of installing a USB expansion PCI card to passthrough (though I'd rather avoid buying one).

Update 3:
I believe I can send a whole USB hub to the VM with the following XML (IDs from lsusb) but the end result of this is just that neither the host nor guest can see any peripherals plugged into the hub.

<hostdev mode="subsystem" type="usb" managed="no">
  <source startupPolicy="optional">
    <vendor id="0xXXXX"/>
    <product id="0xXXXX"/>
    <address bus="X" device="X"/>
  </source>
  <address type="usb" bus="0" port="4"/>
</hostdev>

Update 4:
I have purchased a PCI-e card with a couple USB ports to route to the guest. While I made a mistake and got one that isn't really compatible with macOS, it shows up in the list of PCI devices for this guy. Nothing shows up in the list for me, even though the host loses access to the card once the VM is started. I guess this is a separate problem, but just throwing it out there that it doesn't seem to be an easy (but not free) fix to this problem for myself.

Update 5: I tried sharing the iPod from the host to guest via USB Network Gate. It's just a free trial, but the guest recognises the device. You know what, I'll take it.

Update 6:
Maybe be cautious with that software. The host system barely survived a reboot and USB Network Gate doesn't work anymore. Issues include read-only file system (edit: this was my fault and unrelated) and issues with DNS resolution.

Update 7:
Welp, I guess networking won't work again. How about passing a USB Wi-Fi adapter to the VM? Nah, that would be too easy. Tried a couple, but their drivers aren't compatible with Catalina.

Update 8:
I tried FlexiHub instead of USB Network Gate but it just said my free trial has expired, probably because I'm using a virtual machine. So that was no use, but I'm back to USB Network Gate now anyway. I've also got networking kinda working. And USB kinda working. It's all very hacky, but depending on your needs this may be suitable. I'll write an answer to my own question, but if someone finds an actual solution rather than a series of hacks, they deserve the credit.