Orcon NF4V behind EdgeRouter Lite as VOIP ATA

I have been migrating from the Orcon router to an EdgeRouter Lite but am having troubles getting the VOIP service working.

I have the following set up:

Chorus NTU -> EdgeRouter Lite -> Unmanaged Switch -> NF4V

This works! But when I remove the unmanaged switch and connect the NF4V directly into the ERL the VOIP status stays down.

The NF4V can still ping external addresses including voice.orcon.net.nz.

I just don't know what the unmanaged switch could be doing that's making a difference.