The bug was coming from the RTP socket being used to send every ICE error response instead of using the socket where the request was received one.
This had the effect for the media relay to send back what it received on the RTCP socket to our RTP socket, preventing the ICE process from terminating normally.
This bug has always been present but was only triggered when activating sending of dummy STUN packets on RTCP sockets.