This is particularly problematic when calling FD_SET( -1, ... ), but let's
check it in all functions.

This was introduced with the new API and the fact the net_free() now sets the
internal fd to -1 in order to mark it as closed: now using this information.