Fix crash on client exit

This is a backported fix from the 5.7 branch.

Original patches: 93ca929f and d6fe7b61



Change-Id: Iaec5acb314d3bbc199e962a5dc27a70f0d2df655
Reviewed-by: Giulio Camuffo <giulio.camuffo@kdab.com>
Reviewed-by: Paul Olav Tvete <paul.tvete@qt.io>