Actually, the error given was a red herring to the underlying issue.
The wyBuild.wyp for the wyUpdate.exe can only contain the wyUpdate.exe.
I had an .txt in the .wyp to test that something was deployed, as I don't have an updated version of our base wyUpdate.exe to test with at the moment (although created a dummy one for now).
Anyway, it throws an exception when it tries to clean up the unpacking of the updtdetails.udt. As the base folder will contain the wyUpdate.exe and the .txt. As it expects the folder to be empty after deleting the wyUpdate.exe it throws an exception when trying to delete the directory and it isn't empty.
That exception then gets eaten and then resurfaces itself up higher in the flow with the error I pasted above!
And I guess for anyone coming here looking to debug the wyUpdate part of the update, I found adding a MessageBox.Show in the check for “supdf” in the frmMain.cs will essentially pause the app until you press ok given you time to re-attach the debugger for the next execution.