1. Do I need to provide my users with a new updater.exe (something that is not possible with automatic updates), or is this information stored in client.wyc making a regular update sufficient?
The download sites are stored in the "client.wyc" file. So add your new sites (and remove old ones), rebuild wyUpdate, then include the client.wyc file in your latest (and subsequent) updates within wyBuild.
Then when your customers update to the latest version of your app (from your old server addresses), they will get your new client.wyc file with the new server addresses.
Of course the better solution to this problem is to use a domain name (e.g. "yoursite.com") so when you change hosts or IP addresses you don't don't have to do anything (because the domain will be the same).
2. Is it possible to change the window title of the updater from wyupdate to <productname> updater?
Yes, in wyBuild go to File -> Properties -> wyUpdate and set the "Custom title for wyUpdate title bar".