You must keep your original files for your application (the only place they're stored is on the disk -- wyBuild doesn't tuck them away anywhere special).
Regarding the files that wyBuild generates:
- Your wyBuild project (*.wyp): Never delete this. In fact, make sure you make backups of this file. Among other things it include your signing-keys that are used the verify updates actually come from you.
- The client.wyc & wyUpdate.exe files: You can delete them or keep them around. It doesn't matter because you can regenerate them by clicking "Build wyUpdate" inside wyBuild.
- Your "yourapp.X.to.N.wyu" and wyserver.wys files: You can keep them around or delete them -- it's up to you.