Just thought I'd share the results of my trial and error process.
1) I updated the Merge Module and its associated default MSI installer for my new service to share the same installation directory as the primary application. Not a subfolder, not a parallel folder, the same folder. This ended up being critical even though I could have installed the application into at least a subdirectory of the main app. This update actually moves the program suite off of wyUpdate.exe and onto the AutomaticUpdater.dll which runs in the service and not the primary application. This is so the app can be updated while nobody is logged into the product and the normal users have insufficient rights to do so anyway.
2) I played with doing a direct update of the actual files and executing the InstallClass via a public method I exposed to the startup code with an -installservice command line argument. This way the service installed and then started without having to run the Merge Module or MSI default installer of the service. However, I had issues with order of the service install and the primary app restart after the upgrade completed because of the top down approach of the wyBuild. I would have preferred this method, but without renaming my executables I didn't have them running in the correct order.
3) I created 2 batch files and placed them into the Temp wyBuild folder along with the main service MSI (not merge module). The first batch is to install the MSI of the service and second batch is to restart the application after the update. I passed in the wu-temp and basedir parms into the respective batches and they executed fine. Because the MSI was running from a service the batch file is calling msiexec with all the necessary switches to run silently. The disadvantage of this is the MSI for the service will create a second associated program with Add/Remove programs, where the distribution of the application MSI with the service merge module embedded only results in one. This may be more because of a setting on my merge module and the fact the business only wants one installer.
This all worked fine. I am now testing option 4 which will be to call the original InstallClass code I exposed via 2 similar batch files as described in Step 3. The end result should be the same. I am not sure if there are advantages to doing it this way other than I don't have to support 2 installers and the differences that will show up with new installs showing one registered application with the installer and apps that were automatically updated showing two.
One thing is for certain, if you can't control the setup projects for additional apps you are installing the MSI with corresponding batch files in the temp directory will likely be your only option. Honestly, if you have control of your installers for the supplemental programs then the main determining factor is if you want the additional programs to show up as registered applications in Add/Remove programs, or just be part of the existing registered app.
I hope someone else finds this helpful. If anyone has anything to add, or other strategies to share please feel free to comment.
Chris