Puppet should be able to handle agent upgrades, in most environments that is true. Windows gets a bit weird. Windows is a different beast as compared to POSIX - Windows holds locks on files in use where Linux loads everything into memory. While you can completely wipe out everything on POSIX and replace it while you are running a process, you can not do that on Windows.
That is why a separate process should handle the upgrades, one that Puppet can start, but then needs to exit so the separate process can handle the upgrade.
It's fine to recommend using Puppet to upgrade Puppet agents on almost every OS aside from Windows.
We've discussed this in the past but have not come up with any really good solutions, aside from https://forge.puppetlabs.com/opentable/puppetversion, which creates a scheduled task for Windows, but may still interrupt a catalog that is being applied. We had functionality in the reboot module that we were going to pull out at one time and make more generic, to allow for things like an upgrade to occur right after Puppet exited the run. Here's how it was implemented for Reboot - https://github.com/puppetlabs/puppetlabs-reboot/blob/0.1.x/lib/puppet/provider/reboot/windows/watcher.rb
It also needs to ensure that if the upgrade fails, the MSI application rolls back successfully.