As any VisualStudio user knows, the VS installer includes a bunch of separate product installs. The problem is that when you want to uninstall VisualStudio, it leaves all of those "value add" products installed.
I was retiring an old development machine and started removing the components before running the main uninstaller. Well doing this screwed up the VS installer causing the "A problem has been encountered while loading the setup components. Canceling setup." error.
In trying to solve this issue I found a handy tool provided by Microsoft which will undo the havoc that it had wreaked on your machine.
Behold, the AutoUninstallTool.