Troubleshooting Live Unit Test build failure

It happens that I have a .NET 4.6.2 solution including Unit Test projects that builds just fine. However, when starting Live Unit Test it would complain build completed (failed) for some reason. Which is weird since the solution builds fine.

I started searching for the reason of this mysterious build failure. First thing to do is to enable verbose Live Unit Test logging: Tools/Options.../Live Unit Testing -> Logging Level. Once set to verbose it become obvious that Live Unit Test BuildManager had problems with some referenced third party assemblies, specifically (in my case) with NUnit and Moq. Errors were like this:

[09:02:50.728 Verbose] - BuildManager - MY_TEST_PROJECT_PATH\SomeFile.cs(1,7): error CS0246: The type or namespace name 'NUnit' could not be found (are you missing a using directive or an assembly reference?)

But why wouldn't it find these Assemblies where VS has no problem finding them. Note: I added these two (and other) assemblies through NuGet.
I went looking where the problematic references actually point (Solution Explorer, Test Project, References) and saw something odd. Instead of pointing to a some path under NuGet Packages directory they were pointing to build output directory. At some point in the past VS decided that it has to change the path to the assemblies for some reason. Probably it couldn't find them at original location and decided to find the only ones it could - ones from the recent build in the build output directory. Which is a bad move from Visual Studio and one that most certainly causes problems.

At the end the solution was rather easy once I discovered the reason. I simply upgraded NuGet packages to a newer version. Reinstall would also work.