SCons Environment in depth, part 3

I’m going to focus on the Microsoft toolchain, with the aim of being able to put a Microsoft toolchain into a package that can be loaded at build time. The plus side to this is that you don’t need toolchains installed to systems, but it require a little finagling of SCons. And to do that, we need to understand what it’s doing. I covered individual Microsoft-specific tools in the past part, but in isolation, and with less understanding than I have now. So, onwards.

Note – this is super-sketchy and should be filled in. I started keeping notes for myself as I was working on Visual-C++-in-a-package, and afer the initial exploration, I started working. I need to circle back and update this.

How does SCons configure Microsoft Visual C++?

There is a debugging environment variable that you can set that will enable some SCons spew from Tool/MSCommon/vc.py. If you do that with a simple SConstruct

env = Environment(tools=[], platform='win32', MSVC_VERSION='11.0')

then you’ll get some output that will guide you. Since we’re trying to use specific Microsoft products, there are well-known registry keys pointing to each version. Visual Studio 2012 has a registry key pointing to the on-disk location for Visual C++:

Software\Wow6432Node\Microsoft\VisualStudio\11.0\Setup\VC\ProductDir = C:\dev\VC11\VC\

If you don’t specify a Visual C++ version SCons will enumerate every possible version of Visual Studio going back to to the dawn of time, and then pick the first one it finds – since the list it searches is ordered from newest to oldest, this will find the most recent Visual C++ that you have installed.

If you do this while specifying a specific Visual C++ version, you’ll see that it skips the registry scanning and goes straight to enumerating the hard disk. However, something later forgets this, and it scans anyway. This is because vc.msvc_exists() is defective – it uses the (cached) list of versions as proof that Visual C++ exists, but nothing set it up for the case where you bypass it. This is an easy fix. I’ll add to the list of things I want to patch.

Another nit is that find_vc_pdir is not memoized – it’s called at least three times during setup. The only reason I care is that SCons on Linux (even in a VM) is about 0.5 sec faster at startup than on Windows – this might be Python overhead on the two systems, or it could be the Microsoft tools init. I’ll profile it at some point.

Then it finds the magic BAT file that Microsoft supplies for command-line use, that sets up all the environment variables that the toolchains need to run. There is an boolean environment variable MSVC_USE_SCRIPT that lets you disable the use of the Microsoft script – if this is set to False (it defaults to True), then SCons assumes you have done all the setup yourself.

And it scans for installed SDKs. This part is missing a preconfigure step to let you select a specific SDK. In general, SDKs are loosely coupled with the Visual Studio install, but only very loosely.

Visual C++ vcvarsall.bat

This is a batch file that Microsoft has been supplying for a while, as a convenience for configuring an environment for building with Visual C++. It takes an optional architecture parameter that if not supplied defaults to ‘x86′. And if you’re curious, this just runs a different batch file at \bin\amd64\vcvars64.bat, and this  makes registry queries and calls another batch file, Common7\Tools\VCVarsQueryRegistry.bat, which does most of the real work.

If you run it like this

C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcvarsall.bat amd64

then it will set the following environment variables:

CommandPromptType=Native
ExtensionSdkDir=C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs
Framework35Version=v3.5
FrameworkDir=C:\windows\Microsoft.NET\Framework64
FrameworkDIR64=C:\windows\Microsoft.NET\Framework64
FrameworkVersion=v4.0.30319
FrameworkVersion64=v4.0.30319
FSHARPINSTALLDIR=C:\Program Files (x86)\Microsoft SDKs\F#\3.0\Framework\v4.0\
INCLUDE=C:\dev\VC11\VC\INCLUDE;
  C:\dev\VC11\VC\ATLMFC\INCLUDE;
  C:\Program Files (x86)\Windows Kits\8.0\include\shared;
  C:\Program Files (x86)\Windows Kits\8.0\include\um;
  C:\Program Files (x86)\Windows Kits\8.0\include\winrt;
LIB=C:\dev\VC11\VC\LIB\amd64;
  C:\dev\VC11\VC\ATLMFC\LIB\amd64;
  C:\Program Files (x86)\Windows Kits\8.0\lib\win8\um\x64;
LIBPATH=C:\windows\Microsoft.NET\Framework64\v4.0.30319;
  C:\windows\Microsoft.NET\Framework64\v3.5;
  C:\dev\VC11\VC\LIB\amd64;
  C:\dev\VC11\VC\ATLMFC\LIB\amd64;
  C:\Program Files (x86)\Windows Kits\8.0\References\CommonConfiguration\Neutral;
  C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.VCLibs\11.0\References\CommonConfiguration\neutral;
Path=C:\dev\VC11\Common7\IDE\CommonExtensions\Microsoft\TestWindow;
  C:\dev\VC11\VC\BIN\amd64;
  C:\windows\Microsoft.NET\Framework64\v4.0.30319;
  C:\windows\Microsoft.NET\Framework64\v3.5;
  C:\dev\VC11\VC\VCPackages;
  C:\dev\VC11\Common7\IDE;
  C:\dev\VC11\Common7\Tools;
  C:\Program Files (x86)\HTML Help Workshop;
  C:\dev\VC11\Team Tools\Performance Tools\x64;
  C:\dev\VC11\Team Tools\Performance Tools;
  C:\Program Files (x86)\Windows Kits\8.0\bin\x64;
  C:\Program Files (x86)\Windows Kits\8.0\bin\x86;
  C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\x64;
  C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\x64;
  C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools;
  C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\
Platform=X64
VCINSTALLDIR=C:\dev\VC11\VC\
VisualStudioVersion=11.0
VSINSTALLDIR=C:\dev\VC11\
WindowsSdkDir=C:\Program Files (x86)\Windows Kits\8.0\
WindowsSdkDir_35=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\
WindowsSdkDir_old=C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\

If environment variables already exist, it prepends to them.

Now, this may not be entirely accurate, because I had a few environment variables already set for some reason (I’m assuming the Visual Studio installer did this)

VS100COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Tools\
VS110COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\

I removed these from an environment and ran vcvars64.bat for VC11, and got the VS110COMNTOOLS environment variable. I think this comes from the “Visual Studio Tools” folder which contains Spy++ and other top-level tools that you would run from the IDE, not as part of the build environment.

This may be a side-light to you, but I want to package Visual C++ into a downloadable tool that is used by the build system to allow builds on arbitrary machines. Yes, we’ll have to make sure we only do this where we’re appropriately licensed.

HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0\InstallationFolder

Path to the installed Windows SDK, put into environment variable WindowsSdkDir. The default is C:\Program Files (x86)\Windows Kits\8.0\

Alternate locations

  • HKCU\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0\InstallationFolder
  • HKLM\SOFTWARE\Wow6432Node\Microsoft\Microsoft SDKs\Windows\v8.0\InstallationFolder
  • HKCU\SOFTWARE\Wow6432Node\Microsoft\Microsoft SDKs\Windows\v8.0\InstallationFolder

HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0A\InstallationFolder

Path to an older Windows SDK (for Visual Studio 2012), put into environment variable WindowsSdkDir_old.

Alternate locations

  • HKCU\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.0a\InstallationFolder
  • HKLM\SOFTWARE\Wow6432Node\Microsoft\Microsoft SDKs\Windows\v8.0a\InstallationFolder
  • HKCU\SOFTWARE\Wow6432Node\Microsoft\Microsoft SDKs\Windows\v8.0a\InstallationFolder

Environment variables

Microsoft build tools need to have some environment variables set up.

PATH

PATH needs to contain the paths to the various tools that will be invoked. For example, it might look something like this. I edited a tiny bit for clarity, where C:\dev\VC11 is the installation folder for Visual Studio 2012 (typically C:\Program Files (x86)\Microsoft Visual Studio 2011), and C:\dev\SDKs is the installation folder for Microsoft SDKs (typically C:\Program Files (x86)\Microsoft SDKs).

'PATH':
  C:\dev\VC11\Common7\IDE\CommonExtensions\Microsoft\TestWindow
  C:\dev\VC11\VC\BIN\amd64
  C:\windows\Microsoft.NET\Framework64\v4.0.30319
  C:\windows\Microsoft.NET\Framework64\v3.5
  C:\dev\VC11\VC\VCPackages
  C:\dev\VC11\Common7\IDE
  C:\dev\VC11\Common7\Tools
  C:\dev\VC11\Team Tools\Performance Tools\x64
  C:\dev\VC11\Team Tools\Performance Tools
  C:\Program Files (x86)\Windows Kits\8.0\bin\x64
  C:\Program Files (x86)\Windows Kits\8.0\bin\x86
  C:\dev\SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\x64
  C:\dev\SDKs\Windows\v7.0A\bin\x64
  C:\dev\SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools
  C:\dev\SDKs\Windows\v7.0A\bin\
  C:\windows\System32

As mentioned above, the paths come from executing vcvarsall.bat.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>