C++11 and “old” operating systems

We want to switch to C++11. It’s not just that it’s new and shiny, but it has a lot of features that will make for more readable code.

But… there’s runtime support involved in some of the C++11 features. So what to do about older operating systems that vendors aren’t supporting in their C++11 rollout? E.g. Apple is not supporting 10.6 with libc++.

http://blogs.freebsdish.org/theraven/2013/01/03/the-new-c-stack-in-9-1/

FreeBSD has a new low-level library for the C++ standard called libcxxrt, which sits below libstdc++ (GNU) or libc++ (LLVM). It’s open-sourced, so maybe this would be one option, at least for Unix flavors like Mac OS X and Linux.

Interestingly, the claim is that if you link libstdc++ against libcxxrt instead of libsupc++ (the older low-level library), you can mix libraries that use libc++ with libraries that use libstdc++ in the same program. Now, of course, they have done this work for FreeBSD and not other operating systems, but maybe this can be repeated. It’s perhaps even likely, since Clang is gaining more and more momentum, and it’s moved ahead of GNU C++ in terms of C++11 support.

And Clang compiles my large programs much faster than gcc does.

Marking directory paths in Unix shells

This is an interesting and really low-tech but effective approach.

Quickly navigate your filesystem from the command-line

It works on Unix-like systems in sh-derived shells like bash, zsh, tcsh and so on, because it exploits several things

  1. The idea of a home directory
  2. Symbolic links
  3. Ability to get and set path easily

You use it like this (I’m going to show full paths in my prompt to make it clear what’s going on)

~/projects/git/test$ mark test
~/projects/git/test$ cd ~/contrib/ref
~/contrib/ref$ mark ref
~/contrib/ref$ jump test
~/projects/git/test$

His entire “system” is a handful of lines of sh script. In a sh shell, the pwd command returns your current path. So the mark command just creates a symbolic link with your desired mark name pointing to the current directory, the jump command cds to the directory pointed to by the symbolic link, the unmark command just removes the symbolic link, and the marks command just lists the symbolic links in a pretty fashion.

This is nicer than pushd/popd/dirs, at least for me.

SCons tools

SCons has a plug-in-like architecture that they call Tools. There are a large number of built-in tools for compiling, linking, copying, etc. You can add your own, they are Python scripts that go into your site_scons/site_tools directory.

You can see what the default tools are for a particular platform by looking in the environment you create.

base_env = Environment()
log('base_env.txt', base_env, "base_env = Environment()")

where log is a simple function I made

def log(name, env, prompt):
  logfile = open(name, 'w')
  print >> logfile, prompt
  print >> logfile, env.Dump()
  logfile.close

When I run this on a Windows machine, among all the output, I see:

  'TOOLS': [ 'default',
             'mslink',
             'msvc',
             'gfortran',
             'masm',
             'mslib',
             'msvs',
             'midl',
             'filesystem',
             'jar',
             'javac',
             'rmic',
             'zip'],

When I run this on a Mac with XCode 4.4.1 (clang, not gcc), I see:

  'TOOLS': [ 'default',
             'applelink',
             'gcc',
             'g++',
             'gfortran',
             'as',
             'ar',
             'filesystem',
             'm4',
             'lex',
             'yacc',
             'rpcgen',
             'jar',
             'javac',
             'javah',
             'rmic',
             'tar',
             'zip',
             'CVS',
             'RCS'],

I can’t quite figure out how the ‘default’ keyword expands to this list, but it has to be in the SCons/Tool/__init__.py script. It’s finding one of compiler, C++ compiler, linker, assembler, fortran_compiler and ar, as well as some other tools like filesystem, m4, wix and etc. It’s only adding files that actually exist, hence not showing yacc on Windows.

The names of the SCons tools are set in stone, so perhaps we should specifically add names and not use ‘default’? Except… well, it’s probably ok, because if SCons change that much, other parts of our build will break too.

It’s also crazy that SCons on Windows is so slow at finding things – at least, I assume that’s what the difference in performance is on Mac vs Windows. The Mac is about 3x faster per Environment() call versus Windows.

All the built-in tools for SCons are listed here: http://www.scons.org/doc/2.2.0/HTML/scons-user.html#app-tools.

It’s interesting that SCons thinks the D language compiler is on my Mac OS X 10.8 box (the dmd tool). I didn’t think it was. SCons is looking in my path for dmd or gdmd, and I don’t see any such thing. A bug?

coLinux – run Linux alongside Windows

coLinux

One approach is to run a Linux where the kernel is a Windows program.

http://www.colinux.org/

But unlike User mode Linux, it is actually running in ring 0. I’m not sure how supported it is, though, it hasn’t had updates in a while. Specifically, it still refers to the Linux 2.6 kernel.

User-Mode Linux

http://user-mode-linux.sourceforge.net/

This is also still referring to the Linux 2.6 kernel.

Edubuntu WebLive

http://www.edubuntu.org/weblive

This still seems active.

Wubi – “Live Ubuntu”

http://www.ubuntu.com/download/desktop/windows-installer

This seems the most active of all of the “run Linux in Windows” options. But this isn’t really Linux cooperative with Windows, this is a Linux installer that runs in Windows.

 JSLinux

http://bellard.org/jslinux/index.html

This is the craziest of all – a Linux distribution compiled to Javascript and running in a browser! It’s really a PC emulator written in Javascript, then running a small Linux distribution.

Homebrew

I’m not sure why I avoided using Homebrew for so long. I started using it yesterday, and it’s pretty cool.

http://brew.sh/

It has a diagnostic mode that checks to make sure everything is working, and this alone gave me a lot of reassurance. For example, I know that I had some new apps shadowed by the XCode developer tools; this is mostly Apple’s fault. Homebrew reminds me of this, and even prompts me to update!

laphroaig:gameservice bfitz$ brew doctor
Warning: /usr/bin occurs before /usr/local/bin
This means that system-provided programs will be used instead of those
provided by Homebrew. The following tools exist at both paths:

    git
    git-cvsserver
    git-receive-pack
    git-shell
    git-upload-archive
    git-upload-pack

Consider amending your PATH so that /usr/local/bin
occurs before /usr/bin in your PATH.

Warning: Your Homebrew is outdated.
You haven't updated for at least 24 hours, this is a long time in brewland!
To update Homebrew, run `brew update`.
laphroaig:gameservice bfitz$

It’s a little disconcerting to see Homebrew put a git repository naked in /usr/local, that makes it feel like Homebrew wants to own this hierarchy. And when I look closer at things, that’s definitely going on. I suppose that’s ok.

Timing SCons

Note: if you run across this, I’ve been updating it as I go. I know, bad form for blog posts. Oh well.

I’m trying to figure out what SCons spends all its time doing. In all the below, I wrote synthetic tests, then try to calculate the cost of individual actions. Here’s the summary

  • Python: 60 milliseconds
  • SCons: 490 milliseconds
  • AddOption(): 0.05 milliseconds
  • Environment(): 350 milliseconds

Starting up the Python interpreter on my machine has an overhead of about 60 milliseconds.

SCons appears to have an overhead of just under 0.5 seconds. Or, at least, running scons with an empty SConstruct takes about 0.55 seconds on my machine, putting SCons overhead itself at .49 seconds.

import SCons

AddOption is pretty fast too. It’s hard to measure the time accurately, but a file with 10,000 AddOption calls runs in just under 1.0 seconds, so that means AddOption takes just under 50 microseconds per invocation. This means you can pretty much ignore it.

import SCons
AddOption('--opt0', dest='opt0', action='store_true', default=False, help='help for opt0')
AddOption('--opt1', dest='opt1', action='store_true', default=False, help='help for opt1')
...
AddOption('--opt9999', dest='opt9999', action='store_true', default=False, help='help for opt9999')

Creating an environment takes time, around 0.35 seconds on my machine. Tossing an SConstruct with 10 Environment() calls in it runs in 4.01 seconds.

import SCons
env = Environment()
...
env = Environment() # 10 times

The first time you call File() or Dir(), it takes .357 seconds (at least on my machine). Subsequent calls to either are cheap (as in not really measurable). I guess there’s some lazy initialization going on. It made my timing wonky, I kept thinking this was an improvement, but I just shifted the time.