Travis CI

This is interesting

https://travis-ci.org/

Travis-CI is hosted continuous integration and deployment, using Mac OS X. It integrates with GitHub and supports a fair number of languages. So, you can test and deploy Mac targets without needing Macs of your own.

The big challenge for some companies is letting their source code leave their walls, because obviously this is going to be building your source on machines you don’t control. But I think that attitude is going to die out over time, as people realize that it’s the writing of source code that is where the value is, not the source code itself.

Travis uses xctool to drive XCode instead of xcodebuild.

https://github.com/facebook/xctool

Another piece of Travis I think comes from Sauce Labs

https://saucelabs.com/

 

Studying APIs

API design is important, and yet it’s not studied enough. It’s a field full of practitioners but lacking in both theory and measurement.

Here are some papers I’ve been reading:

Seeking the Ground Truth: A Retroactive Study on the Evolution and Migration of Software Libraries from 2012.

Mapping the Space of API Design Decisions fr0m 2007.

Here’s a book I need to buy:

API Design for C++ by Martin Reddy.

This might be a good book, so I’ll read it:

Practical API Design: Confessions of a Java Framework Architect by Jaroslav Tulach, billed as the initial architect of NetBeans.

 

File system madness

It’s interesting trying to read disk catalog entries in some cases. For example, I copied some files from a Mac to a PC, indirectly through an SMB file share (Windows server). When I tried to access them with a Python program that iterated directories with os.walk, I got an error:

WindowsError: [Error 123] The filename, directory name, or volume label syntax 
is incorrect: 'D:/Xcode/usr/share/man/man3\\Git??I18N.3pm'

On the original Mac machine, I see this

drwxr-xr-x  27 bfitz  admin    918 Jun  9  2013 .
drwxr-xr-x   7 bfitz  admin    238 Jun  9  2013 ..
-rw-r--r--   1 bfitz  admin  25681 Feb  6  2013 Git.3pm
-rw-r--r--   1 bfitz  admin   4804 Feb  6  2013 Git::I18N.3pm
-rw-r--r--   1 bfitz  admin   5514 Feb  6  2013 Git::SVN::Editor.3pm
-rw-r--r--   1 bfitz  admin   6966 Feb  6  2013 Git::SVN::Fetcher.3pm
-rw-r--r--   1 bfitz  admin   5018 Feb  6  2013 Git::SVN::Memoize::YAML.3pm
-rw-r--r--   1 bfitz  admin   5622 Feb  6  2013 Git::SVN::Prompt.3pm
-rw-r--r--   1 bfitz  admin   4787 Feb  6  2013 Git::SVN::Ra.3pm
-rw-r--r--   1 bfitz  admin   4617 Feb  6  2013 Git::SVN::Utils.3pm
-rw-r--r--   1 bfitz  admin   6355 Feb  6  2013 SVN::Base.3pm
-rw-r--r--   1 bfitz  admin  63700 Jun  9  2013 SVN::Client.3pm
-rw-r--r--   1 bfitz  admin  22950 Feb  6  2013 SVN::Core.3pm
-rw-r--r--   1 bfitz  admin   7468 Feb  6  2013 SVN::Delta.3pm
-rw-r--r--   1 bfitz  admin  21923 Feb  6  2013 SVN::Fs.3pm
-rw-r--r--   1 bfitz  admin  30291 Feb  6  2013 SVN::Ra.3pm
-rw-r--r--   1 bfitz  admin  15618 Feb  6  2013 SVN::Repos.3pm
-rw-r--r--   1 bfitz  admin  18271 Feb  6  2013 SVN::Wc.3pm
-rw-r--r--   1 bfitz  admin   5867 Feb  6  2013 arch.3
-rw-r--r--   1 bfitz  admin     19 Feb  6  2013 end.3
-rw-r--r--   1 bfitz  admin   2154 Feb  6  2013 get_end.3
-rw-r--r--   1 bfitz  admin   2522 Feb  6  2013 getsectbyname.3
-rw-r--r--   1 bfitz  admin     25 Feb  6  2013 getsectbynamefromheader.3
-rw-r--r--   1 bfitz  admin     25 Feb  6  2013 getsectdata.3
-rw-r--r--   1 bfitz  admin     25 Feb  6  2013 getsectdatafromheader.3
-rw-r--r--   1 bfitz  admin    940 Feb  6  2013 getsegbyname.3
-rw-r--r--   1 bfitz  admin  10232 Feb  6  2013 kld.3

On the Windows box, I see this

D:\projects\git\brian\disk>dir d:\Xcode\usr\share\man\man3
 Volume in drive D is SSD
 Volume Serial Number is B48A-050D

 Directory of d:\Xcode\usr\share\man\man3

09/29/2013  06:58 PM    <DIR>          .
09/29/2013  06:58 PM    <DIR>          ..
02/06/2013  02:11 PM             5,867 arch.3
02/06/2013  02:11 PM                19 end.3
02/06/2013  02:11 PM             2,522 getsectbyname.3
02/06/2013  02:11 PM                25 getsectbynamefromheader.3
02/06/2013  02:11 PM                25 getsectdata.3
02/06/2013  02:11 PM                25 getsectdatafromheader.3
02/06/2013  02:11 PM               940 getsegbyname.3
02/06/2013  02:11 PM             2,154 get_end.3
02/06/2013  02:42 PM            25,681 Git.3pm
02/06/2013  02:42 PM             4,804 Git??I18N.3pm
02/06/2013  02:42 PM             5,514 Git??SVN??Editor.3pm
02/06/2013  02:42 PM             6,966 Git??SVN??Fetcher.3pm
02/06/2013  02:42 PM             5,018 Git??SVN??Memoize??YAML.3pm
02/06/2013  02:42 PM             5,622 Git??SVN??Prompt.3pm
02/06/2013  02:42 PM             4,787 Git??SVN??Ra.3pm
02/06/2013  02:42 PM             4,617 Git??SVN??Utils.3pm
02/06/2013  02:11 PM            10,232 kld.3
02/06/2013  02:43 PM             6,355 SVN??Base.3pm
06/09/2013  10:09 AM            63,700 SVN??Client.3pm
02/06/2013  02:43 PM            22,950 SVN??Core.3pm
02/06/2013  02:43 PM             7,468 SVN??Delta.3pm
02/06/2013  02:43 PM            21,923 SVN??Fs.3pm
02/06/2013  02:43 PM            30,291 SVN??Ra.3pm
02/06/2013  02:43 PM            15,618 SVN??Repos.3pm
02/06/2013  02:43 PM            18,271 SVN??Wc.3pm
              25 File(s)        271,394 bytes
               2 Dir(s)  459,978,698,752 bytes free

D:\projects\git\brian\disk>

The file names are not actually “Git??I18N.3pm” on the Windows box, because that path isn’t actually a legal path. The actual path has some characters that aren’t legal on Windows, and the Mac SMB compatibility layer preserves them so that the files can roundtrip from Mac to Mac. But not Mac to anything else.

In this particular case, the Mac copied file names that have ‘:’ characters in them. In Unix, any character is legal in a filename except ‘/’. Windows is a lot more restrictive, and ‘:’ characters are not legal. Now, interestingly enough, these can be copied on the Windows box by Explorer. I also suspect that some other character than ‘:’ is being stored.

This is a variant of the character encoding issue; if you inadvertently change character encodings when iterating directories, you’ll be unable to manipulate the files, because you aren’t using the actual name.

In this particular case, it’s not just that the Python code is scrambling the name. It’s very unlikely that the name is stored with ?? characters. Either the name has :: in it (which isn’t legal at the high-level, but perhaps legal at the lowest NT level), or there is some other characters being substituted. I suspect the latter, and I think that Samba is doing this. I copied this with a Mac OS 10.8 box which is still using Samba and not Apple’s replacement.

I’ll have to write some Windows code to iterate the directory to see what the names come back as, and then figure out what I want to do with my Python code.

Also interesting in that explorer.exe can copy this directory. I suspect it is either falling back to short names, or doing other heroic actions.

Python argparse

Command-line processing

As of Python 2.7, the preferred command-line parsing library is argparse. The pattern is straightforward – create an ArgumentParser, add all the command-line options, then call it and extract arguments.

import argparse
def commandline():
  parser = argparse.ArgumentParser(
    description='Your command-line description',
    fromfile_prefix_chars='@')

  parser.add_argument('path', nargs='*', help='help string')
  parser.add_argument('--user', action='append', help='help string')
  parser.add_argument('--verbose', action='count', default=0, help='help string')
  parser.add_argument('--quiet', action='store_true', help='help string')

  args = parser.parser_args()

  print '  path   =    %s' % ',\n              '.join(args.path)
  print '  user   =    %s' % ',\n              '.join(args.user)
  print '  verbose= %d' % args.verbose
  print '  quiet = %s' % 'True' if args.quiet else 'False'

Given

sub/one sub/three --user=sub/two --verbose --quiet --verbose

this produces

path    = sub/one
          sub/three
user    = sub/two
verbose = 2
quiet   = True

Arguments are positional or optional. In the example above, all arguments not attached to one of the listed options is gathered up into the ‘path’ positional argument.

More reading

Reference: http://docs.python.org/2.7/library/argparse.html

Tutorial: http://docs.python.org/2.7/howto/argparse.html

2014

Time to (pick one or more of the following):

  • figure out actual artificial intelligence (none of this fake big-data-driven nonsense)
  • change politics for the better
  • start the process of universal computer programming

At least one of those meaningfully started this year.