• How do I show hidden files in the Mac OS X Finder
  • What are .AppleSingle and .AppleDouble files and folders?
  • Why do I see “short names” from Windows for some files? (copied to a file server from a Mac)
  • How do I unlock files (in bulk) on the Mac

How do I show hidden files in the Mac OS X Finder?

defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder

But this only controls showing “dot” files (like .git). Finder Info might also exist and have the hidden bit set (FinderInfo came from the classic Mac OS system). Or, an ACL might exist that doesn’t allow browsing for the current user. Files with a ‘@’ appended to the file mode means “this file has extended attributes”, and if there’s a ‘+’ appended, the file has an ACL. You can show the size and kind of the extended attributes with a ‘@’ parameter to ls, and you can show specific FinderInfo attributes in a friendlier fashion with a ‘O’ parameter.

bfitzair:~ ls -l@Oed /Users/bfitz
drwx------@ 41 bfitz  staff  hidden 1394 Sep 23 21:03 Library
        com.apple.FinderInfo      32
 0: group:everyone deny delete

You can manipulate some flags with chflags

chflags nohidden /path/to/file

or you can remove the FinderInfo from extended attributes with

xattr -d com.apple.FinderInfo /path/to/file

You can manipulate ACLs with chmod. It’s moderately complicated to edit them (see the man page for chmod), but you can nuke one by using the -N

sudo chmod -N /path/to/file

 What are .AppleSingle and .AppleDouble files and folders?

These are files (or files in folders) that preserve Mac resource forks and extended attributes that cannot be otherwise preserved on the file system. For example, if you copy Mac files to a NFS or SMB file server volume, the Mac will by default create these for files with resource forks or extended attributes.

If you don’t ever want this, you can disable this

defaults write com.apple.desktopservices DSDontWriteNetworkStores true

This might be the drastic fix you need, but you’ll be unable to save classic Mac files (which generally have resource forks) or you’ll lose file attributes that you might actually want.


AppleSingle combined everything into one file, in a way similar to the older MacBinary. This has an advantage (single file) but a big disadvantage in that the data fork (usually the one most applications care about) is now shrouded inside the AppleSingle file.

AppleDouble separates the data fork and resource fork + finderinfo into two files. In some instances, the second file is stored as ._{filename}. In other cases, it’s stored in .AppleDouble/{filename}.

In times past, I actually wrote code different platforms to work on their file formats (e.g. converting Apple IIgs resource forks to and from Mac resource forks, or to even do the same between Windows and Mac). It’s a lot easier when you have AppleDouble-style layout.

Why do I see “short names” from Windows for some files? (copied to a file server from a Mac)

Windows has a slightly more strict view of what characters can be in a file name. Unix is pretty permissive, and only bans \0 (nul) and ‘/’ (and even the directory separator can be stored in the low-level file system, it’s just a pain to manipulate it if you do that). However, Windows disallows the following characters from being in a leaf name (file or directory)

\ / : * ? " < > |

(the explorer shell also disallows the typing of any “control” character, e.g a character <= \x1F, but these can be used in filenames). The Mac is a lot more permissive; for example, ‘|’ is a legal character in leaf names.

It gets slightly weird due to a compatibility decision in the transition from Mac OS 9 (“Classic Mac”) to Mac OS X. Before Mac OS X, Apple used ‘:’ as the path component separator character. However, Unix uses ‘/’. Apple decided that the file system would continue to use ‘/’, but the Finder would display these as ‘:’. And conversely, any files with ‘/’ characters in them would be stored in the underlying filesystem as ‘:’ characters.

Now, both characters are not legal for Windows. But SMB will accept the full path and store it, which the Mac side can use. Windows, however, doesn’t like seeing a file name with ‘:’ in it, so you get the synthesized short name when you view the file on a Windows machine. This is better than not being able to store files (from the Mac) or work on them (from Windows), but it’s awkward.

The only solution, if you want to work on files conveniently from both operating systems, is to rename files and directories to the legal subset of characters. In most cases, this means using the Windows set of legal characters as the set of allowed characters for file names. Or, you could write your own explorer replacement?

There are further restrictions in Windows using the Win32 API – files can’t be named CON, PRN, AUX, CLOCK$, NUL, COM[0-9], LPT[0-9].

Of course, there are a few files that you should not rename. For example, a file named “Icon\r” (yes, there is a carriage-return character in the filename) is used by the Mac Finder to display a custom icon for a file. For me, at least, the short name for “Icon\r” is “I7CIPB~N”, so they are easy to recognize (the short name algorithm is deterministic, I hope).

And finally, copying these short-name-aliased files with a Windows machine tends to mutate filenames, because many Windows clients don’t use the underlying NT APIs, and so end up garbling things (e.g. copying an Icon\r file to an SMB share with Mac using SMB and then duplicating it to a different SMB server using the Windows explorer changes it to a non-hidden file named “Icon”.

And more finally, we won’t even talk about real file names that have characters outside the limited ASCII/ISO-646 range. These tend to be garbled very easily, for reasons that may or many not be obvious to you (read up on encoding).

How do I unlock files (in bulk) on the Mac?

Assuming you mean “Mac Finder lock bit” and not Unix permissions (which are set with chmod), you use chflags. For example, to unlock recursively (and see what you unlocked), you would do

chflags -v -R nouchg <folder>

and all files rooted at <folder> would be examined and unlocked if needed. The -v flag will print a line for each file/folder that gets unlocked.

This is actually a FreeBSD command and changes “file flags”, which on the Mac has been interpreted to be finder flags. In fact, the man page on Mac OS 10.7.4 and the man page in FreeBSD look very similar (perhaps even identical). A slightly more Mac-specific page can be found at http://ss64.com/osx/chflags.html.



GLR parsing

GLR is the way to go. Really.


active as of late 2012. http://jeffreykegler.github.com/Marpa/. Paper: http://cloud.github.com/downloads/jeffreykegler/Marpa-theory/recce.pdf. Google groups: https://groups.google.com/forum/#!forum/marpa-parser


last update in 2006? http://accent.compilertools.net/


Not really active any more.

Elkhound home page and Google talk that Scott McPeak gave in 2006. Also 2004 paper. http://www.cs.berkeley.edu/~necula/Papers/elkhound_cc04.pdf



Scannerless GLR


ASF+SDF -> Rascal

http://www.meta-environment.org/ and http://www.rascal-mpl.org/. See http://en.wikipedia.org/wiki/ASF%2BSDF_Meta_Environment too.

GLL Parsing

Maybe GLL is the way to go?

GLL Parsing paper by Elizabeth Scott and Adrian Johnstone. Other papers

Stratego/XT and Spoofax.

http://strategoxt.org/ and http://strategoxt.org/Spoofax.


Dick Grune wrote a book on just parsing, named Parsing Techniques/




Not sure that these are worth studying, I think they are a dead end. Yes, you can compose two PEG grammars together, but a PEG grammar is more limited than LR . This is the canonical page: http://bford.info/packrat/. ANTLR is the most significant parser generator using PEG parsing.


http://dinosaur.compilertools.net/ – home page for Lex and Yacc.

Converge is interesting: http://convergepl.org/

Cling is an interactive C++ interpreter based on LLVM and clang.

Some useful notes on parsing and compilers from Lutz Hamel at University of Rhode Island: CSC 402 – Lecture Notes

Why I prefer LALR parsers – “[elimination of left-recursion] is achieved by converting left-recursion to right-recursion; which will always convert an S-attribute grammar into an L-attribute grammar; and seriously complicate any L-attribute grammar.”



Handling “path too long” on Windows

This is one of my posts of “study the mistakes of others, so you can learn from them and not repeat them directly or indirectly”.

This is common to see, at least for me

The directory name y:\backup-2012-09-01-2230\Applications\Adobe Acrobat 9 Pro\Adobe Acrobat Pro.app\Contents\Plug-ins\PaperCapture.acroplugin\Contents\Frameworks\OCRLibrary.framework\Versions\A\Frameworks\iDRS.framework\Versions\A\Resources\Asian.framework\Versions\A\.AppleDouble is too long.

(ignore the fact I’m cataloging Mac apps on a Windows share).

The 260 character limit is so deeply embedded in the Windows system that it’s been almost impossible for Microsoft to eradicate it; it persists even into theoretically new frameworks like .NET (mainly because .NET isn’t actually a completely new system, it’s in many cases a thin shim on top of the older Win32 libraries). While there are APIs that let you access paths up to 32767 characters long, there’s enough bits in the system that use the older MAX_PATH limit that you really can’t do anything about it. So while you can use Windows NT-style paths in many cases (paths prefixed with \\?) in your own code, some core APIs in Windows use the old functions (like LoadLibrary, for example).

Note that you can use \\? paths with cmd.exe, BUT these must be drive letter paths, UNC paths don’t work (don’t know why, maybe one needs to use the NT file namespace or volume GUID path?), and it’s useless, because the resultant path is still subject to MAX_PATH limitations. You also can’t set the current working directory to a UNC path (unless you employ a registry hack, see Microsoft KB 156276).

There are a few workarounds I’ve used in the past:

  • using working directories
  • use of subst and/or pushd (auto-subst, don’t forget the matching popd)
  • hard links, junctions, and symbolic links (although these modify the filesystem)

All of these create new virtual hierarchies so that you can read a deeply nested file without needing a path longer than MAX_PATH; the first two can at best double the length, though.

There are some wacky things people have done in code.

Starting in Windows Vista, the shell starts shrinking individual elements in the path until the whole thing fits within MAX_PATH. This is why you can browse a really deep hierarchy in Explorer.exe, and usually copy it or delete it. This, of course, requires that NT short name creation is not disabled.

Of course, the real answer is “stop using Windows, and just use Unix (Linux or Mac OS X)”, but there’s still a lot of programs that run on Windows and not Unix…