Doing an Xcode build from within Sublime Text 3

It is possible to create a comfortable build environment for Xcode projects with Sublime Text 3. Here are step by step instructions for how I did it.

First issue - achieving a build.

Sublime can invoke command line programs to do a build. Xcodebuild is a command line tool that comes with Xcode that does it. To do a build, sublime needs a sublime-project file with a recipe in it.

It's a json file that you put in the root folder of your project, and it should look something like this, assuming that the xcodeproj file is in the same folder, and is named MyProject.xcodeproj.

You'll notice that I am overriding SYMROOT and OBJROOT because I am doing an out-of-source build into ~/build. You can omit that if you are fine with xcode putting build products in the usual spots. I greatly prefer an out of source build for reasons, which include being able to make a build utterly clean by simply deleting the build folder.

Command-B will invoke a build, and Control-C will stop it instantly, which is awesome.

MyProject.sublime-project

{
    "folders": [
        {
            "follow_symlinks": true,
            "path": "."
        }
    ],
    "build_systems": [
        {
            "name": "Xcode Build",
            "cmd": [ "xcodebuild",
                     "-project", "$project_path/$project_base_name.xcodeproj",
                     "-configuration", "Debug",
                     "-scheme", "MyProject",
                     "SYMROOT=~/build/Products/Debug", "OBJROOT=~/build" ],

            "variants": [
                {
                    "name": "Xcode Run Debug",
                    "cmd": [ "~/build/Products/Debug/$project_base_name.app/Contents/MacOS/$project_base_name" ],
                },
                {
                    "name": "Xcode Build Release",
                    "cmd": [ "~/local/bin/xctool",
                             "-project", "$project_path/$project_base_name.xcodeproj",
                             "-configuration", "Release",
                             "SYMROOT=~/build/Products/Release", "OBJROOT=~/build" ],
                },
                {
                    "name": "Xcode Run Release",
                    "cmd": [ "$~/build/Products/Release/$project_base_name.app/Contents/MacOS/$project_base_name" ],
                }
            ]
        }
    ]
}

Second Issue - Optional - excess verbiage

The next issue you will notice is that when you do a build, there is an excess of verbiage that appears in the build window, making it exceedingly difficult to find real problems. You might be fine with that, or have magical regexs to find warnings and errors, and if so you can skip this step.

I used git to clone Facebook's xctool, and built it following the instructions on their page.

https://github.com/facebook/xctool

After building, a build folder appeared. A couple of layers deep there is a Products/Release directory. I have a particular place I store /bin and /lib files, and I copied xctool out of Products/Release/bin to my bin, I copied the dylibs from Release/lib to /lib, and I copied the Release/reporters folder to a /reporters folder that I created as a sibling to my /bin and /lib. (To be clear, the result was /reporters/stuff, not reporters/reporters/stuff.)

I changed the cmd in the project from xcodebuild to xctool.

Finally, I entered this

os.environ["PATH"] = os.environ["PATH"]+":/my/local/bin"

to point to my local bin from within Sublime's python console. This is necessary in my case because I haven't yet figured out how to communicate my shell path to launchctl so that the path Sublime knows about is the same as my shell path.

Inconveniently I have to redo that every time I start Sublime, but I'll work out the right answer eventually.

Now, builds are much tidier to read (my eyes are much happier), and the results include timings, which is nifty.

Third Issue - Sublime's Build Window is Pretty Much Useless

This issue could be PEBCAK (Problem Exists Between Computer and Keyboard), but some searching around places like Stack Overflow make me think otherwise.

In a nutshell the build window vanishes if you do pretty much anything interesting, like touching the mouse or keyboard. Also, the only operation it supports during its brief existence is scrolling. Heaven forbid you want to search for the word warning, or error, or something like that.

Also I was not for the life of me able to make the Sublime "go to next error" regex stuff work. It was mostly a world of hang, so I looked for an alternative.

I installed sublime-text-2-buildview, which despite it's name, works fine in Sublime Text 3. It captures the output from the ephemeral build window and redirects it to a normal text editing window, once again, a small bit of awesome. (I used Package Control to install it.)

Bonus Round - xcconfigs

For the bonus round, I moved all of my project settings out of the xcodeproj and into an xcconfig file so that I could control things like search paths, preprocessor settings, and warning levels from the comfort of my text editor.

I followed this tutorial - http://www.jontolof.com/cocoa/using-xcconfig-files-for-you-xcode-project/, setting my files up exactly as he suggests. Note that if you are copying and pasting a setting and it doesn't show up properly in the text editor (ie, if they paste as only the name of the setting without a value), it's because Debug and Release are different. You need to select them individually and paste them into the appropriate xcconfig file.

I didn't rest until all my settings showed up as coming from the xcconfigs.

Conclusion

Once you've got all this set up, a command line build is equally easy to achieve. I put together a little sh script for myself that contains roughly the contents of the cmd field above, and it worked a treat.

Especially when I need my laptop's battery to last a long time, building in Sublime is a huge win.

There's obviously some rough edges in this set up, particularly the environment setting if you use xctool, and it's hard to believe that the build window is as terrible as it seems - have I missed something obvious? Nonetheless, I hope you find this helpful, and would welcome suggestions or improvements from anyone.

Aquattack Review!

The February 1985 issue of Electronic Games had a positive review of my first console game. Click here to read it. I think the author of the article had quite an active imagination since he describes animation like spinning Zeppelin propellors that I didn't actually achieve with the few pixels there were to play with!

Multi-screen action and originality are Aquattack's strong suits. It's packed full of excitement and gaming fun, offering four different scenarios per level of difficulty. Such diversity really puts games through their paces! ... The graphics are nothing short of spectacular... It's worth the effort!

The game was programmed on a repurposed TRS-80 Model II, with great big 8 inch floppies, hand-soldered break-out boards that jacked the Coleco's Z80 into a Textronix logic analyzer, custom EPROM programmers, and a lot of office hijinks and practical jokes. We worked out some pretty unconventional uses for rocket igniters and ketchup packets.

Aquattack was the first side scrolling shooter on the Colecovision. In fact, at the time there were so few side scrolling shooters that I had to think about whether the player should travel towards the right or to the left. Scramble and Zaxxon provided inspiration, but most other games at the time were vertical scrollers. Gradius came out later in 1985 and established how side scrolling shmups should go, definitively, too late for Aquattack though. Of course if I had been paying attention to cinema at that point I would've known the western convention of travel left to right generally indicates going, and travelling right to left generally indicates coming back.  

Ultimately, I picked travelling towards the left because I could scroll the playfield with fewer CPU instructions than if I chose travelling to the right, which meant I could fit a bit more game play into the tiny ROM. Parsimonious cycle counting not-withstanding, I did sneak a couple of easter eggs into the cartridge, including an alternative soundtrack - Lara's theme from Doctor Zhivago. I think I snuck it into the high nibbles of some initialization data somewhere.

Aquattack was the only pixel-smooth scrolling game ever shipped on the Colecovision. We rolled out our games at CES in Las Vegas in 1984. At that CES, I think we might have been the only independent Colecovision game developer, which made us a bit proud. The Coleco engineers stopped by to peer at my game, because smooth background scrolling was supposed to be impossible - I got a kick out of their speculations.

The trick I used was to rewrite the font making up the playfield instead of the playfield itself. Scrolling a single wall piece for example, would scroll all wallpieces on the screen simultaneously. Communicating to video memory was really slow, so updating a few font characters to move the whole screen was way faster than rewriting blocks of memory to redraw the screen in a more obvious way. I overcame a number of hardware limitations with that trick. There are more helicopters in the screen shot below than there were sprites. That was accomplished by rewriting a few lines of a couple of font characters over and over again. Sprites only had a single color, but I tricked up two or three by allocating the sprites carefully per scan line, and switching tables during horizontal retrace. Most Coleco games' sprites flicker pretty badly due to that sort of multiplexing, but I had a pretty slick manager that kept it from ever happening - I also tweaked the game play to play nicely with the limitations of my algorithm.

 

I was really reminded of the level of fun and invention that went into an old 8-bit game during the PS2 era; the restrictions and foibles of the VU in particular were very reminiscent of the mindset you had to get into make these boxes go. It's quite a different world when as a programmer or designer you can hold the whole machine in  your mind and imagine what you can make it do. I'm very jazzed by what I see going on with new small systems like iOS, because the creative sandbox is very much like what we enjoyed way back when.

Stone Soup

Exotic and Rare, the C Compiler

Once upon a time, a C compiler was a rare and exotic thing. They weren't commonly available outside of research labs or well funded corporations, and if you were working on a newly minted microprocessor-based machine, you were out of luck. In those days you were lucky to have an assembler - sometimes your day started by keying your program in address by address, byte by byte, or by laying some rudimentary tools into RAM from a tape and pushing the Go button. Once upon a time, lifting a machine by its bootstraps was an important start of the development process.

3D0G!

The bootstrapping process was well known to old school game coders. We knew all the Apple ][ system vectors and how to lay in boot loaders to hijack the startup process straight into a game. The boot ROM in the disk controller card loaded the first 256 bytes of code and jumped into it; increasingly complicated loaders were overlaid until finally your game was in memory, and you were off. The Apple ][ was a great to-the-metal playground for learning to code. Its primal tool was a marvelous mini-assembler - no symbols or branch calculation helpers, just a simple translator from opcode mnemonics to bytes, and its greatest treasure was the “L” command to list a few more lines of code. It was enough to put together a lot of fun, and reveal the innermost secrets of programming. Steve Wozniak wrote the mini-assembler, he was a teacher from the start, and the Apple ][ monitor program was the best school.

Early Console Hackery

But what about systems not quite so hacker friendly? One of my favorite bootstraps was the one we did at Interphase Technologies on the Colecovision. The Colecovision was an archetypical closed gaming console, popular long before the days of SDKs. When we started we knew nothing about the machine. Some components were sealed under epoxy blobs, some parts had numbers removed. Our first order of business was some radical deconstruction. This involved acids, sledgehammers, microscopes, and drops from high altitudes. It also involved diagramming out circuit diagrams and other down to earth engineering. We worked out the general capabilities of the machine; most importantly we found that it was sporting a Z80. Going back to an unpulverized unit, we desoldered the CPU and wired in a breakout header and carefully reattached the CPU above it. We attached a signal analyzer to the breakout header, and built a latch on the address lines so that we could halt the CPU when certain addresses were requested from the ROM. We wired LEDs to some outputs. Now we had a debugger! We dumped the system ROM worked out what routines and entry points we could use from it, and it was all straight forward from there. We hacked a TRS-80 assembler to target the features of the Colecovision, got some EPROM burners and UV lights to erase the ROMs for the next debugging sessions, and we had a development system. A couple of weeks later, an early success - I was able to display an X in the middle of the screen! It was only a small step from there to a fully functioning game. You can grab my first console ROM, Aquattack, from this link: (currently offline as site is reconstructed). It works in any Colecovision emulator. My proudest moment was when the Coleco engineers came by at CES because they had heard I had managed smooth scrolling. The advantage of not having any documentation or guidance was that we didn't know what the machine couldn't do! From the screenshot, you might notice that Aquattack scrolls in the opposite direction of every side scroller to follow. Since Zaxxon was the only isometric side scroller before it, there wasn't much precedent for which way things could go. I picked the direction where I could rotate the bits in a byte with the fewest cycles.

Coding in Silos

In those days we coded in silos. A little light came in from the top, but you might go weeks or months without actually talking with someone who was doing the same things as you. Occasionally you'd hear rumors of something called Siggraph or what have you, but it could take a really long time to track down those sorts of resources, and in the mean time you did the next best thing which was to work it out for yourself... At that time the C compiler was still a rare and exotic thing, but at last the great day arrived - I managed to dredge up the source code for a C compiler off usenet. It was a stunning surprise to discover that the C compiler was written in... C. The source code was not the most readable thing in the world, but I was determined to work out how it could be that C might be written in C. I turned to Forth, a language I was already familiar with from my work in robotics. Forth is the classic bootstrappable language.

Bootstrapping a Compiler

At its simplest level, Forth doesn't do much. Here's my implementation of the innermost bit, written in 68000 code (it was certainly a luxury to not have to build the assembler first!):

doNext MACRO movea.l (a5)+,a3 ; fetch cfa pointed to by ip, and inc ip 
             adda.l a4,a3     ; convert the base address to absolute 
             jmp (a3)         ; go execute the next word 
ENDM

Every Forth command ends with the doNext macro which picks up the next opcode to execute, adjusts its address via the base pointer in a4, and jumps through it. To start Forth, a few registers are set up in assembly code to point to the first executable bit of Forth code, and the doNext macro is invoked. That drops into the cold start, written as a series of hard coded Forth word offsets, and so it begins.

 
cold bra doColon 
dc.l coldStuff    ; do all the neat power up stuff 
dc.l clrPage      ; clear screen 
dc.l adotqa       ; write the following message
dc.b 53 
dc.b 'FUSION FORTH 1.00',13,10 
dc.b 'by Nick Porcino and Don Palmer' 
dc.b 27,'e'       ; make cursor visible 
dc.b 27,'v'       ; wrap around 
EVEN 
dc.l cr           ; do a carriage return 
dc.l shell        ; start the command line stuff 
dc.l exit         ; end of definition

Once this much code was written in assembly, I wrote a few of the most basic bits such as “compile a Forth word definition”, “save to disk”, and “parse a number”. Little by little enough functionality came on line that more and more sophisticated parts of the language could be written directly in Forth, and the Forth implementation began to compile itself as part of the bootstrap sequence. The file below is part of the start up; it invokes bits of Forth code to keep building up more of the compiler and base libraries. The last file that is loaded is the saving code that allows new Forth programs to be saved out to disk.

( Fusion FORTH compilation link file ) 
" INIT.BLK" ENTER LOAD 
ENTER GRAFIX.4TH LOAD ENTER MISC.4TH 
LOAD ENTER FEN.4TH LOAD ENTER FSEL.4TH LOAD 
CD . CD 
EDITOR ENTER PRINT.4TH LOAD ENTER 
EDFEN.4TH LOAD ENTER ED2.4TH LOAD 
CD . CD CORE ENTER SAVEFORTH.4TH LOAD

If you are interested in digging through the code, a version of Fusion Forth for the Atari ST is available at (currently offline as this site is reconstructed). Ultimately, I enlisted a couple of friends to help, and we built the language out completely, including a GUI, a text editor, a graphical IDE, some games and utilities, and more, and all of it bootstrapped up from only a few lines of initial assembly code. It's a bit like the old parable of Stone Soup. I never did get around to compiling that C compiler though!

Stone Soup

The process of bootstrapping goes on around us all the time, in the form of new platforms and protocols. When I think back on the humble beginnings of the computer revolution, and how recently it was a major effort just to display an X on the screen, I am simply awed at how complexity has enfolded upon complexity to turn the microprocessor into an engine of societal change. 

http://en.wikipedia.org/wiki/Stone_soup:

• A kindly, old stranger was walking through the land when he came upon a village. As he entered, the villagers moved towards their homes locking doors and windows. The stranger smiled and asked, "Why are you all so frightened? I am a simple traveler, looking for a soft place to stay for the night and a warm place for a meal."

  
  

  "There's not a bite to eat in the whole province," he was told. "We are weak and our children are starving. Better keep moving on."

  
  

  "Oh, I have everything I need," he said. "In fact, I was thinking of making some stone soup to share with all of you." He pulled an iron cauldron from his cloak, filled it with water, and began to build a fire under it. Then, with great ceremony, he drew an ordinary-looking stone from a silken bag and dropped it into the water. By now, hearing the rumor of food, most of the villagers had come out of their homes or watched from their windows. As the stranger sniffed the broth and licked his lips in anticipation, hunger began to overcome their fear.

  
  

  "Ahh," the stranger said to himself rather loudly, "I do like a tasty stone soup. Of course, stone soup with cabbage -- that's hard to beat." Soon a villager approached hesitantly, holding a small cabbage he'd retrieved from its hiding place, and added it to the pot.

  
  

  "Wonderful!!" cried the stranger. "You know, I once had stone soup with cabbage and a bit of salt beef as well, and it was fit for a king." The village butcher managed to find some salt beef . . . and so it went, through potatoes, onions, carrots, mushrooms, and so on, until there was indeed a delicious meal for everyone in the village to share. The villager elder offered the stranger a great deal of money for the stone, but he refused to sell it and traveled on the next day. As he left, the stranger came upon a group of village children standing near the road. He gave the silken bag containing the stone to the youngest child, whispering to the group that it was not the stone, but the villagers that had performed the magic.