Mobile Smalltalk

I've been announcing the Open Beta release of my game HexSolve for Android and iOS.  HexSolve was written in Smalltalk and people have been asking me how it's possible to run Smalltalk on mobile devices.  I thought I'd write up a little description of the technology behind it.

I've wanted to be able to run Smalltalk on Android and iOS for many years.  I finally decided to do something about it.  I chose VisualWorks for my development environment.  Unfortunately iOS doesn't support dynamic compilation so getting the regular VisualWorks VM to run on iOS is virtually impossible.  I decided to skirt around the issue by writing my own VM.

My VM is written in C.  It interprets the same bytecodes as those generated by VisualWorks so I can use the VisualWorks compiler to compile all of the code for the mobile Smalltalk.  Being a pure interpreter, I get around the problem of dynamic compilation on iOS.  Apple is okay with an interpreter so long as all of the code it's interpreting is delivered with the product and not downloaded dynamically.

Development of the Smalltalk code begins in VisualWorks.  I have my own Object class (in my own namespace) as well as my own copies of the other base classes (True, False, UndefinedObject, etc.).  I did this because I had better control of the implementation than I would if I just used the VisualWorks base code.  If I used the regular VisualWorks classes, then I would need to bring in more and more of the base code to get those classes to work.  There are just too many "strings attached" to do this effectively.

Methods beginning with an underscore ($_) will have the underscore removed when moving the image to the mobile device.  It allows me to have one version for Mobile Smalltalk and a different version for VisualWorks.  You can imagine that it would be bad if I re-defined at:put: in VisualWorks.

I have an ImageBuilder utility that takes all of the code in the packages "SimTalk Remote Core" (base Smalltalk code) and "SimTalk HexSolve" (the game itself), packages them into a special 64-bit image format and writes the image into a file.  I can then use AndroidStudio or XCode to include this image as an asset to deliver to the device along with the interpreter.

The interpreter itself is a 64 bit image.  It uses immediate objects (tagged objects that fit within the object pointer) for SmallInteger, Character and Float.  This allows me to do faster operations on these objects without allocating memory.

The game uses Smalltalk primitives to interface to OpenGL ES which runs on both Android and iOS. For text, I interface to Freetype which is compiled and delivered with the game.  Freetype will create images for the characters which I can collect and cache in a Smalltalk Image object.  I then render these images as OpenGL textures.

My mobile Smalltalk is single-threaded - there's no such thing as fork, a Process or a ProcessManager.  In fact, Smalltalk runs completely as an OS callback.  When the OS determines that something happened (a touch event, a button or a timeout), it runs a callback in my VM.  The VM creates an Event object, drops it into a known spot in the image and runs the interpreter.  The interpreter then reads the event, dispatches it, re-renders the screen if necessary then calls suspend which returns from the OS callback.  Nothing more happens until the next OS callback.

Some people have asked me when I'll release this technology so they can use it.  I'm sorry to say, but I don't intend to do that.  This is for internal use for me to write Smalltalk code for Android and iOS.  I have several problems if I try to commercialize the development technology:

1. You can't develop code in this environment without a VisualWorks license.  I have such a license with Cincom but it's hard to tell other developers that they need a license for VisualWorks plus a license for my mobile development environment and it's hard to enforce.
2. I've implemented interfaces to API's I need in the OS but not to everything.  Other developers would need probably access to accelerometers, GPS, Internet, cameras and other things.  I can't have a good commercial product without those interfaces and it's too much work to write these interfaces on speculation.
3. Too many companies have died trying to provide tools for developers to enhance Smalltalk environments.  The market isn't big enough to justify the effort.
4. To be honest, software developers make lousy customers.  They tend to be very demanding on everything they need and usually don't like paying money for what they get.  They're used to software development environments being free and open source and don't want to pay for development and maintenance of their toolset.
5. I'd rather create fun profitable games than creating development environments for other people to use to create fun profitable games.

The graphics for HexSolve were all rendered using POV-Ray.

I hope that answers most of your questions.  Happy Smalltalking.

Base source code - the good, the bad and the ugly

Smalltalk is one of the very few development environments where developers get easy access to all the source code of all the frameworks they use.  This comes in very handy when there are problems you need to debug and you need to step through the framework to figure them out.  It also allows you to see how the framework expects to call your code.

Smalltalk also allows you to make changes to that framework.  Environments like VisualWorks even allow you to keep your base changes in your own packages so that they're loaded in when you load the rest of your code from the version control system.

The problem with all of that is that developers begin to feel too free that they can change anything they want.  If they don't like the colors of controls, they can change them.  Do you want to add new options to the controls? Sure, go ahead. Do you want lists that format their contents into columns?  Just write it.  Many companies develop their own frameworks on to of the vendor-supplied framework.

It makes for an incredibly flexible system but there's one huge drawback.  When you upgrade to a new version of the base, the vendor has made changes and now all of your modifications don't work.  It's a long and painful process to find the pieces that are wrong and either fix them or re-write them.

I'm currently working on two upgrades for VisualWorks customers.  Both of them have built their own frameworks on top of the VisualWorks framework.  Both are encountering problems upgrading to the latest version because some very basic things have changed in the base.  Normally, these are details that only the vendor needs to worry about, but when you build your own framework with intimate knowledge of how the base framework operates, you're going to run into problems.

I don't really have a good solution for these customers except to fix the problems as I come across them.  It would have been nice if they didn't have this strong dependency on the base but given that they do, I can't really change that.  They just have to understand that it makes upgrades more difficult and expensive.

Do developers in other languages have these problems as well?  Do you need to make sweeping changes to your Java application whenever a new version of Java or one of your framework libraries comes out?  Do you customize your framework libraries?  Is this just a Smalltalk headache?  Let me hear from you.