Subcanvas with events

What is the picture of? It's a simple test for the Subcanvas graphics / events API. It's Squeak, but with Morphic temporarily disabled and a Subcanvas test running full-screen and reacting to keyboard and mouse events. The blue squares are from mouse move events (offset by a few mm). The big green rectangle is a subcanvas that draws little red squares rather than blue squares on mouse move events. All the other sized squares are from mouse click events. The text is from keyboard events; the little white lines above text are key press events and the white lines below are key release events.

Font rendering here is done very simply. It will be improved.

The performance of this implementation is currently usable with a single, small subcanvas, but the performance will suffer incredibly as subcanvases get bigger. This is because subcanvases are BitBltted in their entirety after each event; the implementation is currently naïve and there is a lot of room for optimisation.

I'm disappointed by Squeak's graphics performance. BitBlts on Forms are very slow. As a result, I'm not going to try to optimise this implementation of Subcanvas (based on Forms and BitBlt) and instead re-implement it in the future using either a raw X11 socket with XRender, XDamage etc, or use OpenGL in 2-D. Subcanvas at heart is an API, so any users of it should be unaffected. Currently I'm favouring reimplementing it to use X11 as most POSIX systems (Linux, Solaris, BSD, Apple with OS/X) have an accelerated X server, and I could bundle something like XMing with MS Windows. Forms, BitBLT and friends could then be completely purged from the VM leaving it headless.

Keyboard events

Mouse events in Subcanvas now mostly work. A canvas's target now receives mouse movement events, mouse button press and release events and canvas enter and exit events. Now, I've moved my attention to keyboard events.

There are three types of keyboard events in Squeak: Key Press, Character and Key Release. Key Press and Key Release should give only the description of which physical key on the keyboard has been pressed or released, including modifier keys (shift, CTRL, etc). Currently the Squeak VM uses the "Mac-Roman" encoding to encode which key was pressed or released, which is less than ideal and still differs per platform. The Character event contains a Unicode character. A single Unicode character may have required multiple key presses and even some UI interaction to be composed.

There are three types of hardware keyboard in the world; the US one (104 keys), the European one (105 keys) and the East Asian one (109 keys). These are very similar to each other and based on the PS/2 keyboard. Almost every keyboard in the world is based on one of these hardware prototypes with different printed letters on it.

I've also found that the USB specification (made by Microsoft) has reasonably elegant "scan-codes" that specify the actual keys pressed with no regard for which character is printed on that key. I believe these codes would provide for a much more elegant way of specifying which keys were pressed in the "press" and "release" events than the Mac-Roman keys currently provided by the Squeak VM. Unfortunately, any notion of the original USB scan codes have long been forgotten by the OS by the time the character arrives at the Squeak VM, so the VM or the image will need to map OS-specific event IDs to the equivalent USB scan codes. It is a bit of a roundabout way to do it, but the end result is an OS-independent manner of doing raw keyboard events. Perhaps SqueakNOS can strip away the layers of abstraction and pass USB scan-codes through directly? :-)

The end result is that an application using Subcanvas can either use the character events, or it can use the key press and key release events to roll its own "Input Method Editor". This gives a good level of initial functionality for most languages in the world by using the Unicode character input directly from the VM, as well as a future growth path for implementing new "Input Method Editors" (or games) using the raw key press and release events.

On another chain of thought, I'm considering making Pango and Freetype "libraries" available in SecureSqueak / UGP. If they're provided as a resource such that Subcanvas does not have any architectural dependencies on them, they can be implemented as a "deprecated on birth" resource that will provide necessary text rendering functionality until pure Smalltalk-based solutions can be written. At this stage, I'm a bit wary of just how complex the rendering of multilingual text can be.

Squeak on ARM

I'm trying to get Squeak running on my HP Jornada 820. I've used an ancient 3.3 VM until now under Windows CE, but I'd like to get it running under Linux to see if I can improve performance a bit, especially with BitBLT which I know can perform a lot faster on this hardware.

I've learned one lesson: embedded programming requires patience. Everything is slow.

So far I have:

• Gotten Linux (2.4) running on the Jornada from a CF card. This was the easy part.
• Installed Debian Sarge (3.3) from the dusty archives of yore. You see, it needs to support the aforementioned 2.4 kernel.
• Gotten GCC working, which needs a swap space on the CF card. I can hear it screaming in pain! "Hello, world!\n" works.
• Nearly managed to get Squeak to compile. It failed on the Rome plug-in, which was internal.
• Stripped Squeak of all plugins and discovered an unnecessary dependency on a Balloon 3D plugin. It's easy to fix, but I have better things to do.
• Decided that I couldn't cause any more suffering on my CF card and now I'm setting up a Scratchbox environment for cross-compilation.

Currently, I'm stuck on the executables from the cross-compilation environment. GCC on the Jornada makes ELF binaries for "GNU/Linux 2.2" that run fine, but the cross-compiler makes ELF executables with no such versioning... which don't run. Google will inform me.

Concurrency

I'm depressed.

Over the last couple of weeks, I've been playing with various Smalltalk implementations. I always had this idea that some of them could use multiple CPU cores. As it turns out, none of them do it, and the ones that do concurrency don't do it well.

VisualWorks is single-threaded. Squeak is single-threaded. Smalltalk/X didn't fully use two CPU cores. Smalltalk/MT should, but didn't. Huemul 0.7 tried... but crashed.

[update: Smalltalk/MT does use multiple cores as Peter Lount points out in the comments, but is severely limitied by the garbage collector.]

Igor's Hydra VM does concurrency the hard way: by making a completely separate VM in a separate thread but in the same process (using OS terminology here): VMs can communicate with each other, but each is single-threaded. Gemstone, I hear, can make multiple Gems which share a transacted memory, but each Gem is single threaded.

Surely it can't be that hard to make a VM that does fine-grained concurrency?

Smalltalk, the language, is ideally suited to concurrency. The language lends itself to creating concurrent abstractions and already has basic support, but there's no implementation which can take advantage of the CPU power.

Smalltalk VMs are not going to be running any faster if we don't start exploring concurrency. The free ride in MHz increases is about to end soon (in theory, anyway). Moreso, the best Smalltalk VM can only use quarter of the power of a quad-core CPU, and this unused potential will increase exponentially as the number of cores increases exponentially.

What am I going to do about it? Well, I'm going to be modifying the Process scheduler in Squeak to simulate multiple CPU cores. Individual Processes will have their speed throttled down so that using multiple forked processes will be required to get a speed increase. Then I will, eventually, start writing concurrent frameworks to take advantage of the "multiple cores". This way, if some smart alec makes me a nice concurrent VM, they have concurrent code to take advantage of it.

Playing with Blocks

Today I learned about blocks. We Smalltalkers all know and love blocks: those bits of code in square brackets that we can do all sorts of dandy tricks with. So what evils are there?


Well, Evil Trick number one:

b := [ ^ Transcript show: 'In the block'; cr ].

... in another method:

b value.
Transcript show: 'After the block evaluation'; cr.

The latter message will not be printed on the transcript. When you evaluate a block that has a return statement in it (a "non-local" return), the current context is abruptly terminated and the method context where the block was defined is returned from.


Evil trick number two:

b := [ ^ 1 ].
[ b value ] fork.

This will cause >>cannotReturn: to be evaluated on the BlockContext. The block has a non-local return in it but it cannot return. The containing MethodContext of that block is not on the call stack, because the block is evaluated in another process which has its another stack. When a non-local return happens, the VM searches down the call stack until it finds the method that the block is defined in.

This will also happen if you get a block with a non-local return by calling a method that defines it and then evaluate that block after the method returns:

someMethod
^ [ ^ 1 ].

self someMethod value. "Fails"


Evil trick number three:

s := Semaphore new.
b := [ s wait ].
[ b value ] fork.
[ b value ] fork.

This will fail in Squeak, although it shouldn't. Squeak has an optimisation that prevents two concurrent evaluations of the same block. I'm not entirely sure why, but it certainly hinders concurrent programming in Squeak (that, and the fact that Squeak is single threaded).

It also happens if you get a block to evaluate itself:

b := [:block | block value: block].
b value: b


Evil trick number four:

[ |b| b := 'bar'] value.
[ |b| ^ b] value.

You would expect to get a nil, but in Squeak, you'll get a 'bar' instead. This is because block variables are defined in their method contexts rather than in their block contexts. This is what the community is complaining about when they complain about the lack of closure support. So, how do you fix it?

[ |b| b := 'bar'] value.
[ |b| ^ b] fixTemps value.

This version does not work: fixTemps disallows a block from doing non-local returns. How about:

| result |
[ |b| b := 'bar'] value.
[ |b| result := b] fixTemps value.
^ result.

Lame, but it actually works like it should. nil is returned.

Low-level graphics in Squeak

It wasn't until I dug into how low-level graphics worked in Squeak that I realised how trivial it was.

To draw on the screen, use the global variable "Display". This is an instance of a subclass of Form. A Form is a bitmap. To draw on Forms, you use "BitBlt" instances, which represent drawing commands.

For handling events from the mouse and keyboard, you use the global variable Sensor, which is an instance of EventSensor. In a loop, you call EventSensor>>nextEvent to get the next event, which is either "nil" or the next keyboard or mouse event. Yes, you need to poll it, and if you don't want to use 100% CPU, then you need to suspend your thread for, say, 20ms on every iteration.

Except for the details, that's all you need to do to draw on the screen and handle events. There's plenty of example code in a stock Squeak image.

Now comes the tricky part: how do you wrestle control of these objects from Morphic? Well, as it turns out, Morphic wasn't designed in the resiliant, secure and reactive way I would have designed it. Morphic is single threaded! When you run your code, you're running it in the same Process that handles event handling and screen drawing! So if you never return from your event polling loop, Morphic never runs (unless a Transcript is open and you write something to it).

I find the single-threadedness of Morphic rather quaint.

Alternatively, you can use:
Project uiProcess suspend. "This suspends the Morphic process; make sure your code runs in another process!"
Project spawnNewProcess " If you terminated the Morphic process, this makes a fresh one. "

SiteBrowser

Last night I started work on a new SiteBrowser. This time, it will only use the Subcanvas API to do drawing and event management.

Hopefully it won't take much more time before I can release SecureSqueak version 0.1. I think I'll revise the schedule to make lots more releases each with fewer new features.

In the meanwhile, Matthew Fulmer has sent a progress report about Squeak 4.0. I'm not sure if that should affect me. My current state of thinking is that I'll be making such vast changes to the kernel that it doesn't really matter which version I begin with.