Software Engineering Matters

Ruby Is Smalltalk Minus Minus:

"--- Challenge to Smalltalk advocates: go ahead and reimplement DwemthysArray? then! The Array is a rather short program so it shouldn't take you too long. It's also rather well known, even celebrated, by now so a ST version will reach many people who wouldn't normally even notice ST advocacy. And an elegant implementation will frankly be a lot more convincing than 'sure, we could do that' assertions. We've been promised Smalltalk DWEMTHY before: http://redhanded.hobix.com/inspect/theRabbitWillDieInSmalltalk.html - who will deliver?"

BIG DISCLAIMER UPFRONT: This is just an exercise and I am not advocating Smalltalk over Ruby or Ruby over Smalltalk. Both have their strong points. I am just doing this out of curiosity and for fun.

I was browsing around the web and glanced upon the snippet above from the c2 wiki. I am not sure how long it has been up there (probably since 2005) but it seems that no one has responded to it. For those unfamiliar with it Dwemthy's Array it's a Ruby meta-programming example that is along the lines of a text-based game. I was familiar with it when it first surfaced but never thought of the need to implement it in Smalltalk since the approach is not very Smalltalk-ish in the first place. However, since I had some time, I decided to explore how it could be done in Smalltalk by mimicking the Ruby implementation.

A sample run of Dwemthy's Array in VisualWorks. Look at the Browser to see how I mimic the Ruby code. Notice that I still need to explicitly write the super initialize.

The gist of the Ruby implementation of Dwemthy's Array is the ability to write code like the following to declare a new class. It's simple and it reads like a mini Domain-Specific Language. Because of this, it also looks very appealing aesthetically.

class ScubaArgentine < Creature
  life 46
  strength 35
  charisma 91
  weapon 2
end

Basically it can be "de-sugared" to look like this which would make it more familiar. Basically the entire code relies on doing class method calls (the self.life) to create

# This is a comment
class ScubaArgentine < Creature # This declared ScubaArgentine as a subclass of Creature
  self.life(46) # This is just a call to the class method life i.e. Creature.life(46)
  self.strength(35)
  self.charisma(91)
  self.weapon(2)
end

The gist of how all of this is done is in the following snippet:

# Advanced metaprogramming code for nice, clean traits
def self.traits( *arr )
  return @traits if arr.empty?

  # 1. Set up accessors for each variable
  attr_accessor *arr

  # 2. Add a new class method to for each trait.
  arr.each do |a|
    metaclass.instance_eval do
      define_method( a ) do |val|
        @traits ||= {}
        @traits[a] = val
      end
    end
  end

  # 3. For each monster, the `initialize' method
  #    should use the default number for each trait.
  class_eval do
    define_method( :initialize ) do
      self.class.traits.each do |k,v|
        instance_variable_set("@#{k}", v)
      end
    end
  end
end

# Creature attributes are read-only
traits :life, :strength, :charisma, :weapon

In my opinion, the original article on Dwemthy's Array doesn't actually explain how things work clearly -- the author goes through great lengths to make it seem magic and the prosy language doesn't help clarify how it works underneath. Unfortunately, if I were to dissect it line by line it would make this article extremely long. So, instead, anyone interested in a better explanation so watch this video presentation or refer to the Pickaxe Ruby Book or the Ruby Way book.

In the following paragraphs, I am going to focus on what I think are the most fascinating issues of the implementation and how it could be done in Smalltalk. The following paragraphs assume familiarity with Ruby and Smalltalk reflection/code generation. This lecture slide from the University of Bern gives a quick refresher on Smalltalk reflection. My implementation is a direct-translation as close as I know how to implement it. So that means that I will be sticking to code generation, using class instance variables and a lot of class methods.

Dynamic code generation

Ruby offers method such as instance_eval and class_eval to make code generation slightly simpler. In addition, it also offers attr_accessor. Smalltalk, on the other hand, does code generation by using the compile: 'some smalltalk code' method. Code generation might be a misnomer since that is basically how everything is defined into the image.

The compile: 'some smalltalk code' is used behind the scenes in Smalltalk for generating accessors and doing other refactorings. I have not actually seen the instance_eval and class_eval used to do any refactorings.

Code generated in Ruby is also hidden meaning that you are not able to see the actual source/implementation of the generated method. This makes things more terse but can make understanding/debugging the program harder -- it really makes things much simpler if you could actually see the methods that are generated.

define_method( someName ) do |val|
  @traits ||= {}
  @traits[someName] = val
end

def someName(val)
  @traits ||= {}
  @traits[someName] = val
end

Every generated code in Smalltalk (using the compile: 'some smalltalk code') appears in the Code Browser like every other method, which brings me to the next point...

File-based vs. image-based implementation

Because Ruby operates in a file-based manner, it re-initializes the meta-programming facility every time you reload the file. For this form of programming, reloading the file has some advantages -- if you screw up you can just easily reload everything and not worry about side effects.

As I was doing the implementation in Smalltalk, I screwed up a couple of times and had to remove some of the generated code. Rather than unloading everything and loading it again (tedious and not very Smalltalk-ish) I implemented a reset method that removes all the dynamically generated code and resets other meta-programming constructs. So using my reset method, I can simulate starting from a clean state each time for testing purposes.

Additionally, one solution that I used to simulate file-based loading is to override the initialize method on the class side so that some action is performed as the code is loaded into the image.

In a nutshell, it is possible to simulate a file-based loading by writing proper reset and reinitialize methods. These methods definitely come in handy as I was doing this exercise since I don't need to unload/load the parcel in each time -- I did do that for the final testing to ensure that everything is working.

Dynamically Adding Instance Variables

In Ruby it seems that you can just define a new instance variable using @some_name or instance_variable_set("@some_name, some_value)and it will immediately be picked up. In Smalltalk, I had to ensure that I declared a new instance variable first using self addInstVarName: 'someName' and then I can change its value using self instVarNamed: 'some_name' put: some_value.

Instance variables:

"Instance variables do not need to be declared. This indicates a flexible object structure; in fact, each instance variable is dynamically appended to an object when it is first assigned."

This two-step approach in Smalltalk hints are the differences on the underlying layer on how instance variables are stored.

Class Instance Variables

Another important (and possibly subtle) point about the approach is the use of class instance variables. Class instance variables offers the ability of class variables without the problems that come about from subclassing. This is important because it allows classes to share common variables without interference from the class hierarchy.

By default, VisualWorks uses/offers class instance variables when declaring a new class. If you ever need a class variable then I think you would actually define what is called a shared variable.

Bottom line, class instance variables are really useful. Without them you would have to resort to a lot of trickery to share data between instance of a class while preventing access/modifications from subclasses.

Class Methods

The Ruby implementation above uses a lot of class methods. Class methods reside on the metaclass of a class. Fortunately Smalltalk follows this paradigm closely so it's not hard to implement these features. It just involves doing compile: 'some smalltalk code' on the right receiver. In my implementation, you will find a lot of the methods in the class side of things. Most of the instance methods are dynamically generated.

Implicit self

Ruby's implicit self does have some advantage here since it reduces the need to repeat self keyword over and over and over and over....

So here's my implementation in VisualWorks Smalltalk. I am not an expert in packaging parcels in VisualWorks so expect potential problems. However, even if you decide not to load it into VisualWorks it is possible to eyeball the code to get the gist of it. As far as I know, this is not a very Smalltalk-ish way to approach the problem. I did it this way to mimic the Ruby implementation as closely as possible.

The Ruby implementation seems very magical because a lot of the generated code is actually hidden. I think that if it were possible to just examine the generated code, it would make it must easier to analyze and reason about. It also enables you to use your normal tools for refactoring/ debugging. Of course, because things are hidden, it also makes the code terse and sometimes easier to read especially if you don't care about the details.

For an alternative approach see Darren Hobbs: Getting Meta. I am not really sure whether his implementation works in the end or not.

Many people hear that the code browser in Squeak (or any Smalltalk for that matter) is designed for code browsing. In other words, the browser is optimized to help you read code.

However, I have found that reading code in the regular Squeak environment can quickly become a big mess especially if you are trying something out for the first time and need to read multiple implementations in different classes. Usually, when this happens, you end up with a proliferation of browser windows that quickly clutter your screen. I am assuming that veteran Smalltalkers don't find this a problem because:

• they know most of the methods so well that there isn't much need to refer to it
• they have LARGE monitors so 20 open browsers is not a problem
• they are just disorganized and don't give a damn about it

In this article, I am going to show you how to use the OmniBrowser with some new enhancements. The same group that made the OmniBrowser has also created a whole bunch of very nifty tools. I'll try to get to each one in separate articles. But before that, let me divert your attention to this very relevant message:

A Squeak Smalltalk Development Example:

"Most developers who use Squeak would have a plethora of extra tools and utilities installed that make developing a much nicer experience than what you see in this tutorial. Do yourself a favor and start your Squeaking with a real developer’s image loaded with all the proper goodies like the SqueakDev image maintained by Damien Cassou."

So at this moment, I am going to insist that you download one of the excellent images maintained by Damien over at his webpage. Don't even bother using the basic squeak image. The basic image is pretty disheartening and discouraging to any new developer who is trying Squeak for the first time. Since Squeakers are usually developers, I have always found it mind-boggling that the standard Squeak image does not include better support for developers out-of-the-box. Instead, it is cluttered with a lot of stuff that most developers don't care about.

For this tutorial I am using the squeak-web/beta version from here. Don't forget to grab the SqueakV39.sources file from here as well.

Here's our scenario: we want to see how to actually implement the relevant methods so that two objects can be compared to one another for sorting purposes. Here's a step-by-step guide to how I would use the OmniBrowser + enhancements to do it.

Open the SmartGroup Browser
Bring up the World Menu in Squeak and select the SmartGroup Browser.

Type in the '<' selector
In the top pane of the SmartGroup Browser type in '<'. The top pane is called the Mercury Pane and it lets you type in class/selector names and quickly displays it in the current browser for you. For more information about the Mercury Pane see this.

New window showing the implementors of '<'
A new windows pops-up showing the implementors. I think it is OK to have a new window pop-up with the search results. Also notice that this actually shows the classes in a hierarchical manner (by tabs). So you can see that Character is a subclass of Magnitude.

Search for 'Float'
Now back in the SmartGroup Browser, I am going to search for the Float class since I am interested in seeing the definition of the method for it. I am going to enter 'Float' in the Mercury Pane.

Split Panes in the Browser!
By doing a Shift+Click on a class/method you split a new pane with it!

Multiple Split Panes showing all the methods
By repeating the Shift+Click process, we can create as many split panes as we want. Usually it's enough to have about 4 of them. Also, don't forget that you can browse back using the '<<' button.

For comparison, this is what it would look like if we had to use the normal tools from the standard Squeak image. Look at the wasted screen estate and how I don't see any nice hierarchical information between the classes.

This is roughly what it would look like to find the '<' method in Squeak 3.10 without the new tools. We start by trying to look for the '<' selector in the Method Finder and then clicking on each method to see its definition. Notice that there can be no end to the list of new windows!

So I claim that by using the new tools, reading code in Squeak is much more organized. Good organization not only helps the developer focus but also makes it more approachable (and less intimidating) for a new user. Personally, I advocate that any new tutorial about Squeak be written with reference to these new tools. These new tools are hidden gems that need to be exposed to the world! Only by getting more people to use them can we:

• improve existing tools based on user feedback and experience
• actually get new tools to appear for Squeak - after all what's the motivation of creating new tools if everyone is just using the old way?
• change people's perception that Squeak's tools are 20-years outdated compared to more polished tools such as Eclipse
• make it easier for first-time developers to use Squeak. Seaside has already done a fantastic job garnering attention for Squeak and we should take this opportunity to make it easier for developers to get started

I also hope that people will be less close-minded about enhancing Smalltalk as evident from some of the comments in the following posts:

• Extending Smalltalk Syntax
• How Should We View Methods?

Of course, changes should be implemented judiciously but without an open-minded attitude toward change, nothing will change.

BOFs or Birds of a Feather Sessions allow people with common interests to gather and just talk about anything and/or everything about their common interest! This time I went for three BOFs, all of them focused around Squeak and Smalltalk.

The first one was on Tuesday evening and it featured several short talks by various people. I have to say that the highlight of that BOF was about the by-now-well-known XO laptop (formerly known as the OLPC laptop). We even had two demo units available for us to try out if we wanted. The XO laptop is a marvelously piece of technology for its price. Undeniably, it is built to withstand all the harsh conditions that it might suffer through. However, as mentioned by the presenters the biggest roadblock for this project might not be technology but the lack of actual educational contents to put onto the device itself. Right now, there are no official educational materials that will be installed onto the machines yet. And for a machine that is supposed to function as the all-in-one textbook replacement, this is a major predicament. In fact, it might be a serious blow toward the success of this system. And no one knows what is the best solution to this problem.

There were also two presentations on language tools for Squeak: OMeta and CAT. OMeta is a project by Alessandro Warth and is already available from SqueakMap. It's pretty compact and has some nice features for parsing languages. Alessandro has actually implemented a Javascript interpreter on top of it but the source is not yet available. CAT is another tool for language recognition by Jamie Douglas. It has more features compared to OMeta -- can support PEG and CFG, has better error messages, automatic AST generation and some AST rewriting. It's currently not released yet but from what I have seen during the BOF demo it is as good as ANTLR and I will definitely be looking into it. It might be fun and useful to create a simple IDE for CAT in Squeak like what AntlrWorks does.

At another BOF that I attended, Dan Ingalls did a presentation of the Lively Kernel project at Sun. Basically the project shows an implementation of a Squeak-like system in Javascript. Since Javascript is supposed to be cross-platform and runs on all modern web browsers, it is one of the best choices to implement this project in. And by taking advantage of the latest SVG features in new web browsers, one can actually create a lot of graphics without all the overhead of loading images. The demo for the Lively Kernel project is available from its website and runs fine on the Safari 3.0 and above.

The final BOF I attended was the Seaside BOF organized by Roger Whitney who is currently on sabbatical at UIUC. Unfortunately I arrived late for this BOF since I thought I have misplaced my keys and had to go to the lost-and-found office to check on that. I must have arrived more than halfway through the BOF since they were already going into the QA session. The attendees had very different backgrounds: there were some newbies like myself who have read about Seaside but have never used it and there were clearly some veterans who have worked extensively on Seaside. It would have been good if a quick walkthrough demo could have been done to introduce the newbies to Seaside development. Some attendees did raise some interesting questions. For instance, how many projects were using Seaside; what is needed to set it up quickly; which databases did it work with; what hosting options were available (besides hosting it yourself). Fortunately there is an up-to-date website, appropriately named seaside.st, that addresses those questions. I must say based on the short demo that web development with Seaside certainly has a different feel. It requires the developer to use all the tools from Squeak (or some other Smalltalk version). However, like Rails, once you have accepted the philosophy of Seaside, web development is certainly better than what it used to be.