Software Engineering Matters

I am planning a series of articles to help people get accustomed to new and better tools in Squeak. This is mostly written just for fun and hopefully it would be useful to students taking Ralph's object-oriented course with Squeak over summmer. Squeak is a very different way of programming when compared to other programming languages and IDE. So, while I am not hoping for some miraculous conversion, I do hope to make it easier for people to work more comfortably within Squeak after using some of the current state-of-the-art IDEs.

Many people hear that the code browser in Squeak (or any Smalltalk for that matter) is designed for code browsing. In other words, the browser is optimized to help you read code.

However, I have found that reading code in the regular Squeak environment can quickly become a big mess especially if you are trying something out for the first time and need to read multiple implementations in different classes. Usually, when this happens, you end up with a proliferation of browser windows that quickly clutter your screen. I am assuming that veteran Smalltalkers don't find this a problem because:

• they know most of the methods so well that there isn't much need to refer to it
• they have LARGE monitors so 20 open browsers is not a problem
• they are just disorganized and don't give a damn about it

In this article, I am going to show you how to use the OmniBrowser with some new enhancements. The same group that made the OmniBrowser has also created a whole bunch of very nifty tools. I'll try to get to each one in separate articles. But before that, let me divert your attention to this very relevant message:

A Squeak Smalltalk Development Example:

"Most developers who use Squeak would have a plethora of extra tools and utilities installed that make developing a much nicer experience than what you see in this tutorial. Do yourself a favor and start your Squeaking with a real developer’s image loaded with all the proper goodies like the SqueakDev image maintained by Damien Cassou."

So at this moment, I am going to insist that you download one of the excellent images maintained by Damien over at his webpage. Don't even bother using the basic squeak image. The basic image is pretty disheartening and discouraging to any new developer who is trying Squeak for the first time. Since Squeakers are usually developers, I have always found it mind-boggling that the standard Squeak image does not include better support for developers out-of-the-box. Instead, it is cluttered with a lot of stuff that most developers don't care about.

For this tutorial I am using the squeak-web/beta version from here. Don't forget to grab the SqueakV39.sources file from here as well.

Here's our scenario: we want to see how to actually implement the relevant methods so that two objects can be compared to one another for sorting purposes. Here's a step-by-step guide to how I would use the OmniBrowser + enhancements to do it.

Open the SmartGroup Browser
Bring up the World Menu in Squeak and select the SmartGroup Browser.

Type in the '<' selector
In the top pane of the SmartGroup Browser type in '<'. The top pane is called the Mercury Pane and it lets you type in class/selector names and quickly displays it in the current browser for you. For more information about the Mercury Pane see this.

New window showing the implementors of '<'
A new windows pops-up showing the implementors. I think it is OK to have a new window pop-up with the search results. Also notice that this actually shows the classes in a hierarchical manner (by tabs). So you can see that Character is a subclass of Magnitude.

Search for 'Float'
Now back in the SmartGroup Browser, I am going to search for the Float class since I am interested in seeing the definition of the method for it. I am going to enter 'Float' in the Mercury Pane.

Split Panes in the Browser!
By doing a Shift+Click on a class/method you split a new pane with it!

Multiple Split Panes showing all the methods
By repeating the Shift+Click process, we can create as many split panes as we want. Usually it's enough to have about 4 of them. Also, don't forget that you can browse back using the '<<' button.

For comparison, this is what it would look like if we had to use the normal tools from the standard Squeak image. Look at the wasted screen estate and how I don't see any nice hierarchical information between the classes.

This is roughly what it would look like to find the '<' method in Squeak 3.10 without the new tools. We start by trying to look for the '<' selector in the Method Finder and then clicking on each method to see its definition. Notice that there can be no end to the list of new windows!

So I claim that by using the new tools, reading code in Squeak is much more organized. Good organization not only helps the developer focus but also makes it more approachable (and less intimidating) for a new user. Personally, I advocate that any new tutorial about Squeak be written with reference to these new tools. These new tools are hidden gems that need to be exposed to the world! Only by getting more people to use them can we:

• improve existing tools based on user feedback and experience
• actually get new tools to appear for Squeak - after all what's the motivation of creating new tools if everyone is just using the old way?
• change people's perception that Squeak's tools are 20-years outdated compared to more polished tools such as Eclipse
• make it easier for first-time developers to use Squeak. Seaside has already done a fantastic job garnering attention for Squeak and we should take this opportunity to make it easier for developers to get started

I also hope that people will be less close-minded about enhancing Smalltalk as evident from some of the comments in the following posts:

• Extending Smalltalk Syntax
• How Should We View Methods?

Of course, changes should be implemented judiciously but without an open-minded attitude toward change, nothing will change.

The podcast of this special event in OOPSLA is actually available from the OOPSLA website. As my friend Munawar puts it, you really have to be there to experience it -- it's almost like a religious experience to hear Fred Brooks speak.

Fred Brooks, the author of the seminal book The Mythical Man-Month and the famous essay No Silver Bullet was invited to give a talk at OOPSLA 07. His Collaboration and Telecollaboration in Design talk addresses the issues that affect designs done in teams. Currently, the designs of complex systems are usually done collaboratively in teams because the design requires more effort and knowledge than what a single architecture can muster. However, design by teams present a new challenge: how do we get great designs when there are so many contributors? After all, it's pretty much a truism that too many cooks spoil the broth.

The key to achieving great design is maintaining conceptual integrity. Conceptual integrity is achieved when the system exhibits a consistent and unifying theme: everything seems to fit together seamlessly. Conceptual integrity is the presence of normalcy and the absence of radical designs in different parts of the system. A possible example of conceptual integrity would be the drag-and-drop metaphor found in most operating systems (done more right in some than others). If the system was design with conceptual integrity then drag-and-drop should just work everywhere. For instance, opening a file would just involve dragging it to the appropriate application; deleting a file would involve dragging it to the trash. In short, conceptual integrity (for the user and developers alike) creates an easy-to-use system with little surprises.

Ideally, conceptual integrity is easy to maintain when there is only one chief architect who oversees everything. This chief architect plays the important role of seeing how the pieces fit together. He might not understand all the details but he certainly knows at a high level how everything works. How would one achieve this in a team? It does not really seem possible to have multiple chief architects (in fact, that sentence seems grammatically wrong since chief already refers to a single person). There could be multiple architects but there is one chief architect who has the authority to make the final decisions. Even in an open source project like Linux, Linus Torvalds still has the ultimate veto power on what to put in and what not to put in. This almost seems like a dictatorship but if that is what is required for conceptual integrity then it is something that has to be done.

While collaboration should definitely be done, the need to actually appoint someone with the authority for making the final decisions should not be overlooked. According to Brooks, collaborative efforts such as pair-programming actually works to get ideas and design decisions out quickly. The important thing to realize is that certain phases of design requires decisions by an individual.

On the other hand, Brooks does not actually think that telecollaboration with all the fancy bells and whistles actually work. The phrase he used to describe this is profound: [telecollaboration sytems are] technologically-pushed rather than application-pulled. So most of the tools out there are just loaded with technology that no one needs to use. In fact, there is no need for fancy teleconferencing equipment: what is needed is a way to share the documents (via e-mail, source control, real-time editing, etc) and voice conferencing. Video conferencing does not bring any additional benefit.

The shared document in the upper panel is useful; the other panels not so much. Image taken from an article on TheGlobeAndMail

Brook's thoughts agree well with what the two second-life developers said during a previous talk. The developers of Second-Life are located all over the world and they need to use telecollaboration as well. They use an application such as SubEthaEdit for real-time code editing. And they use the new 3D voice technology in Second Life to talk to one another while their avatars are positioned in a room inside Second Life itself. They do not use any form of video conferencing. This setup has been found to be effective for their development task and they have been using it for some time.

One final issue that Brooks addresses in his talk is the perceived malleability of software. Non-programmers have the notion that software is very easy to change - it's easy to add a feature, remove a feature or modify the feature. The software marketing industry is partly to blame for this perception since the marketing team is always touting that the latest version of application X and how it has 100+ features and tons of bug fixes. And they do these kinds of promotions annually. Is it any wonder that the masses just think that software is so easy to change?

In my opinion, this mindset causes many problems for developers. Requirements in software tend to change a lot more than any other engineering project especially once it has started. No one would change the design of a bridge halfway especially after all the foundation has been done. And yet, even after the architecture of software has been decided, it is not impossible to imagine the pointy-haired boss coming up to the developers and demanding a radical change. And this is why, agile methods have been so pivotal in the development of modern software systems.

Maybe people would think differently if they had something more tangible such as the printed source code to relate the efforts of software development. After all, it's easy to see how much of the bridge has been completed based on the visible structures. It makes me wonder if there are other areas which suffer from perceived malleability. I think fiction books suffer from the same syndrome since most editors might think it is very easy to rework the story line. However, this is not always true since the astute reader can actually find inconsistencies between the lines when the author changes the story line halfway through the book.

Fred Brooks is undoubtedly a very smart man. And his arguments, as shown in both the The Mythical Man-month and No Silver Bullet, hold true for many decades. In fact, it is unlikely that anything would come to contradict his arguments. After all, it's hard to challenge something that was written based on his observations on the projects that he was involved in. So when he says something about collaboration and telecollaboration, it's advisable for people like us to actually pay attention.

SeemleCode by Semmle is a tool for for Java and XML code querying. It features its own query language, unsurprisingly called, QL for Query Language and a set of UI options for displaying the results of the queries. Currently, the results could be displayed in a table or in graph form (pie chart, bar chart, etc).

from Class c
where c.getPackage().getName().matches("org.jhotdraw%")
select c.getPackage(), c

The above code snipper has been taken from their example page. This query finds all classes in the package org.jhotdraw and its subpackages. So the language is like SQL except that it structured as from... where... select... instead of the more ubiquitous select... from... where.... According to the developers, this ordering makes it easier to enable context-sensitive content assist as the user is typing.

What I like about this system is the fact that the query language can be used to do a lot of useful things. The example script that comes preloaded with SemmleCode has some example metrics that you could run on you system. Here is an example that measures the Depth of Inheritance Tree.

from MetricRefType t, int d
where t.fromSource() and d = t.getInheritanceDepth() and d > 6
select t, d order by d desc

The code snippet above reports reference types whose maximum distance to Object is greater than 6, in decreasing order of distance.

Moreover, given a complex enough query you could even use it as a form of code lint to check for inconsistencies in the code. So this system is extensible and could be used as a tool to query and inspect the source code in a non-invasive manner by the programmer.

SemmleCode comes with an in-memory database to store all the meta-information for querying. However, using this in-memory database for larger projects is really not recommended. Instead you should either install PostgreSQL or Microsoft SQL Server. Since I was on a Mac, I installed PostgreSQL and got it to work. The installation took only an hour or so (mostly because the internet connection was flaky) and PostgreSQL was easily installed on the Mac following the instructions that came with the download.

One caveat though and I discovered this after showing my installation to one of the developers present at the demo: SemmleCode operates on working sets in Eclipse. Unfortunately, those working sets need to be defined as Java Working Sets and not Resource Working Sets. SemmleCode will not recognize the Java files in a Resource Working Set since it is looking for the Java nature. After fixing this, SemmleCode works just fine. I ran it on the Photran source code and it was able to complete the analysis rather quickly.

So far SemmleCode is free to download and use. I suspect that a paid version would include a prepackaged library that has more sophisticated queries that developers might find useful.

I spent the morning in the Educators' Symposium at OOPSLA. It was an interesting session but unfortunately I could only stay for 3.5 activities since I wanted to go for the Dynamic Language Symposium in the afternoon.

We started off with a short session on how to teach recursion to incoming freshmen by Michael Goldwasser from St. Louis University. He had an interesting way for teaching recursion to first year students: he would pick a group of volunteers who would act as nodes in a list that is defined recursively. He would set it up so that each node knew which was the next list (null being the next node for the tail of the list). He would then act as the main program and call out a function like count() which returns the length of the list. His call of the function was symbolized by throwing a real tennis ball. This tennis ball was slit open and contained a piece of paper that was supposed to show the activation record of the current method call. In the case of count() each volunteer would then throw the ball to the next node and await a reply. This is a great way to illustrate how recursion works. However, I was a bit skeptical whether this activity might distract the students from the main concept of recursion. I was also curious whether this method of teaching recursion had permanence in the students's minds. Lastly, I wonder how well this method would scale up to a class that has 100+ students. Either way this was a nice method and it did give me some ideas for things to try on my students next time.

Next we had a session on Scrum. This was my first introduction to Scrum and it was pretty intersting -- it's a high level process that emphasizes customer needs and iterative design but it is not focused entirely on software development. So unlike XP, it is not explicitly required in Scrum to do pair programming or test first development. However, most software developers who use Scrum will augment it with those two practices anyway. It was a fun hands-on introduction to Scrum but again unless there is a decent reflection session after the activity I am curious whether the students will actually understand what Scrum is all about.

As part of an activity at the Educators' Symposium we had to create a zoo shadow box as part of our Scrum training. That is what our shadow box looked like. You have to use some imagination since it is pretty abstract.

After the break, Kathy Sierra gave a nice presentation on how educators can create more motivating materials for their students. There were some nice gimmick tricks such as using bright colored slides, using seductive images, using animation, etc. These are definitely some ideas that I have to remind myself to use next time. Kathy is an engaging speaker and she managed to capture the audience's attention easily. I think she was the perfect person to give the keynote for the Educators' Symposium. Kathy also had a list of books that she thought was entertaining and enriching to read as educators. Here is a partial list of those that I wrote down:

• Flow: The Psychology of Optimal Experience
• A Smile in the Mind
• What the Best College Teachers Do

Before I had to leave, there was a presentation on Extreme Learning by two professors from the University of Utah. Basically it is a form of XP but in the process of developing the software, you had to actually put down goals to learn something. So you are teaching yourself new things as you implement the project. Again, this seemed rather intuitive to me: every time you do any major development you are bound to have to dedicate some time learning something new. The only difference in this case is that the student has to make it part of the user story to learn a new technology. This is definitely something that would be a good approach in a graduate level class since graduate students are often more mature and know how to organize their time to actually learn something new.

The Educators' Symposium is definitely fun and interesting. I wished I could have stayed for more of the sessions but the Dynamic Language Symposium was too compelling to miss.

BOFs or Birds of a Feather Sessions allow people with common interests to gather and just talk about anything and/or everything about their common interest! This time I went for three BOFs, all of them focused around Squeak and Smalltalk.

The first one was on Tuesday evening and it featured several short talks by various people. I have to say that the highlight of that BOF was about the by-now-well-known XO laptop (formerly known as the OLPC laptop). We even had two demo units available for us to try out if we wanted. The XO laptop is a marvelously piece of technology for its price. Undeniably, it is built to withstand all the harsh conditions that it might suffer through. However, as mentioned by the presenters the biggest roadblock for this project might not be technology but the lack of actual educational contents to put onto the device itself. Right now, there are no official educational materials that will be installed onto the machines yet. And for a machine that is supposed to function as the all-in-one textbook replacement, this is a major predicament. In fact, it might be a serious blow toward the success of this system. And no one knows what is the best solution to this problem.

There were also two presentations on language tools for Squeak: OMeta and CAT. OMeta is a project by Alessandro Warth and is already available from SqueakMap. It's pretty compact and has some nice features for parsing languages. Alessandro has actually implemented a Javascript interpreter on top of it but the source is not yet available. CAT is another tool for language recognition by Jamie Douglas. It has more features compared to OMeta -- can support PEG and CFG, has better error messages, automatic AST generation and some AST rewriting. It's currently not released yet but from what I have seen during the BOF demo it is as good as ANTLR and I will definitely be looking into it. It might be fun and useful to create a simple IDE for CAT in Squeak like what AntlrWorks does.

At another BOF that I attended, Dan Ingalls did a presentation of the Lively Kernel project at Sun. Basically the project shows an implementation of a Squeak-like system in Javascript. Since Javascript is supposed to be cross-platform and runs on all modern web browsers, it is one of the best choices to implement this project in. And by taking advantage of the latest SVG features in new web browsers, one can actually create a lot of graphics without all the overhead of loading images. The demo for the Lively Kernel project is available from its website and runs fine on the Safari 3.0 and above.

The final BOF I attended was the Seaside BOF organized by Roger Whitney who is currently on sabbatical at UIUC. Unfortunately I arrived late for this BOF since I thought I have misplaced my keys and had to go to the lost-and-found office to check on that. I must have arrived more than halfway through the BOF since they were already going into the QA session. The attendees had very different backgrounds: there were some newbies like myself who have read about Seaside but have never used it and there were clearly some veterans who have worked extensively on Seaside. It would have been good if a quick walkthrough demo could have been done to introduce the newbies to Seaside development. Some attendees did raise some interesting questions. For instance, how many projects were using Seaside; what is needed to set it up quickly; which databases did it work with; what hosting options were available (besides hosting it yourself). Fortunately there is an up-to-date website, appropriately named seaside.st, that addresses those questions. I must say based on the short demo that web development with Seaside certainly has a different feel. It requires the developer to use all the tools from Squeak (or some other Smalltalk version). However, like Rails, once you have accepted the philosophy of Seaside, web development is certainly better than what it used to be.

The Dynamic Languages Symposium had plenty of mind-stimulating papers and some not-so-stimulating ones. There were a total of nine papers but I missed the three that were presented in the morning since I was at the Educators' Symposium.

One of the presentations that I liked was RPython: Reconciling Dynamically and Statically Typed OO Languages. It was an easy to understand paper and it showed the efforts that are being made to write a Python compiler in Python itself! So RPython is a Restricted Python; it has most of the dynamic features of Python taken out so that is compiles properly to a more static environment like the JVM or the CLR. But you do not lose much by this though. You would use RPython to code the interpreter for Python itself so you will not actually write normal programs with RPython. By restricting the features of Python that you could use, you could potentially make a better interpreter. RPython is a component in the PyPy project.

The second paper that I liked was Highly Dynamic Behaviour Adaptability through Prototypes with Subjective Multimethods. The title sounds complicated but the main concept is simple. The premise for a system like this would be like a system of multiple interconnected devices. So when device A, B and C are present then device D would perform a certain action for method1(). And when only devide B and C are present then D would performa a different action for method1(). This was implemented using multimethods -- each object's type determines what method gets invoked. It's like method overloading in the dynamic sense.

There are other papers that I liked but those two were the most blog-worhty ones.

Lastly, there was a presentation from Jim Hugunin on implementing dynamic languages on .NET. I am generally, as are most other people as well, skeptical about Microsoft's efforts with languages. They have often created a proprietary version of languages such as J# (obviously a bastardized copy of Java) that no one even bothers about. However, with the CLR and the implementation of the Dynamic Language Runtime (DLR) on .NET, Microsoft seems committed to making .NET a better platform for developing languages in comparison to the JVM. For an unofficial count, check out the list of languages for the JVM and CLR. What makes the CLR rather compelling at the moment is the fact that making your language run on the CLR enables you to hook your program to some of the cool Microsoft technology out there: XNA and Silverlight being two of them. Now, if only Microsoft would endorse the Mono project then I would be more compelled to start using the CLR for my projects.

The next few posts will be about OOPSLA 2007 and the title of the articles will be appended with "OOPSLA 07". Those articles will be written in a more colloquial manner since they will be mostly based on my personal experiences there.

I was fortunate enough to have the opportunity to go to OOPSLA this year. I say fortunate because I had to apply for a Canadian visa; and based on past experiences it can either be a painless ordeal or one where you are unsure what you have done to not deserve one. Anyway, all went well, and after 15 hours of driving and 12 hours of recuperative sleep -- I was not able to sleep in the van -- I attended the first day of OOPSLA without any problems.

The first day was relatively easy-going since I was not signed up for any of the symposiums and could freely choose which two tutorials (or none) that I wanted to attend. I chose the tutorials on Software Architecture by Michael Stal of Pattern-Oriented Software Architecture fame.

The first tutorial was during the morning session and was entitled High Quality Software Architecture. It was an interesting tutorial and I think I was the youngest person in the room. The tell-tale signs of balding heads indicated that most of the other attendees were professional software architects that were here to learn the secrets of the trade from Stal himself. Anyway, the first part of the talk was essentially a nice review of the material that I had read about a year ago when I took the Advanced Software Engineering class. In fact, I wrote articles on the books and papers that we read for that class here in this blog. Stal was using the same terms an definitions from the Software Architecture in Practice book. It was interesting to actually see how someone else who interpret that book since it was a rather academic book and it was hard to actually find someone else to discuss about it.

Stal emphasized three main issues that was only lightly touched upon in the book. And I agree with him that these three topics will help promote higher quality software architecture:

• Clear architecture vision: This is similar to the metaphor in XP or the ubiquitous language in Domain Driven Design. Essentially it means that the architecture should be driven by a clear idea of what you are trying to achieve. And the architecture should strive to maintain this goal in every aspect possible.
• Iterative Development: This is really important. Developers usually think of architecture as something that must always be done up-front. There is some truth to this since if you get the architecture (vs the design; see this for an explanation on the differences) wrong then there is a heavy price to pay in terms of development efforts. Stal solves this by introducing the concept of the baseline. The baseline is the core of the architecture that needs to be done right and will not change throughout the entire development cycle. The baseline is determined from a set of prioritized requirements from the stakeholders. Only the most important requirements shape the architecture of the baseline; the remaining requirements will be acted on later in an incremental manner. This balance on what to add first and what to add later makes sense: unlike design, software architecture will definitely permeate the entire system so it is essential to at least get the important aspects right the first time.
• Testing: It is necessary to be able to test that your system supports certain requirements. It is usually commonplace now to have automated tests for most of the functional requirements of the system. Stal claims that it is necessary to have tests as well for the system architecture. These tests need not be automated fully of course but they should exist nonetheless. Either doing it through a code review or a formal inspection would reveal missing requirements in the architecture. I talked to Stal about using executable Architectural Description Languages(ADL) to verify the architecture and he says that most of the time that approach is not feasible since it requires a domain expert on the ADL itself reaffirming what I thought about ADLs in the first place.

The afternoon session was on Refactoring Software Architecture. Stal admitted that what he was going to present was based on preliminary work but nonetheless he felt that it was essential enough to warrant a tutorial. I completely agree with him especially since he too mentioned the Grand Unified Theory of Refactoring at the end of his talk and if we have any chance of finding one we had to start thinking of a lot more about software in terms on refactoring and program transformation. Like Fowler, Stal motivated the indicators of architecture refactoring in terms of architecture smells or what he terms the lack of the following qualities:

• Pattern Density
• Symmetry
• Expressiveness
• Simplicity
• Orthogonality
• Emergent behavior

The list of refactorings that he presented were based in the spirit of the original refactorings by Fowler: he used a pattern language to describe them. When I first saw the first few refactorings that he presented I was confused by the scope of it: were they architecture refactorings or design refactorings? And this is where the distinctions in the paper Architecture, Design and Implementation might be useful. When I asked Stal, he explained that he treats refactorings that could have a percolating impact to be an architectural refactoring. So when you refactor something and it affects things below it then it is probably an architectural refactoring pattern. There is still this nagging feeling inside me that thinks that architectural refactorings might not be behavior preserving -- the quintessential definition of what a refactoring is -- but I just cannot come up with an example of this. So I could be wrong about this.

Stal has a list of roughly 30 refactorings. But the two that most interested me were Aspectize and Integrate DSLs. The former being that you might need to address some cross-cutting concern and use an aspect-oriented approach to solve it. This is actually the few times that I have seen someone promote aspects in an appropriate manner. Usually people are vehemently for or against aspect-oriented programming. Stal however, presented a clear case where aspects can actually be a good approach. The second pattern is all about not being reserved on using a Domain Specific Language for your architecture if you need one. Again, I think that the time for thinking in more than one language has come and this is one way to make good use of that approach. Software architecture can be a complex beast and while it might be usually over-kill to use AOP and DSLs for normal projects, architecture could be the key area where they should both have relevant roles.

Throughout both these presentations Stal mentions the need for iterative design and refactoring. He is clearly a strong supporter of refactoring. He realizes that all future software development can only be successful with decent and continuous refactorings. And he is far-sighted enough to see how refactoring can be applied not only to code but also other artifacts like architecture.

I was talking to Maurice and we both agree how our advisor, Ralph Johnson, could have made one of the greatest contributions to software engineering by introducing the concept of refactoring. Sure he has co-authored the book on Design Patterns but the main concept of refactoring is probably more influential. Thinking of software engineering in terms of refactoring and transformations is definitely how all future software should be developed.

There were a ton of demos going on at OOPSLA this year but I was only able to attend three of them. The three that I chose were all focused on Java or Eclipse development since I was really interested in finding out what we could use to make Eclipse a better environment for tool development.

Jazz

I did not go for the original demo of Jazz but rather stopped by the booth the next day to talk to some of the developers there. From what I heard, the original demo has some technical problems; they were doing a demo of two development team -- one team at OOPSLA and the other team in another part of the world -- and someone accidentally tripped on the ethernet cable and the demo could not go on after that.

Jazz is an all-in-one collaboration tool for Eclipse. It integrates version control, bug-tracking, task-management and even wiki editing! It could potentially do lots more but those were the features that I saw and which I was most interested in. In order to provide a better user experience the creators of Jazz decided to recreate some of the underlying features from scratch instead of relying on existing technologies that users have grown accustomed to. For instance, instead of relying on CVS or SVN, Jazz has its own version control system. When I first heard this I was rather doubtful of how necessary it would be since most developers are already using some system like CVS and might be reluctant to migrate over to something new. However, this issue is resolved by having an importer that takes the current CVS repository and converts it into a Jazz repository. I was at the booth during this demo and the person there actually imported the source code from CVS for the Photran project that we are working on and converted it into a Jazz project. The conversion retained the meta-information from CVS and this would ease the transition for existing projects.

The goal of Jazz is for all its features to work together seamlessly in a consistent manner. To fulfill this goal, the developers deemed that wrapping a myriad of tools such as CVS/SVN, Bugzilla, etc would not provide the best user experience. This might be true since tight-integration is one way to ensure that everything is working together without having to spend countless hours configuring all the tools to work together. Jazz comes with its own fully packaged server that contains all the tools that you would need to get started immediately.

The approach Jazz has taken is different from Mylyn, another popular Eclipse plug-in for collaborative development:

Mylyn FAQ - Eclipsepedia:

"At the EclipseCon and JavaONE 2006 conferences IBM demonstrated previews of Jazz, a collaborative team server and Eclipse-based client. Articles have remarked on the similarities between Mylyn and Jazz because both integrate tasks into Eclipse (Jazz's 'work items' and Mylyn's 'tasks'), and both provide change sets grouped by task. But there are both significant differences and complementary aspects to the two approaches. A key goal of Mylyn is to provide an open source framework to support integration of task management with existing issue trackers and source repositories. According to the presentations, components that come with Jazz include include a next-generation issue tracker and source repository and related lifecycle management tools such as project health. In addition, a driving and unique goal of Mylyn is to focus the UI around a usage-based and degree-of-interested weighted task context, which is complementary to the Jazz platform. Just as it integrates with Mylyn's Task List, Mylyn's Task Focused UI and task context model components are possible to integrate with other kinds of task management systems, such as the one provided by Jazz."

I have only begun using Mylyn recently and have been impressed by what I have seen so far. Our university Computer Science department has an existing JIRA server that we are using for our development needs. Mylyn connects seamlessly to it and works fine. It is this ability to interoperate with existing technology and tools that is the strongest advantage about Mylyn. Though I would really like to use Jazz it is not really practical to convince the technology services group to install a dedicated server for it when they are already using a bunch of existing tools that work well. I suspect the same situation occurs in existing software development companies as well.

The Jazz developers have already done a good job at reducing the initial risk of switching over to it by providing the importers/adapter that convert existing data into a format that Jazz can work with. If the Jazz developers could convince the Eclipse foundation to adopt Jazz instead of using their current tools, then it would make a big impact in convincing other projects to follow suit. Until then, most developers with existing projects might be less inclined to use it.

This post is based on the paper Problem Frames and Software Engineering by Michael Jackson. This is my first exposure to the notion of problem frames so I apologize for any short-sightedness.

In this paper, Jackson addresses the issue of why software engineering (SE) does not generally fit into the notion that one would have of other engineering disciplines such as aeronautical engineering (AE)-- which he uses as his primary example in the paper. While AE has enjoyed a disciplined approach to building planes and other equipment for several years since its inception, SE has yet to discover its own set of disciplined approach. The emphasis here is on the term disciplined approach. While there are certainly many different ways to solve a problem, most engineers will stick with solutions that are commonly employed usually because they are almost guaranteed to work.

There is certainly no shortage of effort to present SE in light of the more traditional engineering disciplines. The existence of formal methods and other specialized math that has been employed in SE is a testimony to that. The specialized math present a means for modeling complex problems in SE. However, even with these rigorous math models, SE has failed to gain the distinction and recognition that other fields of engineering share.

In fact, Jackson identifies the current modeling for SE as a form of detachment from the real world. Jackson asserts that most of the modeling that has been done is influenced by Djikstra's view that computer science should maintains its mathematical purity and distance itself from the pleasantness problem -- whether or not the model conforms to real world requirements. This detachment from the real world is the root cause for the discrepancies that SE has with other engineering disciplines.

To bring SE closer to the realm of traditional engineering, Jackson proposes his theory of problems frames: viewing the problems that SE needs to solve in the context of the real world. While Djikstra eschewed the need for relating software to the real world, Jackson's problem frames theory promotes the real world into a first class component that needs to be considered as part of the model.

Generalized Problem Frame Diagram

Using the problem frames approach, software engineers can strive to model physical objects in the real world as part of the system that has to be considered. After all, real engineers always model the real world when solving the problems. To use the equations for fluid dynamics, an aeronautical engineer needs to set up the appropriate constraints - for instance the acceleration due to gravity, the viscosity of the fluid, etc. The equations for fluid dynamics can stand alone with all their symbols but unless one supplies the real world constraints, those equations remain nothing more than equations -- certainly a marvel of mathematics but useless nonetheless to the real world.

The astute reader will recognize that lots of problems SE might not have a direct correspondence with the real world. For instance, in a compiler, there is no direct real world correspondence. Jackson acknowledges that his problem frames approach would require some modification to be used under those context. Nonetheless, it is undeniable that most SE projects involve some identification with real world objects and thus the problem frames approach might be useful.

The greatest contribution of problem frames is the fact that it is not just another tool for modeling. In fact, I would categorize it as a tool for synthesizing and distilling the real problem that needs to be solved. After applying the problems frames technique of modeling, certain patterns begin to surface. These patterns are useful to help software engineers solve recurring problems.

As an analogy, one seldom solves a problem in its natural context since that would require too many unknown variables. As a greatly simplistic example, consider the problem of counting how many apples one has after frequenting the grocery store. After kindergarten, most of us would be able to just use numbers to represent the apples without actually thinking about the apples themselves. For instance, if one had 3 apples before and bought another 3 later, then one would end up with 3 + 3 apples. We have reduced the problem of counting apples to the problem of adding two numbers together. The ability to add numbers together is more important than being able to count apples since counting is a skill that can be applied to numerous other domains.

Similarly by using the problem frames approach, one is able to distill certain recurring patterns of modeling. Effectively, if one is able to solve one of those patterns, then the same solution can be applied -- with some modifications of course -- to other problems. Jackson himself acknowledges that this is similar to the notion of Software Patterns popularized by the GoF Design Patterns book. However, unlike software patterns, problem frames are language and system agnostic; their solutions should be applicable to different styles of software development.

In engineering, there are two forms of design: normal design and radical design. Normal design is achieved using a set established best practices. For instance, a civil engineer never needs to start from square one when he is designing a new bridge. Instead he follows known conventions for designing a bridge. In fact, the task of designing the bridge can easily be decomposed into several different smaller projects that other engineers can work on simultaneously. Indeed, it is this set of established best practices that enable normal design to occur.

While SE has definitely developed some form of normal design using some best practices -- using design patterns, choosing some framework to prevent reinventing the wheel, and even choosing the architecture to run the system on -- most of the time, some form of radical design is going on behind the scenes. A radical design is a design that is not based on any established set of rules. In fact, a radical design might be just be an ad hoc solution that is never to be seen again.

In SE, the number of radical designs dwarves the number of normal designs. This makes it awkward to call SE an engineering discipline. There are just too many ways to skin a cat. And this abundance of solutions itself is a challenge to this field. The lack of the best practice principles makes it hard for a new software developer to quickly develop a new system to solve similar problems.

In short, this paper views the abundance of radical design versus normal design to be the problem of SE. This overabundance is an indication that the field of SE has yet to mature enough that known solutions to real problems have manifested themselves. Problem frames is proposed as one way to distill the dormant knowledge that permeates those problems and present them in a way that a software developer can reason about.

In my opinion, it might not be possible to find a unifying method for software development. In many other engineering fields there are real physical constraints that must be obeyed for the system to work. There is nothing much you can do differently for a car since it must move along some road system. In SE, it is different. In software, you can do a lot of things differently unbounded by the physical rules. As an example, consider virtual reality games. Such systems are huge and each game tries to model the world closely. But there is nothing to prevent the virtual world to behave radically different. People could walk the other round and the game would still function correctly. In fact, other than the term virtual reality, these games have nothing much to do with reality since anything and everything can happen.

This freedom in computer science to model things differently is what makes it interesting for many people. In software, a lot of solutions are only limited by what creativity the designer can come up with. True, this leads to a whole hodgepodge of different solutions but this also makes this area of software more challenging. As another example, consider developing software for civil engineers. The software developer not only needs to know the domain knowledge -- how civil engineers reason about their tools and how to actually do the calculations -- but also how to write the software itself. To be useful, SE has to be cross-disciplined. Useful programs for other engineering disciplines can actually be done with software. Whereas I find it hard to imagine using some of the ideas from civil engineering to biomedical engineering.

So while problem frames is one way to solve certain problems, it is not going to radically transform the world of SE as we know it. Perhaps it is time to view software development as a different form of engineering -- one that is different from the other older disciplines. And if the word "software engineering" becomes a misnomer then it is only a matter of coming up with a more suitable name for it.

Squeak by Example:

"This book presents the various aspects of Squeak, starting with the basics, and proceeding to more advanced topics. This book will not teach you how to program. The reader should have some familiarity with programming languages. Some background with object-oriented programming would be helpful."

An updated book for Squeak is long overdue and Squeak By Example will fill that void. The book is still work in progress though a beta version of the pdf will be available soon. The Subversion repository of the LaTeX sources is available though and whoever is interested could generate the .pdf from that.

I find this book to be a positive step in introducing people to Squeak. I have commented before that a book is a physical manifestation of the quality and maturity of an open source project. If you dealing with a close source project then the company backing that project up could easily invest money into making documentation or even publishing a book in that technology. However, when dealing with an open source project -- where all documentation is freely available in the form of code as the proponents would claim -- the commitment to create a book shows just how much the community loves the product and want to get other people excited about it too.

A book shows commitment to the project. After all, why waste time writing a book if the project is going to die some time down the road. Now, this does not mean that the project will not die in the future. But it shows that the people behind it are committed enough that they don't foresee this project shutting down anytime soon. In short, the project has enough momentum to sustain itself for some time.

A book shows excitement for the project. And this book practically bubbles with it since it is co-authored by one of the most prominent figures in the Squeak Community, Stephen Ducasse. If a project leader is excited enough about the product to write a book then you can be certain that it's an exciting product.

A book shows the maturity of your project. After all, it's too risky to write a book if things are going to change. Maturity is a sign of stability -- meaning that this product is unlikely to undergo a capricious change in the near future. Thus, if your product depends on Squeak, it is probably not going to suddenly be incompatible in the future.

A book shows that other people are interested. Why write when you don't even have an audience? This is actually a catch-22 since you wouldn't write a book if there is no audience but you wouldn't really have an audience if you don't really have good documentation to inform them adequately in the first place.

In short, I think an updated book is a good sign that the Squeak community is committed and excited about how much quality and maturity Squeak has achieved. I will be looking forward to this book and hopefully be able to incorporate it in the Smalltalk class that I am teaching assistant for.