softeng2023-08-04T05:53:06+00:00http://softwareengineering.vazexqi.comNick ChenSoftware Engineering And Testing Research at UIUC2010-02-18T00:00:00+00:00http://softwareengineering.vazexqi.com/2010/02/18/software-engineering-and-testing-research-at-uiuc<p>
The Department of Computer Science at UIUC will be having its graduate recruitment weekend at the end of February and each research group is supposed to create a simple poster to showcase their ongoing research projects. In case anyone is interested in the projects that we do, I present to you, the Software Engineering & Testing research projects at UIUC....
</p>
<a href="http://www.flickr.com/photos/37092324@N00/4368111550" title="View 'Software Engineering & Testing Research Poster' on Flickr.com"><div style="text-align:center;"><img src="http://farm5.static.flickr.com/4022/4368111550_e147949f89.jpg" alt="Software Engineering & Testing Research Poster" border="0" width="500" height="375" /></div></a>
<p>
The full PDF is available from <a href="https://netfiles.uiuc.edu/nchen/softeng/SEPoster.pdf">here</a>. Sometimes it displays a bit funny on Windows/Linux machine and I am not really sure why (thought it's a PDF). If the color and text do not look like the picture above, you know that something is wrong.
</p>
<p>
The poster was based on the color scheme from the WWDC Poster Templates from <a href="http://www.apple.com/science/poster/WWDC_Poster_Templates.zip">Apple</a>.
</p>
How To Present2009-11-25T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/11/25/how-to-present<a href="http://www.flickr.com/photos/37092324@N00/4134438031" title="View 'How To Present' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2610/4134438031_697f7628fd.jpg" alt="How To Present" border="0" width="500" height="375" /></div></a>
<p>
I gave a talk on "How to Present" a couple of weeks ago as part of the Grad Academic Council at the University of Illinois. Here's the abstract:
</p>
<blockquote>
<em>Making The World A Better Place, One Presentation At A Time</em>
<br/>
<br/>
You've been to your fair share of presentations; some of them were wonderful, some were pedestrian and some were just awful. Sooner or later you will have to give your own presentation, hopefully one that you and your audience will feel good about. So, what can you do to give your best presentation each time?
<br/>
<br/>
In this session, I'll discuss general guidelines on what to do before, during and after the presentation. And I'll illustrate my points by giving real examples (sources anonymized, of course, to protect the innocent) of mistakes that all of us have made from time to time. I'll also focus on specific techniques for CS presentations because we are a different breed of presenters with all our graphs, source code and UML diagrams.
<br/>
<br/>
If you are going to be making a major presentation for the first time, stop by to pick up some new techniques. And if you are a seasoned presenter, stop by to see if I've made any faux pas in my own presentation.
</blockquote>
<p>
Here's the <a href="http://bit.ly/5jpxEe">PDF version</a> of the slides; and here's the <a href="http://bit.ly/7KfTPS">Keynote version</a> (you need to rename the file from to HowToPresent.zip to HowToPresent.key; if you use a browser that automatically unzips the file into a folder, just rename the folder from HowToPresent to HowToPresent.key). The slides are released under the <a href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 license</a>. The pictures on the slides are from <a href="http://www.istockphoto.com/">iStockphoto.com</a> and I ask that you don't copy-and-paste them from the slides but purchase you own copies of them to support the artists.
</p>
<p>
The presentation was recorded and you can watch it <a href="http://bit.ly/7arYtO">here</a>. The high-speed version shows me as I present (in addition to the slides) so you can see the gestures that I am making - my gestures help illustrate the points that I make.
</p>
<p>
I learned a lot from giving this presentation. And I definitely see plenty of ways to improve the next time I give it. The only way to get better at presenting is to practice by giving more presentations! And, of course, by listening to your audience. So, I value any feedback that you have. Please send them to me at nchen_AT_illinois.edu.
</p>
<hr/>
<small>
If you are watching the video, my presentation itself ended at around 40 minutes. After that I was taking questions but I forgot to repeat those questions into the microphone so you can't hear them in the video. I have summarized the important points for you below:
<ul>
<li>Roy Campbell mentioned how for certain job talks, the audience expect to see at least one equation on your slides. On my slides, I said that equations should be avoided unless really necessary. The case that Roy mentioned is an example of an exception. Again, it's all about what your audience expects. So do some investigation first on the expectation of the audience before preparing your slides.</li>
<br/>
<li>The person sitting beside Roy made a comment about how much you should have on the slides. In the extreme case, you really don't want to have anything – that way, everyone's attention would be on you. However, for most CS presentations, it really helps to have some diagram/text. It's much easier to describe a system with UML diagrams. Additionally, sometimes you want filler images on your slides – these actually pique your audience's interest from time to time so that they don't get too bored with what you are saying.</li>
<br/>
<li>Danny made a comment about making sure that the last slide that you leave on the screen is the conclusion slide which summarizes your ideas. You should leave this slide on the screen even if you are taking questions (as opposed to have a slide with the word "Questions?" on it). This gives anyone who comes in late the opportunity to at least see a summary of your work.</li>
<br/>
<li>Danny also made an interesting comment about how you should use the first slide to introduce yourself – introduce where you are from, what's your background, etc. This gives the audience the ability to build some rapport with you. And more importantly, it gives them the chance to familiarize themselves with your accent and style before going into the technical parts of the presentation.</li>
<br/>
<li>One person at the back made a very important comment about displaying graphs: you need to mention the units that you are using on the axes. This gives the audience an idea on what the data actually means.</li>
</ul>
</small>
Lessons From Parallelizing Matrix Multiplication2009-11-23T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/11/23/lessons-from-parallelizing-matrix-multiplication<p>
Before you dismiss this as yet another boring matrix multiplication example, consider that <a href="http://msdn.microsoft.com/en-us/magazine/cc163340.aspx">this</a>, <a href="http://developers.sun.com/solaris/articles/openmp.html">this</a>, <a href="http://blogs.msdn.com/pfxteam/archive/2007/11/29/6558413.aspx">this</a> and a lot of other tutorials on the web all use a <em>naive</em> version of matrix multiplication and then attempt to parallelize that version of it to demonstrate how to speed things up using some technology.
</p>
<p>
What do I mean by a naive version (since there are many naive ways of doing things depending on the system that we are talking about)? This is the basic C++ code for matrix multiplication used in most examples:
<pre>
void matmult(int m, int n, int p, double **A, double **B, double **C) {
for (int i = 0; i < m; i++) {
for (int j = 0; j < n; j++) {
for (int k = 0; k < p; k++) {
C[i][j] += A[i][k] * B[k][j];
}
}
}
}
</pre>
</p>
<p>
And this is the smarter version which shouldn't surprise anyone who has taken a computer architecture course:
<pre>
void matmult(int m, int n, int p, double **A, double **B, double **C) {
for (int i = 0; i < m; i++) {
for (int k = 0; k < p; k++) {
for (int j = 0; j < n; j++) {
C[i][j] += A[i][k] * B[k][j];
}
}
}
}
</pre>
</p>
<p>
What's the difference? The difference is pretty subtle so you might have missed it: the inner loops have been swapped i.e. the j and k iterations.
</p>
<p>
What can changing two lines do? After all, the results are the same right? Well, take a look at this output from the <a href="http://developer.apple.com/mac/library/documentation/DeveloperTools/Conceptual/SharkUserGuide/Introduction/Introduction.html">Shark</a> profiler tool in OS X.
</p>
<a href="http://www.flickr.com/photos/37092324@N00/4130310962" title="View 'Profiling Matrix Multiplication in Shark' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2526/4130310962_0972196b8a.jpg" alt="Profiling Matrix Multiplication in Shark" border="0" width="500" height="191" /></div></a>
<small>L2 Cache Misses Using Shark on OS X 10.5</small>
<p>
The version on the left uses the naive algorithm and has <strong>3 484 436</strong> L2 cache misses for a 1024 x 1024 matrix multiplication. The version on the right uses the smarter algorithm and has <strong>27 176</strong> L2 cache misses. On my Core Duo MacBook Pro, the naive version runs in <strong>52 133</strong> ms and the smarter algorithm runs in <strong>13 730</strong> ms. That's nearly a 4x speedup and we didn't even parallelize anything yet!
</p>
<p>
Hopefully that last line caught your attention. That's the result on my MacBook Pro with a fairly old two-core processor. Let's run some experiments on a 24-core machine (6 cores per socket) using Intel Xeon CPU E7450 @ 2.40GHz running Linux 2.6.18. The code was compiled using g++-4.2 with -O3.
</p>
<p>
The naive version runs in <strong>15 962</strong> ms. The smarter version runs in <strong>1 860</strong> ms. Let's try parallelizing the naive version with OpenMP:
<pre>
void matmult(int m, int n, int p, double **A, double **B, double **C) {
#pragma openmp parallel for
for (int i = 0; i < m; i++) {
for (int j = 0; j < n; j++) {
for (int k = 0; k < p; k++) {
C[i][j] += A[i][k] * B[k][j];
}
}
}
}
</pre>
</p>
<a href="http://www.flickr.com/photos/37092324@N00/4129473951" title="View 'Graph' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2623/4129473951_95f3c9915c.jpg" alt="Graph" border="0" width="500" height="270" /></div></a>
<small>Execution Time of Parallelized Matrix Multiplication.</small>
<p>
Notice that we need to use ~<strong>9</strong> threads to break even with the smarter version. Incidentally, if we use the smarter algorithm and parallelize it using 24 threads, it runs in <strong>151</strong> ms!
</p>
<p>
So what are the lessons from this?
<ul>
<li>Low level details matter for performance - I think it's important to mention those details when parallelizing. We needed quite a few threads (<strong>9</strong> threads) before you can break even with the smarter algorithm. And most machines today don't have that <em>many</em> cores yet. And as long as the future manycore machines continue to use hierarchical memory (L1, L2, L3 caches), such memory optimization techniques will always be useful. In fact such optimizations help improve performance even if you are running Java or .NET on a virtual machine. However, ...</li>
<li>Low level optimizations can get tricky - this example is simple since we are only swapping the iterations to reduce cache misses. But some other optimizations e.g. tiling require more invasive transformations which is why ...</li>
<li>Always advice your users to use well-tuned libraries whenever possible. Matrix multiplication is a well-known algorithm and there are plenty of better ways to do it (look at <a href="http://www.netlib.org/blas/">BLAS</a> and <a href="http://math-atlas.sourceforge.net/">ATLAS</a>). It's a disservice to give users the impression that they should write their own matrix multiplication routine when well-tuned ones are easily available. And that's why, we need to ...</li>
<li>Pick better and more interesting examples to parallelize. Matrix multiplication is embarrassingly parallel; the hard part is trying to wrestle with the memory hierarchy to ensure performance. For some other problems, it's hard to see what each thread should do and how to coordinate those threads. So instead, people should put in effort into documenting those more complicated parallelizing techniques into <a href="http://parlab.eecs.berkeley.edu/wiki/patterns/patterns">patterns for parallel programming</a>. That's the real test of how good your parallel library/framework/language is.</li>
</ul>
</p>
OOPSLA '09: Onward!2009-11-07T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/11/07/oopsla-09-onward<p>
<a href="http://onward-conference.org/">Onward!</a> this year was a bigger conference compared to previous years; it spanned the entire 5 days of OOPSLA. I had a hard time deciding whether to attend the research tracks at OOPSLA or the presentations at Onward! The research tracks at OOPSLA are dedicated to concrete ideas which have been implemented and evaluated; whereas the Onward! tracks are dedicated to newer, interesting and sometimes <em>out-of-this-world</em> ideas with prototypes and preliminary evaluation.
</p>
<p>
Pretty much everything was interesting at Onward! – yes, even the <a href="http://portal.acm.org/citation.cfm?id=1639950.1640069">Harmony-Oriented Programming</a> papers though I have yet to be <a href="http://softwareengineering.vazexqi.com/2009/01/03/a-series-of-unrelated-events">convinced</a> whether it's more of an deliberate (but interesting) juxtaposition of current programming ideas or something new and <em>useful</em>.
</p>
<p>
There are two papers that I found particularly relevant to my interests and I will summarize them here for you. Definitely take a look at the other <a href="http://onward-conference.org/papers">papers</a> too.
</p>
<h3><a href="http://www.oopsla.org/oopsla2009/program/onward/351-essays-1">Software Evolution and the Moving Picture Metaphor</a></h3>
<p>
When we think of software evolution, we usually think of something like the image below ...
</p>
<a href="http://hgbook.red-bean.com/read/a-tour-of-mercurial-merging-work.html#id353683" title="Merging streams of work"><div style="text-align:center;"><img src="http://hgbook.red-bean.com/read/figs/tour-merge-pull.png" alt="Merging in a typical version control system like Mercurial" border="0" /></div></a>
<br />
<p>
... where we have a timeline that shows the different commits to the version control system. However, if you think about it, each of those commits are merely snapshots of the software as it evolves. What happens <em>in between</em> those snapshots is <strong>lost</strong>. We don't have an idea of the debugging sessions, the testing sessions and the trial-and-error sessions that a programmer goes through before committing the code.
</p>
<p>
Viewing the evolution of software through those snapshots is like watching a movie except that we only get to glimpse at snapshots of the movie at certain points. For the most part, we can definitely get a good idea of the story behind the movie – though we might miss some of the more subtle points.
</p>
<p>
It's the same with software evolution. Each commit in the version control system tells part of the story. And sometimes that might be enough for what we need since we only care about interesting chunks of software development. But sometimes we care a bit more about the things that happen between snapshots. Using our movie metaphor again, you can think of this as viewing a movie with the director's commentary.
</p>
<p>
I don't think anyone has explored the idea of capturing software evolution as a movie (instead of snapshots). There has been some <a href="http://vis.cs.ucdavis.edu/~ogawa/codeswarm/">work</a> on visualizing software evolution but no one has tried to capture software evolution itself as a movie. People have tried recording entire development sessions – how people design the architecture on whiteboards, how requirements are gathered, how they communicate with one another, etc. For instance, there is the <a href="https://teambruegge.informatik.tu-muenchen.de/groups/dolli2/">DOLLI</a> project that filmed the entire software development process of a group of university students.
</p>
<p>
So if we were to create a movie of software development, what would it look like? Here are some interesting questions for an idea like this:
<ul>
<li>What to capture? - Do we capture only the code in the IDE, the debugging sessions, the testing sessions, etc? Or we capture more context like what <a href="http://tasktop.com/">Tasktop</a> is doing?</li>
<li>How much to capture? - Capturing too much information will lead to very boring movies very quickly. We need to find out what's really important to the programmer and filter out everything else.</li>
<li>How would we do this non-intrusively? - The programmer just wants to continue programming so we would need to capture this information without affecting how she programs.</li>
<li>How to replay this information? - It will be very boring to play it from the beginning to the end. How would we edit the movie to make it interesting and useful?</li>
<li>Would something like this be useful? - Who would be interesting in using it? The programmer herself when she is reflecting on her program? Or someone learning the system for the first time?</li>
</ul>
</p>
<p>
Like all Onward! papers, this is a preliminary idea. And there is a lot work that needs to be done to see how useful this will be.
</p>
<h3><a href="http://www.pi-programming.org/What.html">Pi: A Pattern Language</a></h3>
<a href="http://www.flickr.com/photos/37092324@N00/4082930991" title="View 'Pi: A Pattern Language' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2447/4082930991_de6095ba88_m.jpg" alt="Pi: A Pattern Language" border="0" width="240" height="189" /></div></a>
<p>
I missed the actual presentation but the authors have been nice enough to post a <a href="http://www.pi-programming.org/docs/%CF%80.mov">.mov file</a> of their presentation online.
</p>
<p>
Short summary: You are trying to solve a problem. You know how to express your problem and solution but your programming language just doesn't let you express it succinctly. Nonetheless you persevere and manage to code it within 500 lines though you have now lost the essence of your solution amidst the lines of code. If only you could easily define some language constructs to capture the essence of the problem you are trying to solve and its solution.
</p>
<p>
This trend isn't new. The DSL folks are very happy to create custom languages. People in Lisp have been doing this for years. And with the rise of interest in <a href="http://en.wikipedia.org/wiki/Intentional_programming">intentional programming</a>, <a href="http://martinfowler.com/articles/languageWorkbench.html">language workbenches</a> and <a href="http://en.wikipedia.org/wiki/Language-oriented_programming">language-oriented programming</a>, we might just see more of such systems.
</p>
<p>
The paper itself does a good job of talking about related work so you can read it and have a good idea of how it compares to a system that you are familiar with. One related work that the authors missed is the <a href="http://bit.ly/3SVLwP">K Framework</a> developed at UIUC by the <a href="http://fsl.cs.uiuc.edu/index.php/Main_Page">FSL</a> group. The K Framework is worth mentioning because it is a nice and modular approach - using continuations - for solving the <em>context</em> problem while designing your own language constructs. As you design your own constructs there is always the risk of <em>interference</em> especially if you are trying to reuse the same symbols to mean different things for brevity.
</p>
<p>
To reflect on this trend, it might be good to read (or re-read) the popular essay by Paul Graham on the 100-year language. As languages evolve, they seem to gravitate toward Lisp.
</p>
<p>Paul Graham, <a href="http://www.paulgraham.com/hundred.html">The Hundred-Year Language</a>:
</p>
<blockquote cite="http://www.paulgraham.com/hundred.html" title="The Hundred-Year Language"> "The trend is not merely toward languages being developed as open-source projects rather than "research", but toward languages being designed by the application programmers who need to use them, rather than by compiler writers. This seems a good trend and I expect it to continue."
</blockquote>
<p>
Now the next question to ask would be how to easily develop tools that can grow as your language grows as well. Imagine having a syntax highlighter that is smart enough to recognize new syntax as you define it. Imagine having <a href="http://portal.acm.org/citation.cfm?id=1639949.1640127">refactoring tools</a> (my colleague, Jeff, is working on something related) that can help you refactor from an older construct to a newer one. How would you test a system like this: you would have to test your language and also your program written in that language. How would you be able to debug something like this?
</p>
<p>
PS: The paper has been picked up by <a href="http://www.reddit.com/r/programming/comments/9yug5/%CF%80_a_pattern_language/">reddit</a> and <a href="http://lambda-the-ultimate.org/node/3662">lambda-the-ultimate</a> as well so it might be interesting to read the comments there.
</p>
OOPSLA '09: Workshop on Refactoring Tools (WRT)2009-10-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/10/27/oopsla-09-workshop-on-refactoring-tools-wrt<a href="http://www.flickr.com/photos/37092324@N00/4049902724" title="View 'Extracing Concurrency' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2540/4049902724_796805f4a4.jpg" alt="Extracing Concurrency" border="0" width="500" height="333" /></div></a>
<p>
There were two major themes at the WRT this year: concurrency and polyglot refactoring.
</p>
<h3>Concurrency</h3>
<p>
I actually prefer the term <em>transforming</em> instead of <em>refactoring</em> for converting <strong>sequential</strong> code to <strong>parallel</strong> code but I will use the term refactoring here since other people in the community have embraced it.
</p>
<p>
Two papers were presented that dealt with refactoring to concurrency in the <a href="http://domino.research.ibm.com/comm/research_projects.nsf/pages/x10.index.html">X10</a> language:
<ul>
<li><a href="https://agora.cs.illinois.edu/x/p0yVAQ">Extracting Concurrency via Refactoring in X10</a></li>
<li><a href="https://agora.cs.illinois.edu/x/nUyVAQ">Concurrency Refactoring for X10</a></li>
</ul>
</p>
<p>
Both papers presented some prototype refactorings. I really liked the presentation of the second paper since it relates directly with the work I am doing on optimization patterns for parallel programming. As you can see from the image above, the tiling operation is one such optimization. It's important to provide such a transformation since you may need to do it to get good performance. Such transformations cannot be done by the compiler since it needs more feedback from the programmer on the size of the tiles which depends on the algorithm and data structures for that particular problem that you are trying to solve. And sometimes tiling might not even solve the problem so you need to know when to tile and when not to tile.
</p>
<p>
We had a breakout session on the topic of concurrency refactorings and here were the interesting issues (read as future research questions) that came up.
<ul>
<li>How to discover <strong>what</strong> to refactor?
<br/>
Profilers and trace based executions can help guide you in this area but they aren't always that intuitive to use. And you don't want to introduce concurrency blindly since it only matters if you are optimizing the real bottlenecks.
</li>
<li>What are some ways to suggest potential refactorings?
<br/>
Now that you know where the bottlenecks are, what are some refactorings that you can apply? I believe that <a href="http://parlab.eecs.berkeley.edu/wiki/patterns/patterns">parallel patterns</a> are the way to go.
</li>
<li>How do you check that your refactoring is correct?
<br/>
Refactoring engines for sequential code have <a href="https://agora.cs.illinois.edu/x/DJwKAQ">bugs</a>. And with concurrency refactoring you have to be even more careful to not introduce subtle concurrency bugs that can be hard to track down. How would you check the correctness of your transformations statically?
</li>
<li>How do you fine-tune your refactorings?
<br/>
Unlike sequential refactorings, concurrency refactorings require more feedback from the user. For instance, the dimensions of the tiles (see picture above). It would help if the IDE can suggest some parameters and then run the profiler automatically to give more feedback.
</li>
</ul>
</p>
<p>
The work on concurrency refactoring is definitely something that my advisor, Ralph Johnson, and his group at UIUC are interested in as part of our vision for an IDE that can help support parallel programming.
</p>
<h3>Polyglot Refactoring</h3>
<p>
Polyglot refactoring is not a standard term. It's one that I coined to describe how refactorings must be aware of all the different components that you use in your program - frameworks, libraries, etc. All these different components are described using different mini DSLs. For instance, the Hibernate configuration files are described in XML, your testing framework uses a set of APIs and conventions that you must conform to, and sometimes you use different languages (Groovy, JRuby, etc) that run on the JVM but share data.
</p>
<p>
It's important that as you perform the traditional refactoring such as rename, move, etc that you preserve the intended behavior in these other components as well. I think this is not an area that many people have looked at. Researchers focus on having new refactorings for the core language but forget that those other components need to be refactored too leaving a broken program that the user has to fix by hand.
</p>
<p>
There are three papers that deal with this issue:
<ul>
<li><a href="https://agora.cs.illinois.edu/x/9kKVAQ">Refactoring in Multiple Representations - Code and Statecharts</a></li>
<li><a href="https://agora.cs.illinois.edu/x/cUWVAQ">Testing-framework-aware Refactoring</a></li>
<li><a href="https://agora.cs.illinois.edu/x/qUKVAQ">Toward a Language-Agnostic, Syntactic Representation for Preprocessed Code</a> – this deals with preprocessors which essentially can be viewed as a different language on top of your base language</li>
</ul>
</p>
OOPSLA '09: Dynamic Languages Symposium (DLS)2009-10-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/10/27/oopsla-09-dynamic-languages-symposium-dls<p>
There were three interesting talks that I attended at the DLS this year:
</p>
<h3>Of Scripts and Programs: Tall Tales, Urban Legends, and Future Prospects</h3>
<a href="http://www.flickr.com/photos/37092324@N00/4049901868" title="View 'How People Use Dynamism in Javascript' on Flickr.com"><div style="text-align:center;"><img src="http://farm4.static.flickr.com/3512/4049901868_33df8d65ab.jpg" alt="How People Use Dynamism in Javascript" border="0" width="500" height="333" /></div></a>
<br />
<small>
(Unfortunately, I missed the first 10 minutes of the talk where Jan was talking about his experiences with the Pluto language).
</small>
<p>
Jan <a href="http://www.hpi.uni-potsdam.de/hirschfeld/dls/dls-09/invited/index.html">presented</a> the investigation that he and his team did to measure how much <em>dynamism</em> programmers used in Javascript – the most ubiquitous scripting/programming language on the web today. He had a good sample size, much bigger than most experiments that I have seen. And definitely not like the paltry, non-representative data that most other research papers have in their evaluation sections. His team surveyed major web applications that utilized Javascript heavily including <a href="http://280slides.com/">280slides</a>, GMail, Facebook, Flickr and Apple Me.
</p>
<p>
Some aspects of their evaluation include:
<ul>
<li>Size of code (in megabytes)</li>
<li>Prototype chain lengths – Almost like inheritance chains in Object-oriented languages</li>
<li>Variadity – Javascript allows you to call functions of <em>n</em> arguments with <em>m</em> arguments. So how apps out there use this feature?</li>
<li>eval(...) – which can used sanely to grab data from JSON or used dangerously to evaluate any string as code.</li>
</ul>
</p>
<p>
The conclusion? Some of those apps really use <em>almost every</em> dynamism that Javascript has to provide!
</p>
<p>
His solution? Yet another programming language (but, of course :-)). His language is called <a href="http://www.thorn-lang.org/">Thorn</a>. I don't have the time to look at it yet, but it reminds me a lot of <a href="http://www.strongtalk.org/">Strongtalk</a> language with optional typing.
</p>
<p>
Jan's slides should be available from the DLS <a href="http://www.hpi.uni-potsdam.de/hirschfeld/dls/dls-09/index.html">website</a> soon.
</p>
<p>
I also had the chance to ask someone from Google about how they use Javascript. And he basically said that they used a more disciplined approach to it – certain features just aren't used or they are used very judiciously. Some people really like the dynamism of Javascript and others just use it almost like another Java.
</p>
<h3>Optimization of Dynamic Languages Using Hierarchical Layering of Virtual Machines</h3>
<a href="http://www.flickr.com/photos/37092324@N00/4049901230" title="View 'Lua on the Lua VM on the ActionScript VM' on Flickr.com"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2783/4049901230_66e91b10f6.jpg" alt="Lua on the Lua VM on the ActionScript VM" border="0" width="500" height="310" /></div></a>
<p>
Here's a <em>crazy</em> idea: what if you wanted to run Lua on the Lua VM on the ActionScript VM?! Now, why would anyone do this? To test out their optimization technique for layering VMs, of course.
</p>
<p>
It's a rather circuitous way to evaluate this but it seems to be the easiest route. After all, you don't want to write a VM just to be able to try this out. You just want to use an existing one. And the <a href="http://labs.adobe.com/technologies/alchemy/">Alchemy</a> project provides an easy way to translate the original C-based Lua VM to ActionScript.
</p>
<p>
Of course, the caveat of such an evaluation is that the implementation will be so slow in the first place (the presenter was honest and mentioned this early) that it is relatively <em>easy</em> to get speedups no matter how trivial or naive your technique is.
</p>
<h3>Hosting an Object Heap on Manycore Hardware: An Exploration</h3>
<p>
This is a continuation of Dave Ungar's work which he presented at the <a href="http://softwareengineering.vazexqi.com/2008/10/21/oopsla-08-seaside-and-squeak-bof">Squeak BOF</a> at OOPSLA 2008.
</p>
<p>
In a nutshell, he implemented a simple benchmark (baton passing) between different Squeak Smalltalk VMs running on the 56 (it has 64 cores but some of them are reserved) cores of the <a href="http://www.tilera.com/products/processors.php">Tilera</a> multicore processor.
</p>
<p>
His goal with this experiment was to investigate what kinds of problems we would run if we are going to use manycore (not just multicore) processors. With manycore processors, you have non-uniform memory access and it becomes infeasible to maintain true cache coherence between those cores. In other words, you do need to be aware of the location of your data (much like what the distributed computing folks have been doing).
</p>
<p>
This is in contrast to what Intel's multicore chips are doing. Even in <a href="http://en.wikipedia.org/wiki/Larrabee_%28GPU%29">Larrabee</a> with 10+ cores, all the cores still maintain cache coherence.
</p>
<p>
I liked the talk because it presented a different view from all the other research papers that I have been reading where they try to solve the problem (or push the solution as far as it can go) for multicore systems and just implicitly assume that it will <em>scale</em> to manycore. People are trying to hold on to the shared memory model for as long as they can and trying very hard to not use a message passing system.
</p>
How Complex is "Personal Computing"? ââ¬â by the Fine Folks from VPRI2009-10-22T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/10/22/how-complex-is-personal-computing---by-the-fine-folks-from-vpri<a href="http://www.flickr.com/photos/37092324@N00/4036322430" title="Normal Considered Harmful"><div style="text-align:center;"><img src="http://farm3.static.flickr.com/2716/4036322430_0c0af6a16f.jpg" alt="Normal’ Considered Harmful (Or, How Much Like Frogs Are We Computerists?" border="0" height="80%" /></div></a>
<p>
<a href="http://en.wikipedia.org/wiki/Alan_Kay">Alan Kay</a>, of Smalltalk fame, and a few of his colleagues from <a href="http://www.vpri.org/">Viewpoints Research Institute</a> gave two wonderful talks at the Computer Science department at UIUC today. It's been a while since I have heard such interesting and inspiring talks so I'd summarize them for you :-)
</p>
<h2>How Complex is "Personal Computing"?</h2>
<p>
<strong>The video is now available from <a href="http://media.cs.uiuc.edu/seminars/StateFarm-Kay-2009-10-22a.asx">here</a></strong>.
</p>
<p>
This was a joint talk by four people:
</p>
<h4>Alan Kay</h4>
<p>
Alan started the talk by asking a simple question: how large a t-shirt would we need to contain the essence of Computer Science?
</p>
<div style="text-align:center;"><a href="http://www.zazzle.com/and_god_said_maxwells_equations_tshirt-235628270699537542"><img src="http://rlv.zcache.com/and_god_said_maxwells_equations_tshirt-p235628270699537542q6vb_400.jpg" alt="Maxwell's Equations of Electromagnetism." width="40%" /></a></div>
<p>
It's a serious question. The foundations of electromagnetism is contained in Maxwell's equations which can easily fit onto a t-shirt. What about Computer Science? Imagine trying to fit the source code for Microsoft Windows onto a t-shirt. How many t-shirts would we need?
</p>
<p>
With that in mind, the folks at VPRI are trying to find those fundamental <em>equations</em> for computer science as well. Three of his colleagues present their attempts at three different domains.
</p>
<h4>Dan Amelang</h4>
<p>
If you have seen the algorithms and code for doing antialised rendering, you'd know that it's a very complex component. However, it's also a very important component for computer graphics. Almost everything that we see on our screens now is antialised.
</p>
<p>
How would we describe the fundamentals of antialised rendering in simpler terms? <a href="http://www.vpri.org/html/team_bios/danAmelang.htm">Dan</a> has come up with a set of basic equations to do that. And he even created a domain-specific language called Nile to make it easier to describe those operations. For a glimpse of Nile (and I don't claim to even understand most of it), you can peek at Dan's <a href="http://github.com/damelang/gezira/blob/master/bezier.nl">code</a> on GitHub.
</p>
<p>
Nile is still work in-progress; to compile it into executable code builds upon the work done by ...
</p>
<h4>Alex Warth</h4>
<p>
<a href="http://www.tinlizzie.org/~awarth/">Alex</a> is the creator of <a href="http://tinlizzie.org/ometa/">OMeta</a> – an object-oriented language for pattern matching. In short, it's a language for rapid prototyping of languages. <a href="http://www.cs.ucla.edu/~awarth/papers/dls07.pdf">OMeta</a> allows you to easily model existing programming languages and also create new ones easily to enable you to experiment with new features instead of messing around with different lexers, parser, code generators, etc.
</p>
<p>
By applying <a href="http://en.wikipedia.org/wiki/Occam%27s_razor">Occam's razor</a> to computer languages, Alex hopes to enable everyone to easily grasp the fundamentals of one of the most important components of Computer Science: expressing your ideas in ways that the computer can understand. OMeta is succint, you can express a good portion of javascript in less than 200 lines. And like all good metalanguages, OMeta can be used to describe and bootstrap itself.
</p>
<p>
You can try out the Javascript version of OMeta <a href="http://www.tinlizzie.org/ometa-js/#Sample_Project">here</a>. I also strongly recommend reading Alex's <a href="http://www.vpri.org/pdf/tr2008003_experimenting.pdf">thesis</a> (very readable and very interesting as far as Ph.D theses go) where he talks about OMeta and some of his other work.
</p>
<h4>Hesam Samini</h4>
<p>
<a href="http://www.cs.ucla.edu/~hesam/">Hesam</a> is tackling the problem of program correctness using executable program specifications. What I really like about his work is the idea of using program specifications to fix program behavior whenever possible.
</p>
<p>
Most unit testing today try to test what can go wrong with your program and how your program would handle those aberrant behavior. However, it's very hard to specify everything that can go wrong. It's much easier to say what's the intended behavior of a program. And once you have that, if you ever violate what is means to behave correctly, you can use that <em>correctness specification</em> to guide your program back to correctness.
</p>
<p>
There are many ways to try to <em>correct</em> your program. Here's my example on this: take the case of a malformed string; perhaps the user accidentally puts a space i the string. A naive way would be to randomly just permute the string until you get a version that is right and fits the constraint of correct behavior. A smarter way might be to apply <a href="http://www.st.cs.uni-saarland.de/dd/"><em>delta debugging</em></a> to find the cause of the problem and then rectify it. Or you can try the <a href="http://portal.acm.org/citation.cfm?doid=1297027.1297072">techniques</a> that Martin Rinard uses for rectifying bad input for email messages..
</p>
<p>
In either case, specifying the correct and intended behavior in your program allows people to understand it more easily. Compare this with the arduous task of having to read through lines and lines of if...else and exception handling and having a hard time understanding the crux of your program.
</p>
<h3>Normal Considered Harmful (Or, How Much Like <a href="http://en.wikipedia.org/wiki/Boiling_frog">Frogs</a> Are We Computerists?)</h3>
<p>
This was the second talk by Alan which was intended for a more general audience. This <a href="http://cs.illinois.edu/news/2009/Oct20-1">talk</a> was recorded and is available from <a href="http://media.cs.uiuc.edu/seminars/StateFarm-Kay-2009-10-22b.asx">here</a>.
</p>
<p>
Anyway, here are the three main points that I liked from the talk:
</p>
<h4>Learning from History</h4>
<div style="text-align:center;"><a href="http://en.wikipedia.org/wiki/Albert_Einstein"><img src="http://upload.wikimedia.org/wikipedia/commons/a/af/Einstein1921_by_F_Schmutzer_2.jpg" alt="Who am I?" width="30%" /></a></div>
<p>
According to Alan, if you show people the picture above, they would most likely know that it's a picture of Albert Einstein. Show them the picture below and many won't know that it's a picture of Doug Engelbart, the inventor of the computer mouse among other things. Why is it that Computer Scientists don't know the historical figures in their field?
</p>
<div style="text-align:center;"><a href="http://en.wikipedia.org/wiki/Douglas_Engelbart"><img src="http://upload.wikimedia.org/wikipedia/commons/0/06/Engelbartmice.jpg" alt="Who am I?" width="50%" /></a></div>
<p>
And because we don't bother learning about our history, we miss a lot of grand ideas that came about in the 60s. Here, most of us are spending time reinventing the wheel and not even doing a very good job of it. The ideas from the 60s were revolutionary (pun intended); the ideas of today pale in comparison. We aren't daring enough to explore. And because of that ...
</p>
<h4>We Aren't Solving the Important Problems</h4>
<p>
Research today is very parochial. Look at the papers at most conferences. It's almost like they follow some cookie cutter template: problem(a cleverly invented one sometimes that no one really cares about), motivation, solution, case study, evaluation. Researches pat each other's back for solving related problems. However, look closer and you would see that the problems being solved are small and narrow. The big picture is missing.
</p>
<p>
People aren't exploring revolutionary ideas because the system (tenure, conference acceptance, funding, grants) doesn't welcome those ideas. Doing something revolutionary is risky. Of course it is. So people do the easy things and never explore beyond their fields. They become complacent. Instead they need to see that ...
</p>
<h4>All Understanding Begins With Our Not Accepting The World as it Appears – Susan Santog</h4>
<p>
We need to be bolder and explore more. We need to come up with ideas that border on the insane. Some insanity is required. After all, only 1% of the population comes up with such ideas; the other 99% like to keep things in order as they are now. So it's up to those 1% to come up with new ideas that the other 99% will eventually accept.
</p>
A Pattern Language for Screencasting2009-07-09T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/07/09/a-pattern-language-for-screencasting<div style="text-align:center;"><img src="/files/screen.jpg" alt="TV photo purchased from iStockPhoto" /></div>
<p>
I have written a pattern language for screencasting based on my experiences producing and observing various screencasts on the internet. Through my experiences I have noticed various patterns that myself and other screencasters have used. Applying those patterns would certainly help the presentation of your screencast but, more importantly, I believe that the patterns enhance the teaching-learning experience of screencasts. And ultimately that is what matters most.
</p>
<p>
Interested readers may view the patterns <a href="/files/Patterns.pdf">here</a> (older version can be found <a href="/files/screencast_patterns.html">here</a>). The pictures that I have included are based on software for the Mac since that is what I use for my screencasting.
</p>
<p>
At the moment, it's a rather complete version of what I had originally envisioned for such a document. It can definitely be improved but I would like to get a version out on the web for people to read in case anyone is interested in providing feedback or suggestions. This pattern language is in the shepherding stage of <a href="http://hillside.net/plop/">PLoP '09</a>.
</p>
<p>
I do maintain copyright on the document.
</p>
ParaPLoP'09: First Workshop on Parallel Programming Patterns2009-06-02T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/06/02/paraplop09-first-workshop-on-parallel-programming-patterns<div style="text-align:center;">
<img src="http://www.upcrc.illinois.edu/images/para_plop.gif" />
</div>
<br />
<p>
I'll be attending <a href="http://www.upcrc.illinois.edu/workshops/paraplop09/program.html">ParaPLoP 2009</a> over the next few days. It's a patterns workshop along the traditions of <a href="http://www.hillside.net/plop/2009/">PLoP</a> but focusing exclusively on patterns for parallel programming.
</p>
<p>
I am the author and co-author of the following three patterns paper that you can find on the <a href="http://www.upcrc.illinois.edu/workshops/paraplop09/program.html">program page</a>:
<ul>
<li>Barrier Synchronization</li>
<li>Patterns for Collective Communication</li>
<li>Patterns for Topology Aware Mapping</li>
</ul>
</p>
<p>
Most of the patterns submitted are part of the <em>Our Pattern Language (OPL): A Design Pattern Language for Engineering (Parallel) Software
</em> <a href="http://parlab.eecs.berkeley.edu/wiki/patterns/patterns">catalog</a> proposed by Kurt Keutzer (EECS UC Berkeley) and Tim Mattson (Intel). OPL is based on Tim's earlier book <a href="http://www.amazon.com/Patterns-Parallel-Programming-Software/dp/0321228111"><em>Patterns for Parallel Programming</em></a>.
</p>
<p>
While we have already identified quite a few patterns, there's definitely still a lot of work that needs to be done to produce a sizable pattern language that covers most of the patterns that most programmers will encounter in their programming career. However, I think that the layered approach that the Berkeley folks are advocating has great potential. The layered approach allows programmers to focus on different patterns depending on their skill sets and contributions to their project. At the top layer, <em>application programmers</em> focus on understanding the high level patterns and take advantage of the parallelism by using libraries and frameworks. And at the bottom layer, <em>platform programmers</em> focus more on understanding the low level patterns and develop libraries and frameworks to be used by other programmers.
</p>
<p>
Writing good software is hard. And writing good parallel programs is even harder. Patterns help make the task easier by showing the best practice principles that novice parallel programmers can learn from. And it's great that both UIUC and UC Berkeley are working together to catalog such patterns.
</p>
<p>
Ralph Johnson is leading the patterns project as part of the <a href="http://www.upcrc.illinois.edu/index.html">UPCRC</a> at UIUC. And one of his projects is to mine for patterns by examining how different algorithms are expressed in different languages and frameworks, in particular those developed at UIUC (e.g. <a href="http://dpj.cs.uiuc.edu/DPJ/Home.html">Deterministic Parallel Java</a>, <a href="http://charm.cs.uiuc.edu/">Charm++</a>, <a href="http://osl.cs.uiuc.edu/af/">Actor Foundry</a>). Kent Beck, Samira Tasharofi, Amin Shali and I will be contributing to this project and our preliminary results can be found <a href="https://agora.cs.illinois.edu/display/transformation/Results+Matrix">here</a>.
</p>
Newspeak Prototype2009-02-28T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/02/28/newspeak-prototype<p><a href="http://gbracha.blogspot.com/2009/02/newspeak-prototype-escapes-into-wild.html">Room 101: Newspeak Prototype Escapes into the Wild</a>:
</p>
<blockquote cite="http://gbracha.blogspot.com/2009/02/newspeak-prototype-escapes-into-wild.html" title="Room 101: Newspeak Prototype Escapes into the Wild"> "The Newspeak prototype is now available at http://newspeaklanguage.org/downloads/ . "
</blockquote>
<div style="text-align:center;"><img src="/files/newspeak.png" alt="Newspeak, first boot on my machine." width="50%" /></div>
<p>
Yes, the long awaited Newspeak is finally released! It's far from being complete but at least something that runs is out. I was in a class about Actor Programming Languages and Systems last semester and we were supposed to talk about Newspeak (and possibly do a group project on it if people were interested). Unfortunately, at that time, only the language specifications were released. And it's not as fun just reading about the specs and proving things about it.
</p>
<p>
The other good news? It runs on an Intel Mac. I was initially under the impression that it would only run on Windows.
</p>
<p>
Other initial impression: the UI is much improved compared to other Squeak implementations (which it is based on). Like the <a href="http://www.pharo-project.org/home">Pharo</a> project, Newspeak is probably aiming at creating an professional, open-source Smalltalk platform. Unlike Pharo, it still has remnants of eToys though.
</p>
<p>
Looks like I might be canceling some initial reading plans to play around with this over the weekend.
</p>
Statistical Debugging: Using Statistics for Good not Evil2009-02-01T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/02/01/statistical-debugging-using-statistics-for-good-not-evil<p>
<small>This article is a <em>summary</em> of these three papers about Statistical Debugging: <a href="http://research.microsoft.com/apps/pubs/default.aspx?id=73644"><em>Holmes: Effective Statistical Debugging via Efficient Path Profiling</em></a> (technical report from Microsoft to appear in ICSE 2009), <a href="http://pages.cs.wisc.edu/~liblit/pldi-2005/"><em>Scalable Statisical Bug Isolation</em></a> and <a href="http://pages.cs.wisc.edu/~liblit/pldi-2003/"><em>Bug Isolation via Remote Program Sampling</em></a>.</small>
</p>
<p>
Statistics is often used (and misused) as an effective way to finagle people into believing something that isn't really significant. There are many such cases (and you see them a lot on TV too). The one that I remember off the top of my head is a poster by <a href="http://hi5.com/">Hi5</a><a href="#footnote1"><sup>1</sup></a> where they had a nice <a href="http://www.sociableblog.com/2008/07/31/hi5-fastest-growing-top-10-social-network/">bar graph</a> which showed that they were the "fastest growing social network in 2007" compared to other sites such as MySpace, Facebook, etc. You have to read their claim (in quotes) properly to actually see what they are trying to claim and why it isn't really significant. Hint: they needed to use the word "growth". So, if hi5 had 100 members in 2007 and grew to 100 000 members in 2008, then its growth is 1000-fold (pretty impressive). But if Facebook had 100 000 members in 2007 and grew to 1 000 000 in 2008, then its growth is <em>only</em> 10-fold. So hi5 wins in terms of "growth" but in terms of users, that is pretty insignificant (how much more can Facebook grow once almost <em>everyone</em> you know is already on it?) So, what exactly is hi5 trying to sell to the people who are reading the poster? Trying to hire undergraduate students to work for them? Trying to show that they are going to overtake Facebook soon?
</p>
<div style="text-align:center;"><img src="/files/statistics.jpg" alt="Statistics: Good or Evil?" /></div>
<p>
As the example shows, you can use statistics for your own personal gain. And still keep your conscience clean because you are not really lying at all. And it is up to the reader himself to sieve through those numbers and decide whether they even mean anything significant. I believe that inundating the reader with such insignificant numbers is a prime example of using statistics for <em>evil</em>.
</p>
<p>
So it is refreshing to read a paper where statistics is used as part of the fundamentals of the technique itself and presented up-front in a convincing and compelling manner; Statistical Debugging (SD) is the use of statistics for <em>good</em> where numbers can actually <em>help</em> programmers locate pernicious bugs that manifest themselves infrequently and can be hard to track down.
</p>
<p>
SD relies on the fundamental idea of sampling a program. There are many tools out there that can sample programs but they are mostly used in profiling the <em>hotspots</em> of a program; in other words, the parts that are frequently encountered. However, such sampling is not useful for debugging since pernicious bugs are often not in the hotspots of a program. Thus, for SD to work, you need a different kind of sampling – one where it would be possible to sample the infrequently encountered parts of the program itself. This sounds simple but it can be prohibitively expensive if you are trying to observe all those little facts.
</p>
<p>
Solving the issue of collecting useful debugging information is the focus of their first paper, <em>Bug Isolation via Remote Program Sampling</em> where the authors present a novel way of doing the sampling so that they can collect interesting properties of the system. They identify a set of predicates (statements that can be true or false for a particular execution of a program) that is useful for C/C++ programs<a href="#footnote2"><sup>2</sup></a>. And for each run of the program, they would sample each predicate by modeling a Bernoulli process (like flipping a coin whenever each predicate is encountered to see if its needs to be sample). Basically, this means that for each execution of a program, the predicate has a <em>fair</em> chance of being sampled. Contrast this to the typical way of doing a profile i.e. using a periodic timer. If the execution of a program is short, some of the important predicates might be missed. So using a Bernoulli process guarantees a fair sample for each predicate.
</p>
<p>
Instrumenting a program so that we can collect information about each of its predicate can be <strong>very</strong> expensive because they can easily be a million predicates<a href="#footnote3"><sup>3</sup></a> for a typical program. And that is where the authors introduce a novel method for collecting the information which relies on the <a href="http://en.wikipedia.org/wiki/Law_of_large_numbers"><em>Law of Large Numbers</em></a>. It is infeasible to collect all the results of all those predicates from just one person on one machine. Instead it would be better to collect the information from as many people as possible. And each version of the program that runs on a different machine is designed to monitor a <strong>subset</strong> of all the predicates. Given <em>enough</em> machines, it is possible to merge all the results and get a useful matrix.
</p>
<p>
A sample matrix might look like the following:
<table>
<tr>
<th>Executions</th>
<th>Predicate #1</th>
<th>Predicate #2</th>
<th>Predicate #3</th>
<th>Predicate #4</th>
</tr>
<tr>
<td>Pass</td>
<td>True</td>
<td>False</td>
<td>True</td>
<td>Not observed</td>
</tr>
<tr>
<td>Fail</td>
<td>False</td>
<td>False</td>
<td>True</td>
<td>Not observed</td>
</tr>
<tr>
<td>Fail</td>
<td>True</td>
<td>True</td>
<td>Not observed</td>
<td>Not observed</td>
</tr>
<tr>
<td>Pass</td>
<td>True</td>
<td>True</td>
<td>True</td>
<td>False</td>
</tr>
<tr>
<td>..</td>
<td>..</td>
<td>..</td>
<td>..</td>
<td>..</td>
</tr>
</table>
</p>
<p>
For each execution, we check the value of each predicate: true, false or not observed. And from that matrix (which should be much larger) we try to find a correlation between program failure and a particular predicate. The paper <em>Scalable Statistical Bug Isolation</em> discusses how such a correlation is found and how <em>noise</em> is reduced. This technique is usable even if they are multiple bugs in the same program; SD is able to generate a correlation for each different bug. And it's also usable when you have <em>non-deterministic</em> bugs. SD works best when you have a large set of data for each execution of the program.
</p>
<p>
It's important to note that SD does not tell you that a particular predicate is <em>the</em> root cause of a bug. All it tells you is that there is a high <em>probability</em> (80% or higher) that this predicate is the root cause of a particular bug. And the programmer can then start focusing his efforts on figuring out why that particular predicate is a point of failure. Sometimes the bug itself does not lie in that particular predicate, but it is usually within close proximity so examining the predicate itself is a good start.
</p>
<p>
This is useful enough because sometimes it is almost impossible to detect such bugs otherwise since they manifest themselves infrequently. And because the overhead of collecting such information is low (remember that each instrumented program is specialized to only monitor a subset of the predicates) it is possible to run the collection on a user machine where SD can capture the execution of the program under the <strong>actual</strong> environment of a real user (vs. a <strong>controlled</strong> environment on the developer's machine).
</p>
<p>
The latest paper in SD, <em>Holmes: Effective Statistical Debugging via Efficient Path Profiling</em> evaluates using a different set of predicates (path profiles) that might help programmers identify the bug more easily. The previous method of using predicate will report the location (file and line number) of the predicate that is believed to be the root cause. While the predicate is usually within close proximity to the actual bug, the programmer still has a lot of code to sieve through. Path profiles provide more information about the execution of the program and contains the actual execution path that the program went through. This is useful for identifying bugs that have complex control flows through the program.
</p>
<p>
In conclusion, SD is a novel idea in helping programmers find subtle bugs that might be hard to identify otherwise. Good unit tests can usually help eliminate a lot of bugs but it is hard write tests for everything (and the cost of maintaining such detailed tests is very expensive). SD is not perfect nor does it claim to be. Instead it is another useful tool in a programmer's arsenal for hunting down bugs when trying to manually locate them has proven to be unfruitful.
</p>
<p>
The papers also impress me because of their honesty in doing their own evaluations and experimentations; they explicitly state that their experiments are done under controlled environments. And they actually have an <a href="http://www.cs.wisc.edu/cbi/">on-going project</a> where they are evaluating SD in real applications.
</p>
<hr />
<p>
<small>
<ol>
<li><a name="footnote1"></a>I think that it is fair to use this example since it was a public poster that was posted on the public walls of the Siebel Center for CS at UIUC. Moreover, I maintain disinterest in hi5.</li>
<li><a name="footnote2"></a>The set of predicates might be different if we were to conduct the experiment on programs written in different languages or programming styles (procedural vs. OOP).</li>
<li><a name="footnote3"></a>For a C/C++ program the set of predicates includes branches, integer return values and assignments. Integer return values are useful in C/C++ programs because the return value is usually used to indicate if the operation succeeded (return 0) or if it failed (return -1).</li>
</ol>
</small>
</p>
Prediction is very difficult....2009-01-19T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/01/19/prediction-is-very-difficult<p><a href="http://quotationsbook.com/quote/32056/">Niels Bohr </a>:
</p>
<blockquote>
Prediction is very difficult, especially about the future.
</blockquote>
<br />
<div style="text-align:center;"><img src="/files/dice.jpg" alt="Prediction is very difficult...." /></div>
<p>
During New Year's week I was watching tv and happened to switch over to the History Channel. It was showing something about the Bible code. More specifically, the title of the show was <em>The Bible Code: Predicting Armageddon</em>. Don't ask me why the History Channel decided to show something apocalyptic during New Year's week. I guess most people are happy to associate impending doom with the New Year instead of blooming hope.
</p>
<p>
A slight tangent before I get to the gist of this post....
</p>
<p>
For those unfamiliar with the Bible Code, it's a book that postulates that the Hebrew bible contains hidden messages in the form of a code – the <em>Bible code</em> – that hold predictions about the future. The exact details of the postulated cipher can be found on its wikipedia <a href="http://en.wikipedia.org/wiki/Bible_code">page</a>.
</p>
<p>
Anyway, by using the Bible code, the authors claim to be able to find records of all the major historical events that have transpired including the two World Wars, the Holocaust, the assassination of prominent figures, etc. They concluded that there was "strong statistical evidence" that such encodings could not just be <em>random</em>.
</p>
<p>
Interesting. So the Bible code actually encodes all the events that have happened. Then could it be deciphered so we could use it to predict events that have yet to transpire? Sure. But there's a catch. We won't actually know how to look for those predictions. It's easy to look for things that have happened because we have clues and keywords to look for in the code. But for predicting the future, we have no idea on what to actually look for. Catch-22.
</p>
<p>
And that, to me, is a prime example of <a href="http://en.wikipedia.org/wiki/Confirmation_bias"><em>confirmation bias</em></a>. The wikipedia article illustrates this easily with the 2-4-6 problem. We only look for what we seek to discover in the first place. And we conveniently ignore what we don't want to discover (or don't really know about yet). We conduct experiments and case studies but all too often we interpret the results to suit what we want to verify.
</p>
<p>
All right, back to the gist of this post. I wrote this post with a focus on TDD: Test-driven Development. TDD is one of the more controversial practices in agile software development today. And it is also one of the most misunderstood practices.
</p>
<p>In <a href="http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=00951502"><em>Aim, Fire</em></a>, Kent Beck says:
</p>
<blockquote cite="http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=00951502" title=""> "Test-first coding isn't testing."
</blockquote>
<br />
<p>
It's more about design. Writing tests first forces the developer to think about the design of the different units. Each unit should be designed so that it can be unit tested easily (and preferably in isolation) from other units.
</p>
<p>
I'll be honest and say that the first time I heard about TDD, I didn't grasp this fundamental concept. Instead, I too thought that it was all about writing your tests up-front. And, initially, I wasn't very keen on the idea. I believe that adequate testing is definitely useful. But I wasn't really convinced why we needed to do test-first. Wasn't it just as useful to have tests <em>slightly</em> later after the initial design so that your tests actually have a chance of, erm..., passing?
</p>
<p>
So I used to read papers studying the success of TDD with my own confirmation bias. I always looked out for little things that the authors missed that could invalidate their claims of the success of TDD. There weren't hard to find since it was <em>impossible</em> to do a fool-proof study of TDD in any actual environment.
</p>
<p>
But here's the interesting part. Now that I am more in favor of TDD, those little things still cause me to be skeptical on how useful TDD is (especially if the authors forget the part that TDD isn't just about testing first!). The case studies aren't really conclusive enough to help me predict if using TDD is a requirement for good software. N.B. Evaluations on small projects aren't particularly helpful either because when your project is small it is likely to succeed even if you don't have a proper process.
</p>
<p>
Sure, TDD's proponents are still enamored by it. But the views of its opponents (maybe that is too strong a word) cannot be ignored either.
</p>
<p>
Some of the most important things about writing software include delivering the product to the client on time, ensuring that the product has good quality, ensuring that the product fulfills the requirements and also ensuring that the code is maintainable for subsequent releases.
</p>
<p>
And right now, we don't have strong evidence that TDD is essential in accomplishing those tasks. There are teams that do not do TDD (whether for design or testing) and yet produce exceptional code. There are teams that start of being gung-ho about TDD and stop doing it halfway because they run into problems. So what does following TDD actually tell us?
</p>
<p>
And it's not just about TDD. What about things such as refactoring, pair-programming, and all the other pillars of agile development. Or what about all the latest trends in software development such as <a href="http://apsblog.burtongroup.com/2009/01/soa-is-dead-long-live-services.html">SOA</a>, cloud computing, etc.
</p>
<p>
We still don't have a good way to evaluate such things other than to try it out. Trying it out isn't a bad thing but some of these practices cost time and money and could be prohibitively expensive to <em>try out</em> on a whim. And while some would justify it as paying the cost up-front instead of later during the maintenance stage, no one actually knows for sure whether the cost is worth it. And after trying it out, unless we do proper experiments we can't measure the actual merit of that technique. And without proper data, we are inclined to make skew predictions about our ability to replicate the success we had in one software project in our other projects.
</p>
<p>
And when you cannot actually verify those claims, you run into the danger of herd mentality, religious debates, zealotry. And when something new comes along, you either obstinately stick to you old practices or apostatize and switch over to newer paradigms.
</p>
<p>
There needs to be more research on studying how to effectively measure<a href="#footnote1"><sup>1</sup></a> the effects of some software development technique. Now, it <em>could</em> be extremely hard to do or even impossible. But without proper studies, we only have our gut instincts to rely on and that is no better than flipping a coin and letting it predict what software practices to follow....
</p>
<p>
Now I like all the agile development practices. I find that it makes me feel more productive. And it gives me better confidence that I am writing good code. But is that enough as a measurement of how useful a practice is?
</p>
<hr />
Here are some of the TDD papers that I have read that some might find interesting:
<ul>
<li><a href="http://swerl.tudelft.nl/twiki/pub/Main/TechnicalReports/TUD-SERG-2007-014.pdf">Evaluation of Test-Driven Development</a></li>
<li><a href="http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01357925">A Prototype Empirical Evaluation of Test Driven Development</a></li>
<li><a href="http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf">Realizing Quality Improvement Through Test Driven Development: Results and Experiences of Four Industrial Teams</a></li>
<li><a href="http://iit-iti.nrc-cnrc.gc.ca/publications/nrc-47445_e.html">On the Effectiveness of Test-first Approach to Programming</a>. See discussion <a href="http://scruffylookingcatherder.com/archive/2008/01/22/tdd-proven-effective-or-is-it.aspx">here</a>.</li>
</ul>
<small>
<a name="footnote1">1.</a> HCI actually has a good set of <a href="http://www.isrc.umbc.edu/Research/Eval/EvalProject.html">evaluation techniques</a> that they use to evaluate user interfaces. Perhaps we need to develop a set of evaluation techniques for Software Engineering as well.
</small>
A Series of Unrelated Events2009-01-03T00:00:00+00:00http://softwareengineering.vazexqi.com/2009/01/03/a-series-of-unrelated-events<p>
In <a href="http://www.amazon.com/Outliers-Story-Success-Malcolm-Gladwell/dp/0316017922/"><em>Outliers</em></a>, Gladwell presents an interesting chapter about how Asians are better in Math; he tries to bring his point across by illustrating how the traditional Asian habit of planting rice in the paddy field can be seen as an influence on the way Asians approach life (and its problems) and how that approach helps them to be better at math.
</p>
<div style="text-align:center;"><img src="/files/apples.jpg" alt="A Series of Unrelated Events" /></div>
<p>
Like most of Gladwell's books so far, that particular chapter is punctuated with a whole bunch of anecdotes and narratives. They make for an interesting read but do <strong>not</strong> help make the chapter more cohesive. In fact, they even appear to be a series of unrelated events that misguide the reader into thinking that they are more strongly related than they appear.
</p>
<p>
In particular, Gladwell has this story about the way numbers are represented in different languages. He gives the example of this random series of numbers: 4, 8, 5, 3, 9, 7, 6. And he wants the reader to memorize that series. In English, one would read it out as: four, eight, five, three, nine, seven, six. In Malay (another language that I know), one would read it as: empat, lapan, lima, tiga, sembilan, tujuh, enam. And finally, in Cantonese (or other Chinese dialect) it would be: say, putt, mmm, sum, kau, chud, loke. So in terms, of syllables, Chinese trumps English; and both Chinese and English trump Malay which is really inefficient when it comes to counting syllables. Gladwell suggests that the way numbers are pronounced impacts the ability of the user to memorize the series. And this impacts the ability to count and thus perform other math skills such as addition. So, his observations suggests that Chinese users should be able to do math better than other language users.
</p>
<p>
As someone who knows three languages, I find this really hard to believe. If I were to count out loud, I would do it in English (so English is my primary language for math). However, most of the time, I don't need to count out loud. I just do all the calculations in my head with the <strong>most</strong> concise notation possible: numerals e.g. 1, 5, 12, etc. Why would I even care about the number of syllables when I can efficiently use the numeral notation?
</p>
<p>
Things might be very different, if I were forced to use the <a href="http://en.wikipedia.org/wiki/Roman_numerals">roman numeral</a> notation which does not lend itself well for mathematical manipulation. However, as far as I know, most languages will use the decimal numeral system that we are used to. In fact, if Gladwell were to extrapolate his notion of "efficiency in representation", he might be shocked to discover that the Chinese way to represent numbers is fairly inefficient compared to our decimal system. For instance to represent, 999, we use three digits. In Chinese, it would be ä¹ ç™¾ ä¹ å ä¹ (if the text does not come out right, see minute 3:49 of this <a href="http://www.youtube.com/watch?v=IddMHrp6Sn4">video</a>). Now, I don't count in Chinese so I am not sure how much this impacts the ability to add. The decimal system feels natural to me since I can add in each position and just carry over to the next position as necessary. The Chinese could have some special way to do the addition that accounts for the longer notation.
</p>
<p>
Anyway, my point in bringing this up is not really to criticize Gladwell. Rather I want to point out the fact that there are a lot of unrelated events that if taken superficially seem to corroborate something that isn't quite right; it could be because we only have a shallow understanding of the matter and have not even begun to dig deeper into the crux of the problem. Or it could be that we are not defining the problem that we are trying to solve properly.
</p>
<p>
And that brings me to two papers in CS that I have read recently: <a href="http://portal.acm.org/citation.cfm?id=1176625"><em>The Geography of Programming</em></a> and <a href="http://portal.acm.org/citation.cfm?id=1449872"><em>Towards Harmony-Oriented Programming</em></a>. The former was the basis for the latter paper.
</p>
<p>
The first paper, <em>The Geography of Programming</em>, was based mostly on the observations from the book <a href="http://www.amazon.com/Geography-Thought-Asians-Westerners-Differently/dp/0743255356/"><em>The Geography of Thought</em></a> which I have read and do not really agree on. I found the observations and experiments in the book to be jaded and outdated. Most students in the current era have already been affected by the globalization movement (or rather the diffusion of Western thought). Therefore, most students aren't easily classified as Western or Eastern thinkers anymore. In fact, most students today are hybrid thinkers, being able to utilize both Western and Eastern thought processes as necessary.
</p>
<p>
Which is why I don't really agree with <em>how</em> the authors presented in their first paper. I do not have a problem with the idea of creating a less rigid programming paradigm – Harmony-Oriented Programing – that would allow for greater flexibility in creating software. I think that idea is interesting <strong>and</strong> has already been attempted in various forms throughout the years e.g. aspect-oriented programming, conscientious software, agents, etc. What I do not like about the paper is the mythicism that the authors choose to surround their work with. In particular, they choose to shroud their work with invocations of Eastern thinking and philosophy. I found that to be completely unnecessary and confusing. And fundamentally, wrong because they have taken the notion of Eastern philosophy completely out of context. Making it a part of a series of unrelated events to corroborate their work (much the same as what Gladwell has done).
</p>
<p>
As for the second paper, my colleague, Jeff has already written a <a href="http://jeff.over.bz/blog/2008/10/oopsla_08_onward.html">blog post</a> on it during the presentation in OOPSLA 2008. Like I said, I like the work and research that they are doing, but I find that the approach of promoting it to be rather unappealing. It's what I would call a series of unrelated events slapped together to serve as a rather weak metaphor to differentiate it from other existing work.
</p>
<hr />
<small>
<strong>Harmony-Oriented Programming (HOP)</strong>
<br />
The problem that HOP is supposed to address is the rigidity of software. However, Brooks has already pointed out that sometimes, people are too enamored with the <a href="http://softwareengineering.vazexqi.com/articles/2007/11/03/oopsla-07-collaboration-and-telecollaboration-in-design">perceived malleability</a> of software. To promote malleability, HOP advocates the separation of code and data. Inherently code needs to act on data. But instead of binding the data directly to the code, the data is supposed to come in from different sources. One can think of this as being similar to the idea of <em>eval</em> in dynamic languages where a snippet of code is executed in some environment that provides bindings for the variables. However, in HOP, the environment is not provided explicitly -- the environment exists and permeates all the code snippets. All code snippets are supposed to pick up whatever they need from the rich environment. And in turn, those code snippets will contribute back data into the environment that other code snippets can take advantage of (sounds a lot like a proliferation of global variables to me). The authors are trying to create an actual programming environment to evaluate HOP. However, in both papers, they have not yet address the important fundamental issue of how to prevent unwanted interactions between code snippets if all of them will be accessing the same environmental primordial soup of data at the same time.
</small>
Over-reliance on End User Testing2008-11-28T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/11/28/over-reliance-on-end-user-testing<p>
In <em>Classic Testing Mistakes</em>, Brian Marick presents several problems with the way testing is done nowadays. One of the mistakes that I have found most interesting is the <em>over-reliance on beta testers</em> (or just anyone who uses the software).
</p>
<p><a href="http://www.exampler.com/testing-com/writings/classic/mistakes.html">Classic Testing Mistakes</a>:
</p>
<blockquote cite="http://www.visibleworkings.com/papers/mistakes.pdf" title="Classic Testing Mistakes">
"Beware of an over-reliance on beta testing. Beta testing seems to give you test cases representative of customer use - because the test cases are customer use. Also, bugs reported by customers are by definition those important to customers. However, there are several problems...."
</blockquote>
<p>
He gives several reasons for this (the original paper is very readable so you might want to read it first) but I have recently encounter two of them personally:
<ol>
<li>(Someone that I know...)When your product doesn't perform properly, some users not only do <strong>not</strong> report a usability problem, but they also quickly conclude that your product <em>sucks</em>. They don't bother looking on the forums or seeking help; instead they completely conclude that your product is not usable and it's not their fault that they don't RTFM. Now, as a developer you would be interested to at least capture what prompted this review of your software instead of seeing the whole bunch of "x product sucks" on the internet(try it: google for "eclipse sucks", "netbeans sucks", etc). Most problems can be easily resolved by asking on the forums and the data that the developer gets from this could be used to improve the product in the next release to make it more usable.
<br />
<br />
Scenario 1 usually happens when someone is forced to use a particular product. He usually doesn't feel satisfied that he was not given the freedom to choose the product (in particular an IDE). And when things begin to go wrong, instead of thinking that he might be doing it wrongly, he concludes that the product sucks.
<br />
<br />
Of course, Scenario 1 would be a completely different case if the user had actually spent some time seeking help and was told that there is no solution for the problem that he is having. In that case, the product really needs to be fixed.
</li>
<br />
<li>(Personal experience; I feel slightly hypocritical about this...) I needed to use NetBeans over the past week to play around with JavaFX. Because Apple doesn't ship a Java 1.6 JVM for the older 32-bit Core Duo machines, I had to use Windows XP under VMWare Fusion. Anyway, there were a lot of things that NetBeans did not support that could make development <em>easier</em>. I could definitely still use the product and there was nothing seriously wrong with it but there are some features that could definitely make the product more usable. Disclaimer: I am an Eclipse/ IntelliJ user so I might not be familiar with the philosophy of NetBeans. Did I bother reporting such issues? No, unfortunately, because I just wanted to get things done, pack-up all the JavaFX components that I created as a jar that I could then use from Eclipse (as part of my actual plug-in project).
<br />
<br />
So Scenario 2 happens when the user doesn't really need to use the product all that much but is just a casual user (for the time being). However, the problems that the user encounters early as a new user is actually pretty useful for the software developer since it could be used to improve the usability of the product in the future.</li>
</ol>
</p>
<br />
<p>
So there you have it, two personal encounters that demonstrate why it is not really a good idea to rely too much on beta/end users to test your product. I have submitted <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=236740">one</a> usability problem to Eclipse though, and that was only because I use Eclipse so much daily that the issue really bugged me. It was a UI issue and it is one that I have trained myself to circumvent. Nonetheless it bugged me enough the first time I saw it that I decided to report it so that new users will not have to suffer through it.
</p>
<p>
So end users with some form of <em>product loyalty</em> – people who use it constantly by their own volition – are the only people who are likely to report bugs with your product. Or, if you have read <a href="http://www.amazon.com/Tipping-Point-Little-Things-Difference/dp/0316346624"><em>The Tipping Point</em></a>, they are likely to be the <a href="http://en.wikipedia.org/wiki/Maven">mavens</a> who take pride in their knowledge of the product.
</p>
<hr />
<small>Some chapters of <em>The Tipping Point</em> are actually pretty useful for software testing since it gives some idea on who is more likely to help promote your product and who is more likely to report problems. Compared to non-software products, we do have a better way to keep track of all the issues that come in (bugzilla, etc).</small>
OOPSLA '08: DSL Panel ââ¬â The Good, The Bad and The Ugly2008-10-25T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/25/oopsla-08-dsl-panel-ââ¬â-the-good-the-bad-and-the-ugly<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2964839867" title="View 'DSL: The Good, The Bad and The Ugly Panel' on Flickr.com"><img src="http://farm4.static.flickr.com/3029/2964839867_81cdc27aed_m.jpg" alt="DSL: The Good, The Bad and The Ugly Panel" border="0" width="240" height="180" /> </a><a href="http://www.flickr.com/photos/37092324@N00/2965683876" title="View 'DSL: The Good, The Bad and The Ugly Panel' on Flickr.com"><img src="http://static.flickr.com/3038/2965683876_f95443a1e4_m.jpg" alt="DSL: The Good, The Bad and The Ugly Panel" border="0" width="" height="" /></a>
<br />
<small>The panelists: (from left to right) Marjan Mernik, Juha-Pekka Tolvanen, Gabor Karsai, Charles Consel and Kathleen Fisher.</small></div>
<p>
DSLs seem to be a rather popular topic at OOPSLA this year. There was a <a href="http://www.dsmforum.org/events/DSM08/">workshop</a> on it and several tutorials as well. And there was also the <a href="http://www.oopsla.org/oopsla2008/program-overview/panels.html#pnl0000003">DSL panel</a> which I attended this morning. Most of the talks and tutorials on DSLs assume that the audience is using specialized modeling tools to create them. Basically those tools provide a specialized environment for creating the meta-model for your DSL, specifying the grammar of your DSL and helping you specify templates for code generation (which actually makes your DSL useful). Such tools aren't meant to help you create a full-fledged programming language. Instead they help you quickly create a small language that can be refined in an agile and iterative process. One of the panelists commented that this might be more productive (and less intimidating) than using lex and yacc.
</p>
<p>
So, as a summary of the panel, let me present to you the good, the bad and the ugly about DSLs:
</p>
<h3>The Good</h3>
<ul>
<li>Most people are already using DSLs in the industry and it seems that there has been some case studies that strongly suggest that using DSLs actually helps improve productivity. When asked, Juha-Pekka, who presented those results, told the audience that productivity was measured in terms of what the customer told him – so they could be measuring that it used to take 6 hours to complete something and now it takes 1 hour to do it; thus gaining a 6-fold increase in productivity. Such numbers were <strong>actually</strong> reported on his slides.</li>
<li>Most panelists agree that using a <em>simplified</em> language like a DSL helps the user avoid certain mistakes that a general-purpose language might incur. After all, it is not possible to express certain constructs in the DSL that would be possible in a normal programming language thus lessening the chances for making those mistakes.</li>
</ul>
<h3>The Bad and The Ugly (we can just merge them)</h3>
<ul>
<li>As far as I know, it's hard to actually tell what are <em>good</em> qualities of a DSL. So if I am designing a DSL how do I know what direction to steer it to so my customer would benefit more from it? For instance, is DSL A better than DSL B? And in what aspect? The only answer provided to this question is that creating a DSL is an iterative process and depends on what your customers like.</li>
<li>Which brings me to the second point: how to we <em>duplicate</em> success that we had achieve in using one DSL into another project? If we cannot systematically distill the finer points of each DSL, it would be hard to repeat its success on another project. It is also <em>wasteful</em> to start from scratch every time – though the panelists seem to argue that since it's a <strong>domain-specific</strong> language, it might better to create very specialized ones for each customer.</li>
<li>And this leads me to my third point: how to share DSL among different projects? Currently, there is very little being done on researching what is the best way to share DSLs among projects. Also, do we actually need to share the DSL or just the underlying meta-model that the DSL embodies? A simple and naive way might just be to form a standard and have everyone use that. Though, of course, that naturally, leads to a bloated DSL since everyone would have some specific feature that no one else needs. </li>
<li>The panelist also agree that there are currently very few tools to help DSL developers. Currently there aren't many tools to help a Domain-Specific Environment (DSE) that will include all the features of a normal IDE such as code-completion, debugging, etc. But more importantly, there aren't tools to help domain-analysists distill the features of the domain itself – the most important step in creating an <a href="http://softwareengineering.vazexqi.com/articles/2006/10/01/ddd-communication-and-the-use-of-language"><em>ubiquitous language</em></a> as a basis for the DSL.</li>
<li>Also, when asked if they prefer a <em>internal DSL</em> (implemented using the constructs of an existing programming language – common in Ruby) or an <em>external DSL</em>(implementing a new language), the panel seems to agree that it might be better to use an external DSL so that the user will not be tempted to <em>escape</em> from the domain and use constructs outside of the DSL. For instance, if we were to implement an internal DSL using Ruby, we would have at our disposal, all the feature of the Ruby language which can be used in our DSL. This makes it tempting to use existing language features in an ad-hoc manner and <strong>pollute</strong> the actual DSL.</li>
</ul>
<p>
The interesting part of the panel only happened during the second half when the audience was given the opportunity to ask questions. The first half was <em>wasted</em> (pardon the word) on having each panelists spend 10-15 minutes talking about their work and their position on DSLs. In my opinion, that was a serious mistake since it only gave the audience about 30-45 minutes for the interesting questions.
</p>
<p>
Coincidentally, I just found out that Martin Fowler actually has a <a href="http://martinfowler.com/dslwip/">work-in-progress</a> about DSLs. It might be better to read about DSLs from him rather than other DSL enthusiasts since his approach might be a more unbiased view toward how to do DSLs; instead of a DSL-enthusiast who is already too enamored by the very idea.
</p>
OOPSLA '08: Seaside and Squeak BOF2008-10-21T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/21/oopsla-08-seaside-and-squeak-bof<h2>Seaside BOF</h2>
<p>
I didn't get to take any pictures in this session because I came in late and was sitting at an awkward spot. James Robertson from Cincom was there and has posted some pictures on his <a href="http://tinyurl.com/chrruj">blog</a>. I am actually in one of those pictures :)
</p>
<p>
Michael Lucas-Smith did a demo of WebVelocity – Cincom's new in-your-web-browser development environment for Seaside. I missed the first 10 minutes of it but it's similar to the <a href="http://tinyurl.com/6h2kwn">screencast</a> that he had done before. WebVelocity is cool but there are <strong>two</strong> things that I really like about it.
</p>
<p>
First, it has <em>ample</em> documentation provided for Seaside. Besides the base library from Cincom, I think that this is one of the best documented Smalltalk project ever. Not only do they have comments for the classes and method, but they also have a nice getting started guide all built-in and easily accessible. That was really fantastic! And to make it even better, they had an integrated search widget that allowed you to search through the class names, methods names and, now, comments! Something that was missing from Smalltalk before.
</p>
<p>
Second, WebVelocity bravely goes where no Smalltalk has gone before: it lets you edit your source code inside <strong>a single</strong> window – in this case, your web browser. You can actually see all your methods in one editor without having to open multiple windows! I think this is one of the most interesting (and smart) approaches that Cincom can take to get people to use Smalltalk. I believe that they have reduced the entry barrier for Smalltalk by presenting it in a more familiar environment to newcomers. Hopefully they get some positive feedback and comments on this and use that to help them structure their tools to suit both new and veteran users.
</p>
<h2>Squeak BOF</h2>
<div style="text-align:center;">
<a href="http://www.flickr.com/photos/37092324@N00/2962730393" title="View 'Squeak BOF' on Flickr.com"><img src="http://farm4.static.flickr.com/3002/2962730393_bc0d845032_m.jpg" alt="Squeak BOF" border="0" width="240" height="180" />
</a><a href="http://www.flickr.com/photos/37092324@N00/2963574328" title="View 'Squeak BOF' on Flickr.com"><img src="http://farm4.static.flickr.com/3233/2963574328_c5555967b2_m.jpg" alt="Squeak BOF" border="0" width="240" height="180" /></a>
<br />
<small>I took the pictures using my iPhone so they are all pretty bad actually. The only way I could salvage them was by turning them into black and white pictures....</small>
<br />
</div>
<p>
There were three presentations in this year's Squeak BOf.
</p>
<p>
First Goran presented his project on creating a a lightweight Simple-CGi replacement for Seaside development in Squeak. He calls it Blackfoot and at the time of the demo he still had some bugs in getting Seaside to function. However, he is confident that he will be able to fix it and publish the code soon. His primary goal with this project was to create a replacement that was small, simple and fast. And the micro-benchmarks that he had showed that it was about twice as fast.
</p>
<p>
Next, Dave Ungar presented <em>The Birth of Manycore Squeak</em>. Basically he demonstrated what he and Sam Adams have been working on at IBM Research: writing a new Squeak VM that could run on top of the <a href="http://www.tilera.com/products/TILE64.php">Tilera 64</a> multicore chip. Right now he has hacked the VM so that it can actually run the MVC UI in Squeak. It's also able to do simple object migration from core to core. However, as he emphasized, this is still work in progress and there's lots of things to be done. In particular, it might require adding some new primitives to Smalltalk to make multicore concurrent programming easier. And it might also require changing the programming model to make it easier to program things concurrently.
</p>
<p>
And finally, Jecel Assumpcao Jr. talked about <em>Issues in Smalltalk Hardware Design</em>. He gave a lengthy introduction to the various attempts at creating specialized hardware to execute Smalltalk throughout the years. Most (if not all) of those projects are now dead and obsolete. However, he is interested in creating a modern implementation for Squeak. From the looks of it he already has a draft of the architecture and ISA that he is planning to support. The spec for his Squeak bytecode processor, Plurion, is available from his <a href="http://siliconsqueak.org/">Siliconsqueak</a> web page. It wasn't clear from his presentation if he intents to create a multicore bytecode processor or just a single core one.
</p>
<p>
There weren't as many people at this year's BOF as I had expected. It was cool that the Cincom guys were there this time around though.
</p>
<p>
<strong>update:</strong> The videos are now available from Goran's <a href="http://goran.krampe.se/blog/OOPSLA/OOPSLA08-more.rdoc">site</a>.
</p>
OOPSLA '08: Lisp50@OOPSLA2008-10-21T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/21/oopsla-08-lisp50-oopsla<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2960998550" title="View 'Lisp50@OOPSLA' on Flickr.com"><img src="http://farm4.static.flickr.com/3192/2960998550_bb188bbefd.jpg" alt="Lisp50@OOPSLA" border="0" width="500" height="375" /></a>
<br /><small>The Evolution of Lisp</small></div>
<p>
This year marks the 50th anniversary of Lisp. While I am not a Lisp hacker, I have dabbled with it (and lambda calculus) enough that I do take interest in things that happen in the Lisp world. There was the <a href="http://www.lisp50.org/">Lisp50@OOPSLA </a>workshop today but I only had time to attend the first talk of the day by Guy Steele and Dick Gabriel.
</p>
<p>
Guy Steel and Dick Gabriel presented their combo-talk on <em>The Evolution of Lisp</em>. And like every combp-talk by Guy and Dick, it was entertaining and informative. I had never actually realized how many versions of Lisp there were until today! The slides that Guy and Dick showed contain the history of Lisp from the 1960s till now. They could even divide the different implementations geographically across the United States and showed how different version of Lisp from different states influenced one another. While the primary centers for Lips development were at MIT and Stanford, there have been many different efforts outside of those two universities as well.
</p>
<p>
The presentation must have been endearing for most of the Lisp hackers in the crowd. For me, it was also an insightful look into how one of the oldest programming languages had evolved and to see its impact on other modern programming languages.
</p>
<p>
For me, the most memorable part of the presentation was when Dick emphasized that though there were so many different versions of Lisp, each with its own little variation (making integration and code sharing hard) the explosion of those different implementations actually fueled a lot of interest and research into making better implementation for Lisp. Now that there are standards for Lisp, fewer dialects have popped up. And so by gaining standardization, you do lose out on some of the innovation because no one is interested in creating something that doesn't conform to the standards anymore. I thought that was something interesting to think about since we usually take it that standardization is always the best route. But Lisp has shown that innovation can often times blossom through many different home-brew implementations.
</p>
OOPSLA '08: Designed by Designer2008-10-21T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/21/oopsla-08-designed-by-designer<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2962729923" title="View 'Designed as Designer' on Flickr.com"><img src="http://farm4.static.flickr.com/3062/2962729923_33a9a01d68.jpg" alt="Designed as Designer" border="0" width="500" height="375" /></a></div>
<br />
<blockquote style="text-align:right;">
"Brooks was fooled."
<br />
– Dick Gabriel
</blockquote>
<p>
In OOPSLA 2007, Fred P. Brooks gave a keynote entitled <em>Collaboration and Telecollaboration in Design</em>. I actually wrote a blog <a href="http://softwareengineering.vazexqi.com/articles/2007/11/03/oopsla-07-collaboration-and-telecollaboration-in-design">entry</a> about it. Overall, I thought that it was a fantastic keynote.
</p>
<p><a href="http://softwareengineering.vazexqi.com/articles/2007/11/03/oopsla-07-collaboration-and-telecollaboration-in-design">Software Engineering Matters : OOPSLA '07: Collaboration and Telecollaboration in Design</a>:
</p>
<blockquote cite="http://softwareengineering.vazexqi.com/articles/2007/11/03/oopsla-07-collaboration-and-telecollaboration-in-design" title="Software Engineering Matters : OOPSLA '07: Collaboration and Telecollaboration in Design"> "There could be multiple architects but there is one <strong>chief</strong> architect who has the authority to make the final decisions. Even in an open source project like Linux, Linus Torvalds still has the ultimate veto power on what to put in and what not to put in. This almost seems like a dictatorship but if that is what is required for conceptual integrity then it is something that has to be done."
</blockquote>
<p>
However, there was one over-simplification that Fred Brooks used; he claimed that all of the most innovative products in the world today are created by one individual (or at most two). As examples, he used Albert Enstein, Thomas Edison and the Wright Brothers (the only group of two that Brooks cited).
</p>
<p>
A couple of people, including Ralph Johnson, pointed out after Brooks's talk that it wasn't necessarily true; in fact, a lot of excellent products today are not created just by one individual. Instead the product usually created by a committee or group and the product would not have been possible without those other individuals.
</p>
<p>
I definitely agree that Brooks might have oversimplified his examples to get his points across. But then again, he isn't really to blame. Society has always ascribed greatness to <em>one</em> particular individual for each accomplishment. That is just how the world works. And though some might say it isn't unfair for just one individual to claim all the credit, it certainly makes more sense to say that Albert Einstein came up with the theory of relativity than saying that Einstein collaborated with Tom, Dick and Harry and they all helped him to come up with that theory. Besides being a mouthful, it is hard to trace which individuals actually influenced Einstein. If you wanted to, you could even optimistically claim that it was the series of all his interactions with different people that enabled him to formulate his theory. But no one does that because while people realize that such works of marvel aren't the works of an individual, it's best to attribute it to the individual who seemed most responsible for it.
</p>
<p>
Most people who have left it as that. However, some people might ponder about it for a few more days. And a small minority might even think about it for a few more weeks. But there is likely to be only one individual who would ponder about it for a whole year and write an essay on it – Dick Gabriel : )
</p>
<p>
So the title of his essay this year is <em>Designed by Designer</em> in which he cites numerous examples (and personal interviews) on why most grand works of art are seldom, if ever, just the work of one careful individual seeing everything from beginning to end. It is insufficient (and unfair) to study the mind of the perceived genius who created that work; instead we need to also study his collaboration with his colleague and how possibly, his work could have just appeared <em>randomly</em>(!) or the work was already there waiting to be discovered and the first person to do so was given the title of genius.
</p>
<p>
For me, Dick's presentation today was a very interesting one. It was definitely well-planned and well-executed in Dick Gabriel's style – it was artsy, akin to watching a National Geographic or Discovery Channel documentary. It wasn't really a counter-argument to Brooks; rather, I see it as a valid alternative perspective to what Brooks had said. After all, no one really knows what is the best way to create such grand works of art (and similarly no one actually knows how to create complex software perfectly each time). There is always the possibility that a small agile and egalitarian team could produce software of equal quality to that of a big group with organization and bureaucracy. And there is always the possibility that a group wrapped with organizational chains can also produce software of equal quality to that of a small agile group.
</p>
<p>
Some groups function better with the <em>illusion</em> of a romantic genius to guide them into unknown territories. Other groups prefer to approach everything as a team without a leader – who needs a leaders after all? we could all just take turns since none of us is infallible – and approach each new challenge as a group.
</p>
<p>
Dick's essay will be available in the OOPSLA proceedings. I will be spending more time reading (and appreciating) it when I get back. There simply isn't enough time during OOPSLA for deep, contemplative reading.
</p>
OOPSLA '08: Smalltalk Superpowers2008-10-20T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/20/oopsla-08-smalltalk-superpowers<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2960998338" title="View 'Smalltalk Superpowers' on Flickr.com"><img src="http://farm4.static.flickr.com/3252/2960998338_d1325d59bf.jpg" alt="Smalltalk Superpowers" border="0" width="500" height="375" /></a>
<br/>
<small>It seems like John (in the green shirt) is about to use some of his superpowers and fly off into the screen....</small>
<br />
</div>
<p>
<a href="http://smalltalk-superpowers.org/">Smalltalk Superpowers</a> has to be one of the better workshops at OOPSLA that I have attended - well, I have only attended two so far so that doesn't say too much. What I liked about the workshop was the abundance of actual code (mostly short snippets) that were actually used in real systems. There wasn't the usual nonsense hand-waving; on the other hand we saw actual code that people wrote and had used. Some of the code snippets were beautiful, some were atrocious, and some were just too damn heretical that they should remain as skeletons in the closet. However, all of them had a purpose (though some were rather questionable and were just used to demonstrate the Smalltalk-hackery-kung-fu).
</p>
<p>
I was already familiar with most of the main concepts that were used but I lacked the ingenuity to use those concepts as others had done. The usual suspects include using <tt>become:</tt>, <tt>changeClassTo:</tt>, <tt>doesNotUnderstand:</tt> and the <tt>thisContext</tt> variable. So I could easily grasp the main ideas behind those superpowers even though I haven't attempted trying them for myself (I have used <tt>doesNotUnderstand</tt> and <tt>thisContext</tt> before though).
</p>
<p>
Many of the code snippet aren't actually published (for good reason too) since you really don't want to write such code. However, they are useful nonetheless for those dire situations that require a quick throw-away solution.
</p>
<p>
There were a couple of Javascript, Self and Ruby programmers in the group so we had a good mix of different superpowers. Michael Lucas-Smith made an interesting observation about the state of doing "meta-programming" in Smalltalk and how it differs from what the Ruby (and Rails) people were doing -- <strong>transient generated code</strong> where we can always return to a <em>clean</em> slate and remove all the generated code from the image (or program). Smalltalk doesn't really encourage such techniques (see the <em>Dynamic Code Generation</em> section of my article on <a href="http://softwareengineering.vazexqi.com/articles/2008/05/28/dwemthys-array-in-smalltalk">Dwemthy's Array</a>) and it will be nice to see what superpowers people will acquire once Smalltalk makes this easier to do. He suggested something simple like marking/annotating the code that has been generated (I just put mine in a category called <em>generated</em>) and being able to completely clear it when the system is saved/loaded (so it can start from a clean slate). The code should remain for <em>some</em> duration so that we can use the existing Smalltalk browser tools to navigate through the code for testing and debugging.
</p>
<p>
I only stayed for the morning session but I picked up enough little tricks to add to my bag of tricks. And there is enough interesting things that I should be able to play around for some time.
</p>
<p>
Goran was doing a video recording of the workshop and that should be on the web some time soon. <strong>update:</strong> The links to the torrent are already up on his <a href="http://goran.krampe.se/blog/OOPSLA/OOPSLA08.rdoc">site</a>. The videos (albeit at lower quality) is also available from <a href="http://www.veoh.com/videos/v163138695pJEMGmk">veoh</a>. Thanks Goran!
</p>
OOPSLA '08: Workshop on Refactoring Tools2008-10-19T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/19/oopsla-08-workshop-on-refactoring-tools<p>
We spent the majority of the day at the Workshop on Refactoring Tools (WRT) here at OOPSLA. The papers are all available from the <a href="https://agora.cs.uiuc.edu/display/wrt08">WRT '08 wiki</a> and will soon be in the ACM digital library as well.
</p>
<div style="text-align:center;">
<a href="http://www.flickr.com/photos/37092324@N00/2956781457" title="View '' on Flickr.com"><img src="http://farm4.static.flickr.com/3236/2956781457_9d2794aa49.jpg" alt="" border="0" width="500" height="334" /></a>
<br />
From left to right: Michael Klenk , Bill Opdyke and Bob Fuhrer. Photo taken by Munawar Hafiz : )
</div>
<p>
Here are some of the highlights of the workshop:
<ul>
<li><strong>Writing refactorings for IntelliJ </strong><br />
AFAIK, IntelliJ IDEA has the most comprehensive support for refactoring across Java, the popular JVM languages (Scala, Groovy, Ruby) and Java frameworks. This doesn't mean that it is bug free but it is way ahead of other popular IDEs like Eclipse and NetBeans in providing comprehensive support that allows you to take <em>refactoring for granted</em> – your program with multiple components (including those not in Java) will continue to work after the refactorings.
<br /><br />
To achieve this, all language plug-ins in IntelliJ utilize the same representation: Program Structure Interface (PSI). Having this uniform representation allows refactoring across languages and frameworks to be expressed easily without the need for a lot of adapting/wrapping. For instance, it is easy to add a <em>link</em> from a node that uses a variable to the original node that defines that variable (even if it spans different languages). This makes it easier to perform the analysis for a refactoring since every node has the same interface.
<br /><br />
This differs from how Eclipse does it – or rather how Eclipse doesn't do it. Eclipse doesn't impose a uniform program structure representation for plug-in writers. Therefore, you get a myriad of different representations and this makes it hard to actually integrate tools for different languages together. Which is what made my technique on extending refactoring support across Java and XML slightly more complicated – and in the end feel hackish in some way.
</li>
<li> <strong>Better UI for refactoring</strong> <br />
Dustin Campbell from Developer Express/ Microsoft presented their commercial tool for Visual Studio that improves the usability of refactoring tools. It does this via two main methods: <strong>detection</strong> and <strong>preview</strong>. It is able to detect and suggest pertinent refactorings based on the current position of the cursor. And it is able to perform those refactorings <em>live</em> inside the current editor without forcing you to open an external dialog (like what Eclipse, IntelliJ and NetBeans does).
<br /><br />
There was also some discussion about the current state of refactoring tools and how people just aren't using them. So even if we successfully implement our refactoring tool, it's not going to be useful since no one is actually using it. One approach to solve this problem would be to suggest to the user about the possibility of doing such a refactoring and actually showing the results of performing the refactoring in a user-friendly manner.
<br /><br />
I am not going to put any picture up since it is copyrighted but you can find some screenshots in his paper<a href="https://agora.cs.uiuc.edu/download/attachments/17474131/WRT_DesigningRefactoringToolsForDevelopers.pdf"> here</a>.
</li>
<li> <strong>Eclipse Meta-Tooling Platform (IMP)</strong> <br />
There wasn't really a demo of <a href="http://www.eclipse.org/imp/">IMP</a> but it was used to implement the Domain-Specific Environment (DSE) for SIAL Parallel Computational Chemistry language.
<br /><br />
Using IMP enabled Beverly Sanders and her group to quickly create an IDE for the SIAL language and include refactoring support to optimize the program. What impressed me the most is the ability to quickly create a usable DSE with minimal effort.
<br /><br />
JetBrains also has similar system called <a href="http://www.jetbrains.com/mps/index.html">Meta Programming System</a> that I intend to investigate as well.
<br /><br />
As a side note, I wanted to download and try out IMP immediately so that I can talk to Bob Fuhrer who is one of the program committee members for WRT and also one of the developers of IMP. Unfortunately, the convention center for OOPSLA this year is rather ill-equipped. There is no free wifi in the individual conference rooms (for that you would have to pay $10 a day); there is an official free OOPSLA wifi in the atrium at the bottom floor.
</li>
</ul>
</p>
<p>
Overall it was an interesting experience and I did learn a lot of interesting things from talking to the different participants. There are definitely plans to have another workshop on refactoring tools next year.
</p>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2957460912" title="View 'IMG_0501' on Flickr.com"><img src="http://farm4.static.flickr.com/3286/2957460912_66a00d1534.jpg" alt="IMG_0501" border="0" width="500" height="375" /></a>
<br />
<small>Dinner with some of the participants of the workshop. Hoping to see them again in the next WRT.</small></div>
OOPSLA '08: On our way2008-10-17T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/17/oopsla-08-on-our-way<div style="text-align:center;"><img src="http://www.oopsla.org/oopsla2008/templates/oopsla2008-2/images/header-oopsla2008_conference_name2.png" alt="OOPSLA 2008 Logo"/></div>
<br />
<p>
So, we are headed off to <a href="http://www.oopsla.org/oopsla2008/">OOPSLA</a> again this year in the famous <strike>"Ralph Johnson Van"</strike>. There was some mix up at the car rental company so we ended up with one car (the white one in the back of the photo below) and Ralph's own mini van. Our drive will take about 6 hours.
</p>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2951127214" title="View 'OOPSLA Mini Van' on Flickr.com"><img src="http://farm4.static.flickr.com/3216/2951127214_51ceb21833.jpg" alt="OOPSLA Mini Van" border="0" width="500" height="375" /></a>
<br />
<small>We had to use the "Ralph Johnson Mini-van" instead.</small>
</div>
<br />
<p>
This time around, I have submitted a <a href="https://netfiles.uiuc.edu/nchen/www/wrt08.pdf">short paper</a> and <a href="https://netfiles.uiuc.edu/nchen/www/wrt08Poster.pdf">poster</a> for <a href="http://refactoring.info/WRT08">WRT: Workshop on Refactoring Tool</a>s. In that paper, I present the need for supporting cross-language refactorings – specifically across Java and XML - since many different popular frameworks already use both languages and our current tool doesn't support that very well. I then present our technique for solving this problem for the common cases that programmers will encounter. Our solution (and tool) hooks into the Eclipse refactoring participant system and <em>participates</em> every time a refactoring is done in Java; we then propagate a suitable refactoring to the XML side.
</p>
<p>
I hope I'll be able to continue with the blogging that I did with OOPSLA '07 this year around too. My friend Maurice won't be joining the group this year so he'll be feeding off what I write : )
</p>
<p>
I'm particularly looking forward to some Smalltalk events (<a href="http://news.squeak.org/2008/10/07/squeak-bof-at-oopsla-2008/">Squeak BOF</a> and <a href="http://smalltalk-superpowers.org">Smalltalk SuperPowers</a>) this year especially after the <a href="http://news.squeak.org/2008/10/11/smalltalk-is-cool-again-says-gartner/">resurgence</a> of <a href="http://blogs.gartner.com/mark_driver/2008/10/09/remember-smalltalk/">interest</a> in Smalltalk.
</p>
Reflections|Projections '08: Keep Your Shoes On2008-10-05T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/05/reflections-projections-08-keep-your-shoes-on<p>
<a href="http://pragdave.pragprog.com/">Dave Thomas</a>, of Ruby and Pragmatic Programmers fame, was invited as one of the speakers for the Reflection | Projections conference this year. Ralph had already set up plans for lunch with him a few days ago and a few of us tagged along. It was great to actually meet Dave in person. He was very friendly and it was obvious that he is definitely a very <em>pragmatic programmer</em>. Over lunch, we talked about programming (naturally), Perl 6 (from the talk the night before), writing small maintainable programs, concurrent programming, cell phone development on the iPhone and Google's Android phone and GPS devices.
</p>
<a href="http://www.flickr.com/photos/37092324@N00/2915615973" title="View 'Dave Thomas at the ACM Reflections | Projections 2008 Conference' on Flickr.com"><div style="text-align:center;"><img src="http://farm4.static.flickr.com/3099/2915615973_c03f98fd85.jpg" alt="Dave Thomas at the ACM Reflections | Projections 2008 Conference" border="0" width="500" height="357" /></div></a>
<p>
Dave was scheduled to give a talk that night on "keeping your shoes on". He chose not to use any slides; instead he sat in the middle of the stage and talked directly to the audience. He started off with this point: every time we fly now, we have to take off our shoes. We have to take off our shoes whether we like it or not. And we have to take them off even if there isn't any real evidence on how doing so will help prevent anything. We just take them off because we are motivated by a false sense of security.<strong> We are doing it just for the sake of doing it</strong>. And we just accept it.
</p>
<p>
He then compared this to how some companies do software nowadays. Many companies are scared to change how they do things even if the current way of doing things is not working. They would rather believe the abundance of rumor out there about how a new technique couldn't actually succeed instead of trying the technique out for themselves and evaluating the result. His main example was about Ruby and Enterprise Java. Many developers initially did not believe that you could write <em>enterprise</em> programs with Ruby; you could only do that with the popular statically-typed languages and their humongous frameworks and libraries. Then, Rails comes along and shows that there is a viable alternative. Rails shows how it is possible to rely on a dynamic language like Ruby and a small framework to get the job done.
</p>
<p>
But because of <a href="http://en.wikipedia.org/wiki/Fear,_uncertainty_and_doubt">FUD</a>, many people are paralyzed with fear and never bother evaluating alternative solutions. Instead they would rather stick with the monstrous "tried-and-true" solutions even though something better (and cheaper) is just around the corner. And it isn't necessary the case that their tried-and-true solutions would even work in the first place.
</p>
<p>
In his talk, he alluded to the <a href="http://www.wowzone.com/5monkeys.htm">monkey, ladder, banana and water spray</a> <em>experiment</em>. I couldn't actually find any solid references to such an experiment so I suspect that it is just a story that someone made up. It is an interesting anecdote nonetheless. You can read the story but the gist of it is that sometimes we just accept a lot of things without actually understanding the logic behind it. In other words, we just choose to accept (and maintain) the status quo without good reasons; <em>very similar to how religions work -- blind faith</em>.
</p>
<p>
He summed it up best by referring to the importance of <em>courage</em> in eXtreme Programming. In <a href="http://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0201616416"><em>eXtreme Programming eXplained</em></a>, Kent Beck makes it clear that all the principles of XP work together to alleviate the fear of change. For instance, the fear of breaking existing code through refactoring can be alleviated by having automated tests. That way, when you refactor something you can always run your tests to check if anything is broken.
</p>
<p>
Dave wasn't advocating that we ignore our fears and just rush ahead blindly with software development. Instead he was advocating that we be cognizant of those risk and take appropriate measures to address them. We shouldn't let those risks dictate how we develop software, we should instead evaluate those risks and see how we can change to alleviate them. Developing software is inherently risky. But as long as we keep those risk in-check, we can proceed to create great software that developers like writing/maintaining and that customers will enjoy using.
</p>
<p>
Dave wrote something similar on his <a href="http://pragdave.pragprog.com/pragdave/2005/07/rails_and_fud.html">blog</a> and also gave a <a href="http://vodpod.com/watch/308809-dave-thomas-on-rails-fud-and-terrorism">talk</a> about this before. So you can actually peruse those two resources while the video recordings for the Reflections | Projections are <a href="http://www.acm.uiuc.edu/conference/2008/videos">being put up</a>. In fact, halfway through his talk, the microphones stopped working so this particular talk might not have been recorded properly.
</p>
Reflections|Projections '08: Studies in the Ballistic Arts2008-10-03T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/10/03/reflections-projections-08-studies-in-the-ballistic-arts<p>
The ACM chapter at UIUC is hosting the annual <a href="http://www.acm.uiuc.edu/conference/2008/">Reflections | Projections</a> conference again this year. And like every year before, we have a impressive selection of speakers for the event.
</p>
<p>
<a href="http://www.wall.org/~larry/">Larry Wall</a>, the creator of Perl, was speaking on the <a href="http://www.acm.uiuc.edu/conference/2008/speakers#LarryWall">opening night</a>. I have heard (or maybe read) about him and how he is a funny and interesting person. His talk tonight definitely confirmed that. He even did a little dance as he came in. I don't know the name for that dance, but it is the one where you would hold a bar horizontally between your hands as you skipped across the stage.
</p>
<p>
The first half of his talk was all about the philosophy of Perl. His term for this was that the process of creating Perl was a "ballistic art" (hence, the title for the talk). His best metaphor for ballistics was the game of golf, where you hope to have the ball go into the hole but more often than not, the ball moves around in weird ways that don't always take it closer to the hole. And to him that is how designing a language is. Designing a language, is almost like a weird science that does not have definite steps. While trying to add or remove a feature you are bound to please some people and displease other people.
</p>
<p>
He took a few stabs at Python and Ruby but they were hilarious nonetheless and who could blame him. He even said that he was surprised that even though Lisp is the most beautiful language in the world, every big Lisp program out there just looks ugly. This is obviously meant as a joke though I personally think that there is truth to this (at least on the syntax level).
</p>
<p>
In the second half of his talk, he went through the features of Perl 6 -- but he cleverly avoided setting a definite release date for it :-) He even joked that it would take 40 - 8 more years. 40 being the number of years that the Israelites had to be spend in the dessert before reaching the promised land. And 8 being how he has been working on Perl 6. I don't really use Perl but I have seen it enough that his examples made sense. He started off with the easier/common Perl constructs and showed how some of them are going to be standardized in Perl 6.
</p>
<p>
One thing that bugged me about Perl (and this is also present in some extend in Ruby) is the use of sigils to denote the characteristic of variables. For instance $ are for scalar values, @ for arrays and % for dictionaries. Maybe it just takes some getting used to; perhaps once you have used it long enough, you would embrace the philosophy. But anyway, it seems that Perl 6 is going to make the use of sigil more standardized so there should be less confusion.
</p>
<p>
Now, I have never really dwelled too much into the internals of Perl 6 but I have heard rumors about how bloated it is. And the more advanced features that Larry showed toward the end of his talk seemed to confirm my suspicions. I mean the features are really cool but they seem to contribute to a rather bloated language. And all these features are defined at the language level -- meaning that the core of the language has to support those features. And there are a <strong>lot</strong> of features to support....
</p>
<p>
At the other extreme end of this spectrum would be languages like Smalltalk and Lisp where the core of the language is simple but it is powerful enough to allow you to define the other constructs of the language using that core. And at the middle of the spectrum would the languages like Java and C# that most programmers are familiar with.
</p>
<p>
This means that maintaining/evolving the language later could be harder since once you have committed to something as part of the language, you generally have to support it in future versions. While thinking about this, I am reminded of the fact of how Fortran has evolved to the beast that it is today. It is really bloated because it has accumulated so much craft and never bothered to remove any because it wanted to be <em>fully</em> backwards-compatible.
</p>
<p>
Anyway, from the talk, it was obvious that Larry was passionate about Perl. He created Perl to fill the niche between C and shell scripting in Unix. And now it has evolved to a language that is being used by many different people. Notice that I use the term people instead of programmers because Perl has this appeal that even normal people can use it. They might not write the best code (heck, they might not even know what a function is the way based on the way that functions work in Perl) but they can definitely get stuff done with Perl with its low entry barrier.
</p>
<a href="http://www.flickr.com/photos/37092324@N00/2910773065" title="View 'Where Perl sits' on Flickr.com"><div style="text-align:center;"><img src="http://farm4.static.flickr.com/3081/2910773065_69eaefe99b.jpg" alt="Where Perl sits" border="0" width="323" height="262" /></div></a>
<br />
<small>Larry had an interesting slide on where Perl fit into the Unix World. I am pretty sure I spelt the words on the axes.</small>
<p>
So we'll just have to wait to see when Perl 6 comes out to see if it continues with its philosophy of manipulexity + whip-up-titude.
</p>
Eclipse Ganymede Review Follow-up2008-08-15T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/08/15/eclipse-ganymede-review-follow-up<p>
About two months ago, I wrote my review for Eclipse 3.4 Ganymede and participated in the <a href="http://www.eclipse.org/ganymede/aroundtheworld.php">Ganymede Around the World Contest</a>. As one of the first few reviewers, I am going to receive a Eclipse shirt. I have not seen the shirt yet but I have already provided my mailing address to them.
</p>
<p>
Also, I was fortuitous enough to be randomly selected as one of the 5 participants to receive an Eclipse jacket. So I'll be looking forward to that as well.
</p>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/vazexqi/2766023504/" title="Eclipse Debugger Screencast by vazexqi, on Flickr"><img src="http://farm4.static.flickr.com/3012/2766023504_06a86b099e_m.jpg" width="240" height="230" alt="Eclipse Debugger Screencast" /></a><br />
<small>Eclipse Java Debugger screencast</small></div>
<p>
Anyway, as a follow-up, I said that I will be releasing a screencast to showcase some of the new features once the official version of Ganymede has been released. The first (I am hoping that I wlll have time to create more) has been released on the CS 427 Fall 2008 wiki page: <a href="https://agora.cs.uiuc.edu/display/cs427fa08/Eclipse+Debugger">Eclipse Debugger</a>. It shows the new <i>Enhanced Debug Hover</i> feature introduced in Eclipse 3.4. The screencast is catered toward students who have never used a debugger in an IDE before (I might be a bit presumptuous on this...)
</p>
<p>
Hopefully someone else might find the screencast useful.
</p>
Eclipse Ganymede Features Worth Talking About2008-06-05T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/06/05/eclipse-ganymede-features-worth-talking-about<p>
In Fall 2008, the Software Engineering I class at UIUC would make extensive use of the Eclipse 3.4 Ganymede as part of their class projects. In the beginning of the semester, students will start out by familiarizing themselves with Eclipse as <em>one of the best</em> tools for Java development. After familiarizing themselves with Eclipse as a Java IDE, they will then be using Eclipse as a development platform for creating their own plug-ins. Specifically, students will be contributing plug-ins for the <a href="http://www.eclipse.org/photran/">Photran IDE</a>. Eclipse is a big ecosystem – <a href="http://www.eclipse.org/org/press-release/20070627_europarelease.php">up to 17 milion lines of code</a> – so it makes an appropriate exposure for students so that they get some experience working on a large code base compared to some small and unrealistic assignment.
</p>
<p>
We followed a similar plan last year as well but there are some features of Eclipse 3.4 Ganymede that I think will help make the process a little simpler this time around. Eclipse has definitely been growing extensively – some might even argue that it is extremely bloated right now. However, each release has been getting consistently better with new features that are making it more and more useful as <em>the single</em> IDE for different programming languages, frameworks and platforms. IntelliJ IDEA still has some advantages over it but Eclipse is certainly catching up.
</p>
<p>
So here are my <strong>top eight</strong> features that I like about Eclipse and why they are useful from the point of view of a teaching assistant for a software engineering class. All pictures are based on the Photran project since that is the current project that I have which has been ported to Eclipse 3.4. I gleaned the list of features from the <a href="http://download.eclipse.org/eclipse/downloads/drops/S-3.4M7-200805020100/eclipse-news-M7.html">milestone pages</a> since the official documentation for Eclipse 3.4 is not out yet.
</p>
<h2>1. Plug-in Spy</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2555386550" title="View 'Plug-in Spy' on Flickr.com"><img src="http://static.flickr.com/3193/2555386550_c133a5bb5c.jpg" alt="Plug-in Spy" border="0" /></a><br />
<small>The plug-in spy running inside Photran itself; it lets you identify which plug-in contributed to the UI feature via a simple point and click.</small>
</div>
<br />
<p>
Those of us who are accustomed to the debug halos in Squeak now have a similar – albeit less functional – version of that in Eclipse 3.4 called the <em>plug-in spy</em>. By using the keyboard shortcut Shift + Opt + F1 you can click on any view, dialog, editor or preference pane and identify which plug-in contributed that functionality.
</p>
<p>
This makes it much <em>simpler</em> to actually investigate and explore the plug-in system of Eclipse. On previous versions of Eclipse, you had to actually browse the documentation or examine the plugin.xml file to see which plug-in contributed that feature. Now all you have to do, for the most part, is point-and-click.
</p>
<p>
This is the first version of the plug-in spy and it will definitely improve over time. Right now, it doesn't let you inspect the buttons on the toolbars or the menu items in the menubar.
</p>
<h2>2. Extension Point Renaming</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2555386630" title="View 'Extension Point Renaming' on Flickr.com"><img src="http://static.flickr.com/3040/2555386630_5ce4d82fe5.jpg" alt="Extension Point Renaming" border="0"/></a>
<br />
<small>Renaming an extension point in your plug-in.</small>
</div>
<br />
<p>
This is a simple feature but it will encourage students to rename their extension point to something more descriptive as they evolve their plug-in projects. Previously, if you changed the name of an extension point, you had to find all references of it yourself and then manually rename them.
</p>
<h2>3. Breadcrumb Navigation in the Java Editor</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2555386680" title="View 'Breadcrumb navigation in the Java editor' on Flickr.com"><img src="http://static.flickr.com/3122/2555386680_3cdc83c515.jpg" alt="Breadcrumb navigation in the Java editor" border="0" /></a>
<br />
<small>Save precious screen estate with the breadcrumb navigation in the Java Editor.</small>
</div>
<br />
<p>
In the class, most students work from their 13 - 15 inch laptops. So anything that can save some screen estate in Eclipse will definitely be helpful. With the breadcrumb navigation, you no longer need to dedicated precious screen estate to the Package Explorer and Outline views – both are now consolidated into the breadcrumb navigation bar. You can easily access the project and package structure as well as the individual classes, fields and methods from the bar itself.
</p>
<h2>4. Enhanced Debug Hover</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2555386730" title="View 'Enhanced Debug Hover' on Flickr.com"><img src="http://static.flickr.com/3036/2555386730_a5ce00f348.jpg" alt="Enhanced Debug Hover" border="0" /></a>
<br />
<small>View the value of a variable right from the editor itself!</small>
</div>
<br />
<p>
Students don't really know how to use debuggers. In fact, most of the think of debugging as a chore that is only performed as a last resort. But the debugger is not only for fixing bugs. It's a great tool for learning, experimenting and reverse engineering how your program works.
</p>
<p>
Though Eclipse already provides wonderful support for Java debugging (there is no need to jump into the console and use jdb), students don't use the debugger enough. So any improvement, no matter how small, to the debugger to make it more friendly is definitely an advantage.
</p>
<p>
In Eclipse 3.4, you can now hover over a variable and have its value pop-up in a dialog box all from the editor view itself. Previously, you had to go to the Variables view to examine the values of variables. It's just a step closer to the editor now.
</p>
<p>
If you have used Xcode before, then you immediately notice the similarities except that the debugger in Eclipse has many other interesting features that Xcode lacks.
</p>
<h2>5. Import/Export Launch Configurations</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2554562529" title="View 'Import/Export Launch Configurations' on Flickr.com"><img src="http://static.flickr.com/3098/2554562529_3a2903ffa0.jpg" alt="Import/Export Launch Configurations" border="0" /></a>
<br />
<small>Share your launch configurations with your team.</small>
</div>
<br />
<p>
Last year, we had to write out a step-by-step guide on how to setup a proper launch configuration for Photran. It was tedious and students tend to make mistakes while following the instructions.
</p>
<p>
This time, there is no need for such meticulous instructions. All the teaching staff needs to do is export the launch configurations and have the students import it. And we have just eliminated one <a href="http://en.wikipedia.org/wiki/Accidental_complexity">accidental complexity</a> from the assignment.
</p>
<h2>6. Eclipse Communication Framework (ECF) Bundled</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2554562557" title="View 'Eclipse Communication Framework (ECF) Bundled' on Flickr.com"><img src="http://static.flickr.com/3141/2554562557_3f3099339b.jpg" alt="Eclipse Communication Framework (ECF) Bundled" border="0"/></a>
<br />
<small>Now you can communicate with you group from within Eclipse itself.</small>
</div>
<br />
<p>
<a href="http://wiki.eclipse.org/Eclipse_Communication_Framework_Project">ECF</a> is now bundled with the RCP release of Eclipse. It allows you to have IM, IRC and BitTorrent (?) from within Eclipse itself. We have a couple online students taking the software engineering class and such features will make it easier for them to do pair programming and collaborative development over the web.
</p>
<p>
The roadmap for ECF is pretty ambitious <http://wiki.eclipse.org/ECF_Ganymede_Roadmap#Development_Themes> with hopes of supporting collaborative editing – all from within Eclipse itself. There is no definite timeline for those features yet but it seems that they are definitely being planned. There is definitely support for collaborative real-time editing now but I am not sure if the final features will be incorporated into the final release of Eclipse Ganymede.
</p>
<p>
I can't show you my IM list but at least I can link to an image from the ECF wiki showing the Contacts view and the supported features:
</p>
<div style="text-align:center;">
<a href="http://wiki.eclipse.org/Screen_Captures_over_IM"><img src="http://wiki.eclipse.org/images/a/a1/Scap1.png" border="0"/></a>
</div>
<br />
<h2>7. Highlight Read and Write Occurrences</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2554563555" title="View 'Read and Write Occurrences' on Flickr.com"><img src="http://static.flickr.com/3130/2554563555_4b39b660da.jpg" alt="Read and Write Occurrences" border="0" /></a>
<br />
<small>Now you can see where the variables are being read or written to.</small>
</div>
<br />
<p>
This is another small feature but it definitely makes it much easier to see where variables are being read/written easily. Previously, Eclipse offered the ability to highlight all occurrences of a variable but now it distinguishes between read and writes for you.
</p>
<p>
This is much more effective than having to do a text search of the variable and then jumping back and forth between each location like what you have to do in other <em>less-developed</em> IDEs.
</p>
<h2>8. JUnit Execution Times</h2>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2554563585" title="View 'JUnit Execution Times' on Flickr.com"><img src="http://static.flickr.com/3072/2554563585_23ea6c8148.jpg" alt="JUnit Execution Times" border="0"/></a>
<br />
<small>Ever wondered how long your tests take to run? Well, wonder no more since the times are now displayed right in front of you.</small>
</div>
<br />
<p>
Again another simple feature but this feature makes it really easy to eyeball if something is wrong with your tests. This is an easy way to check if your test are suffering from the <a href="http://xunitpatterns.com/Slow Tests.html">slow tests</a> smell.
</p>
<hr/ >
<br />
<p>
Eclipse has certainly come a long way in terms on the features it offers. As someone <a href="http://softwareengineering.vazexqi.com/articles/2008/03/31/better-code-browsing-in-squeak#comments">commented</a> on one of my previous articles about the OmniBrowser in Squeak, Eclipse definitely has a more polished and modern feel to it. Just as Eclipse has been learning and improving from the Smalltalk tools (most of the features that Smalltalk revolutionize are now part of Eclipse), Smalltalk implementations could also learn a thing or two from Eclipse. Eclipse has definitely set the bar high for competing IDEs.
</p>
Dwemthy's Array in Smalltalk2008-05-28T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/05/28/dwemthys-array-in-smalltalk<p><a href="http://c2.com/cgi/wiki?RubyIsSmalltalkMinusMinus">Ruby Is Smalltalk Minus Minus</a>:
</p>
<blockquote cite="http://c2.com/cgi/wiki?RubyIsSmalltalkMinusMinus" title="Ruby Is Smalltalk Minus Minus"> "--- Challenge to Smalltalk advocates: go ahead and reimplement DwemthysArray? then! The Array is a rather short program so it shouldn't take you too long. It's also rather well known, even celebrated, by now so a ST version will reach many people who wouldn't normally even notice ST advocacy. And an elegant implementation will frankly be a lot more convincing than 'sure, we could do that' assertions. We've been promised Smalltalk DWEMTHY before: http://redhanded.hobix.com/inspect/theRabbitWillDieInSmalltalk.html - who will deliver?"
</blockquote>
<br />
<p>
<strong>BIG DISCLAIMER UPFRONT: This is just an exercise and I am not advocating Smalltalk over Ruby or Ruby over Smalltalk. Both have their strong points. I am just doing this out of curiosity and for fun.</strong>
</p>
<p>
I was browsing around the web and glanced upon the snippet above from the c2 wiki. I am not sure how long it has been up there (probably since 2005) but it seems that no one has responded to it. For those unfamiliar with it <a href="http://poignantguide.net/dwemthy/">Dwemthy's Array</a> it's a Ruby meta-programming example that is along the lines of a text-based game. I was familiar with it when it first surfaced but never thought of the need to implement it in Smalltalk since the approach is not very Smalltalk-ish in the first place. However, since I had some time, I decided to explore how it could be done in Smalltalk by mimicking the Ruby implementation.
</p>
<div style="text-align:center;"><a href="http://www.flickr.com/photos/37092324@N00/2533337888" title="View 'Dwemthy's Array in Smalltalk' on Flickr.com"><img src="http://farm3.static.flickr.com/2062/2533337888_b92a147f46.jpg" alt="Dwemthy's Array in Smalltalk" border="0" width="500" height="313" /></a>
<br />
<small>A sample run of Dwemthy's Array in VisualWorks. Look at the Browser to see how I mimic the Ruby code. Notice that I still need to explicitly write the <tt>super initialize</tt>.</small>
</div>
<br />
<p>
The gist of the Ruby implementation of Dwemthy's Array is the ability to write code like the following to declare a new class. It's simple and it reads like a mini Domain-Specific Language. Because of this, it also looks very appealing aesthetically.
</p>
<code>
<pre>
class ScubaArgentine < Creature
life 46
strength 35
charisma 91
weapon 2
end
</pre>
</code>
<p>
Basically it can be "de-sugared" to look like this which would make it more <em>familiar</em>. Basically the entire code relies on doing class method calls (the <tt>self.life</tt>) to create
</p>
<code>
<pre>
# This is a comment
class ScubaArgentine < Creature # This declared ScubaArgentine as a subclass of Creature
self.life(46) # This is just a call to the class method life i.e. Creature.life(46)
self.strength(35)
self.charisma(91)
self.weapon(2)
end
</pre>
</code>
<p>
The gist of how all of this is done is in the following snippet:
</p>
<code>
<pre>
# Advanced metaprogramming code for nice, clean traits
def self.traits( *arr )
return @traits if arr.empty?
# 1. Set up accessors for each variable
attr_accessor *arr
# 2. Add a new class method to for each trait.
arr.each do |a|
metaclass.instance_eval do
define_method( a ) do |val|
@traits ||= {}
@traits[a] = val
end
end
end
# 3. For each monster, the `initialize' method
# should use the default number for each trait.
class_eval do
define_method( :initialize ) do
self.class.traits.each do |k,v|
instance_variable_set("@#{k}", v)
end
end
end
end
# Creature attributes are read-only
traits :life, :strength, :charisma, :weapon
</pre>
</code>
<p>
<em>In my opinion, the original article on Dwemthy's Array doesn't actually explain how things work clearly -- the author goes through great lengths to make it seem <em>magic</em> and the prosy language doesn't help clarify how it works underneath</em>. Unfortunately, if I were to dissect it line by line it would make this article extremely long. So, instead, anyone interested in a better explanation so watch this <a href="http://www.infoq.com/presentations/metaprogramming-ruby">video presentation</a> or refer to the <a href="http://www.pragprog.com/titles/ruby/programming-ruby">Pickaxe Ruby Book</a> or the <a href="http://www.amazon.com/Ruby-Way-Second-Addison-Wesley-Professional/dp/0672328844">Ruby Way book</a>.
</p>
<p>
In the following paragraphs, I am going to focus on what I think are the most fascinating issues of the implementation and how it could be done in Smalltalk. The following paragraphs assume familiarity with Ruby and Smalltalk reflection/code generation. This <a href="https://www.iam.unibe.ch/scg/svn_repos/Lectures/Smalltalk/10Reflection.ppt">lecture slide</a> from the University of Bern gives a quick refresher on Smalltalk reflection. My implementation is a <em>direct-translation</em> as close as I know how to implement it. So that means that I will be sticking to code generation, using class instance variables and a lot of class methods.
</p>
<h3>Dynamic code generation</h3>
<p>
Ruby offers method such as <tt>instance_eval</tt> and <tt>class_eval</tt> to make code generation slightly simpler. In addition, it also offers <tt>attr_accessor</tt>. Smalltalk, on the other hand, does code generation by using the <tt>compile: 'some smalltalk code'</tt> method. Code generation might be a misnomer since that is basically how <em>everything</em> is defined into the image.
</p>
<p>
The <tt>compile: 'some smalltalk code'</tt> is used behind the scenes in Smalltalk for generating accessors and doing other refactorings. I have not actually seen the <tt>instance_eval</tt> and <tt>class_eval</tt> used to do any refactorings.
</p>
<p>
Code generated in Ruby is also <em>hidden</em> meaning that you are not able to see the actual source/implementation of the generated method. This makes things more terse but can make understanding/debugging the program harder -- it really makes things much simpler if you could actually see the methods that are generated.
</p>
<p>
<code>
<pre>
define_method( someName ) do |val|
@traits ||= {}
@traits[someName] = val
end
</pre>
</code>
actually generates the following simple code:
<code>
<pre>
def someName(val)
@traits ||= {}
@traits[someName] = val
end
</pre>
</code>
But you never get to see that code!
</p>
<p>
Every generated code in Smalltalk (using the <tt>compile: 'some smalltalk code'</tt>) appears in the Code Browser like every other method, which brings me to the next point...
</p>
<h3>File-based vs. image-based implementation</h3>
<p>
Because Ruby operates in a file-based manner, it <strong>re-initializes the meta-programming facility</strong> every time you reload the file. For this form of programming, reloading the file has some advantages -- if you screw up you can just easily reload everything and not worry about side effects.
</p>
<p>
As I was doing the implementation in Smalltalk, I screwed up a couple of times and had to remove some of the generated code. Rather than unloading everything and loading it again (tedious and not very Smalltalk-ish) I implemented a <strong>reset</strong> method that removes all the dynamically generated code and resets other meta-programming constructs. So using my reset method, I can simulate starting from a clean state each time for testing purposes.
</p>
<p>
Additionally, one solution that I used to simulate file-based loading is to override the <tt>initialize</tt> method on the class side so that some action is performed as the code is loaded into the image.
</p>
<p>
In a nutshell, it is possible to simulate a file-based loading by writing proper reset and reinitialize methods. These methods definitely come in handy as I was doing this exercise since I don't need to unload/load the parcel in each time -- I did do that for the final testing to ensure that everything is working.
</p>
<h3>Dynamically Adding Instance Variables</h3>
<p>
In Ruby it seems that you can just <em>define</em> a new instance variable using <tt>@some_name</tt> or <tt>instance_variable_set("@some_name, some_value)</tt>and it will immediately be picked up. In Smalltalk, I had to ensure that I declared a new instance variable first using <tt>self addInstVarName: 'someName'</tt> and then I can change its value using <tt>self instVarNamed: 'some_name' put: some_value</tt>.
</p>
<p><a href="http://www.rubyist.net/~slagell/ruby/instancevars.html">Instance variables</a>:
</p>
<blockquote cite="http://www.rubyist.net/~slagell/ruby/instancevars.html" title="Instance variables"> "Instance variables do not need to be declared. This indicates a flexible object structure; in fact, each instance variable is dynamically appended to an object when it is first assigned."
</blockquote>
<p>
This two-step approach in Smalltalk hints are the differences on the underlying layer on how instance variables are stored.
</p>
<h3>Class Instance Variables</h3>
<p>
Another important (and possibly subtle) point about the approach is the use of <a href="http://www.martinfowler.com/bliki/ClassInstanceVariable.html"><em>class instance variables</em></a>. Class instance variables offers the ability of class variables <em>without</em> the problems that come about from subclassing. This is important because it allows classes to share common variables without interference from the class hierarchy.
</p>
<p>
By default, VisualWorks uses/offers class instance variables when declaring a new class. If you ever need a class variable then I think you would actually define what is called a <strong>shared variable</strong>.
</p>
<p>
Bottom line, class instance variables are really useful. Without them you would have to resort to a lot of trickery to share data between instance of a class while preventing access/modifications from subclasses.
</p>
<h3>Class Methods</h3>
<p>
The Ruby implementation above uses a lot of class methods. Class methods reside on the metaclass of a class. Fortunately Smalltalk follows this paradigm closely so it's not hard to implement these features. It just involves doing <tt>compile: 'some smalltalk code'</tt> on the right receiver. In my implementation, you will find a lot of the methods in the class side of things. Most of the instance methods are dynamically generated.
</p>
<h3>Implicit self</h3>
<p>
Ruby's implicit self does have some advantage here since it reduces the need to repeat <tt>self</tt> keyword over and over and over and over....
</p>
<hr />
<br />
<p>
So <a href="/files/DwemthyArray.zip">here's my implementation</a> in VisualWorks Smalltalk. I am not an expert in packaging parcels in VisualWorks so expect potential problems. However, even if you decide not to load it into VisualWorks it is possible to eyeball the code to get the gist of it. As far as I know, this is not a very Smalltalk-ish way to approach the problem. I did it this way to mimic the Ruby implementation as closely as possible.
</p>
<p>
The Ruby implementation seems very magical because a lot of the generated code is actually hidden. I think that if it were possible to just examine the generated code, it would make it must easier to analyze and reason about. It also enables you to use your normal tools for refactoring/ debugging. Of course, because things are hidden, it also makes the code terse and sometimes easier to read especially if you don't care about the details.
</p>
<p>
For an alternative approach see <a href="http://www.darrenhobbs.com/archives/2005/05/getting_meta.html">Darren Hobbs: Getting Meta</a>. I am not really sure whether his implementation works in the end or not.
</p>
EuroPLoP '08: Shepherding experience2008-05-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/05/27/europlop-08-shepherding-experience<p>
About a month ago, I volunteered as a shepherd for EuroPLoP '08. I shepherded an interesting paper which I hope will be accepted at the conference. I don't think I can talk about the paper here because of confidentiality issues but I can at least describe my experiences with the shepherding process.
</p>
<img src="/files/sheep.jpg" alt="Shepherding - Image purchased from iStockPhoto"/>
<p>
I did the shepherding process pretty much based on my own experiences both as a teaching assistant and a peer reviewer for my colleagues' papers. Even though this was my first time shepherding, I <em>think</em> I did a good job based on what my sheep and PC member said. Today, I was pretty much surprised to discover that someone had actually created a pattern language for shepherding -- <a href="http://hillside.net/language-of-shepherding.pdf"><em>The Language of Shepherding</em></a>. I was also pleasantly surprised that most of my actions were inline with the patterns from the article.
</p>
<p>
The following paragraphs are titled with a pattern from the <em>The Language of Shepherding</em> article and my experiences with that pattern.
</p>
<h2>The Shepherd Knows the Sheep</h2>
<p>
<blockquote>
Start immediately to build a relationship with the author, and maintain it throughout the
shepherding. Contact the author right away, even before reading the patterns, and tell the author
about yourself. Tell the author what to expect from you (such as three iterations), and establish what
you expect from them in return. Above all, follow through on your commitments!
</blockquote>
I found that this works very well. In addition, I also asked the sheep about their own experiences with patterns so that I can tailor my comments to their experiences. I also asked about their long-term goals for their pattern; perhaps they are interested in compiling the pattern into a book or actually implementing the pattern in a working system. Having such questions help the me as the shepherd understand how to tailor the comments. Such questions also help remind the sheep of what they need to do to achieve their own goals for the paper.
</p>
<h2>Three Iterations</h2>
<p>
There is usually little time for reviews between the start of the shepherding process and the deadline for submission. Therefore, it is a good idea to establish a <em>schedule</em> at the beginning of the shepherding process. I have found that having an iteration be about 1.5 weeks long works out well -- a week is usually a bit too short. This allows for some flexibility in the schedule in case some other commitment pops-up halfway through the shepherding process. Once you have established a schedule (make this concrete by listing the actual dates that you expect an iteration) both parties should agree to stick with it. Having a complete paper at the end of an iteration (vs just having chunks here and there) is also better since it allows both parties to gauge the current state of the paper in its entirety and examine its cohesion.
</p>
<p>
It also helps to have a plan for each iteration. The first iteration should focus on the big picture of the pattern (and whether or not the paper is <em>really</em> a pattern). Subsequent iterations would be geared toward polishing the paper and making it more cohesive.
</p>
<p>
The hardest thing about iterations is to be mindful of consistencies. As the paper undergoes various rewrites, some parts can become inconsistent and need to be adjusted. For instance making sure that the same terms are used consistently throughout the paper is an important consistency to check for.
</p>
<h2>Half a Loaf</h2>
<p>
I found that small manageable chunks of feedback work well. Small comments tend to be more detailed and concrete and are easier for the sheep to follow. Thus, as the shepherd you are less likely to ramble on too much or dwell into some meta-argument of the paper.
</p>
<p>
As the original article suggests, it might also be necessary to give a full loaf sometimes. For instance, it is important to provide <em>important feedback as soon as possible</em> even if this might be a big comment. For instance, some changes are big and need to be made since the entire paper hinges on it. It's best to deliver such feedback early even if it would require a lot of explanation/reworking.
</p>
<p>
This makes it possible to produce a polished paper within the time constraints by addressing the most important issues first.
</p>
<h2>Small Patterns</h2>
<p>
<blockquote>
Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.
<br />
-- Antoine de Saint-Exupery, French writer (1900 - 1944)
</blockquote>
As the pattern progresses through comments from the shepherd and other potential peer reviewers, there is the tendency for the pattern to accrue too much description. It then becomes extremely hard to distill the gist of the pattern.
</p>
<p>
So it is prudent to keep the pattern short. It is best to prune pathological cases from the pattern that practitioners of the pattern are unlikely to encounter. Or it could also be a symptom that the pattern should be broken down into smaller ones. Anyway, a <em>huge</em> pattern is a kind of "pattern smell" and should be examined thoroughly.
</p>
<hr />
<br />
<p>
Here are some of my own techniques that were not actually mentioned in the original article but which I found helpful.
</p>
<h2>Sheep Bleats</h2>
<p>
Usually as a shepherd you would be familiar with the domain of the paper. However, sometimes there will be certain details that you are not familiar with. Or maybe the sheep makes a very compelling argument for something that you are almost certain is <em>wrong</em>. Nonetheless you want to avoid unnecessary conflicts/arguments.
<br />
<div style="text-align: center;">* * *</div>
<br />
<em>Therefore</em>, give your sheep the benefit of doubt. Ask the sheep for resources to justify and support their claims. Not only will this help make the paper stronger but as the shepherd you also get the chance to improve your own knowledge on the domain. Even if the sheep is wrong, this experience has helped clarify any misconceptions in a non-threatening way.
</p>
<h2>Multiple Examples</h2>
<p>
It's hard to explain something just by giving a meta-argument of the issue.
<br />
<div style="text-align: center;">* * *</div>
<br />
<em>Therefore</em>, spend the extra time to give a few examples on the issue and let the sheep experience an "aha" moment. Being able to give multiple examples usually clarifies the issue for both the sheep and the shepherd.
</p>
<h2>Interlaced Comments</h2>
<p>
<small>This appeared as a candidate patlet in the original article.</small>
<br />
You want to point out a particular section of the pattern that requires work by the author. Usually what you would do is write down the page and paragraph number and then describe the problem in the e-mail. However this has some potential problems:
<ol>
<li>it becomes tedious and confusing to refer to multiple sections.</li>
<li>there is the potential for miscommunication since people in different countries use papers of different sizes. So the page and paragraph number might be off.</li>
</ol>
<br />
<div style="text-align: center;">* * *</div>
<br />
<em>Therefore</em>, put the comments in-line in the original document. This could be done using pencil/pen/colored markers/ etc and then have the document scanned for sending. Or it could be done using a tool like <em>Adobe Acrobat</em> or <em>Microsoft Word</em> which allows comments to be embedded. Using such tools also provides automatic comment tracking -- to see which issues have been addressed.
</p>
<p>
If the use of such tools is not possible it might also be prudent to label the comments so that they can be referred to easily in subsequent follow-ups.
</p>
<p>
Using a pencil/pen or a tool also allows the embedding of non-text comments e.g. images. drawings, circles which might make it easier to communicate.
</p>
<h2>Modern Communication</h2>
<p>
Sometimes it is hard to express something in text. It could either get very verbose or you are afraid that there is a chance for misinterpretation. After all, e-mail doesn't really convey emotion that well.
<br />
<div style="text-align: center;">* * *</div>
<br />
<em>Therefore</em>, when possible it might be a good idea to offer other ways of communicating i.e. using Skype. Verbal communication is usually better when both parties are fluent in the same language but can be problematic otherwise.
</p>
<p>
I offered this option to my sheep but we never got around to using it. My friend Maurice had a chance to use it with his sheep and he told me that it worked well. Setting up a Skype session takes a lot prior arrangement from both parties since it requires you to set up a meeting that could span multiple time zones.
</p>
<hr />
<br />
<p>
As far as I know, <em>The Language of Shepherding</em> has not been revised/updated since its inception in 1999. So hopefully the details of my experiences will highlight the fact that the article is still very much relevant for use circa 2008. Moreover, perhaps someone would also benefit from some of my suggested patlets.
</p>
Better Code Browsing in Squeak2008-03-31T00:00:00+00:00http://softwareengineering.vazexqi.com/2008/03/31/better-code-browsing-in-squeak<small>
I am planning a series of articles to help people get accustomed to <em>new</em> and <em>better</em> tools in Squeak. This is mostly written just for fun and hopefully it would be useful to students taking Ralph's object-oriented course with Squeak over summmer. Squeak is a very <strong>different</strong> way of programming when compared to other programming languages and IDE. So, while I am not hoping for some miraculous conversion, I do hope to make it easier for people to work more comfortably within Squeak after using some of the current state-of-the-art IDEs.
</small>
<p>
Many people hear that the code browser in Squeak (or any Smalltalk for that matter) is designed for <em>code browsing</em>. In other words, the browser is <em>optimized</em> to help you read code.
</p>
<p>
However, I have found that reading code in the regular Squeak environment can quickly become a big mess especially if you are trying something out for the first time and need to read multiple implementations in different classes. Usually, when this happens, you end up with a proliferation of browser windows that quickly clutter your screen. I am assuming that veteran Smalltalkers don't find this a problem because:
<ul>
<li>they know most of the methods so well that there isn't much need to refer to it</li>
<li>they have LARGE monitors so 20 open browsers is not a problem</li>
<li><em>they are just disorganized and don't give a damn about it</em></li>
</ul>
</p>
<p>
In this article, I am going to show you how to use the <a href="http://smallwiki.unibe.ch/hermion/">OmniBrowser with some new enhancements</a>. The same group that made the OmniBrowser has also created a whole bunch of very nifty tools. I'll try to get to each one in separate articles. But before that, let me divert your attention to this very relevant message:
</p>
<p><a href="http://onsmalltalk.com/programming/smalltalk/a-squeak-smalltalk-development-example/">A Squeak Smalltalk Development Example</a>:
</p>
<blockquote cite="http://onsmalltalk.com/programming/smalltalk/a-squeak-smalltalk-development-example/" title="A Squeak Smalltalk Development Example"> "Most developers who use Squeak would have a plethora of extra tools and utilities installed that make developing a much nicer experience than what you see in this tutorial. Do yourself a favor and start your Squeaking with a real developer’s image loaded with all the proper goodies like the SqueakDev image maintained by Damien Cassou."
</blockquote>
<p>
So at this moment, I am going to insist that you download one of the excellent images maintained by Damien over at his <a href="http://damien.cassou.free.fr/squeak-dev.html">webpage</a>. Don't even bother using the <a href="http://squeak.org/Download/">basic squeak image</a>. The basic image is pretty disheartening and discouraging to any new developer who is trying Squeak for the first time. Since Squeakers are usually developers, I have always found it mind-boggling that the standard Squeak image does not include better support for developers <strong>out-of-the-box</strong>. Instead, it is cluttered with a lot of stuff that most developers don't care about.
</p>
<p>
For this tutorial I am using the squeak-web/beta version from <a href="http://squeak.ofset.org/squeak-web/beta/">here</a>. Don't forget to grab the SqueakV39.sources file from <a href="http://squeak.ofset.org/squeak-dev/">here</a> as well.
</p>
<p>
Here's our scenario: we want to see how to actually implement the relevant methods so that two objects can be compared to one another for sorting purposes. Here's a step-by-step guide to how I would use the OmniBrowser + enhancements to do it.
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2379630720" title="View 'Open the SmartGroup Browser' on Flickr.com"><img src="http://farm3.static.flickr.com/2418/2379630720_850deb861d.jpg" alt="Open the SmartGroup Browser" border="0" width="500" height="322" /></a>
<br />
<strong><small>Open the SmartGroup Browser</small></strong>
<br />
<small>Bring up the World Menu in Squeak and select the SmartGroup Browser.</small>
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2379630798" title="View 'Type in the '<' selector' on Flickr.com"><img src="http://farm4.static.flickr.com/3173/2379630798_a57aeed05a.jpg" alt="Type in the '<' selector" border="0" width="500" height="322" /></a>
<br />
<strong><small>Type in the '<' selector</small></strong>
<br />
<small>In the top pane of the SmartGroup Browser type in '<'. The top pane is called the Mercury Pane and it lets you type in class/selector names and quickly displays it in the current browser for you. For more information about the Mercury Pane see <a href="http://lists.squeakfoundation.org/pipermail/seaside/2007-January/010397.html">this<a>.</a></a></small>
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2378793781" title="View 'New window showing the implementors of '<'' on Flickr.com"><img src="http://farm4.static.flickr.com/3003/2378793781_f30e186658.jpg" alt="New window showing the implementors of '<'" border="0" width="500" height="322" /></a>
<br />
<strong><small>New window showing the implementors of '<'</small></strong>
<br />
<small>A new windows pops-up showing the implementors. I think it is OK to have a new window pop-up with the search results. Also notice that this actually shows the classes in a hierarchical manner (by tabs). So you can see that Character is a subclass of Magnitude.</small>
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2378793945" title="View 'Search for Float' on Flickr.com"><img src="http://farm3.static.flickr.com/2352/2378793945_6a40baaccd.jpg" alt="Search for Float" border="0" width="500" height="322" /></a>
<br />
<strong><small>Search for 'Float'</small></strong>
<br />
<small>Now back in the SmartGroup Browser, I am going to search for the Float class since I am interested in seeing the definition of the method for it. I am going to enter 'Float' in the Mercury Pane.</small>
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2379630998" title="View 'Split panes in the Browser!' on Flickr.com"><img src="http://farm3.static.flickr.com/2016/2379630998_4f399fd40c.jpg" alt="Split panes in the Browser!" border="0" width="500" height="322" /></a>
<br />
<strong><small>Split Panes in the Browser!</small></strong>
<br />
<small>By doing a Shift+Click on a class/method you split a new pane with it!</small>
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2379631048" title="View 'Multiple Split Panes showing all the methods' on Flickr.com"><img src="http://farm3.static.flickr.com/2306/2379631048_5f7bdcdedb.jpg" alt="Multiple Split Panes showing all the methods" border="0" width="500" height="322" /></a>
<br />
<strong><small>Multiple Split Panes showing all the methods</small></strong>
<br />
<small>By repeating the Shift+Click process, we can create as many split panes as we want. Usually it's enough to have about 4 of them.
Also, don't forget that you can browse back using the '<<' button.</small>
</p>
<p>
For comparison, this is what it would look like if we had to use the normal tools from the standard Squeak image. Look at the wasted screen estate and how I don't see any nice hierarchical information between the classes.
</p>
<p>
<a href="http://www.flickr.com/photos/37092324@N00/2378778609" title="View 'Finding out how to implement < in Squeak 3.10' on Flickr.com"><img src="http://farm3.static.flickr.com/2156/2378778609_80f4c9c4bb.jpg" alt="Finding out how to implement < in Squeak 3.10" border="0" width="500" height="322" /></a>
<br />
<small>This is roughly what it would look like to find the '<' method in Squeak 3.10 without the new tools.
We start by trying to look for the '<' selector in the Method Finder and then clicking on each method to see its definition.
Notice that there can be no end to the list of new windows!</small>
</p>
<p>
So I claim that by using the new tools, reading code in Squeak is much more organized. Good organization not only helps the developer focus but also makes it more approachable (and less intimidating) for a new user. <strong>Personally, I advocate that any new tutorial about Squeak be written with reference to these new tools</strong>. These new tools are hidden gems that need to be exposed to the world! Only by getting more people to use them can we:
<ul>
<li>improve existing tools based on user feedback and experience
</li>
<li>actually get new tools to appear for Squeak - after all what's the motivation of creating new tools if everyone is just using the <em>old</em> way?
</li>
<li>change people's perception that Squeak's tools are 20-years outdated compared to more polished tools such as Eclipse</li>
<li>make it easier for first-time developers to use Squeak. Seaside has already done a fantastic job garnering attention for Squeak and we should take this opportunity to make it easier for developers to get started</li>
</ul>
</p>
<p>
I also hope that people will be less close-minded about enhancing Smalltalk as evident from some of the comments in the following posts:
<ul>
<li><a href="http://www.cincomsmalltalk.com/blog/blogView?showComments=true&entry=3273165966">Extending Smalltalk Syntax</a></li>
<li><a href="http://www.cincomsmalltalk.com/blog/blogView?showComments=true&printTitle=How_should_we_view_methods&entry=3381673662">How Should We View Methods?</a></li>
</ul>
</p>
<p>
Of course, changes should be implemented judiciously but without an open-minded attitude toward change, <em>nothing will change</em>.
</p>
OOPSLA '07: Collaboration and Telecollaboration in Design2007-11-03T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/11/03/oopsla-07-collaboration-and-telecollaboration-in-design<small>
The <a href="http://www.oopsla.org/oopsla2007/index.php?page=podcasts/">podcast</a> of this special event in OOPSLA is actually available from the OOPSLA website. As my friend Munawar puts it, you really have to be there to experience it -- it's almost like a religious experience to hear Fred Brooks speak.
</small>
<p>
Fred Brooks, the author of the seminal book <a href="http://en.wikipedia.org/wiki/The_Mythical_Man-Month"><em>The Mythical Man-Month</em></a> and the famous essay<a href="http://en.wikipedia.org/wiki/No_Silver_Bullet"> <em>No Silver Bullet</em></a> was invited to give a talk at OOPSLA 07. His <em>Collaboration and Telecollaboration in Design</em> talk addresses the issues that affect designs done in teams. Currently, the designs of complex systems are usually done collaboratively in teams because the design requires more effort and knowledge than what a single architecture can muster. However, <em>design by teams</em> present a new challenge: how do we get great designs when there are so many contributors? After all, it's pretty much a truism that <em>too many cooks spoil the broth</em>.
</p>
<p>
The key to achieving great design is maintaining <em>conceptual integrity</em>. Conceptual integrity is achieved when the system exhibits a consistent and unifying theme: everything seems to fit together seamlessly. Conceptual integrity is the presence of normalcy and the absence of radical designs in different parts of the system. A possible example of conceptual integrity would be the <em>drag-and-drop</em> metaphor found in most operating systems (done more right in some than others). If the system was design with conceptual integrity then drag-and-drop should just work <em>everywhere</em>. For instance, opening a file would just involve dragging it to the appropriate application; deleting a file would involve dragging it to the trash. In short, conceptual integrity (for the user and developers alike) creates an easy-to-use system with little surprises.
</p>
<p>
Ideally, conceptual integrity is easy to maintain when there is only one chief architect who oversees everything. This chief architect plays the important role of seeing how the pieces fit together. He might not understand all the details but he certainly knows at a high level how everything works. How would one achieve this in a team? It does not really seem possible to have multiple chief architects (in fact, that sentence seems grammatically wrong since <em>chief</em> already refers to a single person). There could be multiple architects but there is one chief architect who has the authority to make the final decisions. Even in an open source project like Linux, Linus Torvalds still has the ultimate veto power on what to put in and what not to put in. This almost seems like a dictatorship but if that is what is required for conceptual integrity then it is something that has to be done.
</p>
<p>
While collaboration should definitely be done, the need to actually appoint someone with the authority for making the final decisions should not be overlooked. According to Brooks, collaborative efforts such as <em>pair-programming</em> actually works to get ideas and design decisions out quickly. The important thing to realize is that certain phases of design requires decisions by an individual.
</p>
<p>
On the other hand, Brooks does not actually think that telecollaboration with all the fancy bells and whistles actually work. The phrase he used to describe this is profound: [telecollaboration sytems are] <em>technologically-pushed</em> <strong>rather than</strong> <em>application-pulled</em>. So most of the tools out there are just loaded with technology that no one needs to use. In fact, there is no need for fancy teleconferencing equipment: what is needed is a way to share the documents (via e-mail, source control, real-time editing, etc) and voice conferencing. Video conferencing does not bring any additional benefit.
</p>
<img src="http://images.theglobeandmail.com/archives/RTGAM/images/20061128/tq-video28/video800big.jpg" width="400" height="257" />
<br />
<small>The shared document in the upper panel is useful; the other panels not so much. Image taken from an article on <a href="http://www.theglobeandmail.com/servlet/story/RTGAM.20061128.tq-video28/BNStory/GlobeTQ/home" title="globeandmail.com: Is it real, or is it a video conference?">TheGlobeAndMail</a></small>
<p>Brook's thoughts agree well with what the <a href="http://www.oopsla.org/oopsla2007/index.php?page=sub/&id=190" title="OOPSLA 2007">two second-life developers</a> said during a previous talk. The developers of Second-Life are located all over the world and they need to use telecollaboration as well. They use an application such as SubEthaEdit for real-time code editing. And they use the new 3D voice technology in Second Life to talk to one another while their avatars are positioned in a room inside Second Life itself. They do not use any form of video conferencing. This setup has been found to be effective for their development task and they have been using it for some time.</p>
<p>
One final issue that Brooks addresses in his talk is the <em>perceived malleability</em> of software. Non-programmers have the notion that software is very easy to change - it's easy to add a feature, remove a feature or modify the feature. The software marketing industry is partly to blame for this perception since the marketing team is always touting that the latest version of application X and how it has 100+ features and tons of bug fixes. And they do these kinds of promotions <em>annually</em>. Is it any wonder that the masses just think that software is so easy to change?
</p>
<p>In my opinion, this mindset causes many problems for developers. Requirements in software tend to change a lot more than any other engineering project especially once it has started. No one would change the design of a bridge halfway especially after all the foundation has been done. And yet, even after the architecture of software has been decided, it is not impossible to imagine the <a href="http://en.wikipedia.org/wiki/Pointy_Haired_Boss" title="Pointy-Haired Boss - Wikipedia, the free encyclopedia">pointy-haired boss</a> coming up to the developers and demanding a radical change. And this is why, agile methods have been so pivotal in the development of modern software systems.</p>
<p>
Maybe people would think differently if they had something more tangible such as the printed source code to relate the efforts of software development. After all, it's easy to see how much of the bridge has been completed based on the visible structures. It makes me wonder if there are other areas which suffer from perceived malleability. I think fiction books suffer from the same syndrome since most editors might think it is very easy to rework the story line. However, this is not always true since the astute reader can actually find inconsistencies between the lines when the author changes the story line halfway through the book.
</p>
<p>Fred Brooks is undoubtedly a very smart man. And his arguments, as shown in both the <em>The Mythical Man-month</em> and <em>No Silver Bullet</em>, hold true for many decades. In fact, it is unlikely that anything would come to contradict his arguments. After all, it's hard to challenge something that was written based on his observations on the projects that he was involved in. So when he says something about collaboration and telecollaboration, it's advisable for people like us to actually pay attention.</p>
OOPSLA '07: Software Architecture Tutorial2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-software-architecture-tutorial<small>
The next few posts will be about OOPSLA 2007 and the title of the articles will be appended with "OOPSLA 07". Those articles will be written in a more colloquial manner since they will be mostly based on my personal experiences there.
</small>
<p>I was fortunate enough to have the opportunity to go to <a href="http://www.oopsla.org/oopsla2007/index.php?page=home/" title="OOPSLA 2007">OOPSLA</a> this year. I say fortunate because I had to apply for a Canadian visa; and based on past experiences it can either be a painless ordeal or one where you are unsure what you have done to not deserve one. Anyway, all went well, and after 15 hours of driving and 12 hours of recuperative sleep -- I was not able to sleep in the van -- I attended the first day of OOPSLA without any problems.</p>
<p>The first day was relatively easy-going since I was not signed up for any of the symposiums and could freely choose which two tutorials (or none) that I wanted to attend. I chose the tutorials on Software Architecture by <a href="http://stal.blogspot.com/" title="Hitchhiker's Guide to Software Architecture and Everything Else - by Michael Stal">Michael Stal</a> of <a href="http://www.amazon.com/Pattern-Oriented-Software-Architecture-System-Patterns/dp/0471958697" title="Amazon.com: Pattern-Oriented Software Architecture Volume 1: A System of Patterns: Books: Frank Buschmann,Regine Meunier,Hans Rohnert,Peter Sommerlad,Michael Stal"><em>Pattern-Oriented Software Architecture</em></a> fame.</p>
<p>The first tutorial was during the morning session and was entitled <em>High Quality Software Architecture</em>. It was an interesting tutorial and I think I was the youngest person in the room. The tell-tale signs of balding heads indicated that most of the other attendees were professional software architects that were here to learn the secrets of the trade from Stal himself. Anyway, the first part of the talk was essentially a nice review of the material that I had read about a year ago when I took the Advanced Software Engineering class. In fact, I wrote articles on the books and papers that we read for that class here in this blog. Stal was using the same terms an definitions from the <a href="http://www.amazon.com/Software-Architecture-Practice-Len-Bass/dp/0201199300" title="Amazon.com: Software Architecture in Practice: Books: Len Bass,Paul Clements,Rick Kazman"><em>Software Architecture in Practice</em></a> book. It was interesting to actually see how someone else who interpret that book since it was a rather academic book and it was hard to actually find someone else to discuss about it.</p>
<p>Stal emphasized three main issues that was only lightly touched upon in the book. And I agree with him that these three topics will help promote higher quality software architecture:
<ul>
<li><em>Clear architecture vision</em>: This is similar to the metaphor in XP or the ubiquitous language in Domain Driven Design. Essentially it means that the architecture should be driven by a clear idea of what you are trying to achieve. And the architecture should strive to maintain this goal in every aspect possible.</li>
<li><em>Iterative Development</em>: This is really important. Developers usually think of architecture as something that must always be done up-front. There is some truth to this since if you get the architecture (vs the design; see <a href="http://softwareengineering.vazexqi.com/articles/2006/08/31/architecture-design-and-implementation" title="Architecture, design and implementation">this</a> for an explanation on the differences) wrong then there is a heavy price to pay in terms of development efforts. Stal solves this by introducing the concept of the <em>baseline</em>. The baseline is the <em>core</em> of the architecture that needs to be done right and will not change throughout the entire development cycle. The baseline is determined from a set of prioritized requirements from the stakeholders. Only the most important requirements shape the architecture of the baseline; the remaining requirements will be acted on later in an incremental manner. This balance on what to add <em>first</em> and what to add <em>later</em> makes sense: unlike design, software architecture will definitely permeate the entire system so it is essential to at least get the important aspects right the first time.</li>
<li><em>Testing</em>: It is necessary to be able to test that your system supports certain requirements. It is usually commonplace now to have automated tests for most of the functional requirements of the system. Stal claims that it is necessary to have tests as well for the system architecture. These tests need <a href="http://www.testing.com/writings/classic/mistakes.html#automating_all_tests" title="Classic Testing Mistakes">not be automated fully</a> of course but they should exist nonetheless. Either doing it through a code review or a formal inspection would reveal missing requirements in the architecture. I talked to Stal about using <a href="http://en.wikipedia.org/wiki/Architecture_description_language" title="Architecture description language - Wikipedia, the free encyclopedia">executable Architectural Description Languages</a>(ADL) to verify the architecture and he says that most of the time that approach is not feasible since it requires a domain expert on the ADL itself <a href="http://softwareengineering.vazexqi.com/articles/2006/12/05/archjava" title="ArchJava">reaffirming</a> what I thought about ADLs in the first place.</li>
</ul>
</p>
<p>The afternoon session was on <em>Refactoring Software Architecture</em>. Stal admitted that what he was going to present was based on preliminary work but nonetheless he felt that it was essential enough to warrant a tutorial. I completely agree with him especially since he too mentioned the <strong>Grand Unified Theory of Refactoring</strong> at the end of his talk and if we have any chance of finding one we had to start thinking of a lot more about software in terms on refactoring and program transformation. Like Fowler, Stal motivated the indicators of architecture refactoring in terms of <em>architecture smells</em> or what he terms the lack of the following qualities:
<ul>
<li>Pattern Density</li>
<li>Symmetry</li>
<li>Expressiveness</li>
<li>Simplicity</li>
<li>Orthogonality</li>
<li>Emergent behavior</li>
</ul>
</p>
<p>The list of refactorings that he presented were based in the spirit of the original refactorings by Fowler: he used a pattern language to describe them. When I first saw the first few refactorings that he presented I was confused by the scope of it: were they architecture refactorings or design refactorings? And this is where the distinctions in the paper <a href="http://softwareengineering.vazexqi.com/articles/2006/08/31/architecture-design-and-implementation" title="Architecture, design and implementation"><em>Architecture, Design and Implementation</em></a> might be useful. When I asked Stal, he explained that he treats refactorings that could have a percolating impact to be an architectural refactoring. So when you refactor something and it affects things below it then it is probably an architectural refactoring pattern. There is still this nagging feeling inside me that thinks that architectural refactorings might not be behavior preserving -- the quintessential definition of what a refactoring is -- but I just cannot come up with an example of this. So I could be wrong about this.</p>
<p>Stal has a list of roughly 30 refactorings. But the two that most interested me were <em>Aspectize</em> and <em>Integrate DSLs</em>. The former being that you might need to address some cross-cutting concern and use an aspect-oriented approach to solve it. This is actually the few times that I have seen someone promote aspects in an appropriate manner. Usually people are vehemently <em>for</em> or <em>against</em> aspect-oriented programming. Stal however, presented a clear case where aspects can actually be a good approach. The second pattern is all about not being reserved on using a Domain Specific Language for your architecture if you need one. Again, I think that the time for thinking in more than one language has come and this is one way to make good use of that approach. Software architecture can be a complex beast and while it might be usually over-kill to use AOP and DSLs for normal projects, architecture could be the key area where they should both have relevant roles.</p>
<p>Throughout both these presentations Stal mentions the need for iterative design and refactoring. He is clearly a strong supporter of refactoring. He realizes that all future software development can only be successful with decent and continuous refactorings. And he is far-sighted enough to see how refactoring can be applied not only to code but also other artifacts like architecture.</p>
<p>I was talking to Maurice and we both agree how our advisor, Ralph Johnson, could have made one of the greatest contributions to software engineering by introducing the concept of refactoring. Sure he has co-authored the book on Design Patterns but the main concept of refactoring is probably more influential. Thinking of software engineering in terms of refactoring and transformations is definitely how all future software should be developed.</p>
OOPSLA '07: SemmleCode Demo2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-semmlecode-demo<p>
<a href="http://semmle.com/product/">SeemleCode</a> by Semmle is a tool for for Java and XML code querying. It features its own query language, unsurprisingly called, QL for <strong>Q</strong>uery <strong>L</strong>anguage and a set of UI options for displaying the results of the queries. Currently, the results could be displayed in a table or in graph form (pie chart, bar chart, etc).
</p>
<p>
<code>
<pre>
from Class c
where c.getPackage().getName().matches("org.jhotdraw%")
select c.getPackage(), c
</pre>
</code>
The above code snipper has been taken from their <a href="http://semmle.com/product/ql/examples/">example page</a>. This query finds all classes in the package org.jhotdraw and its subpackages. So the language is like SQL except that it structured as <code>from... where... select...</code> instead of the more ubiquitous <code>select... from... where...</code>. According to the developers, this ordering makes it easier to enable context-sensitive content assist as the user is typing.
</p>
<p>
What I like about this system is the fact that the query language can be used to do a lot of useful things. The example script that comes preloaded with SemmleCode has some example metrics that you could run on you system. Here is an example that measures the <a href="http://www.aivosto.com/project/help/pm-oo-ck.html">Depth of Inheritance Tree</a>.
<p>
<code>
<pre>
from MetricRefType t, int d
where t.fromSource() and d = t.getInheritanceDepth() and d > 6
select t, d order by d desc
</pre>
</code>
The code snippet above reports reference types whose maximum distance to Object is greater than 6, in decreasing order of distance.
</p>
<p>
Moreover, given a complex enough query you could even use it as a form of code lint to check for inconsistencies in the code. So this system is extensible and could be used as a tool to query and inspect the source code in a non-invasive manner by the programmer.
</p>
<p>
SemmleCode comes with an in-memory database to store all the meta-information for querying. However, using this in-memory database for larger projects is really not recommended. Instead you should either install <a href="http://www.postgresql.org/">PostgreSQL</a> or <a href="http://www.microsoft.com/sql/editions/express/default.mspx">Microsoft SQL Server</a>. Since I was on a Mac, I installed PostgreSQL and got it to work. The installation took only an hour or so (mostly because the internet connection was flaky) and PostgreSQL was easily installed on the Mac following the instructions that came with the download.
</p>
<p>
One caveat though and I discovered this after showing my installation to one of the developers present at the demo: SemmleCode operates on <em>working sets</em> in Eclipse. Unfortunately, those working sets need to be defined as Java Working Sets and not Resource Working Sets. SemmleCode will <strong>not</strong> recognize the Java files in a Resource Working Set since it is looking for the Java nature. After fixing this, SemmleCode works just fine. I ran it on the Photran source code and it was able to complete the analysis rather quickly.
</p>
<p>
So far SemmleCode is free to download and use. I suspect that a paid version would include a prepackaged library that has more sophisticated queries that developers might find useful.
</p>
OOPSLA '07: Jazz Demo2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-jazz-demo<p>
There were a ton of demos going on at OOPSLA this year but I was only able to attend three of them. The three that I chose were all focused on Java or Eclipse development since I was really interested in finding out what we could use to make Eclipse a better environment for tool development.
</p>
<h2>Jazz</h2>
<p>
<small>I did not go for the original demo of Jazz but rather stopped by the booth the next day to talk to some of the developers there. From what I heard, the original demo has some technical problems; they were doing a demo of two development team -- one team at OOPSLA and the other team in another part of the world -- and someone accidentally tripped on the ethernet cable and the demo could not go on after that.</small>
</p>
<p>
<a href="https://jazz.net/pub/index.jsp">Jazz</a> is an <em>all-in-one</em> collaboration tool for Eclipse. It integrates version control, bug-tracking, task-management and even wiki editing! It could potentially do lots more but those were the features that I saw and which I was most interested in. In order to provide a better user experience the creators of Jazz decided to recreate some of the underlying features from scratch instead of relying on existing technologies that users have grown accustomed to. For instance, instead of relying on CVS or SVN, Jazz has its own version control system. When I first heard this I was rather doubtful of how necessary it would be since most developers are already using some system like CVS and might be reluctant to migrate over to something new. However, this issue is resolved by having an <em>importer</em> that takes the current CVS repository and converts it into a Jazz repository. I was at the booth during this demo and the person there actually imported the source code from CVS for the <a href="http://www.eclipse.org/photran/">Photran</a> project that we are working on and converted it into a Jazz project. The conversion retained the meta-information from CVS and this would ease the transition for existing projects.
</p>
<p>
The goal of Jazz is for all its features to work together seamlessly in a consistent manner. To fulfill this goal, the developers deemed that wrapping a myriad of tools such as CVS/SVN, Bugzilla, etc would not provide the best user experience. This might be true since tight-integration is one way to ensure that everything is working together without having to spend countless hours configuring all the tools to work together. Jazz comes with its own fully packaged server that contains all the tools that you would need to get started immediately.
</p>
<p>
The approach Jazz has taken is different from <a href="http://www.eclipse.org/mylyn/">Mylyn</a>, another popular Eclipse plug-in for collaborative development:
</p>
<p><a href="http://wiki.eclipse.org/index.php/Mylyn_FAQ#JIRA_Connector">Mylyn FAQ - Eclipsepedia</a>:
</p>
<blockquote cite="http://wiki.eclipse.org/index.php/Mylyn_FAQ#JIRA_Connector" title="Mylyn FAQ - Eclipsepedia"> "At the EclipseCon and JavaONE 2006 conferences IBM demonstrated previews of Jazz, a collaborative team server and Eclipse-based client. Articles have remarked on the similarities between Mylyn and Jazz because both integrate tasks into Eclipse (Jazz's 'work items' and Mylyn's 'tasks'), and both provide change sets grouped by task. But there are both significant differences and complementary aspects to the two approaches. A key goal of Mylyn is to provide an open source framework to support integration of task management with existing issue trackers and source repositories. According to the presentations, components that come with Jazz include include a next-generation issue tracker and source repository and related lifecycle management tools such as project health. In addition, a driving and unique goal of Mylyn is to focus the UI around a usage-based and degree-of-interested weighted task context, which is complementary to the Jazz platform. Just as it integrates with Mylyn's Task List, Mylyn's Task Focused UI and task context model components are possible to integrate with other kinds of task management systems, such as the one provided by Jazz."
</blockquote>
<p>
I have only begun using Mylyn recently and have been impressed by what I have seen so far. Our university Computer Science department has an existing <a href="http://www.atlassian.com/software/jira/">JIRA</a> server that we are using for our development needs. Mylyn connects seamlessly to it and works fine. It is this ability to interoperate with existing technology and tools that is the strongest advantage about Mylyn. Though I would really like to use Jazz it is not really practical to convince the technology services group to install a dedicated server for it when they are already using a bunch of existing tools that work well. I suspect the same situation occurs in existing software development companies as well.
</p>
<p>
The Jazz developers have already done a good job at reducing the initial risk of switching over to it by providing the <em>importers</em>/<em>adapter</em> that convert existing data into a format that Jazz can work with. If the Jazz developers could convince the Eclipse foundation to adopt Jazz instead of using their current tools, then it would make a big impact in convincing other projects to follow suit. Until then, most developers with existing projects might be less inclined to use it.
</p>
OOPSLA '07: Educators' Symposium2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-educators-symposium<p>
I spent the morning in the <a href="http://www.oopsla.org/oopsla2007/index.php?page=program/educators/" title="OOPSLA 2007 Educator's Symposium">Educators' Symposium</a> at OOPSLA. It was an interesting session but unfortunately I could only stay for 3.5 activities since I wanted to go for the Dynamic Language Symposium in the afternoon.
</p>
<p>We started off with a short session on how to teach recursion to incoming freshmen by <a href="http://www.oopsla.org/oopsla2007/index.php?page=presenters/&id=1#Michael_H_Goldwasser" title="OOPSLA 2007">Michael Goldwasser</a> from St. Louis University. He had an interesting way for teaching recursion to first year students: he would pick a group of volunteers who would act as nodes in a list that is defined recursively. He would set it up so that each node knew which was the next list (null being the <em>next node</em> for the tail of the list). He would then act as the main program and call out a function like <code>count()</code> which returns the length of the list. His call of the function was symbolized by throwing a real tennis ball. This tennis ball was slit open and contained a piece of paper that was supposed to show the activation record of the current method call. In the case of <em>count()</em> each volunteer would then throw the ball to the next node and await a reply. This is a great way to illustrate how recursion works. However, I was a bit skeptical whether this activity might distract the students from the main concept of recursion. I was also curious whether this method of teaching recursion had permanence in the students's minds. Lastly, I wonder how well this method would scale up to a class that has 100+ students. Either way this was a nice method and it did give me some ideas for things to try on my students next time.</p>
<p>Next we had a session on <a href="http://en.wikipedia.org/wiki/Scrum_(development)" title="Scrum (development) - Wikipedia, the free encyclopedia">Scrum</a>. This was my first introduction to Scrum and it was pretty intersting -- it's a high level process that emphasizes customer needs and iterative design but it is not focused entirely on software development. So unlike XP, it is not explicitly required in Scrum to do pair programming or test first development. However, most software developers who use Scrum will augment it with those two practices anyway. It was a fun hands-on introduction to Scrum but again unless there is a decent <em>reflection</em> session after the activity I am curious whether the students will actually understand what Scrum is all about.</p>
<a href="http://www.flickr.com/photos/vazexqi/1702446110/" title="Photo Sharing"><img src="http://farm3.static.flickr.com/2313/1702446110_d8bb61a5b0.jpg" width="500" height="375" alt="Zoo Shadow Box" /></a>
<br />
<small>As part of an activity at the Educators' Symposium we had to create a zoo shadow box as part of our Scrum training. That is what our shadow box looked like. You have to use some imagination since it is pretty abstract. </small>
<p>After the break, <a href="http://headrush.typepad.com/" title="Creating Passionate Users">Kathy Sierra</a> gave a nice presentation on how educators can create more motivating materials for their students. There were some nice <em>gimmick</em> tricks such as using bright colored slides, using seductive images, using animation, etc. These are definitely some ideas that I have to remind myself to use next time. Kathy is an engaging speaker and she managed to capture the audience's attention easily. I think she was the perfect person to give the keynote for the Educators' Symposium. Kathy also had a list of books that she thought was entertaining and enriching to read as educators. Here is a partial list of those that I wrote down:
<ul>
<li><a href="http://www.amazon.com/Flow-Psychology-Experience-Mihaly-Csikszentmihalyi/dp/0060920432">Flow: The Psychology of Optimal Experience</a></li>
<li><a href="http://www.amazon.com/Smile-Mind-Beryl-McAlhone/dp/0714838128">A Smile in the Mind</a></li>
<li><a href="http://www.amazon.com/What-Best-College-Teachers-Do/dp/0674013255">What the Best College Teachers Do</a></li>
</ul>
</p>
<p>Before I had to leave, there was a presentation on <a href="http://www.oopsla.org/oopsla2007/index.php?page=sub/&id=357" title="OOPSLA 2007"><em>Extreme Learning</em></a> by two professors from the University of Utah. Basically it is a form of XP but in the process of developing the software, you had to actually put down goals to <strong>learn</strong> something. So you are teaching yourself new things as you implement the project. Again, this seemed rather intuitive to me: every time you do any major development you are bound to have to dedicate some time learning something new. The only difference in this case is that the student has to make it part of the user story to learn a new technology. This is definitely something that would be a good approach in a graduate level class since graduate students are often more mature and know how to organize their time to actually learn something new.</p>
<p>The Educators' Symposium is definitely fun and interesting. I wished I could have stayed for more of the sessions but the Dynamic Language Symposium was too compelling to miss.</p>
OOPSLA '07: Dynamic Languages Symposium2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-dynamic-languages-symposium<p>
The Dynamic Languages Symposium had plenty of mind-stimulating papers and some not-so-stimulating ones. There were a total of nine papers but I missed the three that were presented in the morning since I was at the Educators' Symposium.
</p>
<p>
One of the presentations that I liked was <em>RPython: Reconciling Dynamically and Statically Typed OO Languages</em>. It was an easy to understand paper and it showed the efforts that are being made to write a Python compiler in Python itself! So RPython is a <em><strong>R</strong>estricted</em> Python; it has most of the dynamic features of Python taken out so that is compiles properly to a more static environment like the JVM or the CLR. But you do not lose much by this though. You would use RPython to code the interpreter for Python itself so you will not actually write normal programs with RPython. By restricting the features of Python that you could use, you could potentially make a better interpreter. RPython is a component in the <a href="http://wiki.python.org/moin/PyPy">PyPy</a> project.
</p>
<p>
The second paper that I liked was <em>Highly Dynamic Behaviour Adaptability through Prototypes with Subjective Multimethods</em>. The title sounds complicated but the main concept is simple. The premise for a system like this would be like a system of multiple interconnected devices. So when device A, B and C are present then device D would perform a certain action for <code>method1()</code>. And when only devide B and C are present then D would performa a different action for <code>method1()</code>. This was implemented using multimethods -- each object's type determines what method gets invoked. It's like method overloading in the dynamic sense.
</p>
<p>
There are other papers that I liked but those two were the most blog-worhty ones.
</p>
<p>
Lastly, there was a presentation from <a href="http://blogs.msdn.com/hugunin/archive/2007/04/30/a-dynamic-language-runtime-dlr.aspx">Jim Hugunin</a> on implementing dynamic languages on .NET. I am generally, as are most other people as well, skeptical about Microsoft's efforts with languages. They have often created a proprietary version of languages such as J# (obviously a bastardized copy of Java) that no one even bothers about. However, with the CLR and the implementation of the Dynamic Language Runtime (DLR) on .NET, Microsoft seems committed to making .NET a better platform for developing languages in comparison to the JVM. For an unofficial count, check out the list of languages for the <a href="http://www.robert-tolksdorf.de/vmlanguages.html">JVM</a> and <a href="http://blogs.ugidotnet.org/nettools/articles/8060.aspx">CLR</a>. What makes the CLR rather compelling at the moment is the fact that making your language run on the CLR enables you to hook your program to some of the cool Microsoft technology out there: <a href="http://msdn2.microsoft.com/en-us/xna/default.aspx">XNA</a> and <a href="http://silverlight.net/">Silverlight</a> being two of them. Now, if only Microsoft would <em>endorse</em> the Mono project then I would be more compelled to start using the CLR for my projects.
</p>
OOPSLA '07: BOFs2007-10-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/10/29/oopsla-07-bofs<p>
BOFs or <em>Birds of a Feather Sessions</em> allow people with common interests to gather and just talk about anything and/or everything about their common interest! This time I went for three BOFs, all of them focused around Squeak and Smalltalk.
</p>
<p>
The first one was on Tuesday evening and it featured several short talks by various people. I have to say that the highlight of that BOF was about the by-now-well-known <a href="http://laptop.org/">XO laptop</a> (formerly known as the OLPC laptop). We even had two demo units available for us to try out if we wanted. The XO laptop is a marvelously piece of technology for its price. Undeniably, it is built to withstand all the harsh conditions that it might suffer through. However, as mentioned by the presenters the biggest roadblock for this project might not be technology but the lack of actual educational contents to put onto the device itself. Right now, there are no official educational materials that will be installed onto the machines yet. And for a machine that is supposed to function as the all-in-one textbook replacement, this is a major predicament. In fact, it might be a serious blow toward the success of this system. And no one knows what is the best solution to this problem.
</p>
<p>
There were also two presentations on language tools for Squeak: <a href="http://map.squeak.org/package/dea7935d-2497-45df-936c-69f0daedd136">OMeta</a> and CAT. OMeta is a project by <a href="http://www.cs.ucla.edu/~awarth/">Alessandro Warth</a> and is already available from SqueakMap. It's pretty compact and has some nice features for parsing languages. Alessandro has actually implemented a Javascript interpreter on top of it but the source is not yet available. CAT is another tool for language recognition by Jamie Douglas. It has more features compared to OMeta -- can support PEG <strong>and</strong> CFG, has better error messages, automatic AST generation and some AST rewriting. It's currently not released yet but from what I have seen during the BOF demo it is as good as ANTLR and I will definitely be looking into it. It might be fun and useful to create a simple IDE for CAT in Squeak like what <a href="http://www.antlr.org/works">AntlrWorks</a> does.
</p>
<p>
At another BOF that I attended, <a href="http://www.squeakland.org/community/biography/ingalls.html">Dan Ingalls</a> did a presentation of the <a href="http://research.sun.com/projects/lively/">Lively Kernel</a> project at Sun. Basically the project shows an implementation of a Squeak-like system in Javascript. Since Javascript is <em>supposed</em> to be cross-platform and runs on all modern web browsers, it is one of the best choices to implement this project in. And by taking advantage of the latest SVG features in new web browsers, one can actually create a lot of graphics without all the overhead of loading images. The demo for the Lively Kernel project is available from its website and runs fine on the Safari 3.0 and above.
</p>
<p>
The final BOF I attended was the Seaside BOF organized by Roger Whitney who is currently on sabbatical at UIUC. Unfortunately I arrived late for this BOF since I thought I have misplaced my keys and had to go to the lost-and-found office to check on that. I must have arrived more than halfway through the BOF since they were already going into the QA session. The attendees had very different backgrounds: there were some newbies like myself who have read about Seaside but have never used it and there were clearly some veterans who have worked extensively on Seaside. It would have been good if a quick walkthrough demo could have been done to introduce the newbies to Seaside development. Some attendees did raise some interesting questions. For instance, how many projects were using Seaside; what is needed to set it up quickly; which databases did it work with; what hosting options were available (besides hosting it yourself). Fortunately there is an up-to-date website, appropriately named <a href="http://www.seaside.st/">seaside.st</a>, that addresses those questions. I must say based on the short demo that web development with Seaside certainly has a different feel. It requires the developer to use all the tools from Squeak (or some other Smalltalk version). However, like Rails, once you have accepted the philosophy of Seaside, web development is certainly better than what it used to be.
</p>
Problem Frames: Helping Software Engineers See the Real World?2007-09-23T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/09/23/problem-frames-helping-software-engineers-see-the-real-world<small>
This post is based on the paper <a href="http://mcs.open.ac.uk/mj665/PFrame7.pdf" title="Problem Frames and Software Engineering"><em>Problem Frames and Software Engineering</em></a> by Michael Jackson. This is my first exposure to the notion of problem frames so I apologize for any short-sightedness.
</small>
<p>
In this paper, Jackson addresses the issue of why software engineering (SE) does not generally fit into the notion that one would have of other engineering disciplines such as aeronautical engineering (AE)-- which he uses as his primary example in the paper. While AE has enjoyed a disciplined approach to building planes and other equipment for several years since its inception, SE has yet to discover its own set of disciplined approach. The emphasis here is on the term <strong>disciplined approach</strong>. While there are certainly many different ways to solve a problem, most engineers will stick with solutions that are commonly employed usually because they are almost guaranteed to work.
</p>
<p>
There is certainly no shortage of effort to present SE in light of the more traditional engineering disciplines. The existence of formal methods and other specialized math that has been employed in SE is a testimony to that. The specialized math present a means for modeling complex problems in SE. However, even with these rigorous math models, SE has failed to gain the distinction and recognition that other fields of engineering share.
</p>
<p>
In fact, Jackson identifies the current modeling for SE as a form of detachment from the real world. Jackson asserts that most of the modeling that has been done is influenced by Djikstra's view that computer science should maintains its mathematical purity and distance itself from the <em>pleasantness problem</em> -- whether or not the model conforms to real world requirements. This detachment from the real world is the root cause for the discrepancies that SE has with other engineering disciplines.
</p>
<p>
To bring SE closer to the realm of traditional engineering, Jackson proposes his theory of problems frames: viewing the problems that SE needs to solve in the context of the real world. While Djikstra eschewed the need for relating software to the real world, Jackson's problem frames theory promotes the real world into a first class component that needs to be considered as part of the model.
</p>
<p>
<a href="http://www.flickr.com/photos/vazexqi/1430197702/" title="Photo Sharing"><img src="http://farm2.static.flickr.com/1050/1430197702_dbd5a0f217.jpg" width="500" height="71" alt="Generalized Problem Frame Diagram" /></a>
<br />
<small>Generalized Problem Frame Diagram</small>
</p>
<p>
Using the problem frames approach, software engineers can strive to model physical objects in the real world as part of the system that has to be considered. After all, <em>real</em> engineers always model the real world when solving the problems. To use the equations for fluid dynamics, an aeronautical engineer needs to set up the appropriate constraints - for instance the acceleration due to gravity, the viscosity of the fluid, etc. The equations for fluid dynamics can stand alone with all their symbols but unless one supplies the real world constraints, those equations remain nothing more than equations -- certainly a marvel of mathematics but useless nonetheless to the real world.
</p>
<p>
The astute reader will recognize that lots of problems SE might not have a direct correspondence with the real world. For instance, in a compiler, there is no direct real world correspondence. Jackson acknowledges that his problem frames approach would require some modification to be used under those context. Nonetheless, it is undeniable that most SE projects involve some identification with real world objects and thus the problem frames approach might be useful.
</p>
<p>
The greatest contribution of problem frames is the fact that it is not just another tool for modeling. In fact, I would categorize it as a tool for synthesizing and distilling the real problem that needs to be solved. After applying the problems frames technique of modeling, certain patterns begin to surface. These patterns are useful to help software engineers solve recurring problems.
</p>
<p>
As an analogy, one seldom solves a problem in its natural context since that would require too many unknown variables. As a greatly simplistic example, consider the problem of counting how many apples one has after frequenting the grocery store. After kindergarten, most of us would be able to just use numbers to represent the apples without actually thinking about the apples themselves. For instance, if one had 3 apples before and bought another 3 later, then one would end up with 3 + 3 apples. We have reduced the problem of counting apples to the problem of adding two numbers together. The ability to add numbers together is more important than being able to count apples since counting is a skill that can be applied to numerous other domains.
</p>
<p>
Similarly by using the problem frames approach, one is able to distill certain recurring patterns of modeling. Effectively, if one is able to solve one of those patterns, then the same solution can be applied -- with some modifications of course -- to other problems. Jackson himself acknowledges that this is similar to the notion of Software Patterns popularized by the GoF Design Patterns book. However, unlike software patterns, problem frames are language and system agnostic; their solutions should be applicable to different styles of software development.
</p>
<p>
In engineering, there are two forms of design: normal design and radical design. Normal design is achieved using a set established <em>best practices</em>. For instance, a civil engineer never needs to start from square one when he is designing a new bridge. Instead he follows known conventions for designing a bridge. In fact, the task of designing the bridge can easily be decomposed into several different smaller projects that other engineers can work on simultaneously. Indeed, it is this set of established best practices that enable normal design to occur.
</p>
<p>
While SE has definitely developed some form of normal design using some best practices -- using design patterns, choosing some framework to prevent reinventing the wheel, and even choosing the architecture to run the system on -- most of the time, some form of radical design is going on behind the scenes. A radical design is a design that is not based on any established set of rules. In fact, a radical design might be just be an ad hoc solution that is never to be seen again.
</p>
<p>
In SE, the number of radical designs dwarves the number of normal designs. This makes it awkward to call SE an engineering discipline. There are just <em>too many ways to skin a cat</em>. And this abundance of solutions itself is a challenge to this field. The lack of the best practice principles makes it hard for a new software developer to quickly develop a new system to solve similar problems.
</p>
<p>
In short, this paper views the abundance of radical design versus normal design to be the problem of SE. This overabundance is an indication that the field of SE has yet to mature enough that known solutions to real problems have manifested themselves. Problem frames is proposed as one way to distill the dormant knowledge that permeates those problems and present them in a way that a software developer can reason about.
</p>
<p>
In my opinion, it might <a href="http://en.wikipedia.org/wiki/No_Silver_Bullet" title="No Silver Bullet - Wikipedia, the free encyclopedia">not be possible</a> to find a unifying method for software development. In many other engineering fields there are real physical constraints that must be obeyed for the system to work. There is nothing much you can do differently for a car since it must move along some road system. In SE, it is different. In software, you can do a lot of things differently unbounded by the physical rules. As an example, consider virtual reality games. Such systems are huge and each game tries to model the world closely. But there is nothing to prevent the virtual world to behave radically different. People could walk the other round and the game would still function correctly. In fact, other than the term virtual reality, these games have nothing much to do with reality since <em>anything</em> and <em>everything</em> can happen.
</p>
<p>
This freedom in computer science to model things differently is what makes it interesting for many people. In software, a lot of solutions are only limited by what creativity the designer can come up with. True, this leads to a whole hodgepodge of different solutions but this also makes this area of software more challenging. As another example, consider developing software for civil engineers. The software developer not only needs to know the domain knowledge -- how civil engineers reason about their tools and how to actually do the calculations -- but also how to write the software itself. To be useful, SE has to be cross-disciplined. Useful programs for <em>other</em> engineering disciplines can actually be done with software. Whereas I find it hard to imagine using some of the ideas from civil engineering to biomedical engineering.
</p>
<p>
So while problem frames is one way to solve certain problems, it is not going to radically transform the world of SE as we know it. Perhaps it is time to view software development as a different form of engineering -- one that is different from the other older disciplines. And if the word "software engineering" becomes a misnomer then it is only a matter of coming up with a more suitable name for it.
</p>
Squeak by Example - The Book2007-09-03T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/09/03/squeak-by-example-the-book<p><a href="http://www.iam.unibe.ch/~scg/SBE/">Squeak by Example</a>:
</p>
<blockquote cite="http://www.iam.unibe.ch/~scg/SBE/" title="Squeak by Example"> "This book presents the various aspects of Squeak, starting with the basics, and proceeding to more advanced topics. This book will not teach you how to program. The reader should have some familiarity with programming languages. Some background with object-oriented programming would be helpful."
</blockquote>
<p>
An updated book for Squeak is long overdue and <i>Squeak By Example</i> will fill that void. The book is still work in progress though a beta version of the pdf will be available soon. The Subversion repository of the LaTeX sources is available though and whoever is interested could generate the .pdf from that.
</p>
<p>
I find this book to be a positive step in introducing people to Squeak. I have commented before that a book is a physical manifestation of the quality and maturity of an open source project. If you dealing with a close source project then the company backing that project up could easily invest money into making documentation or even publishing a book in that technology. However, when dealing with an open source project -- where <i>all</i> documentation is freely available in the form of code as the proponents would claim -- the commitment to create a book shows just how much the community loves the product and want to get other people excited about it too.
</p>
<p>
<i>A book shows commitment to the project</i>. After all, why waste time writing a book if the project is going to die some time down the road. Now, this does not mean that the project will not die in the future. But it shows that the people behind it are committed enough that they don't foresee this project shutting down anytime soon. In short, the project has enough momentum to sustain itself for some time.
</p>
<p>
<i>A book shows excitement for the project</i>. And this book practically bubbles with it since it is co-authored by one of the most prominent figures in the Squeak Community, Stephen Ducasse. If a project leader is excited enough about the product to write a book then you can be certain that it's an exciting product.
</p>
<p>
<i>A book shows the maturity of your project</i>. After all, it's too risky to write a book if things are going to change. Maturity is a sign of stability -- meaning that this product is unlikely to undergo a capricious change in the near future. Thus, if your product depends on Squeak, it is probably not going to suddenly be incompatible in the future.
</p>
<p>
<i>A book shows that other people are interested</i>. Why write when you don't even have an audience? This is actually a catch-22 since you wouldn't write a book if there is no audience but you wouldn't really have an audience if you don't really have good documentation to inform them adequately in the first place.
</p>
<p>
In short, I think an updated book is a good sign that the Squeak community is committed and excited about how much quality and maturity Squeak has achieved. I will be looking forward to this book and hopefully be able to incorporate it in the Smalltalk class that I am teaching assistant for.
</p>
Erlang2007-07-23T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/07/23/erlang<p>A History of Erlang:
</p>
<blockquote>
This I did first in Smalltalk and then in Prolog. This was fairly sensible at the time, since I literally interpreted Bjarne's directive to "solve all of Ericson's software problems" as "program POTS in Smalltalk".
</blockquote>
<p>
This post is based on the paper <i>A History of Erlang</i> by Joe Armstrong which was recently presented at <a href="http://research.ihost.com/hopl/HOPL-III.html">HOPL-III</a>.
</p>
<p>
About a year ago, I stumbled upon the article <a href="http://www.defmacro.org/ramblings/concurrency.html"><i>Erlang Stye Concurrency</i></a>. It piqued my interest in Erlang and I decided that it would be an interesting language to learn. However, I put it off due to the lack of a good book. Now, a year later, there is a good book for it: <a href="http://www.pragmaticprogrammer.com/titles/jaerlang/">Programming Erlang</a> by the Pragmatic Programmers. I've managed to cover the first few chapters of the book and thus was acquainted with the basic syntax of the language. Erlang reminded me of a cross between <a href="http://maude.cs.uiuc.edu/" title="Maude">Maude</a> and <a href="http://www-swiss.ai.mit.edu/projects/scheme/" title="The Scheme Programming Language">Scheme</a>.
</p>
<p>
Anyway, I had enjoyed a couple of papers presented at HOPL before and was delighted to see that Erlang had made the list for the latest conference. The paper is fairly easy to read and it does not dwell too much on the technical side of Erlang but it does have some interesting points, some of which I list below.
</p>
<p>
Erlang has this requirement where a variable must begin with an uppercase character. I was a bit appalled when I discovered this since it reminded me Prolog - a programming language that I have yet to appreciate. A identifier that begins with a lowercase character is automatically treated as an atom (or a symbol as one would call it in Ruby or Smalltalk). My suspicions on the connection between Erlang and Prolog were confirmed in the paper. Joe Armstrong, one of the creators of Erlang did indeed write the first interpreter for Erlang in Prolog. This was one of the reasons that the convention for capitalized identifiers as variables persisted in Erlang. It is also interesting to note - though not the fault of Prolog in any way - that there is an upper limit to the number of atoms that one can have in a program since atoms are stored in a fixed-size hashtable. This limit is some ridiculously large number but it is possible to run out of atoms nonetheless.
</p>
<p>
Another interesting aspect of the Erlang language is that the semantics of threads, processes, concurrency are built-in the language itself. In fact, the whole idea of concurrency and its importance at are the center of Erlang. After all, this was a language designed at Ericson to help the company solve its software problems. <strike>Thus, Erlang can be considered a large language which is, in general, frowned upon by language designers since it lead to complications in semantics, implementation, training, etc.</strike> However, the power of Erlang comes from its design since concurrency is already inherent in the language without the need for any other glue code or libraries.
</p>
<p>
Though not discuss extensively in the paper, Erlang has the useful property of being able to support garbage collection while still being able to meet the soft real time constraints that it was designed for. This property cannot be guaranteed 100% but the pathological case of the garbage collector interfering with the real time constraints is so improbable that is non-existent. The key strategy was to use per-process stop-and-copy GC. With many small processes, the cost of garbage collection would be negligible. The more processes that are used, the less likely that the effects of garbage collection would be noticeable.
</p>
<p>
Although Erlang was conceived by the scientists at Ericson, it was actually banned for new product development within the company during the late 90s! The good news is that this ban eventually led to the open source version of Erlang and paved the way for Erlang to spread outside of Ericson. Which is how we could get our hands of an implementation of it to play with.
</p>
<p>
Erlang is indeed an interesting language and it is one that I shall be devoting more time to now that I have purchased the book. Even during my early evaluation of the language, its model for concurrency seems more maintainable and understandable compared to the traditional view of using mutexes and semaphores.
</p>
Finding Code in Squeak2007-05-30T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/05/30/finding-code-in-squeak<p>
<em>This post is inspired by the contents of this <a href="http://wiki.squeak.org/squeak/783" title="FAQ: Finding Code in an Image">entry</a> at the Squeak Wiki FAQ. I just added some pictures to make it more clear.</em>
</p>
<p>
Squeak is an interesting system. All the source code for the system is embedded in the system itself. This means that you can actually go ahead find the code where a certain feature has been implemented rather easily. And if you desire, you can even change that behavior to suit yourself. Of course, before either of that could happen, you need to know <strong>how</strong> to look for the code first. I am going to show you three methods for locating code in Squeak. As our example, we will be looking at ways to find where the "Save and Quit" functionality is implemented.
</p>
<p>
The first method requires the use of the Method Finder (or selector Browser). You can drag an instance of the Method Finder from the tools flap in Squeak. Or you can create a new instance from the World Menu. Just as its name implies, you can use the Method Finder to find methods. You just need to enter a fragment of the method name in the search box and look for it. Sometimes this requires guessing the method name. Fortunately the naming convention in Squeak is fairly consistent and most of the time your guess will help you find the method.
</p>
<p>
<a href="http://www.flickr.com/photos/vazexqi/522194745/" title="Photo Sharing"><img src="http://farm1.static.flickr.com/254/522194745_1772493c33.jpg" width="500" height="371" alt="Finding code in Squeak - 1" /></a><br />
<small>Here I am guessing that there must be a method called "saveandquit" so I enter that into the Method Finder (or Selector Browser) and try to find it. The results show that there is indeed such a method and that two classes have it: TheWorldMainDockingBar and TheWorldMenu.</small>
</p>
Installing packages in Squeak2007-05-26T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/05/26/installing-packages-in-squeak<p>
In my previous <a href="http://softwareengineering.vazexqi.com/articles/2007/05/25/getting-acquainted-with-squeak">post</a> I mentioned some useful packages that do not come installed by default in normal distribution of Squeak. There are two ways that you can get hold of those packages in your image.
</p>
<p>
The first way is to download a specially packaged image that contains most of the developer tools. You can get that image from <a href="http://damien.cassou.free.fr/squeak-dev/">here</a>. The advantage is that you get all the tools that Squeak developers usually use. The disadvantage is that if this is your first time using Squeak, you might not know all the tools. Hence you end up with an image that contains a lot of things that you will not use. And documentation for tools are usually pretty sparse so it is a good idea to start small and selectively choose the tools that you will use.
</p>
<p>
Thus, I present the second method to get hold of those tools: installing them as individual packages. For this process, I am going to start with a clean image to show you how it is done. I will install the <a href="http://map.squeak.org/package/e5c1aec2-0eb9-492d-8cb1-44f64c5b631a">Shout workspace</a> package which gives you syntax highlighting in the workspace.
</p>
<p>
<a href="http://www.flickr.com/photos/vazexqi/515145938/" title="Photo Sharing"><img src="http://farm1.static.flickr.com/214/515145938_e9f9810d59.jpg" width="500" height="375" alt="Installing packages in Squeak - 1" /></a>
<br />
<small>Bring up SqueakMap Package loader first.
</small>
</p>
Getting acquainted with Squeak2007-05-25T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/05/25/getting-acquainted-with-squeak<p>
There are lots of excellent <a href="http://wiki.squeak.org/squeak/377">introductions and tutorials</a> on Squeak on there so this is not an attempt at another tutorial. Instead, this will hopefully be a condensed version of the information I would have liked to know the first time I use something like Squeak.
</p>
<p>
When you first start Squeak, your initial reaction might be something along the lines of: <i>What is this thing!?</i> You are probably in a state of shock and disbelief especially if you have been programming with text editors and the command line or use some nifty IDE. Squeak 3.9 actually has a more appealing interface compared to prior versions of Squeak.
</p>
<p>
After overcoming your initial shock, you are now ready to begin dwelling into Squeak. You should read all the initial documents that pop up when you first start Squeak. They contain useful information on how to get started. You should also be prepared to use a three-button mouse; while you can emulate the three buttons, having direct access to the buttons on your mouse makes you more productive. Squeak windows behave differently compared to normal windows in your operating system. You will need to click on the window using the right-mouse button to gain access to some of the window manipulation tools. Hover your mouse over each icon to get a description of what each does.
</p>
<p>
Here is what a typical development session in Squeak might look like. There will be a bunch of windows open as you develop in Squeak.
</p>
<a href="http://www.flickr.com/photos/vazexqi/513498632/" title="My Squeak setup"><img src="http://farm1.static.flickr.com/191/513498632_e57b7e4965.jpg" width="500" height="372" alt="My Squeak setup" /></a>
<p>
The main thing you should realize is programming in Smalltalk is very different from languages such as Java, Ruby, C#, etc. Programming in Smalltalk entails using its own system. You do not use an external editor, you do not really have a compile/execute phase, and you do not fire up FireFox to read documentation on the web! You even get some primitive form of source control without using CVS or Subversion! Everything for developing in Smalltalk in built-in. So when people refer to Smalltalk, they usually refer to the whole package. And yes, this does mean that you do not really have much of a choice if you would really like to use your favorite text editor.
</p>
<p>
I assume that you have done some of the basic built-in tutorials in Squeak and are familiar with how to navigate in Squeak. In the next few paragraphs, I am going to tell you more about the Squeak image and what you can do with it. I think this is fairly useful and will help you in case you did something wrong in your initial fiddling with Squeak and would like to find out what you can do to rectify it.
</p>
<p>
Everything you do in Smalltalk will eventually be saved in the image. The image is the file that you is named like <i>Squeak3.9-final-7067.image</i>. Accompanying the image are <b>two</b> very important files: the .changes file and the .sources file. Under no circumstance should you attempt to delete them (as I had in the past!) no matter how large they grow. I will explain one way to condense the .changes file in another post but for now leave it alone. For a short description of the purpose of those files, refer to <a href="http://swiki.cs.uiuc.edu/cs598rej/336">here</a>.
</p>
<p>
This image is actually a very useful. The closest thing that I can think of that is similar to a Smalltalk image is the image file that you use in <a href="http://www.vmware.com/">VMWare</a>. Even then there are subtle differences. Everything you do in VMWare is recorded directly into the image whereas in Squeak you have the option to just quit without saving the image. <b>Note: This is a useful feature. If you think you have messed something up, you should always quit without saving</b>. And when you restart the image, it reverts to the last time you actually saved it (where everything was hopefuly working).
</p>
<p>
What if you forget to save the image before quitting and you have just done a whole bunch of work? Does this mean that all your work is gone?! Fortunately it does not. Squeak stores <i>everything</i> that you have done in the .changes file. So you can always start with the last image that you have saved and then use the .changes file to get your stuff back in. This is a pretty cool feature and there is a <a href="http://www.cincomsmalltalk.com/casts/stDaily/2006/smalltalk_daily-11-10-06.html">video</a> showing this in a different version of Smalltalk. To get the same features in Squeak, just left-click on the <i>desktop</i>, select <b>changes...</b>, select <b>recently logged changes...</b> and you have a similar menu where you can see the previous changes that you made.</p>
<p>
What if you want to revert back to a previous version of the image after saving it? This is similar to version control and I will post more on that later. In a nutshell, there are two main ways to do version control: the <a href="http://wiki.squeak.org/squeak/674">changeset</a> and <a href="http://www.wiresong.ca/Monticello">Monticello</a>. If you are new to Squeak, I suggest you just skip the changesets and take a look at Monticello. It is buit-in to Squeak 3.9 and most new packages that you might install are already using the Monticello format.
</p>
<p>
When you are just starting to get familiar with Squeak, it is always a good idea to keep a backup copy of the .image, .change and .source files. Even though there are tools that can help you restore the image, you might accidentally screw it up so bad that it is simply easier to just restore from a backup first and then try to apply your changes again.
</p>
<p>
Over the next couple of days, I will be explaining how to get hold of some tools that will make developing in Squeak more convenient for you. If you cannot wait, you can always try installing them on your own:
<ul>
<li><a href="http://uncomplex.net/ecompletion/">eCompletion</a> - Enables smart auto completion to the squeak system </li>
<li><a href="http://map.squeak.org/package/1314cee7-e1a3-4ca5-a167-b6517bf16367">Shout</a> - Enables syntax highlighting as you type in the code browser</li>
<li><a href="http://map.squeak.org/package/e5c1aec2-0eb9-492d-8cb1-44f64c5b631a">Shout workspace</a> - A workspace that syntax highlights using Shout</li>
<li><a href="http://www.swerlingphoto.com/squeak/svi2/svi2.html">SVI</a> - A text editor for Squeak that supports dozens of VIM and Emacs commands and modes. Useful when you really do not feel like picking up the mouse</li>
<li><a href="http://www.mindspring.com/~dway/smalltalk/whisker.html">Whisker Browser</a> - A browser that allows you to view/read/compare source code more intuitively</li>
</ul>
</p>
<p>
There is also a set of excellent screencasts for VisualWorks (another distribution of Smalltalk) available <a href="http://www.cincomsmalltalk.com/userblogs/cincom/blogView?content=smalltalk_daily">here</a>. While the screencasts revolve around VisualWorks, there are similarities that make it pertain to Squeak as well.
</p>
Creating projects in Squeak2007-05-25T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/05/25/creating-projects-in-squeak<p>
One way to reduce the clutter in Squeak is to create projects. Contrary to its name, a project is more of a organized view of the image. Every project shares the same code in the image; changes made in one project affect code in another project that reside in the same image. So why would anyone want to do this? Like I mentioned, projects are a nice way to organize the things that you are working on.
<p>
A similar analogy is the <a href="http://www.polarhome.com/vim/manual/v70/usr_21.html#21.4"><code>mksession</code></a> command in gVim. It allows you to record the current state of your files, which files are opened, etc. Eclipse has a similar feature called <a href="http://wbeaton.blogspot.com/2005/11/leanin-on-working-sets.html">working sets</a> but it is not as useful since it does not remember the files that are opened in the editor.
</p>
<a href="http://www.flickr.com/photos/vazexqi/513914762/" title="Photo Sharing"><img src="http://farm1.static.flickr.com/225/513914762_971d50989e.jpg" width="500" height="372" alt="Creating a project in Squeak part 1" /></a>
<br />
<small>Click on the <i>desktop</i> to get the World Menu and select projects...</small>
<br />
<br />
<a href="http://www.flickr.com/photos/vazexqi/513914766/" title="Photo Sharing"><img src="http://farm1.static.flickr.com/207/513914766_a9290802a9.jpg" width="500" height="372" alt="Creating a project in Squeak part 2" /></a>
<br />
<small>Select create new morphic project.</small>
<br />
<br />
<a href="http://www.flickr.com/photos/vazexqi/513914768/" title="Photo Sharing"><img src="http://farm1.static.flickr.com/230/513914768_441b8cdf17.jpg" width="500" height="372" alt="Creating a project in Squeak part 3" /></a>
<br />
<br />
<small>Click on the window menu (it looks like a stack of drawers and is on the top left) to select change title...</small>
<br />
<p>
Give the project a name and then hit accept. You can now click on the project to enter it. You can even drag it around. Explore the project... menu to see what else you can do. One quick way to see all the projects and quickly navigate to them is projects... > jump to project...
</p>
<p>
An easier way if you have the navigator flap open is to select it and click the "NEW" button.
</p>
<p>
Once you get used to using projects, you will wish that every IDE has this feature. The ability to remember the <b>exact</b> state of all your open windows and their layout is actually pretty useful to get your brain back into the context it was in while working on your code.
</p>
Updated website to Typo 4.12007-05-24T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/05/24/updated-website-to-typo-4-1<p>
Apparently, the same problem that hit my other <a href="http://blog.vazexqi.com">blog</a> a few months ago has haunted this blog. This morning, all I did was add a new category (the squeak category; new posts on that coming soon). After saving the changes, my entire layout was messed up. The CSS will not work properly and the theme will not even load.
</p>
<p>
Since I have already seen this kind of problem before, I just decided to upgrade to Typo 4.1. This upgrade process was long overdue anyway. After checking the <a href="http://typosphere.org">typosphere.org</a> website, I realize that Typo 4.1.1 was released. Usually I will not bother with a .1 increment release but with bleeding-edge software like Typo sometimes the latest version has bug fixes that will make all the difference between having a smooth transition and a few wasted hours trying to get things to work.
</p>
<p>
Unfortunately, Typo 4.1.1 required the use of a <i>newer</i> version of <code>rubygems</code>. The files no longer use the old <code>require_gem</code> method but the newer <code>gem</code> method. So it was not possible to install Typo 4.1.1
</p>
<p>
Anyway, long story short, I finally got Typo 4.1 installed, updated the theme (I used the default ones so that I will not suffer from any compatibility issues). There is some problems with the new sidebar AJAX interface: I was having trouble dragging and dropping the items; sometimes they refuse to <i>stick</i> to the active sidebar and merely float around the page.
</p>
<p>
Things seem to be working the way I want them to now and I will probably not mess around too much with the layout anymore.
</p>
Aspect-Oriented Programming Refactoring2007-01-30T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/01/30/aspect-oriented-programming-refactoring<p>
In my previous post I mentioned how there aren't any reliable refactoring tools for AOP yet. Seems like this problem is slightly more involved than the refactoring that most of us are used to. I have been reading a bunch of papers on this subject (mostly the top hits from the search for <i>Aspect-Oriented Refactoring</i> in <a href="http://scholar.google.com/scholar?q=aspect+oriented+refactoring&hl=en&lr=&btnG=Search">Google Scholar.</a>). The paper that best summarizes the problems and challenges is <a href="http://aosd.net/workshops/late/2006/later/submissions/hannemann.pdf"><i>Aspect-Oriented Refactoring: Classification and Challenges</i></a> by Jan Hannemann.
</p>
<p>
There are three main issues with refactoring aspects:
<ul>
<li>Aspect aware refactoring tools <br />
If you are using AOP with your OO program, it is no longer safe to just call a <i>normal</i> refactoring like rename method. Calling rename method might actually not respect the aspects that are tied to that particular join point in the code. Thus, you affect the aspects that are advising the code. In all but the most trivial cases, this can have undesirable effects.
</li>
<li>Aspect-oriented programming refactoring <br />
AOP itself is a rich language. As with any language, there is a potential for refactoring to make the code easier to read and maintain.
</li>
<li>OO to AOP refactorings <br />
Most programs are going to be a combination of OO and AOP programming styles. To get the best of AOP, we have to gradually address the points of crosscutting concerns in our OO program and then refactor them to use aspects. There is research being done on <i>aspect mining</i> - automatically detecting crosscutting concerns and presenting the choice for refactoring those concerns into aspects.
</li>
</ul>
</p>
<p>
A fourth issue, I would like to add is being able to refactor from aspects to the normal OO constructs. Sometimes this might be desirable. For instance, an easier and more efficient refactoring to the code might be possible by first pushing the aspects back into normal code, perform some organization on that source code and then refactor into a new aspect.
</p>
<p>
Another paper - <a href="www.old.netobjectdays.org/pdf/03/papers/node/019.pdf"><i>Refactoring of Aspect-Oriented Software</i></a> - claims to have created a prototype to addresses the first bullet point above. However, I am not able to get a hold of their implementation which is a plug-in for Eclipse. The paper has identified the steps for each refactoring to be performed by hand (and also to be verified so that it behaves correctly). It is a good start and it is written along the lines of the first thesis on refactoring by William Opdyke.
</p>
<p>
Aspect-Oriented programming brings with it the interesting issue that it is no longer sufficient to treat the original source code as the <i>only</i> thing that matters. As programs evolve, they will be written with many different languages that are intertwined together. Each language will be complicated enough that it can be refactored to be made better but refactoring would also cause repercussions in other parts of the program. Solving (or at least getting a better understanding of the problem) from AOP refactoring will be useful when we have to deal with the same issue in domain-specific languages.
</p>
<small>
For good introductions to this topic, you can read <a href="http://www.theserverside.com/tt/articles/article.tss?l=AspectOrientedRefactoringPart1">part 1</a> and <a href="http://www.theserverside.com/tt/articles/article.tss?l=AspectOrientedRefactoringPart2">part 2</a> of the Aspect-Oriented Refactoring series.
</small>
Design Patterns and AspectJ2007-01-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/01/29/design-patterns-and-aspectj<p>
This post is based on the paper <a href="http://www.cs.ubc.ca/~gregor/papers/hannemann-OOPSLA2002-aop-patterns.pdf"><i>Design Pattern Implementation in Java and AspectJ</i></a> by Jan Hannemann and Gregor Kiczales. There is also a <a href="http://video.google.com/videosearch?q=Kiczales">video</a> related to AOP and it might be beneficial to watch.
</p>
<p>
I have been reading quite a few papers on Aspect-Oriented Programming recently. Mostly I believe that there is a potential use for aspects. For instance, in the previous post I mentioned how it could be used to extend otherwise closed classes or libraries.
</p>
<p>
This paper is a discussion of implementing the original 23 GOF design patterns in AspectJ. Undeniably, anyone should already know that it is possible to implement the GOF design patterns in <i>any</i> programming language. The more important issue is how easy it is to implement in AspectJ compared to plain old Java? I read the paper and did not feel that it was comparably simpler. My criteria for simplicity is based on how easy it is to understand the code and how many lines of code is needed.
</p>
<p>
The authors argue that one of the drawback of using design patterns in Java is how <i>mangled</i> the code becomes. Furthermore, they argue that additional code needed to implement the design pattern interferes with the domain model underneath. This gets progressively worse as patterns are composed together. They claim that it becomes harder to reason about the code and the interactions because of this mangled structure. And thus, the propose a solution in AspectJ.
</p>
<p>
True, with AspectJ the authors managed to achieve their goal of modularity and reuse. But modularity and reuse do not necessarily make the code simpler to reason about. In fact, artificially imposed modularity makes the code very hard to read and understand. Design patterns avoid this form of modularity. Instead, they present a <i>theme</i> for understanding the code. For instance, in the Observer pattern the most important idea is understanding the roles played by the Subject and the Observer. Once you have identified them, reasoning becomes simpler.
</p>
<p>
In the video (25th minute), Kiczales tries to convince people how aspects can in fact make it easier to reason about the programs <i>once</i> we think at a higher level: <b>what</b> we want to accomplish rather than <b>how</b> we want to do that with objects and other OO design patterns. There were the usual question of whether aspects will create spaghetti code since you can weave things into any class. Whether his argument is convincing or not, it is for you to decide.
</p>
<p>
I downloaded the source code for the examples in this paper and loaded it inside Eclipse with the AspectJ Developement Tools installed. While it does provide some useful features for notifying the programmer which aspects affect the current line of code it may not really be intuitive at first (maybe there is no known paradigm for tools in AOP yet). Refactoring support for AspectJ has not been implemented so it hard to actually make changes. This makes it hard for developers to use AspectJ in a <a href="http://c2.com/cgi/wiki?PiecemealGrowth">piecemeal growth</a> manner.
</p>
<p>
One issue that the authors failed to address properly is how to compose several patterns together. Composing aspects together is a tricky process because undesirable interactions may occur based on the order of weaving and the parameters that are used to identify the pointcuts. Since patterns seldom exist in isolation, the authors should have addressed this issue more carefully in their paper. The source code for the examples do not demonstrate patterns interacting together.
</p>
<p>
My conclusion is that I am more interested in finding out any new patterns that would emerge with the use of AOP. The authors themselves acknowledge that the implementation for some of original 23 design patterns were transparent in more dynamic languages such as Lisp or Scheme. The authors did cite some papers and research that was in progress to address those issues and I will probably be taking a look at them.
</p>
Aspect-Oriented Programming2007-01-23T00:00:00+00:00http://softwareengineering.vazexqi.com/2007/01/23/aspect-oriented-programming<p>
Aspect-Oriented Programming (AOP) is now a popular buzz word. Well since it has survived for nearly 10 years, I figured that it was worth taking a look at it.
</p>
<p>
So far I have read two papers on AOP: <a href="http://portal.acm.org/citation.cfm?doid=1111542.1111554"><i>A Disciplined Approach to Aspect Composition</i></a> and a draft of a paper to be submitted. I will refrain from detailing the draft in this post. Suffice to say it introduces 4 broad patterns that give a better perspective on what AOP can do and how to do it.
</p>
<p>
In general, there is nothing in AOP that OO cannot do. Well, in general, there is nothing in OO that you cannot do with procedural code or functional programming. So, while AOP is nothing that great, it could be something that could be useful for certain programs. If you have heard a bit about AOP you might remember the examples about monitoring and logging. Rest assured that AOP is a lot more than just plain monitoring and logging.
</p>
<p>
For instance, AOP is an interesting way to do program transformation without actually modifying the source code. For instance, you cannot really modify the String class in Java. For instance, you might want to add the method camelCase or snakeCase. Trust me these methods can be handy if you are trying to do some metaprogramming. But since class String is close to modification, you cannot add those methods directly. Instead what you would usually do is subclass the String class or add some String utility class. With AOP you could use <i>introductions</i> to simulate adding that method to the class String. Everywhere you do "some string".camelCase() it actually modifies the code to call a camelCase method somewhere else (not in class String).
</p>
<p>
Programmers in Ruby and Smalltalk will not consider the example above relevant since they are free to change the internals of the classes because its open for modifications. However, consider the potential AOP has for proprietary systems. You will never be able to get the source code for those systems to hack on. But at least now with AOP you can do dynamic woven and get the behavior that you want do. This seems like some ersatz compared to actually modifying the code but at least it provides itself as an alternative.
</p>
<p>
AOP still lacks the tools that make it useful though. For instance for Java and Smalltalk we are used to IDEs that implement "Call hierarchy", "Type hierarchy", "Show all implementors", etc. As far as I know, there are no such tools for AOP yet. Maybe AOP is not mature enough that we know what kinds of tools that we need. For instance, it is not likely to be hard to come up with some modeling language like UML to describe AOP.
</p>
<p>
At the moment, one of the most frustrating things about AOP is the inability to see how your code works. Aspects can be woven in statically or at run-time. And depending on run-time conditions, different aspects might be woven in potentially overwriting previous behavior.
</p>
<p>
An interesting approach to AOP might be design your program in the normal OO way first. And then <b>refactor</b> to aspects. For instance, design your program with all the superfluous monitoring and logging baked in first. And then as you begin to see the underlying domain to your program, you can refactor those that do not really belong into aspects. Therefore, you are not trying to fit your program into the AOP mental model but rather you are trying to get to your domain model by moving things out into aspects. In a nutshell, AOP complements OO design.
</p>
<p>
AOP is not a panacea for all programming problems. However, it might be useful for making certain mundane programming tasks simpler. Personally for me AOP borders on the capabilities of metaprogramming. AOP seems to be an <i>external</i> fix for languages with weaker reflection and metaprogramming capabilities. Instead of baking a heavyweight reflection system into the language, you can add it on with an aspect compiler. So people who do not use AOP will not have to pay for the cost of having that ability built-in.
</p>
Domain Driven Design Quickly2006-12-19T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/12/19/domain-driven-design-quickly<p><a href="http://www.infoq.com/minibooks/domain-driven-design-quickly">InfoQ: Domain Driven Design Quickly</a>:
</p>
<blockquote cite="http://www.infoq.com/minibooks/domain-driven-design-quickly" title="InfoQ: Domain Driven Design Quickly"> "The most complicated aspect of large software projects is not the implementation, it is the real world domain that the software serves. Domain Driven Design is a vision and approach for dealing with highly complex domains that is based on making the domain itself the main focus of the project, and maintaining a software model that reflects a deep understanding of the domain"
</blockquote>
<p>
In this class, we read the excellent book <i>Domain-Driven Design</i> by Eric Evans. The book itself was interesting and useful, but it was too thick and heavy to bring around. And as far as I know, there isn't a legal .pdf copy of it. Well thankfully, InfoQ has been kind enough to condense the tome into a 106 page document in .pdf. This will certainly be useful. I have not read through the entire 106 pages but from the Table of Contents it seems to cover a good chunk of the book. This .pdf will definitely serve as a convenient review and summary of the book for me.
</p>
<p>
Some caveats. The text layout is a bit funky for the book. It definitely has not gone through a good typesetting process (it looks like it was written in Word with left-right text justification turned on) but it is readable. Some of the images are also blurry and that shows how much effort was put into this .pdf. But it is free and it a good effort to getting more people interested in Domain-Driven Design.
</p>
Python Refactoring (Transformation) Tool2006-12-15T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/12/15/python-refactoring-transformation-tool<p><a href="http://mail.python.org/pipermail/python-3000/2006-December/005096.html">[Python-3000] Refactoring tool available (work in progress)</a>:
</p>
<blockquote cite="http://mail.python.org/pipermail/python-3000/2006-December/005096.html" title="[Python-3000] Refactoring tool available (work in progress)"> "In the sandbox I've been working on a refactoring tool, which could form the basis for a Python 2.x -> 3.0 conversion tool. I'd like to invite folks here to give it a try and give me a hand. It certainly needs more work, but I think that the basic infrastructure is sound. Check out sandbox/2to3/: http://svn.python.org/view/sandbox/trunk/2to3/."
</blockquote>
<p>(Via <a href="http://www.oreillynet.com/onlamp/blog/2006/12/guido_announces_refactoring_to.html">Guido Announces Refactoring Tool</a>.)</p>
<p>
I am not a regular Python user but this tool seems pretty interesting. I have not seen any other tool that could potentially refactor (or transform) a language into a new version of its own syntax yet. Most just involve changing the structure of code while preserving the behavior. However the syntax is limited to the current syntax. Using the refactoring tool to help migrate old syntax to new is while preserving behavior is certainly a good idea.
</p>
ArchJava2006-12-05T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/12/05/archjava<p>
This post is based on the paper <a href="http://archjava.fluid.cs.cmu.edu/papers/icse02.pdf"><i>ArchJava: Connecting Software Architecture to Implementation</i></a> by Jonathan Aldrich, Craig Chambers, and David Notkin [ACN]. We heard of ArchJava in my previous <a href="http://softwareengineering.vazexqi.com/articles/2006/11/06/software-architectural-primitives">post</a> on Software Architectural Primitives.
</p>
<p>This paper stresses the importance of tying the software architecture to its implementation. Consistency between the architecture and implementation is important for analysis and reasoning. The current state of software development usually decouples the software architecture from its implementation. [ACN] argue that this is undesirable since it becomes harder to keep the architecture and implementation in sync. To emphasize their point, they introduce the term <i>communication integrity</i>:
</p>
<blockquote>
A system has communication integrity if implementation components only communicate directly with the components they are connected to in the architecture.
</blockquote>
<p>
A system with communication integrity, the authors argue, enable architectural reasoning about its implementation. Thus, they propose ArchJava, an extension to the Java programming language as a way to maintain communication integrity. ArchJava adds new language construct to support <i>components</i>, <i>connections</i> and <i>ports</i>.
</p>
<p>
What really surprise me is the resemblance this extension has with VHDL. VHDL is a hardware description language that uses ports, components and connections to describe electrical components in code. VHDL is meant to mimic the electrical components closely. It was hard to program in VHDL because you need to keep track of the in and out ports that each component provides. However, because of those ports, it was actually easier to ensure that our components had the right specifications.
</p>
<p>
In VHDL IDEs. there are usually tools that help you make the connection between components. You can drag a line to show that one port was connected to the other; it would then generate the code that actually did that connection for you. Apparently there is no such tool for ArchJava and the developer has to manually connect the components using code.
</p>
<p>
One of the problems I have with ArchJava is the fact that if you decide to use it somewhere in your program, you would need to use it <i>everywhere</i>. It is not possible to just describe one part of the system (maybe the more complicated parts) and leave the simpler parts alone. Therefore, existing programs that want to take advantage of ArchJava have to be rewritten. The paper has an example of rewriting. And while rewriting did not take a lot of time, it is not a trivial task either. This could discourage exiting projects from adapting ArchJava.
</p>
<p>
I also suspect that ArchJava makes programming more restrictive. For developers using dynamic languages, developing in static-type languages already feels very restrictive. I suspect that it would feel even more restrictive once ArchJava comes into the picture. It would require discipline to follow the those rigid requirements. This might be a good thing but it is probably hard to write a quick prototype using ArchJava.
</p>
<p>
The main advantage I see with ArchJava is that connections between modules is now explicit. This can be useful for developers when they need to do a quick review if they are actually following the specification.
</p>
<p>
In my opinion, ArchJava is not meant for everyone. If your development project requires stringent adherence to specifications, then ArchJava might be a solution. But for most projects, it is probably too restrictive.
</p>
Documentation for Typo2006-12-02T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/12/02/documentation-for-typo<p>
For the class project in CS527, I am documenting the Typo content management system. Typo is the back-end that powers this weblog.
</p>
<p>
We are supposed to follow the documentation style from the <a href="http://www.amazon.com/Documenting-Software-Architectures-Views-Beyond/dp/0201703726"><i>Documenting Software Architecture Views and Beyond</i></a> book. There is some good advice there but following it blindly will make the documentation too cryptic for even the most enthusiastic readers.
</p>
<p>
I have tried to follow the different views that the book suggests. The end product will have module view, component-and-connector view and allocation view types. Whenever possible, I have also included the code excerpts from Typo as well as links to external sites that describe the rationale behind certain design decisions. Each view is explained in the text and related views and tied together.
</p>
<p>
I do not intend to produce something that mimics the example at the back of the book. The example at the back of the book is meant to cater to various stakeholders so there is an <i>obscene</i> number of views. Moreover, each view follows a template that needs to be described as well.
</p>
<p>The stakeholders for Typo are mainly Rails developers and web designers. Therefore they are more familiar with some of the terms that we will be using. I have written it in such a way that it is possible to read the entire documentation from cover to cover without it being too boring. After all, what is the use of documentation that no one reads. This documentation also serve as a guide on where to obtain more information on Typo. There is no authoritative source of documentation for Typo at the moment, even though there are various tutorials on the web on how to modify certain extension points for it.
</p>
<p>
The advantage of an open source project is the availability of the source code. Thus, this documentation should serve as an overview of the relevant modules in the source code. Moreover, Typo is an evolving project so producing a detailed documentation that will only last for one version is not going to be useful. Instead, by documenting the major modules that are unlikely to change, that documentation remains useful even in the next release.
</p>
<p>
Incidentally, this is the first time I am using LaTeX for writing so there could be some weird artifacts with how the text looks. I describe my <a href="http://blog.vazexqi.com/articles/2006/11/29/latex-and-how-to-learn-it">process</a> of learning LaTeX on my personal weblog.
</p>
<p>
<b>An auto-generated draft is available <a href="http://softwareengineering.vazexqi.com/files/typo_documentation.pdf">here</a></b>. It will be updated frequently but I want to get a version out first since Prof. Johnson requested that we post a copy by the end of the week --- Saturday is the end of the week. The sections on Module views and Allocation Views are pretty complete but the rest are probably going to undergo more changes. Right now the figures have explanations in the text but they do not have a key associated with them. The book suggests that each figure be accompanied by a key. I am still deciding if this is really necessary.
</p>
<p>
Update: The link to the documentation now contains a <i>complete</i> version of the documentation. By complete, I mean that it has all the views that I had planned to address. I have also just read through it and corrected some minor mistakes. Unless I decide to include a new view or receive comments to change certain aspects of the documentation, the final version will be similar to the current one. The first draft is still available from <a href="http://softwareengineering.vazexqi.com/files/document.pdf">here</a> for comparison.
</p>
Worse is neither better nor worse !2006-11-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/29/worse-is-neither-better-nor-worse<p>
This blog post is primarily based on the paper <a href="http://dreamsongs.com/WorseIsBetter.html"><i>Worse is Better</i></a> by Richard Gabriel. I have taken the liberty of reading most of the articles that are linked on that page to get a better feel of this topic. Another article that is worth reading is <a href="ttp://www.artima.com/weblogs/viewpost.jsp?thread=24807"><i>Worse is Worse</i></a> by Jim Waldo.
</p>
<p>
One of the problems (I hear) that most new comers to Lisp have is the fact that the community itself is pretty divided. There is no "one" implementation of Lisp<sup><a href="#fn1">1</a></sup> (the <i>standard</i> is supposed to be Common Lisp). And there is no one "right" way to do things. In fact, having more than one way to do things is probably one of the appealing features of Lisp. Nonetheless to a new comer who is learning a functional language for the first time, the arduous task of learning a new programming paradigm and trying to get things to work in the simplest way possible might be too much. And the worse scenario is when new comers are chagrined for mentioning that which-must-not-be-mentioned: Lisp and its <i>hideous</i> parentheses<sup><a href="#fn2">2</a></sup>.
</p>
<p>
But I digress. The main point of this article is not really about Lisp. It's about the <i>Worse is Better</i> approach to software design.
</p>
<p><a href="http://en.wikipedia.org/wiki/Worse_is_Better">Worse is better - Wikipedia, the free encyclopedia</a>:
</p>
<blockquote cite="http://en.wikipedia.org/wiki/Worse_is_Better" title="Worse is better - Wikipedia, the free encyclopedia"> "Worse is better, also called the New Jersey style, is the name of a computer software design approach (or design philosophy) in which simplicity of both interface and implementation is more important than any other system attribute (including correctness, consistency, and completeness). [Contrast this to the MIT style of software development]"
</blockquote>
<p>
So why did Gabriel choose Lisp as an example to his statement? It's probably because Lisp is purported to be a very <i>powerful</i> programming language. It has correctness, consistency and completeness. Every new programming paradigm can be expressed in Lisp: functional programming, procedural programming, object-oriented programming, etc. However, it's interface and implementation leaves much to be desired. With this great power comes a greater responsibility on the programmer to not shoot his own foot.
</p>
<p>
There has been criticism that Gabriel's essay is not substantiated in any way. His opponents argue that Gabriel's examples with Lisp vs. C and ITS vs. Unix are not real examples that can support his claim. The truth is that, Gabriel himself is <i>not</i> certain of his claim. He has written other essays under the pseudonym Nickieben Bourbaki that rebuts his <i>Worse is Better</i> essays. What Gabriel has done is merely stating the obvious and offering a possible explanation for it. There are a lot of things out there in the world today that are far from perfect and yet they are being used more than their perfect counterparts. It could be because of marketing or price but the truth is that <i>better is not always better</i>.
</p>
<p>
Personally, I agree that Lisp is really powerful. You can make use of macros and meta-programming and define new domain-specific languages inside Lisp: you have the power to define another programming language within this programming language. Why would you need another programming language if you can do it all in Lisp? However, I am more of a fan of using various languages for different tasks. Want to quickly do some string manipulation? A quickly thrown together script with some regular expressions would get the job done. Need to do some logic programming? Using Prolog and the built-in tools would make it simpler.
</p>
<p>
That is the philosophy that the Pragmatic Programmers take. They believe in learning a new language each year. After all, you never know what you might discover. This is the power that you have with variety. The little pieces themselves might not be complete but by <b>combining different pieces together you end up with something simple yet powerful</b>. So standalone products might seem <i>less-than-perfect</i> on their own but when combined with other <i>less-than-perfect</i> components, something good might come about.
</p>
<p><a href="http://www.pragmaticprogrammer.com/loty/">The Pragmatic Programmers Language of the Year</a>:
</p>
<blockquote cite="http://www.pragmaticprogrammer.com/loty/" title="The Pragmatic Programmers Language of the Year"> "Learn at least one new [programming] language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut."
</blockquote>
<p>
<i>Worse is Better</i> is <b>not an excuse for creating sub-standard software</b> as Waldo pointed out in <i>Worse is Worse</i>. It's a guideline that when faced with the conflicting decision between simplicity and power, always choose simplicity. Simplicity allows more people to understand it and with more people using the software, the development team will get better feedback. This feedback allows the developer to improve the areas that <i>really</i> need to be improved. Simplicity also opens the software to a wider range of users and developers. Why limit software to the <i>aristocratic</i> few?
</p>
<p>
Then again, having a provably correct implementation that takes care of all the cases has a certain appeal as well. This is the struggle between being pragmatic and being perfect. But who gets to decide what is <i>worse</i> or <i>better</i>? In what sense is C worse than Lisp? In what sense is Microsoft worse than Linux? After all, <b>one man’s meat is another man’s poison</b><sup><a href="#fn3">3</a></sup>.
</p>
<hr />
<p id="fn1"><sup>1</sup> Another interesting read (as is the case with his other posts) is <a href="http://steve-yegge.blogspot.com/2006/04/lisp-is-not-acceptable-lisp.html"><i>Lisp is Not an Acceptable Lisp</i></a> by <a href="http://steve-yegge.blogspot.com/index.html">Steve Yegge</a></p>
<p id="fn2"><sup>2</sup> In fact in some communities, any questions on the syntax or justification for a particular feature is viewed as a personal attack on the community. The community then would lynch the defector in cyberspace usually through the means of nasty posts and comments. See <a href="http://softwareengineering.vazexqi.com/articles/2006/10/09/software-development-its-a-religion">Software Development: It's a religion</a> </p>
</p>
<p id="fn3"><sup>3</sup> It appears that after a product gets pass the tipping point, regardless of how bad it is, it will survive for many years to come. This scenario will continue regardless of how good a competitor's product is. Finding out the way to break this tipping point will make anyone in marketing very very rich.</p>
Convention Over Configuration Pattern2006-11-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/29/convention-over-configuration-pattern<p>
The final version of the pattern that I wrote for this class is now available. The html version can be found <a href="http://softwareengineering.vazexqi.com/files/pattern.html">here</a> and the pdf version can be found <a href="http://softwareengineering.vazexqi.com/files/convention_over_configuration.pdf">here</a>. </p>
<p>The intent of the pattern:</p>
<blockquote>Design a framework so that it enforces standard naming conventions for mapping classes to resources or events. A programmer only needs to write the mapping configurations when the naming convention fails.</blockquote>
<p>
I admit that the name is not coined by me. Instead it has beeb used actively in the Rails community.</p>
<p>
As I mentioned during the writer's workshop session for this pattern, I was more interested in trying to give a definition to the phrase <i>Convention Over Configuration</i>. The term has been used loosely by various developers and I hope my description will offer developers a better understanding of the underlying concepts and the consequences for using Convention over Configuration.</p>
<p>
One form of convention that I have seen - but not mentioned in the paper because it might make things too complicated - is the fact that code and resource organization can also be a form convention. For instance Rails expects to find all the ActiveRecord classes under the <code>app/models</code> folder. By having a convention for placing code and other resources, Rails does not need to traverse all the directories when it needs to find those files. Also, it does not need a configuration to tell it where to look for those files.</p>
<p>
This is my first attempt at a pattern and it has been an interesting experience. It is very tempting to assume that the reader knows too much or too little. In the former case, I would make the mistake of writing too little and use too many terms. In the latter case, I tend to write too much and the underlying meaning to the pattern might be lost. In this final version, I feel that I have struck a balance between the two and have included sufficient references in case the reader wants to find out more. </p>
<p>
The most important part of writing the pattern is actually getting feedback from other people. I was privileged to have the opinions of four members of the class and that really helped me identify the parts of the writing that need to be clarified.
</p>
<p>
Hopefully somebody finds this pattern and its description useful.
</p>
The Architecture of Complexity2006-11-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/27/the-architecture-of-complexity<p>
The following post is based on the chapter <i>The Architecture of Complexity: Hierarchic Systems</i> from the book <a href="http://www.amazon.com/Sciences-Artificial-Herbert-A-Simon/dp/0262691914"><i>The Sciences of Artificial</i></a> by Herbert Simon.
</p>
<p>
The chapter excerpt that I was reading from is a scanned from the book. That made it really hard to read the words and it could have slightly affected my evaluation of the chapter which was based on a paper of a similar title. Anyway, I felt that this paper was a bit too slow moving and verbose. I do not enjoy interdisciplinary approaches when explaining things especially if the examples themselves are presented with too much detail. Being a book, it caters to a broader audience so including various examples might be more helpful to the reader. But, sometimes these examples masquerade the intentions of the author.
</p>
<p>
Nonetheless, the paper does have some interesting points. The gist of it is that complex systems are usually composed hierarchical components. These hierarchical components can stand alone (and indeed the author calls these <i>stable intermediate forms</i>) but they are more useful when combined into systems that interact. For instance, in Computer Science, we might combine the java.swing packages with the java.io packages when designing our program. By combining those hierarchical components, we actually create even more complex systems albeit in a piece-wise manner so that we can understand the evolution of the system.
</p>
<p>
This decomposition into hierarchical standalone components is <i>a</i> way to tackle the complexity. Once we have a hierarchical structure, it's easier to describe and analyze the underlying complexity. We can take a component and analyze it in isolation first. After that, we could combine components together in different configurations and reason about how they interact. We can continue doing so until we have achieve our objective. Having a way to describe a complex system in simpler terms makes reasoning about the system itself simpler.
</p>
<p>
In design, we strive to decompose components so that they have high cohesion internally and low coupling externally. Complex systems that exhibit this decomposition can be more easily understood by other developers.
</p>
<p>
Indeed, most of the architecture or design that we see in software today is based on some form of hierarchy. Inheritance in object-oriented languages is a form of hierarchy. Classes themselves are a form of components that can stand alone or interact with other classes. In conclusion, decomposing a system into hierarchical components is a useful approach to not only manage the evolving complexity but it is also an useful way to describe the system to someone else.
</p>
<p>
It is also important to realize that hierarchical decomposition is not the only way to handle complexity. It certainly is one of the more well-known methods but there could be other methods yet to be discovered.
</p>
REST Web Architecture2006-11-17T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/17/rest-web-architecture<p>
This blog entry is based on the paper <a href="http://www.ics.uci.edu/~fielding/pubs/webarch_icse2000.pdf"><i>Principled Design of the Modern Web Architecture</i></a> by Roy Fielding and Richard Taylor[FT].
</p>
<p>
The REpresentational State Transfer (REST) architectural style has been generating some buzz over the past few months particularly in the Ruby on Rails community. The new version of Rails, version 1.2 is supposed to <a href="http://www.loudthinking.com/arc/000602.html">discount</a> the importance of traditional web-services and focus more on adhering to the REST standard instead.
</p>
<a href="http://www.loudthinking.com/arc/000585.html"><img src="http://www.loudthinking.com/lt-files/wsdeathstar.png" /></a>
<br />
<small>WS-Deathstar: REST is supposed to replace the ever-more-complicated web services out there</small>
<p>
But what exactly is REST and how is it different from the current workings of the web. What follows will probably be an over-simplification, but I think it captures the notion of what REST is. The most common element, I claim, on the web today is the URL. Even grandma knows to go to <i>www-dot-cnn-dot-com</i> for the latest news. However, the URL could be improved to be more useful using REST.
</p>
<p>
Consider this scenario. There is an interesting article at www.somewhere.com/verynicearticle.html. So that link takes you to the web page where the article is. What if you want a .pdf version of the article and the web author has so kindly provided it for you at www.somewhere.com/verynicearticle.pdf. And what if you want an XML version of that article? Sure enough, the kind web author has provided it for at www.somewhere.com/verynicearticle.xml. Hopefully, we are beginning to see a pattern here. The "real" <i>resource</i> at that page does not really change even if we request it in .html, .pdf or .xml form. Sure, some information might be lost from the conversion, but the main points are definitely still there. So instead of using verynicearticle.html or verynicearticle.pdf or even verynicearticle.xml, we should just use the URL www.somewhere.com/verynicearticle. And each time we request that resource, we also specify what form we want it to be in. The same URL can be used to give us different <i>representations</i> of the same resource!
</p>
<p>
This might not seem like a big deal but trust me, it is! Imagine having a consistent URL to locate the resource that we are interested in. All that changes is the representation that we want. This is really convenient; no more having to invent weird URL names for the same resource. Of course, our request has to include the type of resource that we want but the web server can easily fall back to the default .html representation if we did not specify a representation for it. Moreover, depending on the type of HTTP request (GET, POST, PUT, DELETE) we can actually modify the resource at that URL as well (with proper authentication usually). Simplicity! No more having to to something like www.somewhere.com/delete/verynicearticle_ID or www.somewhere.com/upload/verynicearticle_ID<sup><a href="#fn1">1</a></sup>.
</p>
<p>
So, let's dissect the term "REpresentational State Transfer". We have seen what a representation is: a representation is one form of displaying the information. A web page can be represented as .html or .pdf or .xml. So what is the <i>state</i>? When the web server sends us back the representation for the page, it also embeds some metadata and state in it. Usually, we would not need to make use of this state if we are just viewing the page, but if we need to perform a future request to the server, the embedded state might be useful for communication<sup><a href="#fn2">2</a></sup>. Finally, the <i>transfer</i> occurs when that representation is presented to the end-user usually in the web-browser.
</p>
<p>
REST attempts to keep everything as self-contained as possible. It adheres to the requirement that server requests are stateless. So the server does not need special mechanisms (like cookies or sessions) to keep track of each request. Instead, if state is necessary for the correct behavior, that state should be encoded in the representation. By making everything as self-contained as possible, REST achieves better performance because the same representation can be routed through different proxies or gateways without the server and the client worrying that the state might be lost.
</p>
<p>
This paper was written about six years ago and REST is only beginning to gain more traction in the web community today. It will take some time before full adoption of REST takes place. But because this is the web, it is hard to actually say if it will ever be the de-facto standard for web architecture since it is hard to impose that restriction.
</p>
<p>
The paper does not (nor does it try to) address every concern there about REST. Instead this paper, provides the motivation behind REST and why it is becoming more important. There certainly are a lot of concerns, particularly in the realm of security and how following REST
</p>
<p>
One of the good things with REST or any modern web architecture that aspires to succeed is the fact that it does not preclude the usage of previous standards. REST is not a all-or-nothing type of architecture. It can coexist nicely with the previous standards of the web. Of course, for backward-compatibility, sometimes the latest features of REST might not be employed but that is a small price to ensure that everything remains working as the users expect it to.
</p>
<p>
The internet has reached the tipping point where any radical change to its architecture will have to be incrementally and support. This might be the current trend for most large-scale systems where it is no longer feasible to just start all over from scratch and invent a "pure" architecture for the new system. In other words, modern architectural styles will have to accommodate the previous styles and exist in harmony.
</p>
<hr />
<p id="fn1"><sup>1</sup> Most modern web browsers and servers do not support the PUT and DELETE HTTP requests. In fact, they are allowed to ignore it completely based on the current standards. This is one the reasons why the adoption of REST has not been slow.</p>
<p id="fn2"><sup>2</sup> The <a href="http://www.jabber.org/">Jabber</a> chat protocol encodes each chat message or event as XML. One of the nodes of the XML will contain a special UID that identifies the session. This UID is used to keep track of the sender and each communication with the server has to be checked with this UID. In other words, this UID serves as the <i>state</i> for this particular Jabber connection session between a user and the Jabber server.</p>
Adaptive Object Model2006-11-08T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/08/adaptive-object-model<p>
This blog entry is based on the paper <a href="http://adaptiveobjectmodel.com/WICSA3/ArchitectureOfAOMsWICSA3.pdf"><i>The Adaptive Object-Model Architectural Syle</i></a> by Joseph Yoder and Ralph Johnson [YJ].
</p>
<p>
The Adaptive Object Model [AOM] is a type of architectural style that relies on storing the business or domain model separately from the code. In this architectural style, the code serves merely as "glue" to bring the domain model into an executable form. The domain model is stored in an external database or in XML files. AOM can be viewed as a form of domain-specific language and it relies also on <a href="http://www.martinfowler.com/articles/injection.html">dependency injection</a> to reconcile the code and the externally represented domain model.
</p>
<p>
AOM is useful when flexibility and runtime configuration is essential to the operation of the system. This is more prevalent in the case of static languages that require a "compilation" phase before being executable. AOM is not usable for all forms of languages since it relies on the reflective property of the underlying system. Nonetheless, AOM is a plausible architectural style if the business model is expected to change frequently.
</p>
<p>
While an AOM is similar to a domain-specific language, both of them are not necessary identical. In this paper, the AOM architectural style is presented as conforming loosely to the <i>TypeSquare with Rules</i> style. Thus, practitioners of AOM have a fairly common style that transcends most systems. On the other hand, domain-specific languages usually have language grammars that are distinct to the system that is being modeled. However, just like a domain-specific language, an interpreter for the AOM is necessary. In most cases, the interpreter can be small and simple, but in more complex cases, the interpreter might have to evolve to accommodate the complexity of the AOM language itself. Thus, [YJ] advocates starting with a simple interpreter first and only make it more complex as the system evolves. The main focus of using AOM is to model the software, not to actually make a general purpose language.
</p>
<a href="http://www.flickr.com/photos/vazexqi/292556177/" title="Photo Sharing"><img src="http://static.flickr.com/113/292556177_e5f1ee3f61.jpg" width="500" height="144" alt="Adaptive Object Model" /></a>
<br />
<small>TypeSquares with Rules. The typical form of an AOM</small>
<p>
The presence of an "external" language for describing the domain model is also another maintenance issue that has to be taken care of. Tools that support editing and debugging have to be provided. In addition, if the language is fairly complex, there is the need to provide proper documentation. In short, there is an additional maintenance cost that is incurred for using AOM.
</p>
<p>
There certainly are systems out there that use the AOM architectural style. However, when faced with the deciding between using "traditional" systems and using this AOM system, the usage of AOM might require a greater amount of initial investment to get the system working since it requires creating tools to interpret this new meta-modeling language.
</p>
<p>
I suspect that AOM might be more easily implemented in dynamic languages such as Smalltalk, Ruby and Lisp compared to languages such as C# and Java. The malleable nature of dynamic languages and the ability to run plain text as code during runtime reduces the need for complex interpreters. Here is how adding classes dynamically is <a href="http://www.cincomsmalltalk.com/blog/blogView?showComments=true&entry=3338622442">done</a> in Smalltalk and how it is <a href="http://weblogs.java.net/blog/emcmanus/archive/2006/10/build_your_own.html">done</a> in Java. Then again, for dynamic languages, it might be so easy to modify the system that the need for AOM might be reduced.
</p>
<p>
In conclusion, I suspect that as systems become more complex, the flexibility that AOM offers will become more important. It might no longer be feasible to change the underlying code of the system every time a new subclass is introduced or a new business rule is needed. AOM empowers the domain experts to model the business domain without going into the code.a
</p>
<p>
Moreover, there is a trend now to use domain-specific language to model systems. Martin Fowler has a nice article on <a href="http://www.martinfowler.com/articles/languageWorkbench.html"><i>Language Workbenches</i></a> and how language workbenches will be able to incorporate the facilities that developers are familiar with - code completion, debugger, syntax highlighting, etc - and provide them easily for a new domain language.
</p>
Software Architectural Primitives2006-11-06T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/06/software-architectural-primitives<p>
This blog entry is based on the paper <a href="http://sunset.usc.edu/csse/TECHRPTS/2002/usccse2002-509/usccse2002-509.pdf"><i>Distilling Software Architectural Primitive from Architectural Styles</i></a> by Mehta and Medvidovic [MM].
</p>
<p>
This paper was interesting. However, maybe because of my lack of background in this area - formalizing architectural styles - this paper was not easy to read. I had to reread parts of it many times to actually understand what the authors were saying. Maybe there was a page limit on the paper but I felt that the authors could have made it more clear if they had use more pages or eliminated some examples but elaborated more on a smaller set of examples. The authors assumed that the reader was familiar with various terminology used in the field of architectural description languages. Also, familiarity with the Alloy framework was assumed. So the target audience might not have been readers such as myself.
</p>
<p>
In a nutshell, this paper proposes a way to formalize the description of architectural styles using - what [MM] define as - architectural primitives. A useful analogy of architectural primitives would be assembly language. All programs can be described in terms of assembly language so it forms the primitive for programs. In the same way, [MM] argues that they have been able to describe four common architectural styles (C2, Client-server, Pipe-and-filter and Push-based) using their architectural primitives. They did not say how they actually came up with the set of architectural primitives but they did try to convince the readers that their set of primitives was useful by showing the usage of those primitives in the four architectural styles that they documented. [MM] agree that it is far too early to conclude if the current set of primitives would work on more architectural styles.
</p>
<p>
What is the use of those architectural primitives? The authors argue that currently we lack a formal but also practical way of describing software architectural styles. [MM] contend that the mapping between architectural style and their implementation is generally poorly understood. However, this paper itself did not show how the implementation become more standardized once you have this formal description. Implementing the software architectural style in C++ might still be different from Java. [MM] did not present a way to evaluate if two separate developers implementing the system using the provided description actually made systems that conform to the architectural style.
</p>
<p>
Anyway, I agree that the paper was interesting. Is the research useful? That is hard to say. I am inclined to say that it is <i>not</i> really that useful. I still have the notion that architectural styles are like patterns. They are high-level description of languages. As such, they are seldom applied in their pure form. Instead, these architectural styles are adapted to suit their needs. Furthermore, I am not really convinced why staying true to an architectural style is really that important. Do most clients make it part of their requirement to state what kind of architectural style they expect? Does using an architectural style in its <i>pure</i> form actually reduce the number of bugs? How do you deal with describing hybrid architectural styles? Since there is currently no <i>formal</i> description for many architectural styles (many different descriptions might exist) how do you ensure that you are indeed capturing the <i>right</i> version?
</p>
<p>
There might be a pragmatic use for software architectural primitives but this paper does not motivate that point.
</p>
<p>
Readers interested in finding out more about architectural description languages should take a look at <a href="http://archjava.fluid.cs.cmu.edu/index.html">ArchJava</a>. There is a working implementation of it so it is easier to test it out and see what functionality it provides.
</p>
A Pattern Language2006-11-02T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/11/02/a-pattern-language<p><a href="http://wiki.cs.uiuc.edu/cs527/A+Pattern+Language">cs527: A Pattern Language</a>:
</p>
<blockquote cite="http://wiki.cs.uiuc.edu/cs527/A+Pattern+Language" title="cs527: A Pattern Language"> "A pattern language is a collection of inter-related patterns that can be combined together"
</blockquote>
<p>
I had some time (not a whole lot) before class and I saw Maurice's post on the class wiki that he has some materials that would be nice to read before his presentation. He has a lot of materials listed down above, but I was only able to go through <a href="http://wiki.cs.uiuc.edu/cs527/The+City+is+Not+a+Tree"><i>The City is Not a Tree I & II</i></a> and also the <a href="http://www.gardenvisit.com/landscape/architecture/3.1-patternlanguage.htm">discussion</a> on it .
</p>
<p>
From those two articles, I come to understand patterns as a new, non-rigid way of classification of solutions. When people encounter a complex system (it need not be a computer program; it could be a new concept, new theory, etc) their natural tendency is to clump things together into non-overlapping concepts. Doing so aids their understanding of the new material. This works well for most systems that we deal with. A tree-like classification system is an example of classification that favors non-overlapping composition. A concept is in one branch of the tree but not both.
</p>
<p>
However, by separating the system into non-overlapping concepts, we might actually miss the interactions between them. Most concepts do not exist in isolation. One branch of the tree might be related somehow to the other branch. That <i>tree</i> now should evolve into a <i>graph</i>, where concepts can be interconnected.
</p>
<p>
We have a natural tendency to favor the <i>tree</i> over the <i>graph</i> for organizing ideas. However, solutions to most interesting problems do not appear in isolation. There is seldom a single solution to problems. Good solutions do not exist in isolation from one another; rather, solutions are usually applied in the context of other solutions in some combination.
</p>
<p>
Therefore, patterns are a way to understand new concepts and suggest solutions to them. Patterns must be presented within a context - where this pattern had occurred. Moreover, it is useless to propose a solution without understanding what the real problem is so the problem should always accompany the solution. Also, the solution must be presented in honesty: the advantages and disadvantages of the solution should be stated.
</p>
<p>
This is my understanding of a pattern language from reading the articles above. I am interested in what Maurice will be talking about since he has listed a whole bunch of related materials. It will probably have little to do with the design patterns that most computer scientists are acquainted with. I suspect that he will be dwelling more on the origins of patterns, the pervasiveness of them, and end with a discussion of the usefulness of patterns. It's very tempting to go into the metaphorical and psychological details of patterns but that would be a bit hard for me to swallow. I hope he goes into the classifications of patterns as well. It is very easy to have an explosion of patterns in one particular area and then it becomes hard for the mind to actually see the connections between those patterns.
</p>
<p>
Update: Overall I liked the presentation a lot. At the end of it, I asked the question on what patterns will turn the programmers into a satisfied <a href="tp://headrush.typepad.com/creating_passionate_users/2006/04/announcing_the_.html">Emotional Programmer</a>. Prof. Johnson said that following along the lines of XP usually made programmers happy. I suspect that there are some other patterns out there that could make programmers happy but I have not had a chance to experience all of them yet. For instance, using <a href="http://www.martinfowler.com/bliki/HumaneInterface.html">HumaneInterfaces</a> will certainly help make my day better.
</p>
Social Issues and Software Architecture2006-10-31T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/31/social-issues-and-software-architecture<p><a href="http://members.aol.com/acockburn/papers/softorg.htm">On the Interaction of Social Issues and Software Architecture</a>:
</p>
<blockquote cite="http://members.aol.com/acockburn/papers/softorg.htm" title="On the Interaction of Social Issues and Software Architecture"> "Architects do not like being told their clean design is a result of accounting for social forces."
</blockquote>
<p>
And yet, it is often the case that the architecture does not just evolve devoid from all social dynamics. On the other hand, the architecture of the system is related to the quality of the development team. We have seen in the first chapter of SAIP how the architecture business cycle is affected by the stakeholders who give feedback to the architect. We have also seen how it is not uncommon for an architect to pick a system architecture based on the organization and knowledge of his team of developers.
</p>
<p>
In this article, Cockburn uses a case study to show the dynamics between social issues and software architecture. The case study involves adding a new, transaction-based program to a workstation that is connected by phone lines to the central office. The development team is contemplating on using an object-oriented approach for the system but none of them have done an object-oriented project before. While this case study serves as the basis for the article, Cockburn does not actually provide concrete examples from it. I felt that the paper would be more convincing if Cockburn could provide concrete and motivating examples from the case study.
</p>
<p>
Cockburn motivates the use of his patterns by identifying 6 different external forces and presenting the principle by which to alleviate those forces:
<ol>
<li>Things change -> Variation Behind Interface</li>
<li>People have differing skills and specialties -> Subsystem by Skill</li>
<li>People get confused if ownership is unclear -> Owner per deliverable</li>
<li>Teams have different interests and design points -> Subclass per team</li>
<li>User interface requirements change a lot -> User interface outside</li>
<li>Changing an inter-team interface is expensive -> Facade</li>
</ol>
</p>
<p>
From the list, only items 2, 3 and 4 have an explicit connection with the social issues in a team. The other items might be weakly related to the team's social issues but the connection, if present, is not really that visible. Items 2, 3 and 4 seem to be organizational patters — they suggest ways to split the architecture up into smaller modules that fit the skills of each developer or teams of developers. Depending on the prior knowledge that developers have, there might be more ways to subdivide the tasks and their corresponding developers.
</p>
<p><a href="http://members.aol.com/acockburn/papers/softorg.htm">On the Interaction of Social Issues and Software Architecture</a>:
</p>
<blockquote cite="http://members.aol.com/acockburn/papers/softorg.htm" title="On the Interaction of Social Issues and Software Architecture"> "Often the situations they hoped to anticipate never happen, so the interface serves no useful purpose. The trick in good design is to correctly anticipate the changes, or the cost of the interface against the cost of the change."
</blockquote>
<p>
Cockburn argues that good design anticipate change. Anticipating change correctly, however, is hard. Designing a system to anticipate those changes is even harder and is a task that often leads to over-engineering. The advocated approach to handle change now, is not to anticipate too much, but to refactor the system to adapt to changes as they come. Much of the appropriate refactorings have been discussed extensively in DDD.
</p>
<p>
Besides the principles presented here, the patterns that we have looked at in DDD are also useful for handling social issues. In fact, there are clear connections between the principle that Cockburn lists and the patterns that Evans writes in his book:
<ul>
<li>Variation between Interface <-> Intention Revealing Interfaces</li>
<li>User Interface Outside <-> Layered Architecture</li>
<li>Facade <-> Anti-corruption layer</li>
</ul>
</p>
<p>
All in all, the title of Cockburn's piece is not reflected in the contents of the article. True, there are some principles that are motivated by the social interactions amongst the team members but the rest of the principles are independent of the developers in the team. Cockburn might have unintentionally left out some of the details of the case study that provides the link between social issues and software architecture.
</p>
<p>
Fortunately, we have seen enough from the SAIP and DDD book that we can agree with Cockburn that group dynamics and social issues do shape the design of the software architecture. To deny that connection would be foolish endeavor.
</p>
Code that appeals to us humans2006-10-31T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/31/code-that-appeals-to-us-humans<p><a href="http://cs527.wordpress.com/2006/10/31/human-readable-code/trackback/">Human Readable Code</a>:
</p>
<blockquote cite="http://cs527.wordpress.com/2006/10/31/human-readable-code/" title=""> "In Refactoring to Patterns (pg. 13), Josh mentions that while looking over code before a workshop, he saw code that read:
november(20, 2005)
And called a method that looked like:
public void Date november(int day, int year)"
</blockquote>
<p>
I completely agree that code should be human readable. In fact, this was one of the reasons why I like languages like Ruby so much. Martin Fowler has written a good article on this and since I don't think I will be able to produce something of similar quality at this moment, I refer you to his post entitled <a href="http://www.martinfowler.com/bliki/HumaneInterface.html">"HumaneInterface"</a>.
</p>
Big Ball of Mud2006-10-31T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/31/big-ball-of-mud<p><a href="http://www.laputan.org/mud/">Big Ball of Mud</a>:
</p>
<blockquote cite="http://www.laputan.org/mud/" title="Big Ball of Mud"> "BIG BALL OF MUD might be thought of as an anti-pattern, since our intention is to show how passivity in the face of forces that undermine architecture can lead to a quagmire. However, its undeniable popularity leads to the inexorable conclusion that it is a pattern in its own right. It is certainly a pervasive, recurring solution to the problem of producing a working system in the context of software development. It would seem to be the path of least resistance when one confronts the sorts of forces discussed above. Only by understanding the logic of its appeal can we channel or counteract the forces that lead to a BIG BALL OF MUD."
</blockquote>
<p>
I first read this essay about a year ago when I stumbled across it on the Internet. Reading it now for the second time revealed to me how much truth is contained within it.
</p>
<p>
Chapter 2 (p. 22) of SAIP, mentions that "every computing system with software has a software architecture". So, indeed <i>Big Ball of Mud</i> is not only <i>a</i> form of software architecture, but it is <i>the</i> predominant one. However, it is also <i>the</i> software architecture that most software engineers want to avoid. It lacks elegance and it is a maintenance nightmare. And yet, even the most ambitious and well-coordinated software projects might eventually evolve to <i>Big Ball of Mud</i> if careful steps are not taken.
</p>
<p>
To encourage the reader to read the original essay, I will list out the patterns that were discussed here but not give away too much of it.
<ul>
<li>Throwaway Code <br />
In <i>Mythical Man-Month</i>, Fred Brooks warns developers to "plan to throw one away; [since they] will anyway". Throwaway code is great for testing and prototyping but, as its name suggests, it should never be sneaked in as part of the production code.
</li>
<li>Piecemeal Growth <br />
Favor iterative development over grand up-front design.
</li>
<li>Keep it Working <br />
A working simplistic software is better than software with a solid architecture but does not work (yet, as its proponents will claim).
</li>
<li>Shearing Layers</li>
<li>Sweeping it Under the Rug <br />
When some piece of code is really bad but there is nothing that can be done about it, at least try to cordon it off from the rest of the code to prevent more dependency on it.
</li>
<li>Reconstruction <br/ >
When all else fails, maybe it is time for starting from scratch. However, this should be the last resort since reconstruction puts the software at a great handicap against its competitors. Joel Spolsky has a <a href="http://www.joelonsoftware.com/articles/fog0000000069.html">discussion</a> on this.
</li>
</ul>
</p>
<p>
The patterns discussed in this paper, when used in judiciously in isolation from one another are actually very useful. When developers need a quick solution for testing or prototyping, coming up with <i>Throwaway Code</i> is a natural thing to do. Or when developers are uncertain about what to do next, they will proceed in small steps, cautiously. Therefore, it is natural to follow <i>Piecemeal Growth</i>. The problem occurs when developers combine the patterns discussed here without realizing the interactions between them and do not take steps to counteract each pattern's negative effects.
</p>
<p>
<a href="http://www.amazon.com/Extreme-Programming-Explained-Embrace-Change/dp/0201616416"><i>Extreme Programming Explained</i></a> actually describes how XP functions in terms of these forces. Each pillar or practice in XP works by counteracting the side effects left by using the other pillars. For instance, XP advocates "doing the simplest thing possible to get it to work". This is analogous to <i>Piecemeal Growth</i>. However, to counteract the negative effects of <i>Piecemeal Growth</i>, XP also advocates continuous integration and refactoring. This keeps the code in-check and prevents code rot. However, continuous integration and refactoring might actually break the code. Therefore, XP promotes the heavy use of unit testing. However, unit testing.... And it goes on and on. Every pillar of XP has its advantages and disadvantages. By combining those best practices together XP actually mitigate the negative side effects from using the best practices individually. <i>Extreme Programming Explained</i> is a well-written easy to read book and anyone interested in agile development should definitely read it.
</p>
<blockquote cite="http://www.laputan.org/mud/" title="Big Ball of Mud"> "[Foote & Yoder 1998a] went so far as to observe that inscrutable code might, in fact, have a survival advantage over good code, by virtue of being difficult to comprehend and change. This advantage can extend to those programmers who can find their ways around such code. In a land devoid of landmarks, such guides may become indispensable"
</blockquote>
<p>
The quote above offers some enlightenment as to why, <i>Big Ball of Mud</i> is still the most popular form of software architecture in the world today.
</p>
<img src="http://www.phdcomics.com/comics/archive/phd092306s.gif" />
<br />
<small>You become indispensable when you are the remaining few who know how to operate a legacy system</small>
<p>
This essay is actually a very interesting piece of work. It has engaging sentences decorated with short stories. And the images were chosen to complement what was stated in text. My short description of the essay does not do it justice so I highly encourage the reader to go read that <a href="http://www.laputan.org/mud/">essay</a>.
</p>
<small>
As a side note, I came across the phrase <a href="http://www.laputan.org/dfc/discussion.html#O2isEasy">"freedom from choice"</a> in this paper and could not help but notice its similarity to the pattern that I am writing for this class. I call my pattern "Convention over Configuration" and it focuses on relieving the developer from configuring a framework down to very tiny detail by establishing some conventions that the developer is encouraged to adhere to.
</small>
Does following the patterns make us domain-driven developers?2006-10-18T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/18/does-following-the-patterns-make-us-domain-driven-developers<p>
I have been reading a few interesting articles about agile methods these past few weeks. There have been various issues on it - too much to actually fit in a post. Fortunately , there is a nice <a href="http://www.infoq.com/articles/agile-values-and-practices">article</a> that summarizes all that have been going on. It will be best to read it because otherwise the rest of this post will not make much sense.
</p>
<p>
The question the article asks: by using agile practices, does it make the project an agile project?
</p>
<p><a href="http://www.infoq.com/articles/agile-values-and-practices">InfoQ: Do Agile Practices Make it an Agile Project?</a>:
</p>
<blockquote cite="http://www.infoq.com/articles/agile-values-and-practices" title="InfoQ: Do Agile Practices Make it an Agile Project?"> "There's a down side, however: Agile also risks what one practitioner calls 'dying the death of a thousand copies,' if teams copy practices rather than growing them, implementing what worked elsewhere without understanding how to use continuous improvement to make the process suitable for their own unique context."
</blockquote>
<p>
And I have the same thoughts about actually doing domain-driven design. There are a ton of patterns (or best practices) in the book. Some of them are really useful and some are less useful. It's extremely easy to get overwhelmed with all the patterns and start implementing them all over the place that the patterns themselves become a maintenance nightmare.
</p>
<p>
The same goes for a novice programmer who has just read Design Patterns. He or she is suddenly bestowed with the ability of x-ray vision: the ability to spot potential use for patterns everywhere!
</p>
<p><a href="http://www.codinghorror.com/blog/archives/000380.html">Coding Horror: Head First Design Patterns</a>:
</p>
<blockquote cite="http://www.codinghorror.com/blog/archives/000380.html" title="Coding Horror: Head First Design Patterns"> "The beginner will learn that this is not so, that all designs should be as simple as possible. Complexity and patterns should only be used where they are needed for practical extensibility."
</blockquote>
<p>
The main to realize with all this new faze thing is that there is <a href="http://www-inst.eecs.berkeley.edu/~maratb/readings/NoSilverBullet.html">no silver bullet</a>. Plain and simple. Once you realize that, you know that at most all those snazzy new techniques can only help you so much. There is no panacea for everything.
</p>
<p>
So to answer the question: does following the patterns make us domain-driven developers? Blindly reading the patterns and applying them everywhere will not only lead to a team that is too conscious of the development process but also jeopardize future development because the project becomes a plethora of domain-driven patterns without anyone actually knowing the motivation behind those patterns.
</p>
<p>
Filter out the patterns; you do not need all of them. And incorporate only those that address an inherent problem in your current development process.
</p>
<p>
Update: Paul just told me that a lot of what I say on this blog sounds like what Fred Brooks says in <i>The Mythical Man-Month</i>. I read that book about 2 years ago so probably I can still remember the main points there. But since I do not have the book with me I cannot quote the revelant chapters. That said, I think 2 years is a sufficient time to go back and read the book for new insight. I will probably do that during Thanksgiving break.
</p>
DDD: Large-scale Structure2006-10-18T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/18/ddd-large-scale-structure<p>
Large-scale systems are hard to coordinate. Moreover, it is even harder to come up with a unifying theme that can get everyone thinking on the same page. When this unifying theme is missing, developers find it hard to communicate and understand the system. The previous patterns discussed - refactoring to deeper insight, maintaining model integrity and distillation - help but these patterns do not really provide a unifying theme for everyone.
</p>
<p>
What's needed is a a set of rules or patterns that can span the entire system and thus allow developers to grasp the system as a whole. Evans's patterns in this chapter are more of recommendations based on what he has observed in various projects.
</p>
<p><i>Large-Scale Structure:</i></p>
<blockquote>
"This whole area is still undeveloped. I've observed a few general patterns of large-scale structures that have emerged on various projects. I'll discuss four in this chapter. One of these may fit your needs or lead to ideas for a structure tailored to your project."
</blockquote>
<p>
In this chapter, Evans finally confronts the issue of the <b>System Metaphor</b> in XP and how it relates to the model. I would have appreciated more if he had a linked from his earlier chapters to this one. As some of you might recall, I was having some <a href="http://softwareengineering.vazexqi.com/articles/2006/10/01/ddd-crunching-knowledge">issues</a> on how the system metaphor relates to the model in model-driven design.
</p>
<p>
Another important point in this chapter is the fact that the strategy for large-scale structures need to evolve with the project. It is disastrous to be rigid and stick with just one of those patterns. Unfortunately, there is the resistance to change because once the structure for a large system is established, it is really hard to change. And that is something that we do not have a good rule of thumb for.
</p>
<p>
Those of you who are interested in large-scale structures might also be interested in <a href="http://www.sei.cmu.edu/uls/">Ultra-Large Scale Systems</a>. So far the only work on it, as far as I can tell is the one done by CMU. In the <a href="http://wiki.cs.uiuc.edu/SAG">SAG group</a>, we had a discussion on the paper once but unfortunately no one recorded our discussion down.
</p>
DDD: Maintaining Model Integrity2006-10-15T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/15/ddd-maintaining-model-integrity<p>
What Evans has been advocating about model-driven-development so far has mainly focused on maintaining one consistent model throughout the development team. However, what happens when you have multiple teams working on different parts of the system? For instance, it is not uncommon to outsource part of the development to another group. What happens to the underlying model then? Should all teams still have the same model? Evans argues that they should not. Instead they have different models but aspects of the model that are important to everyone should stay unified and the parts that are less important need not be. Evans presents some patterns that might help. [Patterns are described in the middle portion of this post; interesting aspects of the chapter are at the end of this post]
</p>
<hr />
<p>
<b>Bounded context</b>. Every model should be accompanied by the relevant context to which it applies. The context refers to team organization, usage within specific parts of the application, code and the database schema. The model should be kept consistent within this context at all times regardless of external influences. The question is: how much "context" is needed to actually bound a model? Also, what does one do when there are overlapping contexts?
</p>
<p>
Even within a bounded context, there might be undetected overlaps. For instance, when Team A talks about "Charge" and Team B talks about "Charge" are the two "Charges" the same? What aspects do they differ in and what aspects are they similar in? Recognizing these differences are very important to ensure that your contexts are really bounded properly.
</p>
<p>
<b>Continuous Integration</b>. Continuous integration helps maintain the cohesion within each context and reduces internal fragmentation. How would one do continuous integration? One has to ensure that a strict schedule for integration builds and automated testing is being carried out. Furthermore, within the same context, developers should strive to use the Ubiquitous Language. Continuous integration solves the problem of fragmentation within the same bounded context, but what should one do for multiple bounded contexts?
</p>
<p>
<b>Context map</b>. When multiple bounded contexts exists, it is important to find the point of contact of each context with one another. Each bounded context should be identified by a communicative name and that name should be made part of the Ubiquitous Language. Once the name has been decided, then one can create a map that shows the relationship between each bounded context. The names give identity to each context so that everyone on the team can refer to them clearly. The map gives a picture of how everything relates together.
</p>
<p>
<b>Shared Kernel</b>. Sometimes, it is hard to identify a boundary between different context. Even if this boundary exists, it might seem artificial. In that case, one should treat the intersection of the different contexts as a shared kernel. A shared kernel should undergo some form of continuous integration as well to maintain its coherence between the different contexts. But because this shared kernel, as its name implies, is shared between contexts, integration will not be done as frequently since it has to be agreed upon by members of both teams. A shared kernel is a good choice in place of a context map especially if two or more models are closely related. Using a context map in that case will only lead to an unnecessary artifact that serves only to confuse.
</p>
<p>
<b>Customer/ Supplier Development Teams</b>. This pattern deals more with how different teams should work out dependencies between them. For example, Team A might depend on Team B's work to accomplish the task. Team A might accuse Team B of not following the specifications and vice versa. Also, if this is not the case of a shared kernel, then it becomes harder to integrate since the two teams might not use the same model. The Customer/ Supplier Development Teams pattern helps mitigate this internal conflict between teams by establishing a set of guidelines. Using automated tests that check for interface compliance, arguments can be avoided (most of the time).
</p>
<small>
I am not really sure why this is a pattern; this is common sense. If two teams are going to work together, then they must agree to some specification to make their parts work. It's hard to justify when a pattern is common sense and when it is a profound revelation to some. However, I am pretty sure that this is the case of a common sense pattern. A more useful pattern would be how to actually do the specification without spending too much time and how to ensure that teams to do not go at each other's throat.
</small>
<p>
<b>Conformist</b>. I suspect this pattern happens more often than the previous one. And for good reason too. Having two teams come together to agree on an interface is just asking for trouble (most of the time). Designate one team as the upstream team and have the other team comply to the interface needs of the upstream team.
</p>
<p>
In a setting where there is one management or one chief architect, the Conformist pattern is easy to apply. Some might think it is rule by dictatorship but this saves a lot of time for everyone. Using the Customer/ Supplier Development Teams is a waste of resources. If one wishes to maintain fairness, then rotate the team members so that they can be in the upstream team or downstream team.
</p>
<p>
<b>Anti-corruption Layer</b>. This is just the Facade or Adapter pattern. The only difference is the intent. The Anti-corruption Layer provides clients with functionality in terms of their own domain model. This makes communication within the group easier since the interface now complies with the model.
</p>
<p>
<b>Separate Ways</b>. Recall the Standalone Class pattern from Chapter 12. This is similar. If one can get a bounded context that does not interact with other contexts, then it is usually best to maintain that isolation. Bounded contexts in isolation are free to evolve on their own and thus do not incur the overhead of a context map or shared kernel or the other patterns that deal with interdependence between models.
</p>
<p>
<b>Open Host Service</b>. Define a protocol that gives access to your subsystem as a set of services. This can be formalized using the Published Language pattern. I am not really sure what this pattern is. <i>Is it just a well-published API for the model, or a more heavy duty system like a Web Service?</i>
</p>
<p>
<b>Published Language</b>. Use a Published Language to communicate your domain information to people who are not in the development team. It is best to conform (the Conformist pattern?) to a language that other developers in the field are using. Not only does this aid in communication efforts but is also saves developers the trouble of coming up with their language and the tools to translate them into. Published Language like the Ubiquitous Language share some common goals.
</p>
<hr />
<p>
This is a very long chapter with many patterns. At the end of the chapter, Evans refers to all of them again and defines the relation between them. Some of these patterns seem very abstract and it is hard to actually determine if you are following the pattern or just relying on intuition. In fact, some even almost seem managerial. There is nothing wrong with managerial patterns, but those are patterns that I am not really interested in.</p>
<p>
I must give credit to Evans for the elephant example that he has at the end of the book. It makes so much more sense than the shipping or banking or accounting examples he has been using throughout the other chapters. The elephant example seems to hint that there must be a chief architect for most of the patterns here to work. If every team develops in isolation then it is hard to integrate things properly. There must be someone who has a broader view of the entire development process. And that person is inherently missing from the chapters of the book so far.
</p>
<p>
Another thing that I like about this chapter is the Transformation sections. I found this to be the most important underlying factor to these patterns. Without the transformations, each pattern will just be static commitment. But for a real development effort those patterns will have to evolve depending on the insights offered by continuous refactoring and integration.
</p>
DDD: Distillation2006-10-15T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/15/ddd-distillation<p>
<i>Upon seeing the infamous Maxwell's Equations left on the blackboard from a previous class, my complexity theory professor immediately ordered us to erase them with "extreme prejudice" [sic]</i>. A short anecdote to show that not everyone cares for electromagnetism, even if they have a degree in ECE.
</p>
<p>
The book's title is Domain-Driven Design: Tackling Complexity in the Heart of Software. And this is the chapter that truly details how to do it. The previous chapters in the book were advocating best practices for model-drive design and how to cope with shallow models, multiple models, and the usual office politics. But it is in this chapter, I believe, that we deal with the real complexity in software: complexity in the model itself. And Evans has referred to this process as distillation. In this chapter, he <i>attempts</i> to define how to distill the core domain. Unfortunately, I don't really buy everything he says mostly because I do not have enough experience with this and because his description is not all that clear.
</p>
<p>
As you refactor to deeper insight, there is little that you can do to prevent the model from growing. As you understand more of the domain, your model will begin to reflect that. Soon, the model grows to the size that not everyone can understand it. New developers brought on to the team will have a hard time adjusting. By distilling the model, you end up with the <b>core domain</b>: the set of features that are essential to the system. How does one define what <i>essential</i> mean? Evans thinks it is the real business asset of the entire system.
</p>
<p>
Therefore, I can see the rational behind having a <b>domain vision statement</b>. Evans proposes that it be a page long. But a good domain vision statement should be summarized in as few sentences as possible. And this vision statement should be displayed prominently where everyone on the team can see it. It points the way to the <b>core domain</b>. And your team can supplement this with a <b>highlighted core</b>.
</p>
<p>
I really agree with Evans on how often engineers get distracted by a complex problem that might not actually be part of the <b>core domain</b> of the application. Not only do engineers like dealing with a more "well-defined" problems such as time zone conversion but they also find it very hard to deal with problems that have <i>arbitrary</i> rules such as accounting or business software. These problems are not <i>elegant</i> since there is no <i>elegant</i> solution like Maxwell's solution to them. This is really where analysis patterns come in. Even if there is no <i>elegant</i> solution, at least there are well-tested solutions that can tackle some of the <i>arbitrariness</i>.
</p>
<p>
Because engineers do not like to get into the hairy details of business and domain logic, the <b>core domain</b> usually gets pushed to the less skilled developers who might not have an inkling of how things are supposed to work in the first place. Because of this, no matter how good the <b>generic subdomains</b> are, the core domain still suffers.
</p>
<p>
Again, I found that this chapter had way too many "patterns" or terms. In fact, because I found the terms hard to distinguish, they actually made reading the chapter harder. I was constantly thinking of where else to apply these terms but none came to mind. I will probably change my mind when I reread this chapter again in the future when I have dealt more with some of the issues that Evans mentions. I will probably write my newfound insights after the class on this chapter.
</p>
DDD: Supple Design2006-10-11T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/11/ddd-supple-design<p>
<i>Supple Design (p. 244):</i>
</p>
<blockquote>
"A lot of overengineering has been justified in the name of flexibility. But more often than not, excessive layers of abstraction and indirection get in the way. Look at the design of software that really empowers the people who handle it; you will usually see something simple."
</blockquote>
<p>Evans goes on to say that the early version of software will probably be rigid and stiff. And that many do not even acquire the suppleness that he is talking about.</p>
<p>What Evans says is a good point. But it seems to be very <strong>hard</strong> to achieve. I have seen systems with a lot of overengineering and lots of accompanying abstraction layers piled onto it. It's hard to understand it all at first. But after looking at it for a while, trying things out and actually reading the source code, things are not that bad overall. And when you and the other developers get into this mindset where you understand the code so well, you find it really hard to be able to break out of the box and actually evaluate the complexity of the design. At least until it has become some big ball of mud and then no one knows what to do with it anymore.</p>
<p>In this chapter, Evans offers some patterns that lend suppleness to a design. Combined with the advice from the previous chapter on how to actually dig deeper at a domain model, this might be able to illustrate what a supple design is and how to achieve it.</p>
<p>
<ul>
<li><strong>Intention-Revealing Interfaces</strong> <br />
This is actually similar to a combination of <a href="<a href="http://wiki.java.net/bin/view/People/SmellsToRefactorings">">Fowler's</a> <em>uncommunicative name</em> and <em>inconsistent names</em> code smell. You can eliminate them by renaming the class, method or interface to something that quickly conveys the intent of the code without forcing the developer to go into the details of the implementation. Also, the new name should be consistent with the ubiquitous language.
</li>
<Li><strong>Side-Effect-Free Functions</strong> <br />
<em>Seems like this pattern should be called Side-Effect-Free Operations since in this section, Evans already defines the term function as an operation that returns results without producing side effects</em>.
<br />
<br />
Sometimes, certain operations need to be combined together to achieve a desired result. The order which they are combined might matter because each operation might propagate side effects that "prepare" the system for the next operation. For instance, when reading from a file, you have do first open the file, scan to the segment that you want to read from and then start reading to the points where you are interested in. The file stream has a certain state that needs to be taken into consideration. When the order matters, the developer has to take special care in combining those operations, forcing him/her to go into the details of the code to find out the order of doing things.
<br />
<br />
To solve this problem, you should favor side-effect-free operations i.e. functions. And when it is possible to do so, abstract complex values into their own Value Object.
<br />
<br />
Ruby has this convention of using the ! character for methods that change the internal state. This makes it more explicit which functions have side effects.
<pre style="background: #333333; color: #ffffff;">
<font color="#fcf305">1 </font>array = [<font color="#fc8000">'</font><font color="#ffa0a0">a</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">b</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">c</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">c</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">d</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">e</font><font color="#fc8000">'</font>, <font color="#fc8000">'</font><font color="#ffa0a0">e</font><font color="#fc8000">'</font> ] <font color="#80a0ff"># create array of characters</font>
<font color="#fcf305">2 </font><font color="#80a0ff"># uniq removes duplication in a array but maintains order</font>
<font color="#fcf305">3 </font>array.uniq <font color="#80a0ff"># => ["a", "b", "c", "d", "e"] without modifying array</font>
<font color="#fcf305">4 </font>array <font color="#80a0ff"># => ["a", "b", "c", "c", "d", "e", "e"]</font>
<font color="#fcf305">5 </font>array.uniq! <font color="#80a0ff"># => ["a", "b", "c", "d", "e"] array is now different</font>
<font color="#fcf305">6 </font>array <font color="#80a0ff"># => ["a", "b", "c", "d", "e"] a</font>
</pre>
</li>
<li> <strong>Assertions</strong> <br />
Sometimes it is not possible to use side-effect-free operations especially when dealing with Entities. In such cases, you can help make those side-effects more explicit by placing assertions for pre-conditions, post-conditions and invariants.
<br />
<br />
Languages such as <a href="http://archive.eiffel.com/eiffel/nutshell.html">Eiffel</a> support assertions as part of the language. For languages that do not support them natively, Evans suggests making unit tests to enforce those assertions. So, right now, unit test do not only test for correctness, but they also convey implicit details to the developers. Unit tests are concrete examples that show how to actually use different objects and methods together.
<br />
<br />
Assertions are also part of the <a href="http://www.artima.com/intv/contractsP.html">design-by-contract</a> methodology.
</li>
<li> <strong>Conceptual Contours</strong> <br />
This pattern deals with how you should decompose/ compose your components (classes, methods, modules) so that they do not feel fragmented. For instance, making your components very fine-grained might allow for a lot of flexibility in usage, but it also leads to an explosion of details that need to be managed. On the other hand, making your components very high-level makes it easy to achieve common tasks but it becomes hard to do other tasks. For example, doing bit manipulation in Java is hard because it does not have good support for that.
<br />
<br />
The important thing is to realize how your developers will be using those classes. Java developers do not do a lot of bit manipulation so it is all right to provide them with functions that deal with integers instead of bits and bytes. You find out how your developers use certain components by observing how the system has evolved through refactorings. After various refactorings, you will find that certain components are fairly stable; they are your conceptual contours. However, they are not set in stone. Future refactorings might remove those contours for better ones.
</li>
<li><strong>Standalone Classes</strong> <br />
Classes that depend too much on one another promote unhealthy coupling. Coupling makes it hard for developers to understand the code and make changes to it since one change could easily affect another class. In the extreme case, some classes should be able to function in isolation.
<br />
<br />
A good way to detect for coupling, is to check for <em>inappropriate intimacy</em>, <em>feature envy</em> and <em>message chain</em> code smells. Some metric suites might also be able to detect unhealthy coupling between classes.
</li>
<li><strong>Closure on Operations </strong><br />
I think this pattern is based on the principle of least surprise. For instance, if I give a method some argument, I would expect that the method also returns something that is conceptually similar to the type of the argument. If I give the argument 1 - an <code>int</code> - to the function increment, I would expect it to return another integer but with the value incremented. It might return a <code>double</code> but I would not expect it to suddenly return a <code>char</code>. It's this coherence between the argument and the return values that makes it easy to see what the operation is actually doing.
</li>
</ul>
</p>
<p>Some of these refactorings that have been suggested are pretty small and some are more involved. But they do impact the overall readability and maintainability of the code. The most important thing when doing these refactorings is to understand the fact that these are only rules of thumbs. Refactoring is not a one way process in this case. Sometimes, a previous refactoring might work better and you would have to revert back to it.</p>
<p>Evans also metions Domain-Specific Languages in this chapter as a way to tie the model and the implementation more tightly. He cautions that while the binding between the model and the implementation might be better, it might be a maintenance nightmare to keep the domain-specific language in tune with the ubiquitous language. Also, certain languages like Scheme, Smalltalk and Ruby are more suited for domain-specific language than others like Java and C#. </p>
<pre style="background: #333333; color: #ffffff;">
<font color="#fcf305">1 </font>puts <font color="#40ffff">Time</font>.now <font color="#80a0ff">#=> Tue May 10 17:03:43 CDT 2005 </font>
<font color="#fcf305">2 </font>puts <font color="#ffa0a0">20</font>.minutes.ago <font color="#80a0ff">#=> Tue May 10 16:43:43 CDT 2005 </font>
<font color="#fcf305">3 </font>puts <font color="#ffa0a0">20</font>.hours.from_now <font color="#80a0ff">#=> Wed May 11 13:03:43 CDT 2005 </font>
<font color="#fcf305">4 </font>puts <font color="#ffa0a0">20</font>.weeks.from_now <font color="#80a0ff">#=> Tue Sep 27 17:03:43 CDT 2005 </font>
<font color="#fcf305">5 </font>puts <font color="#ffa0a0">20</font>.months.ago <font color="#80a0ff">#=> Thu Sep 18 17:03:43 CDT 2003 </font>
</pre>
<br />
<small>Domain-Specific Languages should be readable (verbally) and still make sense</small>
DDD: Relating Design Patterns to the Model2006-10-11T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/11/ddd-relating-design-patterns-to-the-model<p>
I am actually a bit surprised that this chapter is in the book. It did not even occurred to me that there might be a separation between design patterns and the patterns mentioned in this book. When I read the chapter that says that there is a close binding between the model and the implementation, I was already convinced that certain design patterns could be applied when modeling. After all, all of the design patterns are language agnostic so they are not tied down to implementation details. Also, we have already been looking at patterns from various patterns book, so I convinced that design patterns are high-level enough for domain modeling.
</p>
<p>
For instance, when the book talked about Aggregates and Repositories, I was under the impression that the Facade pattern might come in useful when we are modeling since Facade provides a higher-level interface that makes the subsystem easier to use. Or if my system will be relying on information from another system, I might use the Adapter or Proxy pattern between those two systems to make the two systems work together.
</p>
<p>
The chapter elaborates on why the connection pattern and the design pattern might be blurred. Some people have the notion that Design Patterns are technical hacks that one applies to overcome certain limitations in the programming language. Instead, I see them as general ways to think of how to design things. The technical details are important, of course, but the intent of the design pattern is even more important.
</p>
<p>
So, either I am completely off and the book is stating something very different. Or I am right in my assumption on how to apply design patterns and the book is merely stating the obvious.
</p>
DDD: Applying Analysis Patterns2006-10-11T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/11/ddd-applying-analysis-patterns<p>
The term Analysis Patterns come from Fowler's book, <a href="http://www.amazon.com/Analysis-Patterns-Reusable-Object-Oriented-Engineering/dp/0201895420"><i>Analysis Patterns: Reusable Object Models</i></a>. In this chapter, Evan tries to show how applying those patterns can actually help create a more mature domain model. Utilizing past experiences in the field help reduce the mistakes that beginner designers make when dealing with a field for their first time.
</p>
<p>
Drawing on the experiences of other software developers is actually very useful. It is not <i>the</i> solution to your problem but at least it shows one way that might work. Furthermore, because Analysis Patterns is described in a way that developers are able to understand it, developers are actually able to come to grasp with the model more quickly. You see terms used in the analysis pattern that can become part of your development teams' ubiquitous language.
</p>
<p>
My only concern with analysis patterns is that they might be hard to read. They focus on one field in great detail. It is not easy to actually explain an entire accounting system without some basic background on the reader's part. And for readers like me who have no idea how accounting actually works, this chapter was actually rather hard to read. However, it is this detail that makes it valuable to developers working on the project. It's just that most people will not pick up this pattern to read for fun.
</p>
<p>
Fowler has posted a .pdf of some more <a href="http://www.martinfowler.com/apsupp/accounting.pdf">accounting patterns</a> on his website for those who are interested.
</p>
Software Development: It's a Religion2006-10-09T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/09/software-development-its-a-religion<p><a href="http://www.codinghorror.com/blog/archives/000699.html">Coding Horror: Software Development: It's a Religion</a>:
</p>
<blockquote cite="http://www.codinghorror.com/blog/archives/000699.html" title="Coding Horror: Software Development: It's a Religion"> "But software development is, and has always been, a religion. We band together into groups of people who believe the same things, with very little basis for proving any of those beliefs. Java versus .NET. Microsoft versus Google. Static languages versus Dynamic languages. We may kid ourselves into believing we're 'computer scientists', but when was the last time you used a hypothesis and a control to prove anything? We're too busy solving customer problems in our chosen tool, unbeliever!"
</blockquote>
<p>
To the list above, we could easily add agile vs. non-agile development process , object-oriented vs. non-object-oriented, etc. Does refactoring code actually improve anything? Is the waterfall process truly abysmal? This was precisely what <a href="http://wiki.cs.uiuc.edu/cs527/Paul+Adamczyk">Paul Adamczyk</a> <a href="http://pinky.cs.uiuc.edu/~padamczy/lectures/2006/cs527/saip11-12.ppt">talked</a> about the other day when he substituted for Prof. Johnson. How much of software development can we actually prove as being more effective? Why do people flock to Process X instead of Process Y? And then why do the people in X flock to W when it comes out?
</p>
<p>
So why do people do research in software engineering? Well, like religion it's an interesting field. You never know when you will become an apostate your faith. Or how long you will hold on to your dying sect. And on the day of judgment (the release date for software developers) you will truly find out if your God has abandoned you.
</p>
<p><a href="http://blog.blainebuxton.com/2006/10/what-makes-ruby-roll.html">Blaine Buxton: Positronic Vibrations</a>:
</p>
<blockquote cite="http://blog.blainebuxton.com/2006/10/what-makes-ruby-roll.html" title="Blaine Buxton: Positronic Vibrations"> "As long as the people who have big checks are running on the CLR and JVM Ruby will have to crossover to those platforms to succeed. Business and economics were the downfall of Smalltalk, not natural selection. The 'arrogance of the smalltalk communities sealed the lid'."
</blockquote>
<p>
As long as everyone plays nice together, all software development techniques, methodologies, tools can coexist together. <i>Hopefully</i>. Passion for something is good but over-zealousness is just plain unhealthy.
</p>
<p>
For all we know, software engineering might just be the <a href="http://en.wikipedia.org/wiki/The_Answer_to_Life,_the_Universe,_and_Everything">answer</a> we need to life, the universe and everything.
</p>
DDD: Using the language2006-10-08T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/08/ddd-using-the-language<p>
<i>Design Patterns, Elements of Reusable Object-Oriented Software</i>:
</p>
<blockquote>
"The key to maximizing reuse lies in anticipating new requirements and changes to existing requirements, and in designing your systems so that they can evolve accordingly."
</blockquote>
<p>
This is an interesting chapter. While it is more of a case study and a summary of all the patterns and techniques that have been mentioned so far, it also serves to illustrate the important concept of reuse. The scenarios presented on p. 173 give examples of changes that need to be handled. I can see how a layered architecture will help maximize the reuse of the system domain and how well it can accommodate changes. Without the discipline to enforce a layered architecture from day one, the model will quickly become littered with all sorts of application logic that will hinder changes to the model.
</p>
<p>
However, I still find it very hard to come up with a model that is this robust. Evans already mentions that he did not show the evolution of the model in this chapter since he wanted to keep the chapter short and focused. Yet, it still seems hard to me how he came up with it in the first place. No doubt talking to domain experts must have really helped. They are the people who have the experience and can tell you how the customers will be using the software. And they are the people who can help you greatly in encapsulating changes. I like how the chapter also hints at the complexity of software that have to be used in different countries: the tax ID cannot be used as a unique key to the customer in an international company (p. 167). With changes that might even require handling international cargo, the model really becomes important. A simplistic model might work well for the first few iterations but will fail when new unprecedented requirements come in.
</p>
<p>
From reading this chapter, I got a good grasp of what it means to accommodate the different changes. It is not sufficient to only prepare for the initial version 1.0 release of the software. Instead a good software developer must be able to take the software in new directions from there without rewriting everything. And the techniques that Evans has presented serve as good guidelines for that purpose. Of course, refactoring is also going to be very important and that is the focus of the next section.
</p>
<small>There is this rather controversial book entitled <a href="http://www.oreilly.com/catalog/prefactoring/"><i>Prefactoring</i></a> that seems to go against the principle of YAGNI: You Ain't Gonna Need It. I have not had the pleasure of reading it but it also seems to offer some guidelines on how to initially design the system based on past experiences. A short introduction to prefactoring can be found <a href="http://www.oreillynet.com/lpt/a/6327">here</a>.</small>
DDD: Making Implicit Concepts Explicit2006-10-08T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/08/ddd-making-implicit-concepts-explicit<p>
This chapter was a bit long and I felt that it should be split into two chapters. Overall the contents of the chapter fits well with the title but the first part focused on making iterative steps to refine the model and the second part focused on specifications.
</p>
<p>
The first part of the chapter mentions some useful methods for identifying missing concepts from the domain. For instance, by paying attention to the daily conversations between developers and domain experts, one might be able to pick up discrepancies in model and the description that the domain experts use. These discrepancies hint at possible concepts that might enhance the model. What else should you pick up from conversations with the domain experts? Pay attention to the contradictions between domain experts. These contradictions might be signs of different approaches to the problem and you should be prepared to accommodate multiple solutions if necessary.
</p>
<p>
Moreover, you should also not discount the fact that reading a book, a paper or some article on the subject might pave the way for a better domain model. Some domain experts do not have the time to sit through every meeting with the developers. In such cases, it is best to pick up the literature and peruse it. It might also be beneficial to see how other developers have approached this problem in the past.
</p>
<p>
Another quick way to diagnose potential problems with the model is by checking for <a href="http://wiki.java.net/bin/view/People/SmellsToRefactorings"><i>code smells</i></a>. By focusing on the smells between classes, you can identify potential trouble hotspots that might require some refactoring. Of particular interest are smells such as <i>divergent change</i>, <i>shotgun surgery</i> and <i>parallel inheritance hierarchies</i> since those smells affect a large portion of the source code.
</p>
<p>
Refactoring toward a breakthrough is a hard enough process. But actually finding the end goal for <i>refactoring to</i> can also be complicated. What Evans has done is present a list of rules of thumb and illustrate them using concrete examples. Even with these guidelines, Evans does not fail to emphasize the importance of trial and error. Sometimes, it takes multiple session of trial and error before getting the <i>right</i> model. Experience certainly helps but I would assume that a great deal of perseverance and an excellent version control - for reverting code - helps significantly as well.
</p>
<p>
In the second part of the chapter, Evans focuses on the <i>Specification</i> pattern. The <b>Specification</b> pattern is a predicate that determines if an object does or does not satisfy some criteria. By making this predicate its own object, it becomes clearer to people that this is requirement rather than some arbitrary behavior of the object or model. I felt that this was the main theme of the chapter: making implicit concepts explicit in an intention-revealing manner.
</p>
<p>
A Specification object takes care of handling the testing of whether some criteria have been met. There are three main uses for specifications: validation, selection and generation. These three uses are on the same conceptual level.
</p>
<p>
Specifications should not be confused with <i>Assertions</i> (presented in the next chapter) which ensure that certain pre-conditions, post-conditions and invariants are maintained through the program. Use a specification to test if an object fulfills some criteria that is part of the domain logic but use an assertion to make explicit the side effects from calling some procedure. Specifications and assertions both make the intent more explicit but they do so for different purposes. Used together, both can communicate the model better.</p>
DDD: Breakthrough2006-10-08T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/08/ddd-breakthrough<p>
Martin Fowler made the term <i>refactoring</i> popular. His book <a href="http://www.amazon.com/exec/obidos/ASIN/0201485672"><i>Refactoring: Improving the Design of Existing Code</i></a> offers micro-refactorings that help reduce the stench of the code smells in the code. These micro-refactorings can usually be automated as exemplified by increase of automated refactorings in popular IDEs. Taking it up a notch is the concept of <a href="http://www.amazon.com/Refactoring-Patterns-Addison-Wesley-Signature-Kerievsky/dp/0321213351"><i>Refactoring to Patterns</i></a> introduced by Joshua Kerievsky in a book of similar title. Refactoring to Patterns describes the process of modifying existing code to embody established design patterns. I am halfway through that book and it does offer good linkage between design patterns and refactoring.
</p>
<p>
Nonetheless, Evans wants to "kick it up a notch" (to quote TV's celebrity chef). The refactorings he wants to achieve not only make the code more readable, but it is also offers the "aha" feeling to the design when it is done right. This is the breakthrough. At this point, not only is code being affect, the entire model might be affected as well. This is a change that will ripple through the entire system. Thus, it is a costly change that can take weeks (not hours or even minutes like the micro-refactorings). But, with its cost, comes greater insight into the design and model more suited to address future requirements (or handle existing one better).
</p>
<a href="http://www.flickr.com/photos/vazexqi/264218417/" title="Photo Sharing"><img src="http://static.flickr.com/107/264218417_3bf43499b9_m.jpg" width="240" height="194" alt="Breakthrough" /></a>
<p>
<small>Figure 1: Returns from refactorings are not linear </small>
</p>
<p>
This chapter presents the motivation for refactoring the domain. While this chapter ended with a happy ending, I will not be surprised if some refactorings had dire consequences instead of beneficial ones. Nonetheless, those dire consequences will open up new insights that can be used in a future iteration.
</p>
DDD: Life Cycle of a Domain Object2006-10-04T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/04/ddd-life-cycle-of-a-domain-object<small>
This post includes a lot of examples from the Ruby on Rails framework simply because I am working on it for my documentation project for this class. So I could easily see where these patterns where used in that framework. I am sure that other frameworks provide similar features and I am not advocating one over the other.
</small>
<p>
It's funny how sometimes when you read about something and that something pops up somewhere else. So, I was reading about the ActiveRecord (it's like the Active Record pattern in P or EAA but has more features so it might be a misnomer ) module in the Rails framework and they mentioned about using aggregation in the models. Here's the comment blurb directly from the source code:
</p>
<p><i>rails-trunk/activerecord/lib/active_record/aggregations.rb</i>:</p>
<blockquote>
Active Record implements aggregation through a macro-like class method called +composed_of+ for representing attributes as value objects. It expresses relationships like "Account [is] composed of Money [among other things]" or "Person [is] composed of [an] address". Each call to the macro adds a description of how the <b>value objects</b> [emphasis added] are created from the attributes of the entity object (when the entity is initialized either as a new object or from finding an existing object) and how it can be turned back into attributes (when the entity is saved to the database).
</blockquote>
<p>
More information can be found on Rails <a href="http://api.rubyonrails.org/classes/ActiveRecord/Aggregations/ClassMethods.html">aggregation</a> here. Validation support is also <a href="http://api.rubyonrails.org/classes/ActiveRecord/Validations.html">built-in</a> to the models.
</p>
<p>
I am sure that other frameworks out there support aggregations out-of-the-box as well but it is interesting to know that Rails does support some of the best practices in domain modeling. And how I would have used aggregations in Rails without even bothering with the fact that there was a pattern called Aggregations. Frameworks that make such patterns almost transparent are a great way to pick up patterns even without realizing it.
</p>
<p>
Aggregates are great for simplifying the issue of maintaining and handling transactions. Even if your database supported pessimistic or optimistic offline locking you would have to be very careful to order your database accesses to avoid deadlocks and. By using an aggregate properly, you can get away with some of that responsibility since all access to the database can only be done in larger chunks (the aggregate).
</p>
<p>
After discussing Aggregates, the chapter goes on to Factories. Factories is a generic term applied to the creational patterns such as the ones mentioned in Design Patterns: factory method, abstract factory and builder. Evans does not discuss each one in turn but shows how the concept of Factories are important to encapsulate the process of object creation so that that process of creation does not complicated the underlying domain. Some objects are hard to construct so expressing its construction process as a complicated region of code might distract the reader from the real domain underneath the system.
</p>
<p>
Finally the book presents Repositories. Repositories seem like a wrapper to me around the complex process of object retrieval. Just like object creation, object retrieval itself can be a complex process. For instance, if you are using a database, it is not uncommon to have to do joins across tables to actually get at the data that you want. However, executing a raw SQL query is not very clean; it exposes the encapsulation of what your tables are. Evans made sure to include the fact that sometimes, the framework already does a decent job of information retrieval for you. At least, that is the case with the Rails framework using reflection and meta-programming.
</p>
<pre style="background: #333333; color: #ffffff;">
<font color="#fcf305">1 </font> <font color="#ff80ff">class </font><font color="#60ff60"><b>Post</b></font> < <font color="#40ffff">ActiveRecord</font>::<font color="#40ffff">Base</font>
<font color="#fcf305">2 </font> has_one <font color="#40ffff">:author</font>
<font color="#fcf305">3 </font> <font color="#ff80ff">end</font>
<font color="#fcf305">4 </font>
<font color="#fcf305">5 </font> <font color="#ff80ff">class </font><font color="#60ff60"><b>Author</b></font> < <font color="#40ffff">ActiveRecord</font>::<font color="#40ffff">Base</font>
<font color="#fcf305">6 </font> belongs_to <font color="#40ffff">:post</font>
<font color="#fcf305">7 </font> <font color="#ff80ff">end</font>
</pre>
<smal>Listing 1: Ruby code that expresses the relationship between classes</small>
<pre style="background: #333333; color: #ffffff;">
<font color="#fcf305"> 1 </font> <font color="#ffff60"><b>CREATE</b></font> <font color="#fc8000">TABLE</font> posts (
<font color="#fcf305"> 2 </font> id int(<font color="#ffa0a0">11</font>) <font color="#ffff60"><b>NOT</b></font> <font color="#fc8000">NULL</font> auto_increment,
<font color="#fcf305"> 3 </font> title <font color="#60ff60"><b>varchar</b></font> <font color="#fc8000">default</font> <font color="#fc8000">NULL</font>,
<font color="#fcf305"> 4 </font> PRIMARY KEY (id)
<font color="#fcf305"> 5 </font> )
<font color="#fcf305"> 6 </font>
<font color="#fcf305"> 7 </font> <font color="#ffff60"><b>CREATE</b></font> <font color="#fc8000">TABLE</font> authors (
<font color="#fcf305"> 8 </font> id int(<font color="#ffa0a0">11</font>) <font color="#ffff60"><b>NOT</b></font> <font color="#fc8000">NULL</font> auto_increment,
<font color="#fcf305"> 9 </font> post_id int(<font color="#ffa0a0">11</font>) <font color="#fc8000">default</font> <font color="#fc8000">NULL</font>,
<font color="#fcf305">10 </font> name <font color="#60ff60"><b>varchar</b></font> <font color="#fc8000">default</font> <font color="#fc8000">NULL</font>,
<font color="#fcf305">11 </font> PRIMARY KEY (id)
<font color="#fcf305">12 </font> )
</pre>
<small>Listing 2: The SQL table that represents the relation in Listing 1</small>
<p>
With that you can do is:
</p>
<pre style="background: #333333; color: #ffffff;">
<font color="#fcf305">1 </font><font color="#80a0ff"># find_by_title is a method generated automatically based on your table schema</font>
<font color="#fcf305">2 </font>post = <font color="#40ffff">Post</font>.find_by_title(<font color="#fc8000">"</font><font color="#ffa0a0">First</font><font color="#fc8000">"</font>)
<font color="#fcf305">3 </font>post.author <font color="#80a0ff"># this will return the right author for that post </font>
</pre>
<p>
It works for many-to-many relationships as well. More information can be found <a href="http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html">here</a>.
</p>
<p>
What's interesting is that the three patterns discussed here all help to make sure that the underlying model of the system remains as uncluttered as possible. The implementation and the model should be bound together as advocated in Chapter 3, but it's also important that details of implementation - that might be the inherent limitation of the framework or programming language - do not interfere with the model. If there is too much interference, then no matter how you try to use a Layered Architecture, it will still be hard to extract the underlying logic for the model from the code.
</p>
DDD: A Model Expressed in Software2006-10-04T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/04/ddd-a-model-expressed-in-software<p>
This chapter begins to define the importance of associations among different models and how to refine those associations. Refinement makes models that are simpler to understand. Furthermore, careful refinement also reveals overly complex associations that might not actually exist in the domain that we are modeling. It is best to prune those redundant associations early to have a model that is both simple and accurate.
</p>
<p>
To facilitate describing associations between objects (not models), Evans presents four patterns: Entities, Value Objects, Services and Modules.
</p>
<p>
<b>Entity</b> is a pattern used when you need to distinguished an object by its identity, rather than its attributes. For instance, in a grading application, you want to distinguish students based on their name. You cannot use any of the other attributes such as their total points, their discussion section, their major. These variables may change but the name of the student is unlikely to change. That is the identity of that student. Maintaining this identity is important because that is the <i>only</i> way to determine if you are really who you claim you are.
</p>
<p>
If you are familiar with databases, you realize that in each table, there is a primary key that will uniquely identify the row on that table. Even if that row is deleted, the same primary key will should not be used again for a new row since it might be confusing to the application. That is why most databases provide the ability to auto-increment the primary key to guarantee that it is unique for all rows and that no row will have to use a previously issued primary key. No chance of a mistaken identity with a deleted row.
</p>
<p>
In a blogging software, each post should be identified by GUID (global unique identifier). This allows you to track changes to that post. For instance, the author might decide to modify the contents of that blog post, but it still the same blog post. Good RSS readers make use of the GUID to determine if a post is new and will mark it as such. Those same RSS readers can also determine if the post is just a modified post and will show the difference between the previous version (from local cache) and the new version (from the server).
</p>
<p>
In short, the Entity pattern lets you identify an object uniquely even if the attributes of that object might evolve. As the examples above show, there is a whole lot of use for entities. However, there is also the temptation for over-using the Entity pattern. Sometimes some objects just do not care about their identities. For that you should use the Value Object pattern.
</p>
<p>
The <b>Value Object</b> pattern is useful when you care about the attributes of the object instead of its identity. For instance, when using numbers to add, you do not care if you do not distinguish between '7' and another '7'. All you care about is the value. A '7' is a '7' is a '7'. You would care if suddenly a '4' became a '7'. That is why, value objects are immutable. Once you create a number '4' you expect it to remain that way. Immutable objects are better for performance since you never have to keep track of their identity. In fact, the chapter talks about ways to make value objects faster. One method involves using the Flyweight pattern.
</p>
<p>
Not everything can fit into a Entity or a Value Object. It would be very constraining to have everything be either one of those objects. Instead, the book mentions a third pattern: <b>Services</b>. Use the Service pattern when your object does not really fit with the definition of an Entity of a Value Object. There is the temptation to make everything a "service" and eschew good object-oriented design patterns. Use the Service pattern only when it becomes awkward to actually represent the object as an Entity or Value object. Even then, try to employ other design patterns such as the Facade or Singleton to keep the design manageable. A Mediator might be necessary for more complex systems.
</p>
<p>
The final pattern that was mentioned is Modules. <b>Modules</b> should be used to separate code into groups that are decoupled from other groups but have cohesiveness within the same group. You can have groups nested in groups. Most developers already use modules. For instance, in Java, the concept of packages is common because it is a convention. However, the focus here is on naming the modules correctly so that they reflect the ubiquitous language discussed in Chapter 2. Refactoring modules by renaming or moving files is not practiced much because it can cause a cascading effect to the source control. As such, careful naming should be done early.
</p>
<p>
This chapter has discussed the next step in model-drive design. And that is how to represent the model in software. While the book's primary focus is on using object-oriented languages, the end of this chapter also presents some methods for combining different languages together. The patterns help designers express the model in software.
</p>
DDD: Isolating the Domain2006-10-03T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/03/ddd-isolating-the-domain<p>
To have a successful domain model, we need to separate the different concerns from the model. A domain model is easily cluttered with code from the UI, database and other supporting code. When this clutter happens, it becomes harder to sieve through the code and determine what's the actual domain. The current trend to mitigate this problem is by separating the system into different layers and minimize the interactions between those layers.
</p>
<p>
This chapter presents a general layered architecture that comprises:
<ul>
<li>User interface layer</li>
<li>Application Layer</li>
<li>Domain Layer</li>
<li>Infrastructure layer</li>
</ul>
The user interface does not communicate directly with the domain layer. Instead it does so through the application layer. This separation is based on the Model-View-Controller compound pattern and makes use of the heuristic that the view (the user interface layer) is more likely to change compared to the model. Therefore, instead of putting code that talks to the model directly in the view, we have a middle-man: the controller (application layer). The infrastructure layer provides the technologies for persisting the domain objects and presentation.
</p>
<p>
Two patterns from the Patterns of Enterprise Application Architecture (P of EAA) were briefly mentioned:
<ul>
<li><a href="http://www.martinfowler.com/eaaCatalog/unitOfWork.html">Unit of Work</a></li>
<li><a href="http://www.martinfowler.com/eaaCatalog/transactionScript.html">Transaction Script</a></li>
</ul>
I found it useful to skim through those patterns from the P of EAA book while reading this chapter.
</p>
<p>
This chapter also gives some warning about architectural frameworks that might seem to be useful but actually stymie the development of a domain model because the framework demands that files fit certain constraints. This is especially true if you are following the framework blindly. Steve McConnell talks about this in <a href="http://cc2e.com/">Code Complete 2</a> when he mentions programming <i>into</i> a language instead of programming <i>in</i> a language. McConell was using Visual Basic to program the UI and Visual Basic makes it really tempting to place all the business code into the forms (.frm files). In fact, Visual Basic <i>advocated</i> keeping everything in the form file and did not make it easy to delegate functions from the .frm file to the associated .bas file. But that did not stop McConnell. He came up with his own convention to minimize the business logic in the .frm files. The moral of the story: do not just program in the framework, program <i>into</i> it.
</p>
<p>
The "Smart UI" anti-pattern is a very common pattern for novice UI designers. When using a visual editor that allows drag and drop of widgets and auto-generation of code, most novices just dump all the code into the auto-generated placeholders. For instance, if you are using Visual Studio's Form editor, it is easy to just double click on the button and be transported to the callback code that is to be executed when that button is clicked. Without analyzing further, a user might be tempted to put all the code inside that callback handler instead of delegating the function to some controller.
</p>
<p>
Often, auto-generated code leads to code that is hard to maintain and hard to understand. Auto-generated code does not try to be "smart"; it tries to be safe. When creating new functions, the safest thing would be to call them function1, function2 and so on. This helps eliminate the chances of a name clash. But these names make no sense to a developer.
</p>
<p>
What the "Smart UI" anti-pattern is trying to show, is that you can still have a cluttered code base if you are not careful with the tools you use. A visual editor is suppose to make it easy to generate the user interface, but it is up to the developer to be skeptical about how to organize the code. Do not just blindly follow the suggested auto-generated placeholders.
</p>
<p>
In conclusion, this chapter focuses on the need for separation of concerns between the different layers. Leaving the model clean from all extraneous code helps developers focus their effort on creating a richer model instead of wasting time trying to figure out what actually belongs in the model and what does not.
</p>
DDD: Crunching knowledge2006-10-01T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/01/ddd-crunching-knowledge<small>
The class has finished the SAIP book and is now moving over to the a new book: <a href="http://domaindrivendesign.org/books/index.html#DDD">Domain-Driven Design</a> by Eric Evans. The reviews have been good so far and I expect this book not only to be a good read but also to be educational as well without all the dry formalisms of a textbook. All posts on readings from this book will be prepended with the acronym DDD.
</small>
<p>
While reading the introduction to this book, I was curious as to how much the model referred to in this book is similar to the metaphor in XP. I was not able to find anything in the index about XP or metaphor so I assume that this book does not dwell on that issue. However, this issue of a model and metaphor is pretty interesting (to me at least).
</p>
<p>
In <i>Extreme Programming Explained</i>, Kent Beck defines the system metaphor as:
</p>
<blockquote>
A story that everyone - customers, programmers, and managers - can tell about how the system works.
</blockquote>
<p>
A model for the domain seems to be fulfilling some of that task. With a suitable domain model, one can more easily implement and design the system to correspond to the model. The model itself also serves as the basis for communicating amongst the developers when they refer to parts of the system. Instead of calling the method funcA, they will call it applyFilter for instance. Moreover, the model is an abstraction of the most important features of the system; it allows one to describe the internal workings of the system in terms of these abstractions. With these abstractions, the developers can communicate better with the people who are using the software. You do not talk to Photoshop users about the convolutions involved mathematically to change the image; you talk to them in terms of filters and masks.
</p>
</p>
Unfortunately, since less than 2 pages were allocated to discuss the system metaphor in XP, most people do not actually know the real meaning of the metaphor and it is hard to compare it with the concept of a domain model.
</p>
<p>
Anyway, Chapter 1: Crunching knowledge presents a process of effective modeling. It's called knowledge crunching because like number crunching, you start of with globules of unrelated data and try to synthesize something. And the best knowledge crunchers do not work alone; they talk to domain experts. Want to create software that deals with gene structures? Talk to a cellular biologist. The best knowledge crunchers also see "real" pervasive links (not fragile links) between the clumps of knowledge. They can formulate new relationships and remove obsolete ones. Their knowledge of the system grows as they become more experienced.
</p>
<p>
Evans has outlined the methods for effective modeling:
<ol>
<li>Binding the model and the implementation <br />
While abstractions are nice for communicating and getting the initial idea across, building concrete instances are also important. Building something forces you to think about the parts that are still missing from your domain model.
</li>
<li>Cultivating a language based on the model</li>
<li>Developing knowledge-rich model</li>
<li>Distilling the model <br />
Do not be afraid to throw something out or formulate a new model if the old models do not scale well. Your previous models serve as guides and pointers for you. Your new one model will build upon those lessons that you have learned.
</li>
<li>Brainstorming and experimenting <br />
New paradigms come and go. You need to know when to embrace them.</li>
</ol>
</p>
<p>
In the example about shipping books, Evans shows how important it is to capture important domain knowledge and not let it be hidden in the code. In his example, the concept of overbooking was being implemented using a simple if... then clause in the code. While this certainly works, it does not immediately convey the message of overbooking to the code reader. By abstracting that overbooking if... then clause into a different class (a strategy), it makes it clearer what the if... then clause actually serves. Clear code is not only easy to read and understand, but it also conveys important underlying features across clearly.
</p>
DDD: Communication and the Use of Language2006-10-01T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/01/ddd-communication-and-the-use-of-language<p>
If you have ever played the game where a group of people sit in a row and you try to to pass a message verbally from the first person to the last person, you know how fun this activity can be. Not only is the sentence deformed in every way imaginable, but the original meaning is usually completely lost.
</p>
<p>
This chapter emphasizes the importance of communication. All major software projects involved more than one developer, so it is important that every developer communicates clearly to one another. One way to do this is to use well-established terminologies. A queue to a data programmer might mean a data structure but to a non-programmer, it can validly refer to "a braid of hair worn at the back". So it best to make sure that everyone is on the same wavelength before using a term. As the book claims, a project faces serious problems when its language is fragmented across the team members.
</p>
<p>
And thus the concept of a ubiquitous language arises. With this ubiquitous language, the same terms can be applied in verbal and written communication. You use the same terms in code and in your documentation and in your daily communications. It's unlikely to have this ubiquitous language from day one of the project. But as your project matures, you will find yourself building the vocabulary of the language and refining it to make it more precise.
</p>
<p>
One very interesting theory in anthropology is the <a href="http://en.wikipedia.org/wiki/Sapir-Whorf_hypothesis">Sapir Whorf</a> theory. It says that you can only understand the world if you can describe it using some language (not necessarily written since some languages have no written form). So the canonical example is that the Hopi people are not able to come to grasp with the notion of future and past since their language does not support it. They have trouble understanding what "yesterday" or "tomorrow" means. The same thing can be applied for domain modeling. Without a suitably rich language and vocabulary, developers will never be able to see what they are actually missing. Enhancing your language helps enhance your model as well.
</p>
<p>
There were some terms in the diagrams in the chapter that I did not really understand. For instance, cargold and hazmat. I found out that hazmat referred to HAZardous MATerials but I still have no idea what cargold is. We will probably revisit this diagram soon but it would be good to know what cargold is.
</p>
<p>
Evans dedicates a section for modeling out loud. After reading that section, it felt like a CRC: Classes, Responsibilities, Collaboration session to me. In a CRC session, you have people talk about the problem and try to create suitable classes as models for the problems. This seems like a good way to come up with some models as well. CRC sessions are highly interactive and everyone gets a chance to say something. We can use the general idea of CRC cards without getting too much into the details.
</p>
<p>
UML is good for getting ideas across quickly in a diagrammatic manner. However, Evans is quick to realize that a diagram is just a diagram and nothing more. You can definitely represent a lot of things in a diagram but there are certain things that are easier to describe using words. In effective communication, the key is to convey the intent as clearly and quickly as possible. When you have to scrutinize the picture for "hidden" meanings, then you might as well use text. Over-advocating one tool is always a bad idea. All tools have their limitations and knowing those limitations enables you to make judicious use of them.
</p>
<p>
All in all, by using an ubiquitous language in as many parts of the development as possible, we can come up with a consistent way of describing the system that makes sense to the stakeholders. As the book says: "[O]ne model should underlie implementatoi, design and team communication. Having separate models for these separate purposes poses a <em>hazard</em>."
</p>
DDD: Binding Model and Implementation2006-10-01T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/10/01/ddd-binding-model-and-implementation<p>
This chapter emphasizes the importance of the model and the implementation. It is useless to use a different model when designing and a different one when actually implementing the system. Not only does this mean that whenever something change you are going to have to modify two models (a prime candidate for introducing inconsistencies and bugs) but you would also have deal with the mapping between the two models. It is always easier to deal with just one model. And from the previous chapter, that one model will help promote the use of an ubiquitous language.
</p>
<p>
The process of binding the model and its implementation is called model-driven design. And together with the ubiquitous language, forms the basis for domain-driven design. Model-driven design is more easily implemented in a language that supports modeling. Object-oriented languages do this well and thus is more suited for this task. Procedural languages hide the model by operating in terms of passing data around between functions.
</p>
<p>
A good article always gives concrete examples. And this book has done exactly that in the example for the PCB board. I have had some experience using CAD tools (<a href="http://www.mentor.com/products/fpga_pld/hdl_design/hdl_designer_series/">HDL Designer</a>) for designing microprocessors in one of my computer engineering class so the terms he used are similar to the ones that we use when designing. Instead of nets we called them signals and in addition to buses, we also had bundles which could contain signals and buses. The CAD tool was helpful in letting us connect nets and buses using the mouse, but that was just a visual representation of it. Underneath all that GUI, the design was still embodied in VHDL (or Verilog) which is stored as a text file. And sometimes, when using buses, you have to manually edit the names of the individual nets in a tabular manner.
</p>
<p>
The mechanistic manner would surely work since everything was basically in VHDL which is very editable by text manipulating scripts. But the model-driven design manner was much cleaner to a programmer. The ability to unit test each the methods were also an added advantage.
</p>
<p>
One thing that was probably enforced but not mentioned explicitly in the text was that each net and bus name were unique. You cannot have buses with the same name in the same layout because that is not legal syntax and would only confuse the simulator. Nets and buses with the same names are going to be treated the same.
</p>
<p>
The example of Internet Explorer's favorites is a good example of the importance of a model that matches the user's mindset. A matching model has a lower learning curve for the user and lets the user make use of the features of the application easily.
</p>
<p>
In conclusion, this chapter shows the importance of having a model that is tied closely to the implementation. Without this tight relationship, the implementations and the underlying abstractions become harder to reconcile as the project requirements evolve leaving the system in a state of code rot.
</p>
Inheritance and relational databases2006-09-29T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/29/inheritance-and-relational-databases<p>
I am reading Martin Fowler's Patterns of Enterprise Application Architecture. I have been flipping between patterns because the project that I am documenting, Typo, makes used of most of them. Actually Typo is built on top of the Rails framework, so it is more correct to say that Rails makes use of them and Typo builds on them.
</p>
<p>
To get a better feel for Rails, I have been perusing <a href="http://www.pragmaticprogrammer.com/title/rails/">Agile Web Development with Rails</a>. It's a nice book. More tutorial-like in contents than a real book on the architecture of Rails. Between reading that book and the source code, I think I have a pretty good idea how Rails is structured. The source code is not as easy to read as I had hoped. Some files are over 2000 lines long because they were comments that were lengthy examples of how some concept works. Also, there is heavy use of meta-programming around so it is easy to miss something since you cannot just do a search for that term that you expect.
</p>
<p>
Anyway, there are three patterns in Fowler's book that deal with how to handle inheritance within databases. I read them all and was really astonish that I have a sense of deja vu for these patterns. Seems like we had already covered these patterns in my database course albeit with different names for them.
</p>
<a href="http://static.flickr.com/84/255672494_8846ca5dec_o.png" title="Handling inheritance in databases" rel="lightbox"><img src="http://static.flickr.com/84/255672494_8846ca5dec.jpg" width="500" height="401" alt="Handling inheritance in databases" /></a>
<p>
So here is how those patterns correspond to the picture above:
<ul>
<li>Single Table Inheritance - The Null Value Approach <br />
Create a super-node kind of table that can accommodate all children of the current class. Fields that are not used will be left null.
</li>
<li>Concrete Table Inheritance - The OO approach <br />
Create a table for each concrete class. Fields common to classes will be duplicated across the tables.
</li>
<li>Class Table Inheritance - The E/R approach <br />
Class Table Inheritance will try to eliminate duplication between elements by specifying a parent class and then using foreign keys to link them together
</li>
</ul>
</p>
<p>
The Class Table Inheritance scheme is potentially going to be able to save space but it is hard to actually retrieve stuff since you have to do multiple joins on the tables.
</p>
<p>
The OO approach is going to be really explosive! For every possible combination of child-parent relationship, you are going to have a table for it. Each kind of relationship will thus only fit in table by default. I can imagine that if you ever need to change you schema, you will probably be in for a lot of trouble.
</p>
<p>
And now, as if that sense of deja vu was not enough, Rails also uses one of the methods above fpr handling complex relationship between tables. Rails has built-in functions for associations and inheritance. It uses the simplest of the three methods above: <a href="http://wiki.rubyonrails.com/rails/pages/SingleTableInheritance">Single Table Inheritance</a>.
</p>
SAIP: Software Architecture in the Future2006-09-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/27/saip-software-architecture-in-the-future<p>
This is the last chapter of the book. Somehow, I feel that most of the contents here should have been placed either in the prologue or in the preceding chapters. For example, the evolution of software from subroutines, modules, objects and frameworks should have been mentioned earlier to give the reader an idea of how software has evolved.
</p>
<p>
This chapter mentions the shortcomings of the some of the techniques introduced in the previous chapters and I am glad that the authors acknowledge those shortcomings. For instance, in Chapter 5: Achieving qualities, readers were presented with a very unclear description on how to proceed once the development team has a plethora of tactics that they want to employ. Chapter 5 suggested that from those tactics, one could easily pick a right type of architectural style to apply. I am doubtful that it is that simple. If it were that simple, then there would not be a need for all this discussion in the first place.
</p>
<p>
The authors also discuss the existence of documentation within a tool environment. Again, I feel that this really should have been mentioned in the chapter about documenting the architecture. I agree that the authors might be wary about mentioning tools that they have neither used nor analyzed closely. But a brief mention of those tools might actually be helpful. After all, those tools are used in <i>practice</i> so they must be beneficial in some way. And more readers will be familiar with it compared to the Dali tool that was discussed extensively.
</p>
<p>
I appreciate the book devoting a sidebar to software architecture in education. However, regardless of how many course you take in the university, software architecture cannot really be <i>taught</i>. You can be exposed to a series of techniques like what was done in the book but then again, you only really learn those techniques if you apply them in real life scenarios. And that's why I am a fan of classes that have lots of projects. There are many things that have been left out from this book (and judiciously so) for the sake of keeping it simple. As the title of the book hints, software architecture is something that exists right now as a form of practice, and there has not been any significant advances in trying to codify the theory of it.
</p>
<p>
This book is an attempt to describe software architecture. It's not easy task, and while the book could be improved, it does a pretty good job at it.
</p>
Why Java Applets did not make it2006-09-26T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/26/why-java-applets-did-not-make-it<p>
There was some discussion in class today about why Java Applets did not take off as much as Sun Microsystems had hoped. While I really do not know the answers, here are my thoughts.
<ul>
<li><b>No broadband</b>. 10 years ago broadband Internet was not as pervasive as it is today. So there wasn't really much need for highly interactive web pages. Imagine trying to download a few megabytes worth of data over a 28.8 kbps modem. Now imagine if every web page out there embedded an animating Java Applet. </li>
<li><b>Heavy JVM</b>. Even today, most people don't really like web pages embedded with Java applets. Windows does not really ship with the latest version of the Java Virtual Machine. Our university uses a classroom system called <a href="compass.uiuc.edu">Compass</a> based on Java. Almost nobody likes it. Enough said.</li>
<li><b>Browser compatibility</b>. Not every browser in the 1990s supported Java Applets. And most people did not bother installing a new one.</li>
<li><b>Java was not a buzzword (yet!)</b>. Most consumers did not know or care about Java. And pointy-haired bosses did not know enough to jump on the bandwagon.</li>
<li><b>No web designer knows the language</b>. This is very important. The first few Java Applets were really hideous compared to what was out there. I suspect that it was because web designers did not get on board yet. To most web designers. Java is a heavy language to learn just to make animated applets. You need to know about threads, classes, methods and objects. Even today, most web designers prefer the simplicity of HTML, CSS and some javascript.</li>
<li><b>It just looks weird</b>. The UI from the AWT looks weird on the browser. And that makes the entire web site feel clunky.</li>
</ul>
</p>
<p>
I am interested in hearing what other people think. Java Applets were something introduced in the wrong time period. Had it been introduced later when it was more mature, it might have appealed to more people. I guess it is a compromise between how early to enter the market and when to wait for maturity of your technology.
</p>
SAIP: J2EE and EJB2006-09-24T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/24/saip-j2ee-and-ejb<p>
Chapters 16 and 17 are case studies on systems that utilize Java and its technologies.
</p>
<p>
Chapter 16 was a description of the architecture of J2EE and how it achieved the quality attributes of its stakeholders for distributed computing. However, it was too detailed for me to appreciate since I have not used J2EE or EJB before. It did point out the robustness of J2EE and how Sun Microsystems invested a lot of time and money to make it handle most of the situations. One interesting aspect of this architecture was the design to support the older standard for distributed computing using CORBA. I found this to be a wise step on the part of Sun Microsystems; no one is likely to switch over everything to just use your company's new technology. It has to be a gradual shift that helps build confidence in the new product. It was because of neglecting this fact that the Air Traffic Control system we saw back in Chapter 6 did not take off.
</p>
<p>
Chapter 17 focused on using J2EE for wearable computing devices. These computing devices had unique requirements that had to be catered to. For instance, some of them had small screens, limited input abilities, low power consumption and limited processing power. Sometimes, these devices were so varied that entirely new input/output devices had to be designed from the ground up. While the company, Inmedius Corporation, decided to use J2EE, I am rather surprised that they did not turn to J2ME, the Java standard for mobile devices. J2ME seemed like a better choice for micro devices that had limited processing power since it offers a simpler API and smaller memory footprint.
</p>
<small>
Though I am no in favor of this, I suspect that an entire CS undergraduate degree can be taught entirely using Java and its technologies. Java has grown into a behemoth with components for distributed computing (J2EE and EJB), embedded systems (J2ME), web architecture (JSP) and it even has enough of the best practices of software engineering (object oriented design, aspect oriented design) thrown in. If this topic is your cup of tea, you might want to read <a href="http://www.joelonsoftware.com/articles/ThePerilsofJavaSchools.html">The Perils of Java Schools</a>.
</small>
SAIP:Building Systems from Off-the-shelf Components2006-09-24T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/24/saip-building-systems-from-off-the-shelf-components<p>
This is one of the more interesting chapters in the book. I am glad that the authors acknowledge that most start-ups do not build everything from scratch; it handicaps their market penetration.
</p>
<p><a href="http://www.secretgeek.net/howtobeaprogrammer.asp">How to be a programmer</a>:
</p>
<pre>
intellisense
<span name='c1' id='c1'>||</span>
<span name='c2' id='c2'>\/</span>
code <span name='c1' id='c1'>></span><span name='c2' id='c2'>></span><span name='c1' id='c1'>></span> compile <span name='c1' id='c1'>></span><span name='c2' id='c2'>></span><span name='c1' id='c1'>></span><span name='c2' id='c2'>></span><span name='c1' id='c1'>></span> run <span name='c2' id='c2'>></span><span name='c1' id='c1'>></span><span name='c2' id='c2'>></span><span name='c1' id='c1'>></span> success <span id='wink'>;</span>-)
<span name='c1' id='c1'>/\</span> <span name='c2' id='c2'>||</span> <span name='c1' id='c1'>||</span>
<span name='c2' id='c2'>^^</span> <span name='c1' id='c1'>\/</span> <span name='c2' id='c2'>\/</span>
<span name='c1' id='c1'>^^</span> errors errors
<span name='c2' id='c2'>^^</span> <span name='c2' id='c2'>\\</span> <span name='c1' id='c1'>//</span>
<span name='c1' id='c1'>^^</span> <span name='c1' id='c1'>\\</span> <span name='c2' id='c2'>//</span>
<span name='c2' id='c2'>^^</span> google
<span name='c1' id='c1'>^^</span> <span name='c2' id='c2'>||</span>
<span name='c2' id='c2'>\\</span> <span name='c1' id='c1'>\/</span>
<span name='c1' id='c1'>\<</span><span name='c2' id='c2'><</span><span name='c1' id='c1'><</span><span name='c2' id='c2'><</span><span name='c1' id='c1'><</span><span name='c2' id='c2'><</span><span name='c1' id='c1'><</span> cut N paste
</pre>
<small>Figure 1: What actually happens during development. Or why no one codes alone anymore.</small>
<p>
Using off-the-shelf components sounds really convenient. After all, why rewrite what you can get for free from open source projects or buy for a small fee from other companies? There are various <i>very real</i> reasons why you do not want to use off-the-shelf components all the time as the book points out.
</p>
<p>
Here are some real risks that have to be addressed before committing to any off-the-shelf component:
<ul>
<li>Maintenance: Whose job is it to maintain that component that you have just borrowed? Will you have to hire someone to maintain that component full-time? What if the company that owns that component goes bankrupt? </li>
<li>Cost: Will there hidden fees in the long run? For instance, annual license fees? Upgrade fees?</li>
<li>Quality: How good is that component? Is it safe, reliable and secure? How would you find out?</li>
<li>Compatibility: Does the component fit with the rest of your system? Or would you have to bend over backwards just to accommodate it?</li>
<li>Long term goals: Can I scale using the current component? In other words, is there a component that I can upgrade to when I need more features?</li>
</ul>
</p>
<p>
As the chapter proposes, the best way to mitigate some of those risk is to isolate the system from directly accessing the off-the-self component. Never access the component directly from code. Instead, use a proxy and provide more general APIs for accessing that component. That way, should you need to change that component (maybe to a different vendor) you will minimize the changes to your system.
</p>
<p>
All in all, though there are clear risks involved with using COTS product, it is unlikely that every company will go build their software from scratch. Even if they intend to establish their own product line architecture, they will seldom do it from scratch. It just takes too long. The time gained can be used to do more testing, or add new features, or come up with the next version of the software. In other words, the time saved is better spend focusing on other parts of the system.
</p>
<p>
I think this form of reuse is most apparent in the web. There is a lot to be gained from sharing your data with people. For instance, by making part your data accessible to other web companies, you can charge them a small fee and also increase traffic to your site. One example would be <a href="http://www.flickr.com">Flickr.com</a>, a photo sharing website. By giving stable APIs that enable people to query their database (not private data) a lot of website can integrate Flickr's services. Not only does this save the website from having to implement their own photo sharing service (who knows how long that will take. And who would actually use it?) but it also helps Flickr get more publicity. So it could be a win-win situation for both parties.
</p>
SAIP: The World Wide Web2006-09-20T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/20/saip-the-world-wide-web<p>
<i>Relationships to the Architecture Business Cycle</i>:
</p>
<blockquote>
By their nature, research organizations tend to generate projects from the bottom up more often than commercial organizations do, because they are dependent on the researchers' originality and creativity and allow far more freedom than is typical in a commercial organization.
</blockquote>
<p>
Freedom. That is the major difference that I have noticed among all the case studies presented so far. The WWW was created based on a series of requirements but none of them were really detailed yet. After all, this was the first time something like this has been done before so you have to rely on your creativity in coming up with new solutions. This makes the problem more interesting. And since this really wasn't a commercial effort, there weren't that many bureaucratic procedures to go through.
</p>
<p>
This has been the most interesting case study because it actually shows the evolution in the implementations of a working system. Moreover, since it is a case study on the WWW, it is easier to follow since most of us are already familiar with the terminology. But more importantly, it avoids the heavy discussion of all the stakeholders, quality attribute scenarios, tactics and all the other procedures that the book had mentioned so far.
</p>
SAIP: Software Product Lines2006-09-20T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/20/saip-software-product-lines<p>
As the books mentions, reuse is great for any business out there. It is much too costly to invest in a "one-trick-pony" kind of architecture that only fits the profile for one kind of product or serve. Reuse justifies the return on investment for a large project that can easily take years to complete and requires thousands of employees working on it. Reuse empowers commercial companies to roll out a new product line quickly by sharing similarities between previous products. And more importantly, reuse reduces the need to reinvent the wheel.
</p>
<p>
So it seems that in this chapter, we have introduced the notion of <i>reusability</i> as a desired quality attribute. The SEI defined reusability as: "the degree to which a software module or other work product can be used in more than one computing program or software system". If reusability is a desired quality attribute, it must also be incorporated as part of the ATAM when evaluating the software architecture. However this is always easy. In fact planning for reusability is <i>hard</i>.
</p>
<p>
Making things reusable takes a bit of prediction (which comes not only from experience but from observing the trend in the market and the technologies that might emerge in the future). I was reading the <a href="http://www.amazon.com/Pentium-Chronicles-Politics-Landmark-Practitioners/dp/0471736171">Pentium Chronicles</a> this summer and the chief architect, Bob Colwell, described the creation of the P6 architecture that would be the product line architecture for the Pentium 3, Xeon and even the Pentium 4. The product line architecture for a microprocessor has to be almost perfect to accommodate any future changes. It is <i>prohibitively</i> expensive to change the fabrication equipment so any chief architect worth his salt must design an architecture that can last a few years down the road. Not only that, this same architecture must work with several generations of chips. To achieve that, Colwell had to be smart on where he made his assumptions when he designed the architecture. Realizing that fabrication technology would continue to improve, Colwell made it a point that his architecture would be able to support many more transistors in the future and that would improve the performance of the chip. My point here: if you want reusability, you have to plan (and make predictions) for it early on.
</p>
<p>
If you are designing the architecture for the first time, there is very little you can do to help you deal with reuse. Knowing the future plans for the company might help so that you can anticipate on the components that are likely to change. So you can make those components more modular. But how would you know if people are actually going to be using your architecture? The best way I can think of is to "<a href="http://en.wikipedia.org/wiki/Eat_one's_own_dog_food">eat your own dog food</a>". Use your own architecture as you develop it. From there you can find the missing abstractions that need to be in for reusability. Or you can release your framework as an open source project and get other people to comment and contribute as well. This is exactly how <a href="http://www.rubyonrails.org/">Ruby on Rails</a> was created. It was initially developed as an internal framework for <a href="http://37signals.com/">37signals</a> who later released it for public use.
</p>
<p>
Also, when do you actually know if you are going to need it again? There is a principle in XP called YAGNI: You ain't gonna need it. So there is a conflicting case for making things reusable and also for keeping the design simple. If you have read my previous post on reuse, you would have also read the <a href="http://www.basilv.com/psd/blog/2006/the-reuse-trap-in-software-design">article</a> I linked to. In it, the author mentions how you should just code as normal and get things working first. Only then do you try to eliminate code duplication by refactoring. The more times you have to eliminate that duplication, the more likely that the code you are dealing with is a candidate for a framework or library so that you can make it more general for reuse in the future.
</p>
<p>
But another problem with reuse is getting people to actually use the same architecture. There are many architecture out there that are overly complex that most people do not need all the features. For instance, many developers are leaving the world of J2EE and EJB for more lightweight architectures that can get the job done. J2EE can definitely accomplish <i>everything</i> you need for your enterprise but because it is so general, there are so many components that you need to know first that it becomes a greater hurdle. What I am trying to say is that there is a limit to what people will be using your framework for. There is a threshold of "enough, that works for me". Going beyond that leads to complexity that defeats the purpose of reuse.
</p>
<p>
It is very hard to make an architecture programming language agnostic and still be deployable in a short amount of time. But if you realize early that you need your architecture to work with different programming languages, then you need to design another layer for it. This is exactly what the .NET platform has done. All programs compile to <a href="http://www.gotdotnet.com/team/clr/about_clr.aspx">CLR</a> so you can code in C#, C++, Ruby or Python and still be able to run that program using the .NET framework.
</p>
<p>
So, reuse has not pervaded the society of computer science because it is just hard to make it work right. There are various other reasons for this, but sometimes one has to ask: is reuse really that important? Not every system out there needs to be reusable. And not every part of a system needs to be reusable. By cutting down on reusability, we can actually create less complex systems.
<hr />
<p>
The subtitle for this chapter reads: "re-using architectural assets". It suddenly dawned to be that in this class, we have looked at the terms architecture, design and implementation. However we have not looked at frameworks. I have not come across a concrete definition for frameworks but from the diagram on p. 478 of the book it would appear that frameworks are a form of abstraction that falls below architecture.
</p>
<p><a href="http://www.theserverside.com/discussions/thread.tss?thread_id=39262">Diff b/w Framework and Architecture</a>:
</p>
<blockquote cite="http://www.theserverside.com/discussions/thread.tss?thread_id=39262" title="Diff b/w Framework and Architecture">
"<b>Architecture</b>: an idea about how to structure your app, possibly shared through natural language documentation and structured methods, like UML.
<br />
<br />
<b>Framework</b>: an implementation of an architecture that someone can use as the basis for a working application. "
</blockquote>
<p>
So loosely speaking, I have seen various frameworks being applied. In fact, in my Software Engineering II class we read a paper on that entitled: <a href="http://st-www.cs.uiuc.edu/users/johnson/cs329/2003/framework97.pdf">Components, Frameworks, Patterns</a> that discusses frameworks are reuse techniques for object-oriented systems. However, I have not really seen the idea of a software architecture actually being reused.
</p>
SAIP: The CBAM2006-09-19T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/19/saip-the-cbam<p>
The CBAM: Cost Benefit Analysis Method is an attempt to quantify the return on investment (ROI) for each architectural strategy. While not perfect, this number serves as a good comparison when faced with different options. In a true business sense, it is most logical to maximize the benefits and minimized the investments that have to be put in. The CBAM is useful because it takes into account what every software development effort must utilize: time and money. Time and money, or the lack of, are usually the factors that determine if a project is going to fail. We have seen examples of projects in our first two case study in this book that never took off because of their cost.
</p>
<img src="http://www.sei.cmu.edu/architecture/CBAM.gif" alt="CBAM from SEI"/>
<p>
The CBAM just like any other metric should exhibit the following qualities:
<ol>
<li>It must be repeatable. So, if the measurement is done by Alice the same result should be achievable by Bob as well. </li>
<li>It must be interpretable. There is no use for a measurement that is so vague that it does not produce any useful comparison or correlation. For instance, Bill Gates was <i>purported</i> to have said that "measuring lines of code is like measuring the weight of the airplane".</li>
<li>It must be easy to perform the measurements. We are using this metric to help us make decisions. If it takes too much resources to perform the metric, we might as well just pick the decision by random.</li>
<li>It must be applicable to more than one system. There is no point having a metric if you cannot use it for comparison on another system.</li>
</ol>
</p>
<p>
Fortunately the CBAM satisfies at least <i>one</i> of those qualities. It can serve as a point of comparison; meaning that people have deemed it to be more useful than a number on a page. I believe that it falls short on the other criteria.
</p>
<p>
The CBAM is not really repeatable. The scenarios that are taken into consideration for calculating the CBMA are voted on by the stakeholders. If we bring in different stakeholders, of if something just happened to a similar competing product, or if a butterfly flaps its wings in Tokyo, we are probably going to have a different vote. This is just my assumption. I have no idea how steadfast or fickle minded real stakeholders are. Moreover we are assigning a lot of responsibility on the stakeholders assuming that they are able to make informed decisions about certain requirements. Then again, if not the stakeholders, who else would we turn to for the answers?
</p>
<p>
The CBAM is not something that is easily conducted. It requires a lot of data as input. And this chapter seems to take it for granted that development groups that intend to use the CBAM need to do a thorough analysis like the ATAM. And while, the iterative nature of the CBAM suggests that it is more accurate, it undeniably means that it is going to take a long time.
</p>
<p>
The results from one CBAM do not necessarily scale to another system. The CBAM is heavily influenced by the stakeholders. And since the stakeholders for each software project is going to be different, it is improbable that any other system will have the same stakeholders under the same circumstances. Nonetheless, the figure for CBAM can serve as a rule of thumb for developers in another project. This is a good thing since it prevents developers from making potential mistakes based on their deviation from a previous project's CBAM measurements. Then again, it might also influence a team to be more reserved and unwilling to take risks in their own projects.
</p>
<p>
All in all, the CBAM is a good enough measure for estimating the cost and benefits before fully committing to something. In the worst case scenario, one can actually fool management with some numbers and the software development can still proceed as hoped. There are apparently no other attempts on anything better than the CBAM as one can conclude from the book's limited further reading list.
</p>
<p>
The best quote from the chapter that I can think of is:
</p>
<blockquote>
"Cost models (especially early in the life cycle) are <b>imperfect</b> [emphasis added] for a wide variety of reasons, but they are the <b>only</b> tools available to constrain requirements. As such, they are invaluable to the architect".
</blockquote>
Software Reuse2006-09-17T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/17/software-reuse<p><a href="http://softwareindustrialization.com/DisplayPost.aspx?PostID=82">The Industrialization of Software: Software Reuse: Enigma or SOP?</a>:
</p>
<blockquote cite="http://softwareindustrialization.com/DisplayPost.aspx?PostID=82" title="The Industrialization of Software: Software Reuse: Enigma or SOP?"> "So how do we implement reuse in our organization? In a nutshell, we Google it. "
</blockquote>
<p>(Via <a href="http://joel.reddit.com/search?q=reuse">joel.reddit.com</a>.)</p>
<p>
Chapter 14 of SAIP talks about reuse. I have not read the chapter itself yet, but I came across these articles on reuse in the past few months that I found rather interesting. If you are interested, you might want to read them before reading Chapter 14 just to get some idea of what other people are thinking.
</p>
<ul>
<li><a href="http://www.basilv.com/psd/blog/2006/the-reuse-trap-in-software-design">The Reuse Trap in Software Design</a></li>
<li><a href="http://www.loudthinking.com/arc/000557.html">Reuse is Vastly Overrated</a></li>
<li><a href="http://butunclebob.com/ArticleS.DavidChelimsky.DiscoveringReuse">Discovering Reuse</a></li>
</ul>
SAIP: The ATAM [Evaluating architecture]2006-09-17T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/17/saip-the-atam-evaluating-architecture<p>
Describing a process dryly is the worst way to explain something. And this chapter has done precisely that: describe a process in the most mechanical way possible. I am curious how someone else can explain this topic interestingly.
</p>
<p>
The ATAM: Architecture Tradeoff Analysis Method is one way to evaluate a system's architecture. To perform an ATAM, a group consisting of representatives from the evaluation team, project decision makers and architectural stakeholders convene and discuss the architecture in mind. This group can easily be total 20 members or more. Just like any formal review, roles are assigned to each member. There are discussion leaders, scribes, timekeepers and questioners. The review itself is divided into four phases: partnership and preparation, evaluation, further evaluation, and follow-up. The entire review can take a few weeks. At the end of it all, a final written report (hmmm... a heavyweight tome for <i>everyone</i> to read) is produced that addresses the issues discussed in the review. Also, the authors claim, the final report is not the only important outcome of the ATAM; it also includes a "palpable sense of community" [sic] among the participants.
</p>
<p>
The authors of this chapter claim that the ATAM is a useful method for evaluating the soundness of an architecture before any product has been produced. The ATAM is neither a code evaluation or an actual system testing. Instead it is meant as a "questioning" method of evaluating the architecture. No concrete measurements can be done because the system has not been built yet. So whatever feedback you get depends on the quality of your reviewers. The more experience they have, the better they can find loopholes in the architecture. What I find really interesting is that these people are so good that they do not even need a real system to begin talking about for a couple of weeks!
</p>
<p>
I guess that the ATAM has not been applied to large scale open source projects (the chapter only mentions cooperate cases). While effective, the ATAM requires a lot of involvement from all the stakeholders. I find it hard to believe that a group of more than 10 people can actually discuss all the important issues of the architecture within one meeting. I find it even harder to believe that they can retain useful information since the evaluations goes on for weeks. The ATAM also requires a lot of money. There is a need for external developers to come and evaluate the system. The authors argue that the cost of bringing in the outside developers is easily offset by the cost of having to fix an architectural problem later in the development phase.
</p>
<p>
While I do not doubt the effectiveness of the ATAM in helping identify missing areas in the architecture, the ATAM is not the most practical for smaller developing companies. I am really interested in seeing what smaller development teams use (if they use any) to evaluate their architecture. Smaller companies do not produce inferior architectures. So what are they using to ensure that quality? Is it true that <a href="http://www.catb.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/ar01s04.html">"with enough eyeballs, all bugs are shallow"</a>? Would a more agile method be available for evaluating software architecture? We should look at those in this class as well. There is an <a href="http://csdl2.computer.org/dl/mags/so/2006/02/s2047.htm">article</a> [subscription required] on IEEE Software that sorts of discuss this.
</p>
<p>
And the ATAM is long. Really long. It can be taxing on all the participants. And quite frankly, I find it hard to believe that the software engineers will be willing to sit through this. There is an interesting article <a href="http://education.guardian.co.uk/higher/research/improbable/story/0,11109,1687547,00.html">here</a> that discusses the negative effects of prolonged meetings. Also, I find it hard that software engineers and managers can actually sit down and have a decent conversation without going at each other's throat.
</p>
<p>
In conclusion, I am skeptical of this approach to evaluating software. I like the general idea of convening people to discuss about the architecture but I do not like the idea that entire weeks are spent just talking about a system without having a concrete implementation to talk about. Many other issues manifest themselves during real development and production. And unless you have tons of experience, a crystal ball, or some divine imp beside you, your guesses are going to be wrong most of the time.
</p>
SAIP: Reconstructing Software Architecture2006-09-14T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/14/saip-reconstructing-software-architecture<p>
My previous experience with reconstructing an architecture was part of the reading for the Software Engineering II class where we read <a href="http://www.iam.unibe.ch/~famoos/handbook/">FAMOOS Object-Oriented Reengineering Handbook</a>. This handbook mentions several tools at the end but when I read it, I was focusing more on the patterns for reverse engineering. There is a subtle difference between being able to extract the architecture and actually understanding the architecture. The latter is definitely more important to the developer who might need to make changes, but the former might serve as a good starting point for an approach to the system.
</p>
<p>
Why is reconstructing architecture important? There are several good reasons listed in the chapter itself. My favorite is using reconstruction as a starting point for re-engineering the system to a new desired architecture. It is hard to make modifications to the architecture if one is not even aware of what it is. Every little glimpse at the original architecture will help one make more informed decisions on how to proceed to make modifications. One does not need to extract <i>everything</i>; just enough to get the job done.
</p>
<p>
This chapter presents four steps for software architecture reconstruction:
<ul>
<li>Information extraction <br />
Having computer tools might help in this part of the process. It would be best to also incorporate some of the patterns in the FAMOOS handbook. For instance, <i>Read All the Code in One Hour</i> will definitely help instead of just blindly trying to rely on a tool to extract information. <i>Skim the Documentation</i> would also help if there was a previous documentation, no matter how shabby it might be. At this stage, the purpose is to get information, not really forming any significant relations between those pieces of information. The chapter suggests using tools such as parsers, abstract syntax trees, lexical analyzers. It might be hard to extract those information and form relations between them the architecture in mind uses different programming languages. Also, static analysis is not that useful for dynamic languages. It would be best if we could run the system with the data model as recommended by the <i>Check the Database</i> pattern.
</li>
<li>Database construction <br />
The previously extracted information is converted into some standard form and stored in a database. This chapter uses a relational database and stores the information in a schema that <i>Dali</i> can understand. <i>Dali</i> is a toolset for architecture reconstruction developed by the Software Engineering Institute.
</li>
<li>View fusion <br />
By manipulating the information in the database, we hope to extract important relationships between the elements. There is some heuristic at work here on what will be useful and what is not. A metrics tool can be used to identify the <i>hot areas</i>. It is in this stage that the tools come into the most use. A tool such as <i>Dali</i> would help if the user provides it with the right SQL statements to manipulate the database.
</li>
<li>Reconstruction <br />
In this stage, the various views and representations of the information extracted can be visualize. This is "where the main work of building abstractions and various representations of the data to generate an architecture representation take place".
</li>
</ul>
</p>
<p>
What I do not like about this chapter is that it focuses too much on Dali such that the important details are lost. I am not familiar with Dali so describing how Dali does it not going to be helpful to me. It would be more useful it the process was described using a patterns approach just as the FAMOOS handbook did.
</p>
<p>
Also, it seems to be that the final artifact produced is actually pretty <i>useless</i>. A series of graphs that show the relation between different entities. What is the use of that? The more useful lesson is actually learned by the person who is going through each of the 4 steps above. That person has to make hypotheses and inference about the architecture. At the end of it all, he comes up with a better understanding of the system from his interaction with it. So the final artifacts actually mean something to him. But what about someone who did not go through the 4-stage process? How would the final artifact help him? It would not. In fact, it is no better than staring at the system for the first time. So, I believe the process of actually reconstructing the architecture is more useful in helping developers understand the system than just generating the final architecture representation of the system.
</p>
<p>
Actually this chapter is pretty much based on the paper <a href="http://www.sei.cmu.edu/publications/documents/01.reports/01tr026.html"><i>Architecture Reconstruction Guidelines</i></a>. People curious about what architecture reconstruction is all about can go read that if they do not have the SAIP book.
</p>
SAIP: Documenting Software Architecture2006-09-13T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/13/saip-documenting-software-architecture<p>
I believe that the chapter on documentation does serve as a good point of reference but is generally lacking. I do not expect myself to be able to actually document a system just by reading that architecture. While the introduction to UML at the end of the chapter is probably good as a refresher, it is not going to be detailed enough to help introduce developers. I was particularly frustrated that I could not actually see the essence of that chapter. It seems like a hodgepodge of every mix on documenting software architecture.
</p>
<p>
I do like this one sentence in the chapter though:
</p>
<blockquote>
Documenting an architecture is a matter of documenting the relevant views and then adding documentation that applies to more than one view.
</blockquote>
<p>
Also, I find the use documentation as a training vehicle for new developers really interesting. As the author of the chapter said, most view the documentation as the end of the development cycle where everything has been completed. However, it is also a starting point for a new development team that might take over the project in the future. So, it has to be written to take account the needs of the new development team.
</p>
<small>
A slight distraction on making mundane manuals more interesting can be found here: <a href="http://headrush.typepad.com/creating_passionate_users/2006/08/why_marketing_s.html">Why Marketing Should Make the User Manuals</a>.
</small>
<p>
In a previous project that I worked on, a friend of mine actually advocated incorporating test cases into the documentation so that developers can quickly look at the test cases to see how to utilize each small function, object or method. Also, the actual inputs for the test cases convey much more information than just describing it in the function header.
</p>
<p>
I would also like to think of documentation as something that evolves as the project progresses. Outdated documentation can be the bane of many programmers. And there is really nothing worse than having to regenerate documentation by hand. Since there are tools for nightly builds, I am sure that automatically rebuilding the documentation can be done as well. Documentation, just like testing, should not be left until the end. Like testing, there are benefits to having documentation available as early as possible.
</p>
<p>
Therefore, I am curious why they did not mention anything about documentation tools such as <a href="http://www.stack.nl/~dimitri/doxygen/index.html">Doxygen</a>, <a href="http://java.sun.com/j2se/javadoc/">JavaDoc</a>, <a href="http://rubydoc.sourceforge.net/">RubyDoc</a>, etc. True, these tools cannot generate all the views that are useful to all the stakeholders but these tools are capable of at least generating documentation that will be useful for the programmers working on the project. Some of them are also able to generate simple class diagrams. For a list of the graphs and diagrams that Doxygen can generate, please click <a href="http://www.stack.nl/~dimitri/doxygen/diagrams.html">here</a>.
</p>
<p>
Anyway, my conclusion after reading the book so far, is that even though this book was last updated in 2003, there are a lot of modern tools and paradigm that are missing from the discussion. As <a href="http://wiki.cs.uiuc.edu/cs527/Paul+Adamczyk">Paul Adamczyk</a> mentioned in the previous class, this book does not even mention refactoring even though that is a pretty common term with development teams now. Nope, it's not even mentioned in the index or in the further reading section. Seems to show that this book is rather traditional and, for lack of better words, outdated and possibly obsolete.
</p>
<p>
There is another book entitled <a href="http://www.amazon.com/Documenting-Software-Architectures-Views-Beyond/dp/0201703726/sr=8-1/qid=1158171779/ref=pd_bbs_1/102-0730306-6825767?ie=UTF8&s=books"><i>Documenting Software Architecture</i> </a>that is optional for this class. I believe that readers will be able to gain much more from reading that book. The same authors who wrote this chapter in the SAIP book also co-authored the <i>Documenting Software Architecture</i> book.
</p>
SAIP: Designing the Architecture2006-09-12T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/12/saip-designing-the-architecture<p>
This chapter talks about about the Evolutionary Delivery Life Cycle (EDLC). This cycle is generally a good thing when compared to the more traditional <a href="http://www.waterfall2006.com/">waterfall</a> <a href="http://en.wikipedia.org/wiki/Waterfall_model">model</a>. An iterative process means that errors and inadequacies in the requirements and architecture could be fixed less expensively as the software develops.
</p>
<p>
I argue that EDLC is a lot like XP but there are some differences. Both of them are definitely iterative. EDLC uses <i>architectural drivers</i> - business goals that have the highest impact on the architecture in order to help shape the architecture. On the other hand, XP uses a set of user stories organized by priority for each iteration.
</p>
<p>
Both EDLC and XP emphasize the importance of the software concept. EDLC calls it the software concept but it is manifested in XP as the metaphor. A clear metaphor promotes a strong architectural vision.
</p>
<p>
XP is agile but is also demands that the team members follow certain <a href="http://www.xpexchange.net/english/intro/practices.html">practices</a>. These 12 practices are there to alleviate the risks involved with software development. The <i>small releases</i> practice will definitely help a team elicit customer feedback as quickly as possible. So, while you can enforce these practices in EDLC, those disciplined practices are not really part of EDLC. Also, XP will work fine for small development groups but EDLC is more scalable to larger development groups.
</p>
<p>
Of particular interest in this chapter is the term <i>Attribute-Drive Design</i>. "Attribute-Driven Design (ADD) is an approach to defining a software architecture that bases the decomposition process on the quality attributes the software has to fulfill". ADD seems like a safe bet on the road to producing usable software because it focuses on satisfying requirements at each stage. It makes use of the scenarios (<a href="/articles/2006/09/06/saip-understanding-quality-attributes">SAIP: Understanding Quality Attributes</a>) that are defined for the system as factors for decomposition.
</p>
<p>
ADD first tries to decompose the entire system into smaller modules. These smaller modules are then refined again. The architectural pattern that can be used as the building point for these smaller modules is decided by the tactics that are used. Generally, one team will focus their efforts on one module.
</p>
<p>
Object-oriented design also makes use of decomposition into modules. A heuristic used for decomposition in ODD is to come up with some use cases for the system. Use cases are similar to quality scenarios except that use cases do not try to make guarantees about qualities; they show how the user would interact with the system. From the use cases the architect can pick out the nouns and turn them into class names. As Prof. Johnson mentions in class, for some domains, the nouns have been standardized and one can just make use of them. Not only is this faster, it also avoid confusion in case you start using non-standard terminology.
</p>
<p>
Another method for ODD makes use of CRC cards. CRC stands for "Classes-Responsibilities-Collaborators". CRC cards, as compared to use cases, have the advantage that very little has to be written down before you can start talking about the design of the system. However, since very little is written down, unless someone jots down the design, it is pretty easy to forget what was discussed between different CRC meetings.
</p>
<p>
Both ADD and OOD help the architect decompose the system into smaller modules. ADD is operating at a higher level since it does not deal with classes and objects yet but just focuses on the modules (might be compositions of objects). The advantage of ADD is that it is not limited to object-oriented programming. In general, developers can still benefit from ADD even when using a non-object-oriented system (not every system out there is suitable for OOD).
</p>
<p>
Moreover, when dealing with OOD, it is not as simple to actually identify the key requirements of the system. Since requirements are not explicitly spelled out, OOD might missed out on the more subtle points. For instance, a proliferation of similar objects might be easier to design but puts a strain on performance<sup><a href="#fn1">1</a></sup>. But because we using an iterative approach this might not be too big a deal since it can be corrected in the next iteration.
</p>
<p>
In my opinion, ADD is more elaborate. It involves capturing a lot of the functional and quality assurance requirements before it can proceed effectively. I think ADD is more suitable when the software development requires the participation of more people in a company. The steps are more defined and the methodology can be easily understood by non-developers. OOD, on the other hand, is more suitable for discussions among developers since developers are familiar with classes and objects and can relate in those terms. It would be hard, if not impossible, to have a CRC session with management.
</p>
<p id="fn1"><sup>1</sup>One way to solve this would be to use the <a href="http://en.wikipedia.org/wiki/Flyweight_pattern">Flyweight</a> design pattern.</p>
SAIP: Air Traffic Control2006-09-10T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/10/saip-air-traffic-control<p>
<em>Air Traffic Control</em>
</p>
<blockquote>
Although this system was never put into operation because of budgetary constraints, it was implemented and demonstrated that the system could meets its quality goals.
</blockquote>
<p>
The past two case studies in this book have never been utilized commercially. This is particularly disturbing since this <i>is</i> a book on software architecture. What use is an architecture if the project never takes off? Would we not consider cost as a non-functional requirement that has to be taken care off also? What's the use of the <i>best</i> architecture out there if no one is using it? How can it be the <i>best</i> if people are still using other products that have various shortcomings?
</p>
<p>
Of course, there is plenty of driving forces that might compel people to use the older software instead of a newer one. This resistance to change itself is an interesting topic to analyze when developing software. As we have seen, one reason will probably be because of cost. The second reason, as presented in the book, is ease of upgrading. The system proposed in the chapter required a more elaborate upgrade and installation path that could not be done in isolation of the other older systems (even though the interoperability software quality attribute was part of the stakeholders's requirements). Another reason that I can think of is backwards compatibility. Data stored in the old format readable with the hackneyed computer is still better than no data on a super fast computer.
</p>
<p>
It would be good if we could address those software quality attributes in this book as well.
</p>
<p>
Undeniably, there is plenty to learn from the two case studies presented in the book so far. I just find it very ironic that we are reading a book on software architecture in <i>practice</i> and the case studies in this book are not being practiced at all.
</p>
<p>
Of course, future cases studies will probably be successful projects. The one in chapter 8 seems successful.
</p>
Usability for non-programmers2006-09-07T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/07/usability-for-non-programmers<small>This might seem tangential to the discussion on architecture, but I think it is pretty interesting to see where this is going.</small>
<p><a href="http://www.joelonsoftware.com/uibook/chapters/fog0000000058.html">User Interface Design for Programmers - Chapter 2</a>:
</p>
<blockquote cite="http://www.joelonsoftware.com/uibook/chapters/fog0000000058.html" title="User Interface Design for Programmers - Chapter 2"> "This turns out to be relatively easy. Just ask them! Pick five random people in your office, or friends, or family, and tell them what your program does in general terms ('it's a program for making web pages'). Then describe the situation: 'You've got a web page that you're working on, and a picture file named Picture.JPG. You insert the picture in your web page.' Then ask them some questions to try and guess their user model. 'Where did the picture go? If you delete Picture.JPG, will the web page still be able to show the picture?'"
</blockquote>
<p>
SAIP states that "usability is concerned with how easy it is for the user to accomplish a desired task and the kind of user support that they system provides".
</p>
<p>
In today's class, we talked about how having models help us discover that the use really needs. I agree that having a model of the system makes it easier for us to anticipate the stimulus and their sources. Anticipating those stimulus gives us a better way to handle them. Security specialists have their threat models and performance experts have their workload model. What do usability experts have as a model?
</p>
<p>
For most software out there that emphasize usability (yeah, Linux is all <i>about</i> usability. really.) the most important layer would be the user interface layer. And yet, this is the layer that most people have trouble with. It is really hard getting the perfect user interface. Even so, Joel Spolsky (quoted above) has made it slightly easier for us to work with this problem - by asking. </p>
<p>
There can <i>never</i> never be a perfect user interface. And <i>perfect</i> is not what we really should be aiming for. What we can do, is try to predict the <b>user model</b> to improve usability. Spolsky suggests just thinking about how the user will like to use it. And, even better, just do a rough survey with your co-workers on how they expect the system to behave. You, as the programmer of the system might be the worst person to design the user interface. Why? Because you have implicit knowledge of the underlying system and you are trying to structure everything in terms of that implicit knowledge. For instance, you know that you are representing the table of contents as a tree internally. So what do you do? You add a button that says: "Add node to table of contents". And your user would be going what??
</p>
<p>
What I suggest is that programmers just try to think in terms of the normal everyday user.</p>
<img src="http://flyingmachinestudios.com/assets/2006/9/6/chuck-interaction-models.jpg" />
<br />
<small>Another way to describe the user model would be the interaction model as mentioned at <a href="http://flyingmachinestudios.com/assets/2006/9/6/chuck-interaction-models.jpg">this</a> website.</small>
<p>
Better yet, if programmers really suck at designing user interface, why not just design your software so that there is a clear separation between the underlying core functionality and the user interface layer? The Model-View-Controller compound pattern comes to mind. With a good enough user interface palette, you can let your "human computer interaction" people go crazy with the positioning of every button and toolbar down to the last pixel. (In case you missed it, this is where the architecture part comes in). So, in short, I am advocating an architecture that separates the user interface from the underlying structure as much as possible. That way, you can have "non-programmers" come in and make a better user model.
</p>
<p>
You might also find this <a href="http://blog.vazexqi.com/articles/2004/12/29/visceral-behavioral-and-reflective">article</a> I wrote about design a couple of years back interesting.
</p>
SAIP: Understanding Quality Attributes2006-09-06T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/06/saip-understanding-quality-attributes<p>
Functionality and quality attributes are orthogonal. I like this phrase. As the chapter claims, this is a bold statement but at the same time it makes sense. Ensuring that software exhibits all the required functionality does not guarantee that it has any of the quality attributes. It may very well work as intended but it might be abysmally slow and just plain hard to maintain in the future. In fact, it can a big ball of mud with everything built-in.
</p>
<p>
So, why is it important to understand quality attributes? Software quality attributes help shape the system. As mentioned above, it would be trivial to construct software that meets all the functionalities but fail miserably. No one might want to use it, developers might detest maintaining it and it might probably is not reliable. Understanding quality attributes help shape the architecture of the system by giving priorities to the qualities that must be achieved. By emphasizing the importance of quality from day one, the architecture will be more robust and appeal to developers and stakeholders alike.
</p>
<p>
This chapter presents three kinds of quality attributes that a "good" system needs to take into consideration: qualities of the system, business qualities and the quality of the architecture itself. The three kinds of qualities influence one another and it is impossible to fulfill one set while completely ignoring the other.
</p>
<p>
Maintainability, availability and modifiability are some of the qualities of the systems that we usually consider. Qualities of the system are hard to qualify. For instance, maintainability as defined by the Software Engineering Institute (SEI) is: "the ease with which a software system or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment". This definition alone does not help developers check if the system at hand possesses that quality. What this chapter proposes is to use quality attribute scenarios to help characterize the qualities that we are seeking. These scenarios will be real cases that the system will encounter and has to react to.
</p>
<p>
Here is a picture to illustrate what a scenario looks like. The terms used are pretty broad and it is easy to have an idea what they mean.</p>
<a href="http://www.flickr.com/photos/vazexqi/236265392/" title="Photo Sharing"><img src="http://static.flickr.com/89/236265392_e73588be38_o.png" width="576" height="198" alt="quality_attribute.png" /></a>
<br />
<small>Figure 1</small>
<p>
Basically, we can model a scenario for each quality attribute that we are interested in by identifying the source, the stimulus to the system, which part of the system is affect, the response from the system and the by measuring the response of the system. And we keep a copy of all these models at hand.
</p>
<p>
While scenarios definitely do not capture everything, having them around gives concrete instance of situations that the system has to handle gracefully. With enough scenarios, we will have a pretty good idea of the requirements that our architecture must fulfill. Consider what would happen instead if we did not have scenarios to model the qualities that we seek. It would be impossible to determine what we mean when we say that we want ensure "portability" in our system. How much "portability" do we mean? What kind of "portability" are we trying to achieve?</p>
<p>
However, one point this chapter does not address is how we would make use of those scenarios effectively to ensure that our architecture enforces those qualities. Now that we have all these scenarios, what is the best way to apply them? Should we consider by the attributes they represent or should we tackle them as a whole?
</p>
<p>
A more interesting kind of quality attribute presented in this chapter is system quality attributes that deal with the architecture itself. <i>Conceptual integrity</i> is mentioned as a desired quality for the architecture. The architecture should do similar things in similar ways. It reminds me of the <i>metaphor</i> that developers strive to achieve in the Extreme Programming software process. Good architectures are supposed to possess this conceptual integrity since it means that everything is operating under the guise of the principle of least surprise: there is a proper place for everything and there is a intuitive way of doing something.
</p>
SAIP: Achieving Qualities2006-09-06T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/06/saip-achieving-qualities<p>
This chapter introduces yet another term: tactics. A single tactic is a design decision that matches a stimulus with a response. There might be several tactics to react to a particular stimulus. A collection of tactics will form an <i>architectural strategy</i> which is not examined in this chapter. While designing the architecture, one has to choose a set of tactics to implement.
</p>
<p>
The bulk of this chapter gives a rather comprehensive list of tactics on how to provide better availability, modifiability, performance, security, testability and usability. While the tactics presented here certainly work, I find it hard to actually get anything meaningful from just reading the list. Though the tactics have been categorized for each quality attribute, it is still hard to relate to their use in a system. I would prefer separate chapters that go more in-depth with certain tactics and discuss the advantages and disadvantages of those tactics. However this chapter does serve as a reference point for tactics when I need to meet certain quality attributes.
</p>
<p>
This chapter also describes how the tactics are used. Based on the choice of tactics to realize, an architect is presented with a selection of design patterns that can be used. There are trade-offs that have to be made since choosing one design pattern might preclude certain tactics by making those tactics extremely complicated to implement. What I would like to see in this book is a catalogue of all the tactics mention and the design patterns that they can map to since the term tactics seem to be an term used exclusively in this book and existing design patterns book might not be organized around tactics. That detail has been left out in this chapter.
</p>
<p>
Using tactics to guide the choice of design patterns and ultimate the architectural style is a sensible thing to do. By capturing the quality attributes with scenarios and then trying to solve those scenarios using well-known tactics, we can come up with a good design that is not gratuitously chosen. Also, by fulfilling the quality requirements, the stakeholders are more likely to be pleased (but not necessarily).
</p>
SAIP: What is Software Architecture?2006-09-04T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/04/saip-what-is-software-architecture<p>
<em>What Software Architecture Is and What It Isn't</em>
</p>
<blockquote>
The software architecture of a program or computing system is the structure of structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them.
</blockquote>
<p>
The definition above fits well with the other definitions that we have seen so far especially the one from Garlan and Shaw's <a href="http://www.inf.ed.ac.uk/teaching/courses/seoc1/2005_2006/resources/intro_softarch.pdf"><em>Introduction to Software Architecure</em></a>. However, there are some subtle yet distinctive points about this particular definitions:
<ul>
<li>Systems can consists of more than one structure. <br />
For large systems, there can be many structures. And not one of them is <em>the</em> software architecture for the system. As we have seen in the <a href="http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/mags/so/&toc=comp/mags/so/1995/06/s6toc.xml&DOI=10.1109/52.469759">4+1</a> paper by Krutchen, there can be indeed different views for the same architecture with each view contributing toward a different structure.
</li>
<li>All computing systems have a software architecture <br />
Even if the architecture is a <a href="http://en.wikipedia.org/wiki/Big_ball_of_mud">Big Ball of Mud</a>. It might not be describable, but an architecture definitely exists. And the definition does not discriminate against bad or good architecture.
</li>
</ul>
</p>
<p>
Three useful terms are defined in the book that will be pivotal to the discussion of software architecture:
<ul>
<li>An <b>architectural pattern</b> is a description of element and relation types together with a set of constraints on how they may be used. <br />
It's interesting to note that this chapter considers client-server as an architectural <b>pattern</b> and not a software architecture. In this book, the term architectural pattern is synonymous with architectural style. I suppose that the patterns describe in <i><a href="http://www.amazon.com/exec/obidos/tg/detail/-/0321127420?v=glance">Patterns of Enterprise Application Architecture</a></i> would fit this category. The original <a href="http://www.amazon.com/exec/obidos/tg/detail/-/0201633612?v=glance">23 design patterns</a> from the Gang of Four might fit in here as well though some of them are pretty low-level and does not address the system as a whole.
</li>
<li>A <b>reference model</b> is a division of functionality together with data flow between the pieces <br />
A reference model is an abstraction of how the system should function. For me, I think of it as a metaphor for the system. For instance, if I want to represent an XML document, I might conceptually think of it as a tree since all the nodes can be nested within one another. So to actually use an XML document, I will need a reader, writer and some internal representation. That is the conceptual idea I have of a an application that uses XML. I do not have a definite design or implementation for the reader, writer or the internal representation yet.
</li>
<li>A <b>reference architecture</b> is a reference model mapped onto software elements and the data flows between them. <br />
It simpler terms, it tries to connect the architectural pattern and the reference model. Once you have that mapping (the reference architecture) you are in a better position to actually solve the problem at hand. For instance, to solve a database problem, I can relate my database schema to the<a href="http://www.martinfowler.com/eaaCatalog/activeRecord.html"> Active Record pattern</a>.
</li>
</ul>
</p>
<p>
Here is a picture to make things clearer.
</p>
<a href="http://www.flickr.com/photos/vazexqi/234055015/" title="Photo Sharing"><img src="http://static.flickr.com/98/234055015_9eceb25647_o.png" width="518" height="184" alt="Relationships.png" /></a>
<br />
<small>Figure 1</small>
<p>
So why do we need software architecture? Krutchen already discuss how it gives a representation of the state of the software to the stakeholders. And in Chapter 1, we saw how an architecture helps stakeholders make decisions about the future stage of the software development.
</p>
<p>
Furthermore, software architecture is also important because it's the earlier manifestation of the design decisions that go into the software. With this software architecture, developers have a better sense on how to organize the structure of software, how to handle changes and how to proceed in the next iteration. Though at the early stage, this manifestation is most probably incomplete and even inadequate, the software architecture serves as an early model for which to base future cost and schedule estimation.
</p>
<p>
Another importance of software architecture as given by the book is the fact that it is a <em>transferable abstraction of the system</em>. While this is definitely possible, I would argue that architectural patterns (defined above) are more transferable than software architecture because in a software architecture the parts are more detailed and concrete already. As can be seen from figure 1 above, software architecture is the product of the reference architecture stage. As you recall, the reference architecture stage provides the mapping between the abstract architectural patterns and the reference model. Therefore, I believe that some level of abstraction has been lost. Instead, I think that the software architecture might provide a good example of the use of the architectural pattern. But the architectural pattern is the high-level description that should be shared amongst developers.
</p>
<p>
This chapter also proposes some alternative views for representing software architecture in contrast to Kruchen's 4+1 view. These forms of representations become increasingly important as systems grow more complex. It is no longer possible to represent the entire architecture with just one diagram. By using multiple diagrams, each structure of the software architecture becomes clearer. It becomes easier to identify the important elements for each structure.
</p>
<small>
This chapter credits the works of David Parnas as one of the most influential figures in defining what software architecture is. I remember reading one of his papers, <a href="http://www.acm.org/classics/may96/"><i>On the Criteria To Be Used in Decomposing Systems into Modules</i></a> during my CS427: Software Engineering I class. Apparently this was the seminal paper that proposed breaking the system into modules to promote reuse.
</small>
SAIP: The Architecture Business Cycle2006-09-04T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/09/04/saip-the-architecture-business-cycle<p>
<small>
The next reading assignments come from the book Software Architecture in Practice (SAIP). Whenever future posts are based on this book, I shall prepend the acronym SAIP to the title.
</small>
</p>
<p>
<em>The Swedish Ship Vasa (p. 4 of SAIP):</em>
</p>
<blockquote >
Faced with an impossible task, Hybertsson had the good sense to die about a year before the ship was finished.
</blockquote>
<p>
One interesting (maybe even fairly obvious) point is that project requirements alone do not determine the architecture of the system. Think of it. Given the same set of requirements, no two developers would go out and develop with the same architecture in mind. And yet many software projects fail simply because all focus has been placed solely on the requirements. By placing all faith on the requirements alone, all other aspects tend to be neglected.
</p>
<p>
Instead, this book proposes that a software architecture is the result of technical, business and social influences. Moreover, just as these three factors influence the software architecture, the software architecture also affects the technical, business and social environments. This chapter of the book has not yet defined architecture, but we may just use the definitions that we have covered in my previous posts.
</p>
<p>
For instance, the architect has to take into consideration the resources that his organization can provide. The organization might have a limited pool of programmers to spare for the project. Therefore, the architect might have to choose a particular tool, programming language and architecture that the programmers are familiar with.
</p>
<p>
Moreover, the experience of the architect himself also plays a great role in determining the architecture of the system. If the architect suffered from a bad experience with some particular approach, he would be wary of using that approach on a future project.
</p>
<p>
As aforementioned, not only does the technical, business and social influences affect the decisions of the architect but the architect will also affect those factors. An example of this would be when the architect produces a prototype for evaluation. With a prototype, the customer would be able to evaluate the feasibility of this piece of software. The customer might then decide to add or remove features to suit their own needs. Adding or removing features has an effect on how the architect should proceed for the next stage.
</p>
<p>
The book itself has a lot more examples of the interaction between the stakeholders and the architect. Most of these might seem like common sense but the authors have done a good job by actually expressing the more general cases on paper.
</p>
<p>
Toward the end of the chapter, the book provides some rules of thumbs on what is a <em>good</em> architecturea. A more mature methodology for evaluation would be based on the Tradeoff Analysis Method that is presented in chapter 12 of the book.
</p>
<p>
One would think that the interaction between software architects and the stakeholders is something that people in the software business should know about. And yet, most software projects do not go as plan simply because the customers throws down a set of requirements for the developers, with some budget and a timeline and expects the developers to produce <em>something</em> by the deadline. Great architecture seldom (if ever) come out as a sequential series of steps. Instead it requires iterative steps that refine trouble spots in the software architecture as both the requirements, stakeholders and architect themselves mature in the life of the software.
</p>
<p>
Hopefully, with more agile development methods such as XP, there is a trend toward making software development more iterative.
</p>
Architecture, design and implementation2006-08-31T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/08/31/architecture-design-and-implementation<p>There is a difference between architecture, design and implementation. Without knowing any better, when asked what architecture means for me during the first class session, I shouted out design. Obviously, I now realize that architecture is more than just design and that the two terms are not synonymous.</p>
<p>In <a href="http://eden-study.org/articles/2003/icse03.pdf"><em>Architecture, Design, Implementation</em></a> the authors hope to disambiguate the different terminologies so that communication amongst developers might be simplified. And also, I think, to curb the haphazard abuse of those terms for marketing purposes. For instance, there is no need to market <span class="caps">UML</span> as an architectural design language when it completely does not fit the profile.</p>
<p>To get the most out of this paper, one must understand the table and definition that follows:</p>
<blockquote>
<table style="border:1px solid black;">
<tr>
<td >architecture</td>
<td>intensional</td>
<td>non-local</td>
</tr>
<tr>
<td>design</td>
<td>intensional</td>
<td>local</td>
</tr>
<tr>
<td>implementation</td>
<td>extensional</td>
<td>local</td>
</tr>
</table>
<p><b>Intensional</b> (vs. extensional) specifications are “abstract” in the sense that they can be formally characterized by the use of logic variables that range over an unbounded domain.</p>
<p><b>Non-local</b> (vs. local) specifications are “abstract” in the sense that they pervade <em>all</em> parts of the system (as opposed to being limited to some part thereof)</p>
</blockquote>
<p>To further elaborate on those terms would require much mathematics and mathematical notation is where normal HTMl fall short. Those who are interested should go read the paper.</p>
<p>I do not have any experience reading other papers that try to discriminate between these three terms so I might not be able to give a comprehensive discussion of the merits and faults of the other definitions to date. However, suffice to say, this paper does fulfill the objective of giving proper definitions to the three terms.</p>
<p>At first, I was a bit critical and skeptical of the use of formal mathematics to define these terms. However, after some thought, I realize that without using formal methods, it will be hard to give a qualitative description for each term. This was one of the reasons that compelled the authors to write this paper; previous definitions were based on quantity (the amount of details in each specification).</p>
<p>Even after reading this paper, I remain slightly skeptical of such formal specifications. As mentioned in the paper itself, there is no way to check this formalization for all design patterns since a catalogue of such patterns does not exist. “A direct proof would require a formalization of “all” design patterns and architectural styles.”</p>
<p>Moreover, in most architectural systems, to render the distinction between architecture, design and implementation might not really be that important. The main purpose of architectural documentations (for me, and with my current knowledge) is to ensure that people who read it can get a grasp of the system. At the moment, architectural documents are still read by human beings who are not too troubled by formal or informal definitions.</p>
<p>Still, the existence of this paper does provide for developers who need such degrees of precision when defining these three terms. And it was interesting to see how one would approach software architecture from a mathematical point of view.</p>
<p>Even more interesting is how the developers would react when something that they have always classified as a design criterion gets upgraded to an architectural style! That is exactly the case with the Law of Demeter. The Law of Demeter is usually thought of as a design decision, but in this paper it has been updated to an architectural style. Somehow this really goes against the principle of least surprise.</p>
<p>And Prof. Johnson pointed out something interesting in lecture. Very few architectural styles exist in pristine form. For instance, it is rare for a Pipes and Filters system to be just pipes and filters. Developers are definitely going to modify the architectural style to suit their domain model. However, if they did that, then it would violate the intensional property and nullified their architectural style; <i>demoting</i> it to the status of design!
</p>
<p>In a nutshell, one can conclude from the paper that <em>architecture</em> is most abstract (in the sense that its description excludes the details of implementation) and also the most encompassing (in the sense that it pervades the entire system). How one actually use this definitions when dealing with other developers not familiar with these definitions is not entirely clear yet.</p>
"4+1" View of Model Software Architecture2006-08-30T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/08/30/4-1-view-of-model-software-architecture<p>If you recall, in my first blog post on software architecture, I mentioned architecture description languages (ADL) as ways to – obviously – describe some software architecture. However, using ADLs is not the only way. In the paper <a href="http://csdl2.computer.org/persagen/DLAbsToc.jsp?resourcePath=/dl/mags/so/&toc=comp/mags/so/1995/06/s6toc.xml&DOI=10.1109/52.469759"><em>Architectural Blueprints—The “4+1” View Model of Software Architecture</em></a> by Phillipe Kruchten<sup><a href="#fn1">1</a></sup>, he examines how developers can use multiple views to capture the software architecture and how these views are more desirable than a single viewport that tries to capture everything. Each view, should ideally, address one specific set of concerns.</p>
<p>For Kruchten, software architecture is the deliberate assembling of various architectural elements to fulfill the requirements of the system. As such, it is possible to describe the architecture using these four views:</p>
<blockquote>
<ul>
<li>The logical view, which is the object model of the design (when an object-oriented design method is used), </li>
<li>The process view, which captures the concurrency and synchronization aspects of the design, </li>
<li>The development view, which describes the static organization of the software in its development environment, </li>
<li>The physical view, which describes the mapping(s) of the software onto the hardware and reflects its distributed aspect.</li>
</ul>
</blockquote>
<p>Each view uses its own specialized set of notations that can convey the main point more succinctly. For the logical view, the class and object diagrams in <span class="caps">UML</span> would be an appropriate choice whereas the sequence diagrams in <span class="caps">UML</span> might be suitable for the process view.</p>
<p>The <strong>fifth view</strong> (the +1 view) would be some <a href="http://en.wikipedia.org/wiki/Use_case"><em>use cases</em></a> for the system. And this fifth view ties the other four views and gives a more concrete description of the system. It is this concrete description that would be most important in a system since the fifth view explicitly states the scenario that needs to be addressed by the system. Most of the details of the “main” problem might be hidden away in the other views since they present a more abstract view of the system.</p>
<p>Furthermore, this fifth view serves an important role in a iterative approach to refine the architecture. The paper presents a sample iterative cycle that uses a specific scenario as the fifth view to help identify the components in the other four views. As each use case is addressed, the other views are also adapted to accommodate that use case.</p>
<p>Finally, these views can also serve as the basis for other forms of written documentation such as the <em>Software Architecture Document</em> and the <em>Software Design Guidelines</em>.</p>
<p>Kruchten himself states the generic nature of these 4+1 views and how other notation systems might be used alternatively. In fact, he is not against dropping certain views if there is no need for them in the system. For instance, he argues that if the logical view and development view are so similar, then there is no need for separate descriptions.</p>
<p>Why does this separation of concerns work? There are various requirements that each stakeholder might be interested in but not every stakeholder cares for what the other stakeholders are interested in. For example, the project manager might be interested in the development view but the systems engineer will be more interested in the physical and process views. By separating the concerns, each stakeholder would be able to quickly find out what they need to know about the architecture and, if necessary, provide feedback.</p>
<p><em>Documenting Software Architecture</em> (another book to read for this class) does mention a brief note about the “4+1” view at page 380 but the book associates it more with the Rational Unified Process (RUP). So, it would be interesting to see whether this “4+1” is used by developers who are not in favor of <span class="caps">RUP</span>.</p>
<p id="fn1"><sup>1</sup> I did not link to the version on <span class="caps">UIUC</span>’s CS server because the paper might still be under copyright.</p>
Software architecture: An Introduction2006-08-27T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/08/27/software-architecture-an-introduction<p><a href="http://en.wikipedia.org/wiki/Software_architecture">Software architecture - Wikipedia, the free encyclopedia</a>:
</p>
<blockquote cite="http://en.wikipedia.org/wiki/Software_architecture" title="Software architecture - Wikipedia, the free encyclopedia"> "A software architecture or software systems architecture is a representation of a software system, as well as the process and discipline for effectively implementing the design(s) for such a system. It is a representation because it is used to convey the information content of the related elements comprising a system, the relationships among those elements, and the rules governing those relationships. It is a process because a sequence of steps is prescribed to produce or change the architecture, and/or a design from that architecture, of a system within a set of constraints. It is a discipline because a body of knowledge or a general set of principles of (software) architecture is used to inform practitioners as to the most effective way to design the system within a set of constraints."
</blockquote>
<p>
<a href="http://www.inf.ed.ac.uk/teaching/courses/seoc1/2005_2006/resources/intro_softarch.pdf"><em>Introduction to Software Architecture</em></a> was a paper written by David Garlan and Mary Shaw in 1994 that still gives its readers a good idea of what software architecture is all about. The excerpt above was taken from Wikipedia's entry on Software Architecture and not much has actually changed from Garlan and Shaw's earlier definition. Garlan and Shaw's definition treats software architecture as a collection of <em>components</em> and the <em>interaction</em> of those components under a set of constraints. To understand a particular software architecture, Garlan and Shaw not only propose identifying the components, their interactions and the constraints that bound them but to also document the specific situation under which that software architecture flourished. Proper documentation was an important part of describing and preserving the nature of the software architecture for future references.
</p>
<p>
The evolution of software proposed by Garlan and Show goes like this: high-level programming languages -> abstract data types -> software architecture. I agree that as we deal with more complex systems, the underlying hardware has been greatly abstracted. We no longer deal in terms of bits and bytes or even integers. In fact, in software architecture terminology, we might not even be dealing with actual classes (in the Object-oriented sense) anymore. What we deal with is in fact even higher level. We are describing the architecture using common phrases such as "three-tier", "pipeline" and "web services" but each phrase conveys a high-level concept that developers can grasp easily.
</p>
<p>
<em>Introduction to Software Architecture</em> lists six common architectural styles. All six examples are still listed on the Wikipedia page as well. One common architectural style that was mentioned is Pipes and Filters. Pipes and Filters are pretty commonplace in Unix systems based on the advocacy of <a href="http://www.catb.org/~esr/writings/taoup/html/ch01s06.html">Eric Raymond's mantra</a>: "This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface." What I find interesting is that although pipes work well for most task such as grep, sed and awk for more specific applications there is an added cost of having to reparse at each step because the textual representation might not be what you expect.
</p>
<p>
Nonetheless as noted in the paper, most of the six common architectural styles do not exists in their pure form in software development. Garlan and Shaw prefer to use the term heterogeneous architectures but I prefer calling them hybrid architectures. To me, heterogeneous means mixing two or more architectural styles together and hope that they work well. For me, the term hybrid is more suited because changes need to be made to each architectural style that you wish to combine for them to work together. The existence of hybrid architectures show that software architecture can borrow ideas from different styles and merge them together. Such evolution means that there will always be new architectural styles to observe and document as developers keep producing software.
</p>
<p>
I enjoyed this paper a lot because it presents some concrete examples of software problems and different approaches for solving the problem. Each problems can be solved using any one of the six common architectural styles, but a better solution usually exists as a conglomeration of two or more styles. As software complexity increases, there is a demand for different paradigms for solving different problems. There is, most definitely, no single <a href="http://en.wikipedia.org/wiki/Golden_hammer">golden hammer</a> for every software challenge out there.
</p>
<p>
The paper includes a summary that advocates the importance of documenting the different architectures so that other developers might be able to analyze architectural styles that fail and reapply architectural styles that work. It proposes the development of proper notations for describing those architectural design. Wikipedia's entry on software architectural has some examples of architectural description languages that might be employed: Wright, Acme, xADL and Darwin.
</p>
<p>
While there are indeed many different definitions for software <a href="http://www.sei.cmu.edu/architecture/definitions.html">architecture</a>, most parties agree that architecture embodies the organization of the components in the software, the interaction and the compelling forces that constraints that style. All in all, software architecture will remain an important facet of software development. As we move into <a href="http://www.sei.cmu.edu/uls/">ultra-large-scale systems</a>, the need for different architectural styles of software development is also at its greatest to sustain the need for continuos development in the future.
</p>
Welcome2006-08-24T00:00:00+00:00http://softwareengineering.vazexqi.com/2006/08/24/welcome<p>
This is a sub-blog created to discuss issues with the papers, book, etc that I have read as part of UIUC's CS 527: Advanced Topic in Software Engineering. This blog will only discuss topics concerning software engineering. It might include humorous articles too.
</p>
<p>
My main blog is at <a href="http://blog.vazexqi.com">blog.vazexqi.com</a>. It also contains a fair amount of software engineering related materials but you will have to dig for it yourself. </p>
<p>
To start things off, here is a picture that most of you will find interesting:
<a href="http://www.flickr.com/photos/vazexqi/224012764/" title="Photo Sharing"><img src="http://static.flickr.com/76/224012764_d91062484f.jpg" width="500" height="375" alt="Software Engineering Explained" /></a>
</p>
<p>
The original post was from Andy Hunt's <a href="http://www.toolshed.com/blog/articles/2006/06/26/software-engineering-explained-perfectly">blog</a>.</p>
<p>
This blog is powered by <a href="http://www.typosphere.org/trac/">Typo</a>, a <a href="http://rubyonrails.com/">Ruby on Rails</a> content management systems what is currently at version 4. Great as it is, Typo does lack proper architectural documentation and would make a great project for CS 527 :)</p>