[ | smalltalk dot org ].
Community and industry meet inventing the future.
The Simplicity and Power Comes from the Smalltalk Syntax
posted by Peter William Lount
minor revisions, 20040914 4:53pm PDT
version 1, 20040914 4:03pm PDT

The last few days I've been digging deeper into PERL and their new register based virtual machine Parrot. It looks like an awesome approach and it could be easy to port Squeak to Parrot! (Anyone up to it?) I'm digging into PERL for a couple of reasons, one being the shock of seeing the highly complex PERL Periodic Table of the Operators (PDF 156K) on the weekend. Wow! Just the sight of this table scares me further away from using PERL since to master PERL is to master all the complexities of the language syntax grammar in the table! My clients have better things for me to do with my time and their money! Talk about complexity on full gory display! Here's a tiny (clickable for larger) version of it:

Nice table by the way. It takes quite a bit of skill to organize and communicate any complex system as well as Mark Lentczner has with his table. I congraduate him for his ability and the resulting clarity he brings to those interested in PERL. Too bad PERL wasn't simplier in the first place so a table like this wouldn't be needed. Well there is always Smalltalk!

The PERL Periodic Table of the Operators sure helps one to understand a language like PERL that has gone the route of using a "complex language syntax" to generate it's power rather than simplifying the syntax and using the language's libraries to express the power as Smalltalk does.

Many language designers place too much complexity into their language syntax. Language syntaxes are most often expressed using a version of Backus-Naur Form (BNF). (I heard that PERL's grammar is so complex that it can't be expressed in a BNF or extended BNF (EBNF) and that one of the goals for PERL6 is to simplify so that it can be. Is this true? Please let me know. Thanks.)

LISP's Special Forms
As I mentioned previously, it was annoying that the surface beauty of LISP was marred by some of its key parts having to be introduced as "special forms" rather than as its supposed universal building block of functions. The actual beauty of LISP came more from the promise of its metastrcutures than its actual model. I spent a fair amount of time thinking about how objects could be characterized as universal computers without having to have any exceptions in the central metaphor. What seemed to be needed was complete control over what was passed in a message send; in particular when and in what environment did expressions get evaluted?
- Alan Kay, The Early History Of Smalltalk, Special Forms.
Smalltalk was in part born out of Alan Kay's observation that LISP, while having a simple syntax, had too many special [syntax] forms (see side bar on "LISP Special Forms"). Alan (and team) saw that it was possible to have a uniform message passing syntax that enables objects to communicate for all computations including meta operations (operations about the objects definitions themselves). A very nice unification bringing together multiple syntaxes into one simple easy to understand form that enables both capabilities thus increasing the power without scraficing clarity. Note that we're not talking about a simple "merging" of two or more seperate syntaxes (PERL seems to be layer upon layer of special syntaxes some of which collide with each other), but a unified and new syntax that enables all the desired language possibilities without the cruft of a syntax merger.

In addtion a major advantage occured for Smalltalk with it's elegant and simple, but not simplistic, language syntax (BNF). That's the shift of "language extensibility" away from the sole and restricted domain of the languge designer as fixed in the "language syntax" (BNF) to the wider and easily accessible domain of the object class libraries and the everyday programmer.

If you think about it when you want to implement a language extension to most computer languages you need to convince the language designers, or become one yourself. Given access to the source code (common these days) it's possible for your extension to be made to the laguage, but getting it into the mainstream release for your favorite language might be a bit of a challenge. Furthermore you'll have to modify the language BNF, source code, possibly regenerate the BNF using complex tools like LEXX and YACC, recompile, link, and test your extensions hoping they won't break the language and introduce unknown bugs.

The process is much simplier when extending a language like Smalltalk where many of the extension dimensions have been shifted from the language syntax BNF to the language class object library. Yes testing is important as you can still make a mess so you have to know what you are doing but it's one hundred times easier, no maybe ten thousand times easier to extend the language when you need to. Sufice to say that it's so much easier that many more people are likely to not only try it but succeed doing it. And you don't have to be a language designer to do it! The language is thus more likely to evolve new and powerful capabilities this way.

Now certainly there are cases where the BNF must be changed for an extension to Smalltalk. A number of descendants and variants of Smalltalk modify the BNF to gain their capabilities. The Self Language, the Squeak Smalltalk, the Slate Smalltalk, and Zoku Smalltalk are a few examples. In the Zoku Smalltalk that I'm writing I've discovered a number of addition dimensions of extensibility that can be shifted from the language BNF to the language class object library. I'm currently working on implmenting the Zoku language compiler.

A number of extensibility domains were sifted to the Smalltalk language class object library including but not limited to: all control and iteration (looping) structures (ifTrue:ifFalse:, whileTrue:, whileFalse:, to:do:, etc...); meta programming; and most importantly "operators" (like the ones listed in the PERL Periodic Table of the Operators).

Yes, that means that basic core language operations like "equality testing" with "=", "greater than testing" with">", "less than testing" with "<", object identity testing" with "==" and all the rest are implemented not in the language syntax (as is the case in almost all languages) but in the Smalltalk object class library! This means that you can add your own operators and extend (or override) the existing ones! Very powerful! While some implementations of Smalltalk have primitives for these operations for performance reasons many of the operators are visible and implemented as Smalltalk level source code! Very powerful as you can see what works and reuse in many ways!

Let's look at an example. First let's create a Person class with three object instance variables: firstName, middleName and lastName. Now let's add our own "equals" method to the Person class as an object instance method that all instances of Person will respond to.
= theOtherPerson

(self firstName) = (theOtherPerson firstName) ifFalse: [
    "First name didn't match"
(self middleName) = (theOtherPerson middleName) ifFalse: [
    "Middle name didn't match"
(self lastName) = (theOtherPerson lastName) ifFalse: [
    "Last name didn't match"
] ifTrue: [
    "First, middle and last names match, so at least the names are the same for these two people."

Note that the ^ hat, ASCII "caret", charcter represents the 'return anObject' operation in Smalltalk, one of the few reserved operators defined in the syntax rather than the object libraries. In the case of the above method it's returning "true" or "false".

Now we can compare Person objects using the equals operator, =, to see if the names are the same:
aPersonA := Person firstName: 'Joe' middleName: 'Bill' lastName: 'Smith'.
aPersonB := Person firstNmae: 'Mary' middleName: 'Jane' lastName: 'Jones'.

aPersonA = aPersonB ifTrue: [...] ifFalse: [...].

Nice and simple comparison using the user defined equality test. We could implement the other tests for less than and greater than and some of the other variants that we'd use. We could also have implemented these using Smalltalk's "keyword" message form using "equals:" or some other keyword of our choosing, hopefully choosen to claify the meaning of the operation.

Today I saw an comment by Vincent Foley posted on James Smalltalk Rants here promoting the "brevity" of PERL code over Smalltalk code. Here is the comment:
Somehow, some people prefer to do

%matches =~ m/foo/bar/g


| re matches |
re := 'foo' asRegexp.
matches := re matches: 'bar'.

Well the Smalltalk portion of the above code comparison
| re matches |
re := 'foo' asRegexp.
matches := re matches: 'bar'.

could potentially be written simplier:
| matches |
matches := 'foo' matchesManyOf: 'bar'.

It depends on the meaning of "matches:" and it's variants.

While PERL has highly evolved and time tested regular expressions they can easily be implemented in Smalltalk to provide similar power and brevity of code. So the following is certainly possible and likely would assist in bringing PERL programmers into the Smalltalk universe.
| matches |
matches := 'foo' regularExpression: 'm/bar/g'.

Now I prefer longer and more clear keyword messages like "regularExpression:" spelled out in full but it is a bit long so something like "regExp:" might be a short cut, but "re:" would be too short and "cryptic" due to a loss of meaning obtainable from a glance. In my view clarity wins over pure brevity, but in this case notice that we have brevity of syntax even though we might use a longer keword message (selector) name. The length of the keywords has no effect on the complexity of the syntax (regularExpression:, regExp: and re: are all syntatically identical) but a well choosen name can often improve the clarity of the program. Remember you are likely not the only one who will have the need to read, comprehend and maintain the programs you write. This is especially true of business applications. That's why I prefer clarity.

By stressing shear brevity over clarity Vincent reveals that he's potentially a member of the "cult of the cryptic" (see Devoted to complexity? Why? and The Cult of the Complex and Cryptic Stack Up Another Win. Languages with complex syntax, such as PERL, enable very compact "codings" of problem solutions by taking advantage of the special features embedded in the very complexity of their syntax. Programming in languages like these is really literally "coding" or "code ciphering" and takes a large amount of comprehension and cognitive focus. Languages with simplier syntaxes, such as Smalltalk, offer compact syntaxes but clairty is usually favorted over pure brevity. It also makes Smalltalk much easier to learn, master and debug than PERL would be due to the vast number of special cases in the PERL syntax. (The complexity of learning the PERL syntax is well known and often commented on in the PERL community).

Yes, Smalltalk requires variables to be defined before use and PERL does not, this is a minor point and more of an "implementation choice" than a fixed feature of the language. It's certainly possible for a Smalltalk version to allow temporary variables to be defined by their first usage (and default to the "nil" object/value or report an possbile unused variable error if the first usage isn't a variable assignment). The current state of the art for "workspaces" (Smalltalk's equilivant of a command line shell but in a two dimensional text gui window) permits the temporary variable definitions to be omitted. So the Smalltalk example simplifies to a one liner as in PERL.
matches := 'foo' matches: 'bar'.

The key to Smalltalk syntax and the source of it's power is the simple and elegant message sending syntax. Rather than putting the power language capabilities in the syntax - and thus the hard to access compiler - powerful Smalltalk language capabilities are placed in the objects making up the Smalltalk Object and Class Libraries. This makes Smalltalk highly extensible in ways that other languages only dream of; and extensible by end users in major ways! PERL is undergoing a major rewrite (to PERL6) due to the inner complexities of it's syntax/compiler/intrepreter system. It's just too difficult and undesireable to add new features to PERL5.

One dimension of Smalltalk's extensiblity is the ability to add new "control structures" to the language when the need arises. The best example is adding "ifNil:ifNotNil:" (and "ifNotNil:ifNil:", "ifNil:", and "ifNotNil:") so that instead of using:
someObject isNil ifTrue: [... "The object exists case" ...] ifFalse: [... "No object case" ...]

we can replace isNil ifTrue:ifFalse: with:
someObject ifNotNil: [... "The object exists case" ...] ifNil: [... "No object case" ...]

or one of the other variations. While this doesn't seem like a big difference it's actually a sea change from boolean logic to "object existance based logic" and it can make a huge difference in thinking approach by being clearly written as existance logic. Also readbility and comprehension improves and thus bugs are less likely.

Ten years ago this "extension" wasn't in the base object libraries of shipping Smalltalk systems. So I'd simply add it in to Object and UndefinedObject. Now this extension is in just about all Smalltalk systems. Actually at the time I received serious grief from other Smalltalkers resisting the adoption of ifNil "existance logic", they preferred to use "boolean logic" since that's the way they'd done it before. If fact after six months of successful use in a very major application I was asked to change all usages of "ifNil:ifNotNil:" and variants back to their "isNil ifTrue:ifFalse:" forms. Actually when I resisted this request they "ordered" me to make the change. I'm, glad to see clearly expressed existance logic in wide use now.

The main take home point with the "ifNil:ifNotNil:" extension is that a very useful extension could be and was added by anyone! As soon as many anyones found it interesting it was included into the default base class libraries. This is one way that Smalltalk evolves through community participation! Everyone in the Smalltalk community has the opportunity to easily make serious contributions to the language itself!

Another example is how I extended "SortedCollection" to incorporate "multi aspect/attribute column" sorting by adding two classes and NOT modifying any system classes at all! The "SortCritter" uses polymorphism to "pretend" it's a "BlockContext" object by implementing the BlockContext messages "value:value:" that are sent to the SortedCollection sort blocks. This innovation hasn't made it into all Smalltalks yet but the invitation is there for all vendors to include it - I'll adapt the licence to suite. In the meantime it's a simple code file in for SortCriteria is available along with an article explaining it's use.

Learning Smalltalk has been a journey of discovery that began for me in 1979 with the Scientific Americian article, deepened in 1981 with the August Byte Smalltalk special issue, and began in urnest in 1984 when I got my hands on a real live Smalltalk system. Twenty years of hands on experience later I'm still learning about the language. Earlier this year I had the privilage of communicating with Alan Kay and Dan Ingalls on the nature of the Smalltalk language syntax and where it's could possible head. Out of this a deeper apprecaition of the language has evolved for me and my ability to explain the language to others. The Zoku variant of Smalltalk that I'm working on has also benefited from these conversations with Alan and Dan since I was able to clarify the Zoku language syntax and the reasons why those variations make sense. Time to forward the future! Thanks guys.

Copyright 1999-2010 by Smalltalk.org"!, All Rights Reserved.