Smalltalk as a Program MetaServer

Understanding a major difference between Smalltalk and Java

Overview

This topic discusses how Smalltalk's execution model and environment is different from a language like Java. It also discusses why it is necessary to understand that execution environment in some detail to understand the semantics and capabilities of the Smalltalk language. Smalltalk is unusual in that it's syntax and "first-level" semantics reveals very little of its extraordinary capabilities.

The following is a reference to the original thread:

• Deja.com message: Smalltalk as a Program MetaServer (was: Scenes, etc.)

Original Posting: Smalltalk as a Program MetaServer

I hope this posting helps some of these threads... it is basically a continuation of the posting by Xavier Alvarez, but it expresses some concepts in a different way.

D'Arcy Smith wrote:
> > > If you are talking strictly about the language then you ignore the JDK
> > > and you ignore anything to do with the Smalltalk environment that
> > > has nothing to do with the language.  This includes ignoring things
> > > like compilers etc...
[SNIP]

Ben Z. Tels wrote:
> If you feel you must, you can drag environments into this. But then you are
> saying that a language cannot exist without whatever the enironment is you
> have chosen. And that is not so; programming languages are NOT hard-linked
> to any one particular tool.

Smalltalk is not hard-linked to a particular Tool, but it is hard-linked to a "Program Server" environment. Smalltalk syntax is linked to a "Program Server" just like SQL is linked to a "Database Server". It makes no sense to talk about SQL without describing the environment (the Server) that SQL will be executed within. SQL does not create a complete program, SQL is executed by a server and by so doing changes the server's behavior/state. We have exactly the same case for Smalltalk. To understand the capabilities of Smalltalk you need to understand what that running Server environment can do [1].

I am not taking the (IMO) extreme position of saying the Development Tools for Smalltalk are necessarily part of that Program Server. Very good Development tools like a source code browser happen to be included with the Server, but that is not what makes Smalltalk different on this aspect. I will assume all we have is VI for either Java or Smalltalk. The tools are now about the same, but what do we use them for in each case?

In most programming languages, you use an edit-compile-run cycle to create an application. You edit the source of the program, compile and link it, and then run the program to make it behave. So we need VI to edit, a compiler to compile, and we get something to run (a JVM plus class files with a main). The actual program is just the JVM plus the class files.

Smalltalk does not work this way. Smalltalk is an already running program and you talk to it to make it behave. The cycle for Smalltalk is run - command* - save. So VI for Smalltalk would be a Command editor (like a SQL editor): VI would help us write a bunch of commands that will be sent to the Smalltalk Program Server. That Server keeps running between commands and remembers its state continuously. This is a dramatically different metaphor, and allows use/reuse of Objects instead of just reuse of Code.

A Smalltalk program is created by sending messages to Program Server objects to change the state of the Server, and then you save that Server's complete state [like taking a Snapshot] and you can get back that Smalltalk Server's state/program at any time. Effectively we have a Program MetaServer that can be "pushed/expanded" in different directions to produce the behavior you need. Again, I am not talking about Development Tools in the editor/browser/inspector/debugger sense: I am talking about how Smalltalk the language works independent of (underneath) the development tools.

The closest equivalent in the Java world would be a Java shell, which provides a running JVM that you can execute commands in. Add the ability to save a complete serialization of that running shell so you can reload the serialization and return to exactly the same state, and you have something that is abstractly like the Smalltalk environment.

Now we can start to describe where Smalltalk's Program MetaServer has capabilities that Java can't even "dream" of and those capabilities may affect productivity dramatically. In any case, the capabilities do dramatically affect the semantics of Smalltalk and make it different from Java in ways that would not be apparent at first blush.

So Smalltalk is linked to this Program MetaServer: what does that mean? What does it allow Smalltalk to do? Well, you might want to look at "Smalltalk-80: The Language" [called the Purple or Blue book] which describes the Smalltalk language along with its capabilities. But I will also give some examples. For much of the following I will use Java syntax with Smalltalk concepts [No type declarations: see [2]] so we don't have syntax as an issue.

If we have an always-running server, we need to send it commands by talking to objects:

   anObject.doSomething();

To be interesting, the first important thing to do is to describe how to define a new class. Maybe something like:

   personClass = Object.class.newSubclassNamed_fieldNames("Person",{"myName","myAddress"});

   class Person extends Object {
       protected String myName, myAddress;
   }

   personClass.addMethodNamed_source("getName","return myName;");

   person = personClass.newInstance();
   println(person.getName());

   personClass.addMethodNamed_source("getAddress","return myAddress;");
   person2 = personClass.newInstance();
   println(person2.getAddress());
   println(person.getAddress());

It also means that: Java: anObject.doSomething(); Smalltalk: anObject doSomething.

looks like just a trivial syntax change (which it is) but it has a dramatic semantic difference because the Smalltalk version executes within the SPMS and survives mutations of the Class of anObject. If you know about JVM rules and Java binary compatibility (and lack thereof), you are aware how fragile the Java version is to any Class changes. You can not significantly evolve a running Java program. [3]

Another example of the Program MetaServer capabilities was also implied above: Class objects are active, useful objects that have certain standard capabilities. Because they are objects themselves, they can also be augmented with new capabilities. Say I have:

   personClass  = Object.class.newSubclassNamed_fieldNames("Person",{"myName","myAddress"});
   companyClass = Object.class.newSubclassNamed_fieldNames("Company",{"myName","myAddress"});

   personClass.newAssociationNamed_to("employee-employer",companyClass);

   person.getEmployer();

will return a person's employer (or 'null' if none has been set yet). The method 'getEmployer' is constructed similarly to before, but it can be automatically generated (even without textual source) from this higher level language. There is no way that Smalltalk's syntax could let you know that this worked where it couldn't possibly work in Java. You have to understand the SPMS to understand Smalltalk.

So to understand Smalltalk you need to understand the Smalltalk Program MetaServer more than the Smalltalk Syntax [syntax is just a finger pointing at the moon]. After becoming familiar with it (and if you were like me fourteen years ago [and today] being EXTREMELY IMPRESSED at its capabilities and elegance), you might then wonder "Does it really work that well?". Is it more productive? Does its power cause problems? Those are good questions, and there are lots of sides to discuss and thoughts by smart people related to that topic (Declarative Smalltalk, Strongtalk, Squeak's distributed projects, stripping issues, SELF, Cecil). But those are different topics and the capabilities of the Smalltalk Program MetaServer has to sink in deeply before they can be discussed.

Smalltalk without an environment isn't Smalltalk

"Ben Z. Tels" wrote:
> Yes, it is. It won't compile, but it's still Smalltalk. Just like Java
> without a compiler is still Java. They are both languages, with a syntax and
> appropriate semantics, with standard class libraries and other relevant
> properties.

I hope you can now see that the Smalltalk Program MetaServer environment is part of the semantics and "other relevant properties" of Smalltalk. Taking the definition of the SPMS away from Smalltalk is like taking the definition of the JVM away from Java (or the EJB Server/Container definitions for EJB). It is not the Tools that matter when comparing Smalltalk to Java... if they are superior it may be because the SPMS is helping them significantly or they just were built by really good programmers. But to compare the languages you have to understand and compare Smalltalk+SPMS+Libs to Java+JVM+Libs.

--Mark
mark.fussell@chimu.com

[1] The only true definition of Smalltalk behavior is from Smalltalk-80 and any evolutionary descendants. If some tool uses the Smalltalk syntax in a different way, it is not really Smalltalk [although it could be a decent product on its own right]

[2] Effectively I am using SmallJava [sort of like JavaScript]. See http://www.chimu.com/publications/smallJava/

[3] A cool, industrial augmentation of the Smalltalk PMS exists within GemStone/S, where under modification of a Class, you can choose whether to migrate existing instance of the Class to the modified version or simply keep objects of each Class (Person.1 & Person.2) independent.

Subsequent Discussions

SQL analogy and levels of language comparisons

"Joern W. Janneck" wrote:
> "Mark Fussell" <mark.fussell@chimu.com> wrote
> > Smalltalk is not hard-linked to a particular Tool, but it is hard-linked
> > to a "Program Server" environment.  Smalltalk syntax is linked to a
> > "Program Server" just like SQL is linked to a "Database Server".  It
> > makes no sense to talk about SQL without describing the environment (the
> > Server) that SQL will be executed within.
>
> i am not sure i agree all that much with your analogy, but assuming it, i
> must say:
>
> wrong.
>
> in fact, it makes eminent, full, great sense to talk about sqp without
> assuming any particular database. it makes sense to define the meaning of
> sql expressions in fully general, where possible even mathematical terms. do
> you find a specific database as a prerequiste in the sql standards docs?

I did not say a particular specific database, I said an enviornment (the "Database Server"). The meaning of any SQL expression must be in terms of this environment: the SQL-environment. And yes, the SQL standard starts with a definition of this SQL-environment. It especially defines modules, catalogs, schemas, etc. and how SQL expressions would be affected by and modify that environment.

Codd's relational model includes the definition of its environment (DBMS) as well. The capabilities this environment provides is how an RQL [Relational Query Language] can be compared to predicate logic. In addition, the RQL can also declare new base relations and other "beyond-logic" rules and state of the DBMS.

> > SQL does not create a complete program,
> well, it does.

[SNIP]
> isee the point about the st program server running, while java starting anew
> each time. then again, isn't the os, whichever it is, exactly the same thing
> to java, as the smalltalk env is to st-programs (program-parts)? is this
> much more than simply drawing the line at a slightly different place?

The OS analogy is fine also... Smalltalk was originally effectively a very sophisticated OS-augmentation language. And a good OS has the same characteristics: you can transform it to do what you want without restarting the OS/machine. The characteristics that enable this are important to understanding any "language" on top of the OS.

The point was solely that Smalltalk the Language can not make sense without understanding the Smalltalk environment (SPMS) and how that environment is different from a Java/JVM environment. The Smalltalk language is just a way to talk to that environment as opposed to constructing a new environment afresh each time you launch a JVM-based program. And the Smalltalk language/environment allows you to restructure the running environment in VERY USEFUL ways [e.g. augmenting existing objects] that do have major impact on the abilities of the language. Ignoring that only lets you go so far in comparing things.

Effectively I have different writings from each of the levels that you could compare Java and Smalltalk [or any two languages]:

   (1) Just the syntax:
          http://www.chimu.com/publications/javaSmalltalkSyntax.html

   (2) Basic semantics ignoring the syntax:
          http://www.chimu.com/publications/smallJava/index.html

   (3) Fuller semantics that includes the definition and affects of the
       language environment.
          http://www.chimu.com/publications/short/spms.html

The thread I was entering had [has] the problem that some programmers want to compare Java to Smalltalk at level (1) or maybe level (2) [1]. But people experienced with Smalltalk know that a full comparison would require getting to (3) [2].

--Mark

[1] I saw no evidence of a true understanding of level two from the original poster.

[2] Further some Smalltalkers argued that

    (4) The Tools Built on (1..3)

Environment vs. Tools, JVM analogy

Mike Spille wrote:
> >     The point was solely that Smalltalk the Language can not make sense
> > without understanding the Smalltalk environment (SPMS) and how that
> > environment is different from a Java/JVM environment.  The Smalltalk
> > language is just a way to talk to that environment as opposed to
> > constructing a new environment afresh each time you launch a JVM-based
> > program.  And the Smalltalk language/environment allows you to
> > restructure the running environment in VERY USEFUL ways [e.g. augmenting
> > existing objects] that do have major impact on the abilities of the
> > language.  Ignoring that only lets you go so far in comparing things.
>
> By your standard then you cannot meaningfully talk about Squeak and
> Visual Works in the same breath, because they have vastly different
> environments.  GNU Smalltalk in that world doesn't exist as all,
> but it has no environment that coincides to what you're talking
> about.
[SNIP]

Please do read my two postings: I am not talking about the tools within the environment, I am talking about the "Engine" (what I called the Program MetaServer). Smalltalk environments' engines are far more similar than different. All the major dialects of Smalltalk do descend from Smalltalk-80 and most of us can easily switch between one environment and another. It is like moving between Oracle and Sybase, except the two "engines" are even more similar and the syntax is identical. This is not about the tools built on top (Oracle or Sybase Control Managers or Programming Tools) it is about the capabilities of the engines and how that actually affects what the environment/languages (Smalltalk or SQL) can do.

> Let me come at this from a different angle.  If I write Java code
> on a puny laptop with a 6-inch by 3-inch keyboard my productivity is
> going to suffer... [SNIP]

The analogy is more like: If you program to JVM 1.0 and I program to JVM 1.2, I have a different environment from you. It is not that Java, the language, had to change[1] it is that the capabilities of the JVM, the environment, changed. Take WeakReference capabilities: these had no syntax effect because they are added in via new Classes. But JVM 1.0 is completely incapable of supporting WeakReferences. JVM 1.2 has a capability (unreflected in the syntax of the language) that makes Java on JVM 1.2 far more capable than Java on 1.0. Some of Smalltalk's engine's capabilities (ignoring syntax) are far beyond JVM 1.2 [maybe JVM 3.x], and these capabilities dramatically affect the "apparent" syntax and semantics of Smalltalk.

--Mark

[1] It did because some of Java's capabilities are very syntax reflected. Smalltalk's capabilities are rarely syntax reflected, so it makes it harder to notice those changes.

Similarity and differences among Smalltalk environments/engines

Mike Spille wrote:
> OK, you draw the line fairly well at the "engine" level.  But let's be
> honest here - the engines aren't all that similar.  "Descend from" is
> not the same as "your code works identically".  Or even you work identically.

No, they are not identical... this is a problem of competitive advantage verses standardization. Same problem as for SQL DBMSs. This is a notable advantage of Java's marketing/deployment approach. It would have been relatively easy to do this for Smalltalk in the mid-late 1980s, but the major vendor did not choose that path.

> You say "The Smalltalk language is just a way to talk to that environment
> as opposed to constructing a new environment afresh each time you launch
> a JVM-based program".  VAJ users might be surprised at your statement there.

I am a heavy VAJ user, I know it well (actually, it is the tool I am using in between sending these postings). VAJ is written in Smalltalk, which is why it can do some of "the magic" it can do. VAJ breaks where "the magic" would be too impressive for the Java standard (e.g. add an instance variable to objects that exist in the currently executing program). But definitely a great tool.

Even if VAJ were written in a different language, it would have effectively reproduced a Smalltalk like environment and reaped the benefits from it. Of course, in this case the advantages are solely for development time.

[SNIP]
> The problem here is that you're assuming the developer's mighty IDE matters
> a wit to the end user - it doesn't.

No programming language matters to the end user. Even computers don't matter to the end user. But the topic was related to comparing computer languages and their capabilities. I am not referring to the IDE, but we have to be talking in terms of the application creation process in some sense.

> Smalltalk is geared towards the developer
> with the highly-mutable/discovable image.  This doesn't help you much when
> you need to deploy it out (indeed, much of what _you_ call Smalltalk is
> stripped out at deployment time, is it not?).

Some you do, some you don't. Depends on the needs of the application. I have deployed applications that dynamically augmented existing objects to support new messages (e.g. when the storage schema changes). On the other extreme, GemStone needs to always have extendend Engine features so it can run 24x7 and maintain data under program changes.

But the point was still the capabilities of the language. A language is either better or worse at doing some tasks, or even for conceiving of certain tasks to begin with. The environment/engine affects those capabilities.

[SNIP]
> > The analogy is more like: If you program to JVM 1.0 and I program to JVM
> > 1.2, I have a different environment from you.  It is not that Java, the
> > language, had to change[1] it is that the capabilities of the JVM, the
> > environment, changed.  Take WeakReference capabilities: these had no
> > syntax effect because they are added in via new Classes.  But JVM 1.0 is
> > completely incapable of supporting WeakReferences.  JVM 1.2 has a
> > capability (unreflected in the syntax of the language) that makes Java
> > on JVM 1.2 far more capable than Java on 1.0.  Some of Smalltalk's
> > engine's capabilities (ignoring syntax) are far beyond JVM 1.2 [maybe
> > JVM 3.x], and these capabilities dramatically affect the "apparent"
> > syntax and semantics of Smalltalk.
>
> As I mentioned, you can get those capabilities in advanced debuggers for
> many languages.  The fact that such debuggers exist doesn't impact the
> underlying language very much.

--Mark
mark.fussell@chimu.com

Smalltalk Syntax

Andrew Carpenter wrote:
> Marcel Weiher wrote:
> > For example, looking at just the language would show you that
> > it is incomplete:  there is no stand-alone syntax for defining
> > classes or methods.
>
> Okay, finally -- that last point is starting to get to the crux of the
> matter. (I can't believe this thread going in circles so much!) So the
> language syntax, by itself, is insufficient to write programs in
> Smalltalk? (not fully develop or complete, just write.)

Yes, you need a Smalltalk Program MetaServer (SPMS) that will execute commands specified by the Smalltalk syntax (or really any other syntax). See my other posting for more discussion.

> why is the Smalltalk syntax defined the way it is? Is there some reason
> it *must* follow the structure it does? Why is that structure better
> than the equivalent syntax from another language?

You might want to ask Alan Kay, look at Squeak (and the Smalltalk-72 version), or read the HOPL-II (History of Programm Languages-II) articles. But as someone said, 'anObject message.' is pretty hard to argue with. In any case, the syntax is good but irrelevant. If all the delimiters and special characters were changed but messages could be sent to objects within the SPMS (Smalltalk Program MetaServer) you would still have a pretty pure [albeit confusing] Smalltalk.

> and to logically follow: is the syntax for Smalltalk identical across
> implementations? If as you say the syntax is largely irrelevant to the
> use of Smalltalk, why can't other 'languages' be substituted?

Yes, the syntax is mostly identical but now the analogy with SQL database servers really starts to show up. The reason Smalltalk is not portable between dialects is because the Program MetaServers have slightly [or dramatically] different capabilities. In some areas, this just does not matter but in other areas the differences causes one to rewrite code. The range of difference is similar to JDK 1.1 vs. JDK 1.2. And the reason is pretty similar to the reason database servers are different: a company's need to differentiate. If Sun did not have a contract with vendors (e.g. Microsoft) to comply with the standard, Java would have splintered far more than Smalltalk ever did.

Other languages certainly could be used within a Smalltalk Server/VM, but now people expect an open VM specification (which Smalltalk does not have) and there is currently a Java "trend". A Universal VM would be a wonderful thing for a lot of programming languages, but it would not be high on Sun's list.

> Oh, and you snipped this. I don't use Smalltalk myself, so I'm honestly
> curious about the answers.
>
> > how compatible are the various vendor releases? If Joe
> > Coder worked on a Squeak project, and moved to a company using
> > VisualWorks, could he take any of his work with him?

Some of it. Squeak is kind of strange in that it stepped back fifteen years and then rushed forward from there. So a lot of progress needs to be rebuilt. It would have been "cooler" if at least the VM of VisualWorks had been used for Squeak, but giving the VM away was not high on that vendor's list.

> > What common ground is there among the vendors?

Historical, ANSI, and convenience. Historically all Smalltalks came out of Smalltalk-80. Some evolved from Smalltalk-80 and so are very compatible. Some just took the spirit or syntax of Smalltalk and so are less compatible. Second there is an ANSI X3J20 standard. Third, it is nice if things stay similar and reconverge if at all possible, and the vendors somewhat recognize this.

--Mark

Further resources

• Java for Smalltalk programmers