bitwise: What are the principal language differences between ‘traditional’ Delphi and Delphi Prism?

mh: I think there’s two principal categories here: (a) language "differences" as in features that are actually present in Delphi and Prism but work or look slightly different and (b) "differences" as in features that are not supported by one or the other.

In general the structure and syntax of the languages is pretty similar, so Delphi developers will feel right at home in Prism, and the vast majority of Delphi-style code will "just work" in Prism, as well.

There are a few subtle syntax differences where Prism decided to go a different route than Delphi, either for aesthetic reasons or dictated by the platform. For example Prism uses := to define default parameters for methods or assign property values in attributes, because we felt := better represented the "assignment" nature of the operation, rather then equality (= is reserved purely as the binary equality operator and for defining constants. Another example is that Delphi allows multiple statements to follow the "else" clause of a "case" statement - we found that’s inconsistent with how "else" works everywhere else in the language, so Prism requires a "begin/end" here, if there’s more than one statement. There’s a few more of those, and we are working on an (officially unsupported) tool called Oxidizer that will automate the task of adjusting some of these, from Delphi to Prism syntax.

Then there are certain language differences dictated by the platform. For example there’s no concept of (and no proper and reliable way to implement) "initialization" and "finalization" sections in .NET, so they don’t exist for Prism. Similarly, there are no "destructors". Also, events function fundamentally differently in .NET than they do in Delphi/Win32, so we have a separate "event" keyword to define them (rather than using properties).

Finally, there is, of course, a wide range of features in Prism that simply don’t exist in Delphi/Win32, and these are largely implemented as a superset of Delphi - meaning they exist without conflicting with exiting language elements.

Examples for these include Class Contracts, which allow you to define pre- and post-conditions for methods, and "invariants" for classes that will be enforced at runtime. There’s extensive support for nullable value types (beyond what’s available in C#), sequences and query expressions (aka LINQ), "future" types and iterators. There are extensions to for loops to support type matching, indexing and running loops in parallel. There’s the ":" colon operator for nil-safe member access (a personal favorite of mine). there’s support for true virtual properties; inline variable declarations; type inference. The list goes on and on.

bitwise: And what are the differences between Delphi Prism and RemObjects’ Oxygene?

mh: Delphi Prism will be a full super-set of what Oxygene is now - it will include the compiler and Visual Studio IDE integration, and all the features you have in Oxygene today. On top of that, there’ll be additional, mainly database-centric, features coming in from the CodeGear side - things like the fully-managed Blackfish SQL database server, or dbExpress- based ADO.NET and ASP.NET providers.

bitwise: Will there be an upgrade path for existing users of Oxygene?

mh: Yes. Users who own an active subscription to Oxygene will automatically see that subscription "upgraded" to Delphi Prism when it is ready to ship. Customers with older versions or expired subscriptions will also have the chance to upgrade through us at the current pricing model for Oxygene. Once customers have received Prism, they will become Embarcadero customers, and any future upgrades or renewals will be handled through Embarcadero.

Delphi .NET

bitwise: Does Delphi Prism completely replace and supersede previous versions of Delphi .NET or will the two products coexist?

mh: Prism completely supersedes Delphi for .NET. Of course the existing Delphi for .NET 2007 will not stop working for customers who own RAD Studio 2007, but as I understand it there will be no "2009" version of Delphi for .NET. Of course the exact details will be in the announcement from Embarcadero.

bitwise: One of the ‘selling points’ of Delphi .NET was that it was broadly compatible with Delphi For Win32. It even had a .NET version of the Delphi Visual Component Library to let people migrate Win32 projects reasonably easily. Will there be any way for Delphi users to migrate existing projects to Delphi Prism?

mh: We are working on two (unofficial) tools that will help people migrate their code from Delphi (be it .NET or Win32) to Prism, where sensible. The first is the before-mentioned Oxidizer, which, at this stage, is a command line utility that you can run over your existing code to adjust it for language differences between Delphi and Prism. This isn’t magically going to make your code all ready to compile and work for .NET, but it will take care of a range of small (but time-consuming to fix) language idiosyncrasies - such as the = vs := differences mentioned above. it should turn your code into something that’s 99% "valid" Prism code. Oxidizer is something we’re developing in-house at RemObjects.

Then there’s ShineOn, which is an open source project started by several Oxygene users, to bring a basic implementation of the RTL and non-visual, non-database VCL to Prism. Think of things like IntToStr, TStringList, etc - all of these have equivalents in .NET, but converting to those can be a time-consuming process. ShineOn’s goal is to produce a base set of the most commonly used functions and classes (in some cases implemented from scratch, in others as wrappers around existing ,NET classes or methods), so that basic Delphi code that uses the RTL and parts of the VCL can work.

ShineOn is not something we recommend that people develop against or use as a runtime library - the .NET Framework Library has plenty of functionality to cover everything ShineOn does, and usually better or more efficient; it’s really just meant as a migration aide.

bitwise: Why was it decided that Oxygene would be a better product for .NET than an updated version of the current Delphi For .NET?

mh: This is probably best, or most honestly, answered by someone from the CodeGear side. Basically, I think it boils down to two main factors: on the one hand was the language itself: the original intent with Delphi for .NET was to provide a language that focused mainly on compatibility with the existing Win32 version. While that might have been a good idea, what, 5 years back, in turned out that this focus on compatibility ended up weighing both versions of the language down - Delphi for .NET could not evolve to make use of the latest and greatest .NET features without worrying about staying compatible to its Win32 cousin, and vice versa.

On the other hand, there was the IDE. As .NET evolves, Microsoft is adding more and more elements to the tool chain that are made to work best (or even exclusively) with Visual Studio. Reproducing these in CodeGear’s Delphi IDE, ‘Galileo’, would have meant a lot of work either to glue in existing tools or to reinvent the wheel altogether. For example, while I don’t have any inside info on this, I’d assume that WinForms support was dropped in Delphi for .NET 2007 mainly because of the tremendous amount of work it would have taken to integrate the new form designer into the IDE - it simply would not have been worth it. And moving forward, the WPF and Silverlight designers are even more tightly coupled with the IDE. Properly supporting WPF inside Galileo would most likely have meant writing a fresh designer from the ground up.

So the logical choice was to look at integrating into the Visual Studio IDE. But that’s not a simple task, either, as I can tell you from our own experience. From the outside, it must look as if the biggest part of Oxygene is the compiler, with integration into VS being the "icing on the cake". Resources-wise, this could not be further from the truth; by far the largest amount of work done on Oxygene (and now Prism) goes into integrating with various aspects of the IDE. Visual Studio provides a great extensibility API, but it is (or was) largely undocumented. For example, we (literally) wrote the book on using the Form Designer from a third party language. No-one had done it before, and it took a lot of trial and error (and help from the great folks on the Visual Studio team and Microsoft) to get it working.

So even integrating the existing Delphi for .NET compiler into Visual Studio would not have been a simple thing to do.

I think what we’ve ended up with is, in fact, a pretty sweet solution for everybody: The Delphi team at CodeGear can concentrate on driving the native development side forward, like with the just-released Unicode support and the upcoming 64-bit version of Delphi, while our team here at RemObjects continues, as we always have, to drive Prism. The two teams will be working together to coordinate future language evolution - where it makes sense - without either of the languages being "held back" by the other.

The result is that Delphi/Win32 and Delphi Prism are (and will remain) different languages and tools, each of them focusing on the strengths they need for their particular platform - native/VCL on the one hand, and .NET and the FCL (Framework Class Library) on the other - while at the same time being similar enough that switching between them is easy, so that Delphi developers will feel right at home in Prism (and vice versa).

bitwise: Are there any plans to have some version of the VCL (Delphi’s ‘Visual Component Library’) for Visual Studio?

mh: There are not plans, currently, to reproduce functionality of the visual or database parts of VCL, nor will there be a "VCL.NET" with accompanying form designers for Prism; the goal is for new GUI development to be done using WinForms or WPF - both of which Prism supports, of course.

But With C#, Who Needs Prism?

bitwise: People who program .NET in Visual Studio largely do so using C#. Given the fact that C# is by far the most widely used .NET language, why would anyone want to use Pascal instead?

mh: I think there are two big reasons why people would choose Prism over C#:

The first reason would be that they prefer the Pascal language, and its well-readable design over C-based languages. For me personally, that’s a big factor. I’m pretty fluent in C#, of course, and we use it for some of our projects (mostly for legacy reasons), but given the choice, I just find it more comfortable to write and work with Object Pascal code. It’s more readable, it’s less ambiguous (although, granted, C# is a big step forward on that front from C and C++), and I like its structure.

I realize that’s the sort of thing that people either will care about or won’t, but is not going to make a vast number of C# users who don’t come from a Delphi/Pascal background switch.

The second, and, in my opinion, more important reason, are language features. First, it’s important to realize that feature wise, Prism is a complete superset of C#. There’s literally not a single thing you can do in C# that you cannot do just as well in Prism. So you’re not giving anything up - that’s important. But then, Prism adds a whole slew of new features on top which make it a joy to work with.

For example, just recently introduced in Oxygene, there are "future" types, "async" blocks and expressions and "parallel" loops; these make it really easy to write applications that scale out to systems with many CPUs or cores. This is something that will get tremendously important in the near future, as CPU speeds no longer increase, but instead systems get more (and sometimes slower) CPUs. Microsoft is adding infrastructure to leverage this in .NET 4.0, called the "Parallel Framework", or "PFX". The PFX is really just a bunch of classes and functions you can call, but for Prism we went the extra mile to integrate these concepts into the language. So for example, you can now declare a variable to be of a "future" type - which means it will behave very much like a normal variable of a given type, except for one thing: it’s value will be calculated at some point in the future, possibly on a different thread.

So you can write, for instance:

What we’re doing here is calculating the sum of a binary tree, by basically calculating the left and right side, and then adding everything together. But rather then first calculating Left and waiting for that to return, we do that asynchronously. As a result, leftSum is not a plain Int32, but a future Int32 - meaning a variable that, at some point in the future, will contain the result of our calculation. The hope is that - given enough CPU cores - this value will be calculated by some other thread, while we’re busy calculating rightSum.

The important part is the last statement. As you can see, we’re using leftSum as part of a longer expression here. The takeaway point is that, even though it’s a future, you can work with leftSum as if it were a plain integer. You can do math on it, you could call it’s member methods, etc. The compiler and the runtime will of course take care that at the time you need the value of leftSum, it will be calculated.

This stuff is going to make it really easy to write multi-threaded code, without having to think about manually creating threads, synchronizing them, getting data back, and all that. At the same time, the compiler and the runtime make sure that the code scales well, so as things get more complicated it will not go and spawn off a thousand threads that just waste resources - while at the same time if your application is running on a 8, 16 or 64-core system, a few years down the road, it will seamlessly be able to leverage all of those cores.

I apologize for going into so much detail on this particular, but I think it provides a great example of how proper language support for a concept can really drive productivity. And futures are just one of many language features that set Prism apart from C#. We have the Class Contracts I mentioned earlier. We have support for nullable types that goes way beyond what you can do in C# - like futures, nullable types in Prism work seamlessly as if they were plain value types - you can use them in expressions (which then turn nullable themselves) and call their members. And you can use the Colon : operator to access nullable types (and reference types), in a nil-safe fashion.

Also, many standard language features that do exist in C# have subtle improvements in Prism. For example, when implementing iterators, you can "yield" off an entire sub-sequence in a single call; for..each loops allow you to access the index of the current iteration or to filter down to only running the loop for members of a given type (think looping a list of all Controls on your form, and performing an action for Buttons only).

So I think we have a pretty good story for language benefits. For people coming from Delphi/Win32 or even Delphi.NET, there will be a sheer breathtaking range of new things they can now do, on the language level. But even for seasoned C# and VB developers, who already are accustomed to all the benefits of the .NET framework itself, there should be a lot to like in Prism.

And we’re not even touching on non-language features here, such as our exhaustive support for building apps for Mac OS X or Linux, using Mono.

The CodeGear Connection

bitwise: CodeGear now has development products hosted by a variety of incompatible IDEs - Delphi and C++ live inside the ‘RAD Studio’ environment; JBuilder and 3rdRail are based on Eclipse; Delphi For PHP has its own standalone environment and now Delphi Prism is integrated into Visual Studio. That sounds as though the products are becoming very fragmented. Are there any plans to achieve some degree of integration? If so, how?

mh: I cannot speak much for CodeGear here, of course, but at least for Delphi Prism, I can say that there are currently no plans being that "back" into the Galileo IDE used by Delphi/Win32 and C++Builder. I think the decision to go with Visual Studio (in addition to being historical because Oxygene has always been integrated into Visual Studio, so that support was "just there") was very much a pragmatic one - all the new infrastructure provided by Microsoft lives inside Visual Studio - whether it’s the WinForms designer, WPF support, and so on.

Having Prism inside the Galileo IDE would mean a constant effort of reinventing the wheel, just to keep up with technologies that come "for free" in Visual Studio. We already do that to a degree with the compiler itself, of course - but there’s a clear benefit there: to be able to use the technologies with the Pascal based language (and, where possible, to integrate them better with the language than C#).

For the IDE, I personally don’t see much benefit. Sure, it’s a matter of preference, and some people may prefer the layout of Galileo while others prefer Visual Studio but, in the end, both are pretty decent IDEs that are comfortable to work with, and there is, in my view, not enough distinction to warrant the substantial amount of resources required for such an endeavor.

Mind you, this does not mean Prism is wed to Visual Studio alone. We are working on integration into MonoDevelop for Linux and Mac developers (which is already working in house and is held back mainly for license concerns, but is something we want to focus on getting resolved in the coming year) and I personally would love to see the Oxygene compiler integrated into Apple’s Xcode IDE.

bitwise: What does Embarcadero get from this deal? They are primarily a database company so I’m guessing that Delphi Prism may be seen as part of a bigger plan to integrate their database tools into Visual Studio. Can you explain how Delphi Prism will do this and will it offer any benefits over and above, say, standard Microsoft technologies such as C# and SQLServer?

mh: I’m not sure if I’m the right person to answer this, but let me try. Since the acquisition of CodeGear from Borland earlier this year, database tools are really just one of two ranges of products, with the development tools, including Delphi and C++Builder and JBuilder being a significant addition to their portfolio. As such, I look at Delphi Prism more as an extension of the developer line of products (i.e. the "CodeGear" products).

That said, just like Delphi/Win32 and C++Builder, Delphi Prism contains database features, developed by CodeGear, that round off the core language product. Basically there will be the Blackfish SQL database engine (which also ships with Delphi/Win32), dbExpress connectivity for ADO.NET, and client support for DataSnap, enabling Prism clients to talk to Delphi/Win32 Midas/DataSnap servers - both of which make it possible to access non-Microsoft data sources.

Of course we at RemObjects will also keep supporting Prism with our own product suite, such as providing a full multi-tier database access solution using Data Abstract for .NET, and we hope Prism users will find Data Abstract something worth looking at, if they do any kind of database access.

bitwise: When was it decided that Prism would be the future face of Delphi on .NET? Was this part of a long term CodeGear strategy or was a decision of the Embarcadero management following their acquisition of CodeGear?

mh: It’s not really my place to say (or speculate) who at CodeGear or Embarcadero made this decision, or if it was influenced by the acquisition. Michael (Swindell, formerly with Borland/CodeGear, now Embarcadero’s Vice President of Product Management) and I first started seriously talking about synergies between our two companies and the direction we both were heading with in regards to our compilers in late spring or early summer of this year. Things evolved pretty rapidly from that. This was certainly before the acquisition by Embarcadero was completed, but after it was already in progress.

bitwise: Now that Oxygene has become Delphi Prism, does that mean it is wholly ’owned’ by CodeGear? Are future developments determined by CodeGear or by RemObjects? And who will do the actual hands on coding of Prism?

mh: RemObjects Software is licensing its Oxygene technology to Embarcadero for exclusive use as part of Prism. Everything that was and is being developed on our side is still owned by RemObjects Software, and we continue developing and controlling it.

Of course we’ll be working closely with the teams at CodeGear, in particular the Delphi/Win32 compiler team, when it comes to future directions for the Oxygene language (and, come to think of it, the Delphi language). It’s very likely that for future language features, we’d consider implementing them in ways that could work for both Prism and Delphi/Win32, and that some of the features currently in Prism might make it into Delphi/Win32, as well (for example, I know internal versions of the Delphi compiler now support the "method" keyword, and I wouldn’t be surprised to see that become available in Commodore or some future version of Delphi).

bitwise: But haven’t you really lost hold of the reins of this project now? Won’t you have to move Prism in the direction that is dictated by the overall Embarcadero product strategy rather than in the direction that might be best for Oxygene/Prism if it were guided purely by its own requirements?

mh: In principle, the compiler team at RemObjects remains in control of where we go with the language (and the product in general); we have pretty strict guidelines and beliefs as to how the language should evolve, what kind of things we would or would not add to the syntax, and future development of the Oxygene compiler will stay true to what we’ve established over the past four years. I believe we owe that to our users - present and future.

The good news is that this is precisely the reason why the guys at Embarcadero approached us; they liked what we’ve been doing with Oxygene over the past years, and they want us to continue doing what we do best. The goal here (from both our and Embarcadero’s side) is not to forcefully transform Oxygene into Delphi for .NET.

Background

RemObjects Software was founded in summer of 2002. Its first product was RemObjects SDK 1.0 for Delphi, the company’s remoting solution which is now in it’s 5th version. In late 2003 RemObjects expanded its product portfolio to add Data Abstract for Delphi, a multi-tier database framework built on top of the SDK.

In 2004, Carlo Kok, who would eventually become Chief Compiler Architect for Oxygene, joined the company. Initial development was begun on for Oxygene (which was then named "Chrome") based on Carlo’s experience from writing the widely used Pascal Script scripting engine.

Chrome 1.0 was released in mid-2005, providing support for .NET 1.1 and .NET 2.0, which was still in beta at the time - making Chrome the first shipping language that supported features such as generics. It was followed up by Chrome 1.5 when .NET 2.0 shipped in November 2005.

In the Summer 2007, RemObjects released Chrome ’Joyride’ which added official support for .NET 3.0 and 3.5. Chrome once again was the first language to ship release level support for new language features supported by that runtime - most importantly Sequences and Queries (aka LINQ).

Development continued and in May of 2008 Oxygene 3.0 was released, dropping the "Chrome" name. Oxygene once again brought major language enhancements, including extensive support for concurrency and parallel programming as part of the language syntax. In October 2008, CodeGear’s Delphi Prism was announced - in effect, the latest version of Chrome/Oxygene.

CodeGear is a division of Embarcadero Technologies. It specialises in development tools such as JBuilder (Java), 3rdRail (Ruby On Rails), Delphi (Pascal) and Delphi For PHP.

CodeGear was previously owned by Borland Software corporation which began to seek a buyer for the development group (later to become ‘CodeGear’) in February 2006, finally selling CodeGear to Embarcadero in May 2008.

Delphi is the name of CodeGear’s best known product/language/IDE. The ‘traditional’ version of Delphi comprises the Object Pascal language implemented in its own visual design and coding environment. The launch of Delphi For PHP in 2007 somewhat confused the ‘Delphi’ nomenclature by applying it to a product using a non-Pascal language (PHP) hosted in a different IDE from the Pascal-based Delphi. The latest addition to the Delphi range, Delphi Prism, retains the Object Pascal language (a variant which, however, is not identical to previous versions of Delphi) but hosted within Microsoft’s Visual Studio.

Useful Links
 RemObjects: http://www.remobjects.com
 CodeGear: http://www.codegear.com
 Embarcadero: http://www.embarcadero.com