After a lifetime of writing C and C++ compilers, Walter Bright decided to go one step beyond with the creation of D. In this exclusive interview, he tells Huw Collingbourne and Dermot Hogan about the past, present and future of the D programming language…

See also: Dermot Hogan's Overview of D : 'C Done Right...?'

Huw: First of all, quite simply, why did you decide to create D? What was really so wrong with C++ anyway? And why didn't Java measure up?

Walter Bright: I really like C++. I was instantly attracted to it back in 1986, and created the first native C++ compiler (Zortech C++). Over the years since, C++ has grown by layering on needed features without disturbing backwards compatibility. Often this results in some very strange rules and surprising corner cases. In the meantime, the rest of the programming world certainly has grown, too, with features and paradigms with proven productivity.

At some point, it just makes sense to look at where we are, and try to design a language that gets there in a straight line, rather than try to be compatible with obsolete decisions made 20 or 30 years ago. (For example, the text #inclusion system was a good idea 30 years ago, but has long passed any shred of sensibility in today's world. For another, the design of C++ is rooted in its ASCII past, making support of modern requirements like UTF difficult.)

C++ has reached a certain level of maturity now, its period of rapid evolution is over. So the time is right to see what can be done with a redesign. Java isn't it, for the simple reason that it isn't powerful enough. For example, a garbage collector cannot be written in Java.

Huw: How big a challenge was it to write Zorland/Zortech C++? If you were doing it these days, what would you do differently?

Walter Bright: It's a huge challenge for anyone to write a professional quality compiler, and C++ is the hardest of all languages to implement. I wouldn't do it these days, as high quality free C++ compilers exist, but back in 1986 or so, there was plenty of opportunity in implementing one.

Huw: What is D's 'unique selling point'? Given all the other languages available, why should a programmer choose D?

Walter Bright: D appeals to programmers who are interested in writing high performance code, want a C++ style language, but need a language that is much easier to master with support for modern techniques like automatic memory management, modules, UTF, etc. It isn't unusual for a D program to have 30% less source code than the equivalent C++, yet run at the same speed or faster. It's simply faster to develop code in D, and faster to get it debugged. Other languages tend to adopt either a different look and feel from C++, or omit necessary power features like pointers. I don't believe any of them have gone as far as D has in developing a well-rounded set of core features.

Dermot: Can you use D for COM interfacing? It would be nice to have something between C# and C++ that didn't use the casting dirty trick that C# uses to query IUnkown.

Walter Bright: Yes, D directly supports building COM classes.

Huw: Is D open source? I see on your web site that you say "the front end" of D is open source. I can't see a mention of the GNU licence anywhere. What are the licence terms and just how open is D?

Walter Bright: The front end of the D compiler is dual license, GPL and Artistic. David Friedman has taken the front end and grafted it on to gcc, and the result is gdc, the gnu D compiler, which is fully GPL. The runtime library is not GPL, but is freely redistributable and usable for any purpose. Many parts of it are even explicitly public domain.

Huw: In the license agreement I see mentions of Zortech and Symantec. What is the involvement of those companies with D?

Walter Bright: Zortech was bought by Symantec and no longer exists. The language in the license agreement is there at the request of Symantec to make clear that Symantec has absolutely no involvement with D or Digital Mars products. I used to work on compilers for Symantec.

Huw: Why is there no IDE (at least on Windows)?

Walter Bright: There are several people in the D community who are working on IDEs. D has a number of characteristics which make writing a good IDE for it easier (like it can be parsed without needing a symbol table).

Dermot: I'm curious about your comment on parsing D. C++ is notoriously difficult to parse, and I'm still feeling a bit raw from writing a LL(k) parser for Ruby, so I've got strong feelings on the subject. Is D much easier to parse than C++? If so, is the language 'tighter' than C++ ?(i.e. less ambiguous and prone to 'blowing your leg off' to re-use a famous quotation).

Walter Bright: C++ isn't hard to parse because it requires arbitrary lookahead; that's an easy problem to solve. It's hard to parse because a particular sequence of tokens can produce multiple totally different parse trees, depending on what some of the symbols are declared as. Even worse, many of those symbols aren't known at parse time, because they aren't declared yet. So it has to be parsed into some indeterminate state that gets "fixed" later.

This issue makes it impossible to correctly parse C++ code without building most of a C++ compiler front end, including all the hard stuff. It's further complicated by the preprocessor - unless the parser has full implementation of the preprocessor, and includes all the #includes and command line #defines, it cannot successfully parse an arbitrary C++ source file. This cripples the utility of source formatters, analyzers, syntax directed editors, documentation generators, source code instrumentation tools, etc. To avoid these problems, the D language was designed with the following rules in mind:

1) no text preprocessor
2) no command line switch can change the syntax
3) no imported file can change the syntax
4) a source file can be tokenized without reference to syntax or semantics
5) a source file can be syntactically analyzed without needing any semantic information

This should make it easy to build tools that work with D source code. Parsing D still requires arbitrary lookahead, but this is an easy problem to deal with. The D lexer and parser are GPL open source, so the proof of that is plain to see.

Dermot: Again, relating to a comment you've made below on programming errors, YACC LALR parsers are notorious for saying things like 'there's an error in here somewhere, but I'm not sure where'. Ruby is particularly bad in this respect - are you saying that D uses a better bag of tricks than YACC?

Walter Bright: I've never used YACC; all the lexers and parsers I've built are hand made. Producing good error diagnostics, and doing good error recovery so parsing can continue without generating a blizzard of spurious messages is much more art than science.

From a theoretical perspective, however, being able to generate a good diagnostic requires that there be redundancy in the syntax. The redundancy is used to make a guess at what was intended, and the more redundancy, the more likely that guess will be correct. It's like the English language - if we misspell a wrod now and then, or if a word missing, the redundancy enables us to correctly guess the meaning. If there is no redundancy in a language, then any random sequence of characters is a valid program.

A good language design must find that sweet spot between enough redundancy to be able to detect errors and issue good error diagnostics, and not so much redundancy that it becomes too wordy and tedious.

Huw: These days, a number of languages (such as PHP, Python and Ruby) offer special support for the development of web applications. How does D shape up against those languages for web development?

Walter Bright: D is very good for developing CGI apps because D has excellent support for strings, arrays, UTF, and regular expressions, as well as automatic memory management.

Huw: Recently both Ruby and D entered the TIOBE programming language 'Top 20'. Even though, syntactically, Ruby and D are very different, they do share some common features (e.g. fairly 'pure' OOP, simple syntax, garbage collection, modules and mixins, closures). Are the similarities between Ruby and D more than skin deep?

Walter Bright: There are many fascinating parallels between Ruby and D. Ruby started out as a reengineering of Perl, and Perl has a lot of parallels with C++. Ruby was started by an independent developer out of sheer love of programming languages, and so has D. Neither language has any agenda other than serving the needs of programmers - and so they've succeeded without needing massive corporate backing or budgets. Probably the most fundamental difference between the two is that Ruby lives in the interpretive, dynamic typing world, and D lives in the native compiled, static typing world.

Huw: Where is D going next? What is the next really important feature that you plan to add to the language?

Walter Bright: D is going wherever the D community wants it to go. I have a special interest in the types of features that lend themselves to better detection of programming errors and better optimization. Andrei Alexandrescu has been particularly generous with his time in helping me to understand the issues and coming up with solutions. These kinds of issues are steadily becoming more and more critical to professional software development. D already has many of these things, like contracts, built-in unit testing, guaranteed initialization, scope guards, etc., but much more can be done.

Huw: You mentioned the parallels between Ruby and D. How closely do you follow the developments of other languages?

Walter Bright: I obviously closely follow developments in C and C++. My interest in other languages is contextual - if I'm working on a better way to do X, and a certain other language does X very well, then I'll check it out. I'm also very interested in why some languages succeed, and others fail. Why did C++ succeed and Modula 2 fail? Why is Eiffel always trailing way behind? Why did Ruby leap so quickly to the fore? Do languages succeed because they have a killer feature, killer marketing, or are just well oiled machines?

Huw: Other than D itself, which modern program languages do you think are most interesting or have the greatest potential?

Walter Bright: There's no doubt about it, the one to watch is Ruby. C, C++, and Java are mature languages that have reached their potential.

See also: Dermot Hogan's Overview of D

May 2006