Metaclasses: Thoughts on Generative C++

Can you simplify C++ with minimal library extension? Herb Sutter shows how metaclasses could make this possible.

Herb recently blogged this note about a new ISO C++ proposal he and colleagues are working on, and we felt it would be of interest to Overload readers. Herb has kindly agreed to let us republish the blog post here as an article, and add the abstract from the current version of the proposal. -Ed.

I’ve been working on an experimental new C++ language feature tentatively called ‘metaclasses’ that aims to make C++ programming both more powerful and simpler. You can find out about it here:

Current proposal paper: P0707R1 [ Sutter17a ]. I hope the first ten pages give a readable motivation and overview. (The best two pages to start with are 9 and 10, which probably means I need to reorder the paper…)
Initial intro talk video: ACCU 2017 (YouTube) [ Sutter17b ]. This is the initial public presentation three months ago. Thank you to Roger Orr, Russel Winder, Julie Archer, and the other ACCU organizers for inviting me and for holding back the video until we could have the ISO C++ summer meeting in mid-July, so it could go live along with a report (herein) on the results of this feature’s first presentation to the ISO C++ committee. And special thanks to Ina and Arvid, the two audience volunteers who graciously agreed to come on-stage to participate in a live mini UX study. There’s a lot of subtle information in their nuanced reactions to the code examples; pay special attention when their responses are different or as their responses evolve.
‘Incomplete and experimental’ prototype compiler. The Clang-based prototype by Andrew Sutton is available as an online live compiler at cppx.godbolt.org, and as source at github.com/asutton/clang. It’s incomplete but can compile a number of the examples in the paper (see the paper for example code links). Thanks to Matt Godbolt for hosting it on godbolt.org!

Abstract from PO707 R1

The only way to make a language more powerful, but also make its programs simpler, is by abstraction: adding well-chosen abstractions that let programmers replace manual code patterns with saying directly what they mean. There are two major categories:

Elevate coding patterns/idioms into new abstractions built into the language. For example, in current C++, range- for lets programmers directly declare “for each” loops with compiler support and enforcement. Templates are a powerful parameterization of functions and classes, but do not enable authoring new encapsulated behavior.

(major, this paper) Provide a new abstraction authoring mechanism so programmers can write new kinds of user-defined abstractions that encapsulate behavior. In current C++, the function and the class are the two mechanisms that encapsulate user-defined behavior. In this paper, $class metaclasses enable defining categories of class es that have common defaults and generated functions, and formally expand C++’s type abstraction vocabulary beyond class / struct / union / enum .

Also, §3 includes a set of common metaclasses, and proposes that several are common enough to belong in std:: . Each subsection of §3 is equivalent to a significant “language feature” that would otherwise require its own EWG paper and be wired into the language, but here can be expressed instead as just a (usually tiny) library that can go through LEWG. For example, this paper begins by demonstrating how to implement Java/C# interface as a 10-line C++ std:: metaclass – with the same expressiveness, elegance, and efficiency of the built-in feature in such languages, where it is specified as ~20 pages of text.

Please see the above paper and video to answer “what are metaclasses and why should I care?”

If you’re the “show me code first, English later” kind of person, try the live compiler and these quick examples: interface, base_class, value (regular type), plain_struct (links are in the paper).

The rest of this article aims not to duplicate any information above, but to provide some context about the broader journey, and what I and others are attempting to accomplish.

A journey: Toward more powerful and simpler C++ programming

Phase 1: By using the existing language better

About five years ago, I started working on long-term effort toward making using C++ simpler and safer.

In the first phase, a small group of us – centered on Bjarne Stroustrup, Gabriel Dos Reis, Neil MacIntosh and Andrew Pardoe – pushed to see how far we could get with ‘C++ as it is’ plus just a few well-chosen library-only extensions, with a particular goal of improving type and memory safety. Bjarne, Neil, and I first publicly reported on this effort in the two CppCon 2015 plenary sessions ‘Writing Good C++14’ [ CppCon15a ] and ‘Writing Good C++14… By Default’ [ CppCon15b ]. The results of that work so far have manifested as the C++ Core Guidelines [ CCG ] and its support library, GSL [ GSL ], that adds a limited number of library types (e.g., span , now being standardized); and I led the Lifetime design in particular (available in the Guidelines /docs folder) which Neil and I and others continue to work on formalizing with the aim of sharing a ‘draft’ static analysis spec later this year.

One of the goals of this phase was to answer the question: “How much progress can we make toward simplifying the existing C++ language with only a few key library extensions?” The answer as I see it turned out to be: “Some solid progress, but probably not a major simplification.” And so that answer led to phase two…

Phase 2: By evolving the language

Two years ago, I started to focus specifically on exploring ways that we might evolve the C++ language itself to make C++ programming both more powerful and simpler. The only way to accomplish both of those goals at the same time is by adding abstractions that let programmers directly express their intent – to elevate comments and documentation to testable code, and elevate coding patterns and idioms into compiler-checkable declarations. The work came up with several potential candidate features where judiciously adding some power to the language could simplify code dramatically.

Of those potential candidate features, metaclasses is the first major piece I picked to propose for ISO C++. We presented it for the first time at the summer ISO C++ meeting earlier this month, and it received a warm reception. [1]

There was (rare) unanimous support for pursuing this kind of capability, but also some concern about how best to expose it and specific design change feedback the committee wants us to apply to improve the proposal. [2]

We’ll work to include in a revision for the November standards meeting as we start the multi-year process of vetting and refining the proposal. So this is good progress, but note that it (only) means encouragement to continue the experiment and see where it leads; it’s far too early to talk about potential ship vehicles.

So do expect change: The proposal is still evolving, and it in turn assumes and builds on the static reflection proposal (P0578 et al.) and the compile-time programming proposal (P0633), both of which are actively evolving in their own right. Incidentally, one of the contributions of Andrew Sutton’s prototype metaclasses compiler is that it is implementing those other proposals too(!), since the metaclasses feature needs them. The aim is to keep the latest compiler and the latest P0707 paper in sync with each other and with those related proposals, but there will doubtless be occasional drift in between syncs.

What’s next

I’ll talk about metaclasses more in my upcoming CppCon 2017 talk this September, and Andrew Sutton will also be giving two CppCon talks about metaclasses – one about implementing them in Clang, and one about using them for a real project.

This is just the beginning, and we’ll see whether it all pans out and leads somewhere, but I hope you enjoy this exploration and I look forward to talking with many of you about it at CppCon this September.

Notes

I actually brought a smaller piece from this same work to the committee at the previous meeting, the winter meeting in Kona: P0515 (consistent comparisons), which proposes adding the three-way <=> comparison operator. P0515 is only about a minor feature, and not one of the most important things that can help improve C++, so normally I wouldn’t have picked that piece to contribute first; but the committee was already continuing to actively discuss comparisons, so I cherry-picked it from my design work and contributed it since I had the design in my pocket anyway. Happily the committee liked what they saw and both EWG and LEWG accepted it, and it is now progressing well and on track to hopefully be voted into draft C++20 in the next meeting or two. Thanks to Jens Maurer and Walter Brown for the heavy lifting of writing the core language and library standardese wording, respectively, for that P0515 proposal.
The committee’s design feedback was primarily about how to wrap up the transformation code: Instead of putting it inside a new ‘meta’ class-like abstraction, how about wrapping the same code inside a compile-time function-like abstraction that takes an input meta::type parameter and returns a generated meta::type return value? This doesn’t affect the proposal’s basic engine, just the shape of its steering wheel – for example, we could change the first line of each example metaclass definition from the class-like syntax
```
    $class interface {
       constexpr {
         // … basically same code …
       }
    };
```
to the decorator-function-like syntax
```
    meta::type interface(const meta::type source) {
      // … basically same code …
    };
```
where the latter has the advantage that it’s easy to see that we’re reading one type and generating another type. Interestingly, I think this dovetails with the mini UX study in the video where most of the difficulty the UX participants seemed to encounter was in understanding the $class syntax, not the metaclass bodies and not later using the metaclasses to author new types.

But we’ll explore this and other options and validate/invalidate it with more experiments… and feel free to drop me a line (or comment on the original blog post [ Sutter17c ]) if you like one of these styles better, or perhaps another variation.