Herb Sutter's Blog, page 9

Trip report: Winter ISO C++ standards meeting (Prague)

A few minutes ago, the ISO C++ committee completed its final meeting of C++20 in Prague, Czech Republic. Our host, Avast Software, arranged for spacious and high-quality facilities for our six-day meeting from Monday through Saturday. The extra space was welcome, because we had a new record of 252 attendees. We currently have 23 active subgroups, and met in nine parallel tracks all week long; some groups ran all week, and others ran for a few days or a part of a day, depending on their workloads.

See also the Reddit trip report, which was collaboratively edited by many committee members and has lots of excellent detail. You can find a brief summary of ISO procedures here.

[image error] ISO C++ committee in Prague, on the occasion of completing C++20 (February 2020)



C++20 is done!

Per our published C++20 schedule, we finished technical work on C++20 at this meeting. No features were added or removed, we just handled fit-and-finish issues including addressing all of the 378 national body comments we received in last summer’s international comment ballot (Committee Draft, or CD). The next step is that the final document will be sent out for its international approval ballot (Draft International Standard, or DIS) and will be published later this year.

In addition to C++20 work, we also had time to make progress on a number of post-C++20 proposals, including continued work on contracts, networking, executors, reflection, compile-time programming, pattern matching, and much more. We also discussed ABI stability and took polls that said we are definitely not willing to guarantee pure ABI stability forever, and we are ready to consider proposals (especially ones that enable performance improvements) even if they may require an ABI break or migration on some platforms for affected types and functions, but that we aren’t ready to take a broad ABI break across the entire standard library. This is an important and meaningful decision, and an engraved invitation for proposal authors to bring proposals (and to bring back previously rejected ones) for their “wish lists” of such potentially-affected features, as soon as our next meeting this June. I’m looking forward very much to seeing how this can spur further C++ standard library innovation for C++23.

Speaking of C++23…

C++23 schedule and priorities

For the first time, we formally adopted a schedule and a planned set of feature priorities for the next round of standardization, C++23, right at its outset.

The schedule for C++23 reaffirms that we’ll use the same meeting and feature-freeze deadline schedule that we used for C++20. Note that this means we are “only” two years away from the feature freeze of the next standard! Two years has a way of going by really quickly – “warning: objects in the schedule are closer than they appear.”

The priorities for C++23’s feature set are to focus our work on the following, emphasizing upgrades to the standard library:

“Finishing C++20” with standard library modules and library support for coroutines. This will let programmers use the standard library via  modules, and easily use coroutines with futures and other common types right “out of the box” (today some additional coding or a helper library is required).Adding executors and the networking library that relies on executors.

On the language side, we will prioritize progressing the following as quickly as possible, for C++23 “if possible” but we’ll know better in a year or two whether they are likely to make it for 23 or not:

Reflection, including introspection to query the program, compile-time programming to manipulate the results, and generation to inject new entities into the program.[1]Pattern matching, which also progressed at this meeting with continued feedback on proposals.Contracts, which we spent another half-day on in SG21 on Friday afternoon.

As a second priority, the wording groups will also prioritize bug fixing higher than in the past, to pay down technical debt faster.

There will also be plenty of work on other features, so do expect C++23 to contain other work too. The purpose of setting these priorities is to mainly to say that at any given meeting we are not going to spend a lot of time working on other proposals until we have made as much progress as possible on the above ones first, that’s all. This way at each meeting we will give these proposals’ authors the maximum help and direction we can, so they can get as much further work done in the gap until the next meeting.

Finally, note that “priorities” doesn’t mean “commitments.” Prioritizing these features is not a commitment that they’ll all be done in time for C++23, though we hope that most of them may likely be. Watch the next two years’ trip reports and you’ll get a good sense of how we’re making progress against these priorities.

Wrapping up

Thank you again to the 252 experts who attended this final meeting of C++20, and the many more who participate in standardization through their national bodies!

But we’re not slowing down… in less than four months we’ll be meeting again in Varna, Bulgaria, for the first meeting to start adopting features for C++23. I look forward to seeing many of you there. Thank you again to them and to everyone reading this for your interest and support for C++ and its standardization.

Notes

[1] For those who are interested in my metaclasses proposal, this is ~98% of metaclasses – as soon as this reflection work lands, the only thing that will be left for me to propose to complete metaclasses is to add a declaration syntax like class(M) as “just” a minor syntactic sugar for invoking a consteval function that takes a meta::info reflection of the class body as input.

Last night’s talk video is online: Quantifying C++’s accidental complexity, and what we really can do about it

The ISO C++ committee is here in Prague this week to finish C++20, and the meeting hosts Avast Software also arranged a great C++ meetup last night where over 300 people came out to see Bjarne Stroustrup, Tony Van Eerd, and me give talks. The videos are already online, see below — they’re really high quality, and it was a great event. Thank you again to everyone who came out to see us! You made us feel very welcome in your beautiful city and your enthusiasm for C++ is contagious (in a good way!).

Mine was a brand-new talk with material I’ve never presented on-camera before. (I gave a beta version at ACCU Autumn last November in Belfast.) I’m really excited about this upcoming work that I’m planning to bring to the committee in the near future, and I hope you enjoy it.

Thanks again to Hana Dusíková for her hard work organizing this meetup and this entire week-long record-shattering C++ standards meeting, by far the largest in history with 250 attendees. But there’s no rest for the competent — she still has to chair SG7 (reflection and compile-time programming) all day tomorrow. :) I’ll be there!

Last night's talk video is online: Quantifying C++'s accidental complexity, and what we really can do about it

The ISO C++ committee is here in Prague this week to finish C++20, and the meeting hosts Avast Software also arranged a great C++ meetup last night where over 300 people came out to see Bjarne Stroustrup, Tony Van Eerd, and me give talks. The videos are already online, see below — they’re really high quality, and it was a great event. Thank you again to everyone who came out to see us! You made us feel very welcome in your beautiful city and your enthusiasm for C++ is contagious (in a good way!).

Mine was a brand-new talk with material I’ve never presented on-camera before. (I gave a beta version at ACCU Autumn last November in Belfast.) I’m really excited about this upcoming work that I’m planning to bring to the committee in the near future, and I hope you enjoy it.

Thanks again to Hana Dusíková for her hard work organizing this meetup and this entire week-long record-shattering C++ standards meeting, by far the largest in history with 250 attendees. But there’s no rest for the competent — she still has to chair SG7 (reflection and compile-time programming) all day tomorrow. :) I’ll be there!

Trip report: Autumn ISO C++ standards meeting (Belfast)

A few minutes ago, the ISO C++ committee completed its autumn meeting in Belfast, Northern Ireland, hosted with thanks by clearpool.io, Archer-Yates, Microsoft, C++ Alliance, MCS Group, Instil, and the Standard C++ Foundation. As usual, we met for six days Monday through Saturday, and we had about 200 attendees. We now have 23 active subgroups, most of which met in nine parallel tracks all week long; some groups ran all week, and others ran for a few days or a part of a day, depending on their workloads.

See also the Reddit trip report, which was collaboratively edited by many committee members and has lots of excellent detail. You can find a brief summary of ISO procedures here.

C++20 is on schedule to be finalized in February

Per our official
C++20 schedule, at our previous meeting in July
we reached feature-freeze for C++20 and sent out the C++20 draft for its
international comment ballot (Committee Draft, or CD) which ran over the summer
and generated 378 comments from national bodies.

At this meeting and the next one (February in Prague), our main job was to work through these comments as well as other fit-and-finish work for C++20. To make sure we were in good shape to finish in Prague, our goal was to make sure we resolved at least half the national body comments at this meeting. Thanks to a lot of hard work across all the subgroups, and especially the subgroup chairs who leveraged our ability to do work in parallel in our nine tracks and domain-specific subgroups, this week we resolved 73% of the national body comments, and made good progress on most of the rest. Here’s a snapshot of national body comment status, breaking out the number that we were able to close even before the end of the week, and the number of CWG (core language) and LWG (standard library) comments whose final resolutions we adopted today:

[image error]

This means we are
in good shape to ship the final text of the C++20 standard at high quality and on
time, at the end of the next meeting in February in Prague.

Because we are in
feature freeze for C++20, no new major proposals were added into C++20 at this
meeting, though we did adopt a few minor design fixes. Most of the changes made
at this meeting were bug-fix level improvements, mostly to the “wording implementation
details” to make sure features were specified correctly and clearly in the
formal specification wording to implement the approved design.

Other progress

In addition to C++20 work, we also had time to make progress
on a number of post-C++20 proposals, including:

the new SG21 (Contracts) study group’s
first meeting;the newly reopened SG4 (Networking) study
group including an evening session on networking security;an evening session on executors;further progress on reflection and compile-time
programming proposals;progress on pattern matching in the main language
evolution design group;and much more.

Thank you again to the approximately 200 experts who
attended this meeting, and the many more who participate in standardization
through their national bodies! Our next step is to finish the final text of C++20
three months from now in February (Prague, Czech Republic) and then send final
C++20 out for its approval ballot.

GotW-ish Solution: The ‘clonable’ pattern

This is the solution to GotW-ish: The ‘clonable’ pattern.

In summary, a distinguished C++ ISO C++ committee expert emailed me to ask:

[To avoid slicing], for each derived class, [I could] write something like

class D: public B {
public:
shared_ptr clone() const {
return make_shared(*this); // not make_shared
}
// ...
};

and then I can write

shared_ptr b1 = /* as before */;
shared_ptr b2 = b1->clone();

and b2 will now point to a shared_ptr that is bound to an object with the same dynamic type as *b1.

However, this technique requires me to insert a member function into every class derived from B, with ugly bugs resulting from failure to do so.

So my question is whether there some way of accomplishing this automatically that I’ve missed?

Let’s take a look.

JG Question

Describe as many approaches as you can think of that could let us semi- or fully-automate this pattern, over just writing it by hand every time as recommended in C++ Core Guidelines #C.130. What are each approach’s advantages and drawbacks?

There are two basic approaches in today’s C++: the Curiously Recurring Template Pattern (a.k.a. "CRTP"), and macros (a.k.a. "ick").

But first let’s consider a class of alternatives that is similar, even though it doesn’t answer this specific question or achieve the basic goal.

Nonintrusive solutions

There are nonintrusive solutions such as using type erasure, which don’t require the class to actually have a clone function. One example currently in the standardization proposal pipeline is P0201: A polymorphic value-type for C++. P0201 leads to code like this:

// The class hierarchy is unaffected

class B {
};

class C : public B {
};

class D : public C {
};

// Wrappers enable writing code that's similar to the question...

polymorphic_value b1(D()); // similar to the target use case
polymorphic_value b2 = poly;

The nonintrusive approaches are interesting too, but they don’t satisfy this particular question about how to automate the intrusive clone pattern. They also generally don’t satisfy the original motivation of the question which was to prevent slicing, because with nonintrusive approaches users can still create objects of the types directly and slice them:

D d;
B b = d; // oops, still works

Only an intrusive solution can make the copy constructor nonpublic or suppressed as part of automating the clone pattern, and all of the intrusive solutions can be extended to do this, with varying degrees of robustness and usability.

So, how can we automate the pattern in the question?

CRTP: The Curiously Recurring Template Pattern

Since C++98, the main recommended method is to use a variation of CRTP, the Curiously Recurring Template Pattern. The idea is that we instantiate a base class with our own type, and the base class provides the boilerplate we want. CRTP leads to code like this (live example — note that all the live examples use reflection to show the code that gets generated):

// User code: using the library to write our own types (many times)

class B : public clonable_base {
};

class C : public clonable {
};

class D : public clonable {
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation typically looks something like this:

// Library code: implementing the CRTP helpers (once)

template
class clonable_base {
public:
virtual std::unique_ptr clone() const {
return std::make_unique(static_cast(*this));
}
};

template
class clonable : public Intermediate {
public:
std::unique_ptr clone() const override {
return std::make_unique(static_cast(*this));
}
};

Advantages include:

It’s standard C++: Works on all compilers.
It semi-automates the pattern.
It’s extensible: It can be directly extended to require nonpublic copying.

Drawbacks include:

It’s incomplete and repetitive: It requires cooperation from the code that uses it to supply the right types. It also violates the DRY principle (don’t repeat yourself). If we have to repeat the types, we can get them wrong, and I did make that mistake while writing the samples.
It makes it harder to diagnose mistakes: If the supplied types are wrong, the error messages can be subtle. For example, as I was writing the live example, sometimes I wrote the template arguments incorrectly (because cut-and-paste), and it took me longer than I’d like to admit to diagnose the bug because the error message was related to the static_cast downcast inside the clonable implementation which wasn’t the root cause.

Macros

And there are, well, macros. They lead to code like this (live example):

// User code: using the macros to write our own types (many times)

class B {
CLONABLE_BASE(B);
};

class C : public B {
CLONABLE(B);
};

class D : public C {
CLONABLE(B);
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation typically looks something like this:

// Library code: implementing the macros (once)

#define CLONABLE_BASE(Base) \
virtual std::unique_ptr clone() const { \
return std::unique_ptr(new Base(*this)); \
}

#define CLONABLE(Base) \
std::unique_ptr clone() const override { \
using Self = std::remove_cv_t>; \
return std::unique_ptr(new Self(*this)); \
}

Advantages include:

It’s standard C++: Works on all compilers.
It semi-automates the pattern: Though less so than CRTP did.
It’s extensible: It can be directly extended to require nonpublic copying.
It’s easier than CRTP to diagnose mistakes, if you have a modern compiler: If the supplied types are wrong, the error messages are more obvious, at least with a compiler that has good diagnostics for macros.

Drawbacks include:

It’s brittle: Macros are outside the language and can also alter other code in the same file. We hates macroses. Sneaky little macroses. Wicked. Tricksy. False.
It’s incomplete and repetitive: Like CRTP, we have to supply information and repeat things, but a little less than with CRTP.

Summary so far

You can find more examples and variations of these proposed by a number of people on the original post’s comments and on the Reddit thread.

Both CRTP and macros have drawbacks. And perhaps the most fundamental is this point from the original question (emphasis added):

However, [writing clone manually] requires me to insert a member function into every class derived from B, with ugly bugs resulting from failure to do so.

Can we do better?

Guru Question

Show a working Godbolt.org link that shows how class authors can write as close as possible to this code with the minimum possible additional boilerplate code:

class B {
};

class C : public B {
};

class D : public C {
};

and that still permits the class’ users to write exactly the following:

shared_ptr b1 = make_shared();
shared_ptr b2 = b1->clone();

Reflection and metaclasses: Basic "starter" solution

Future compile-time reflection will give us an opportunity to do better. The following is based on the active reflection proposals currently in the standardization proposal pipeline, and the syntactic sugar of writing a compile-time consteval metaclass function I am proposing in P0707. Note that draft C++20 already contains part of the first round of reflection-related work to land in the standard: consteval functions that are guaranteed to run at compile time, which came from the reflection work and are designed specifically to be used to manipulate reflection information.

The idea is that we use reflection to actually look at the class and compute and generate what we need. Three common things it lets us do are to express:

Requirements: We can check for mistakes in the users’ code, and report them with clean and readable compile-time diagnostics.
Defaults: We can apply defaults, such as to make member functions public by default.
Generated functions: We can generate functions, such as clone.

Let’s start with a simple direct example that does just answers the immediate question, and leads to code like this live example):

// User code: using the library to write our own types (many times)

class(clonable) B {
};

class(clonable) C : public B {
};

class(clonable) D : public C {
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation is a compile-time consteval function that takes the reflection of the class and inspects it:

consteval void clonable(meta::info source) {
using namespace meta;

// 1. Repeat bases and members

for (auto mem : base_spec_range(source)) -> mem;
for (auto mem : member_range(source)) -> mem;

// 2. Now apply the clonable-specific default/requirements/generations:

auto clone_type = type_of(source); // if no base has a clone() we'll use our own type
bool base_has_clone = false; // remember whether we found a base clone already

// For each base class...
for (auto mem : base_spec_range(source)) {
// Compute clone() return type: Traverse this base class's member
// functions to find any clone() and remember its return type.
// If more than one is found, make sure the return types agree.
for (auto base_mem : member_fn_range(mem)) {
if (strcmp(name_of(base_mem), "clone") == 0) {
compiler.require(!base_has_clone || clone_type == return_type_of(base_mem),
"incompatible clone() types found: if more than one base class introduces "
"a clone() function, they must have the same return type");
clone_type = return_type_of(base_mem);
base_has_clone = true;
}
}
}

// Apply generated function: provide polymorphic clone() function using computed clone_type
if (base_has_clone) { // then inject a virtual overrider
-> __fragment struct Z {
typename(clone_type) clone() const override {
return std::unique_ptr(new Z(*this)); // invoke nonpublic copy ctor
}
};
}
else { // else inject a new virtual function
-> __fragment struct Z {
virtual std::unique_ptr clone() const {
return std::unique_ptr(new Z(*this)); // invoke nonpublic copy ctor
}
};
}
};

Advantages include:

It fully automates the pattern.
It’s extensible: It can be directly extended to require nonpublic copying. (See next section.)
It’s complete and nonrepetitive: The code that uses clonable only has to say that one word. It doesn’t have to supply the right types or repeat names; reflection lets the metafunction discover and compute exactly what it needs, accurately every time.
It’s easy to diagnose mistakes: We can’t make the mistakes we made with CRTP and macros, plus we get as many additional new high-quality diagnostics we might want. In this example, we already get a clear compile-time error if we create a class hierarchy that introduces clone() twice with two different types.

Drawbacks include:

It’s not yet standard C++: The reflection proposals are progressing not but yet ready to be adopted.

But wait… all of the solutions so far are flawed

It turns out that by focusing on clone and showing empty-class examples, we have missed a set of usability and correctness problems. Fortunately, we will solve those too in just a moment.

Consider this slightly more complete example of the above code to show what it’s like to write a non-empty class, and a print test function that lets us make sure we really are doing a deep clone:

class(clonable) B {
public:
virtual void print() const { std::cout << "B"; }
private:
int bdata;
};

class(clonable) C : public B {
public:
void print() const override { std::cout << "C"; }
private:
int cdata;
};

class(clonable) D : public C {
public:
void print() const override { std::cout << "D"; }
private:
int ddata;
};

This "works" fine. But did you notice it has pitfalls?

Take a moment to think about it: If you encountered this code in a code review, would you approve it?

OK, for starters, all of these classes are polymorphic, but all of them have public non-virtual destructors and public copy constructors and copy assignment operators. That’s not good. Remember one of the problems of a nonintrusive solution was that it doesn’t actually prevent slicing because you can still write this:

D d;
B b = d; // oops, still works

So what we should actually be writing using all of the solutions so far is something like this:

class(clonable) B {
public:
virtual void print() const { std::cout << "B"; }
virtual ~B() noexcept { }
B() = default;
protected:
B(const B &) = default;
B& operator=(const B&) = delete;
private:
int bdata;
};

class(clonable) C : public B {
public:
void print() const override { std::cout << "C"; }
~C() noexcept override { }
C() = default;
protected:
C(const C &) = default;
C& operator=(const C&) = delete;
private:
int cdata;
};

class(clonable) D : public C {
public:
void print() const override { std::cout << "D"; }
~D() noexcept override { }
D() = default;
protected:
D(const D &) = default;
D& operator=(const D&) = delete;
private:
int ddata;
};

That’s a lot of boilerplate.

In fact, it turns out that even though the original question was about the boilerplate code of the clone function, most of the boilerplate is in other functions assumed and needed by clone pattern that weren’t even mentioned in the original question, but come up as soon as you try to use the proposed patterns in even simple real code.

Metaclasses: Fuller "real" solution

Fortunately, as I hinted above, we can do even better. The metaclass function can take care of all of this for us:

Apply default accessibilities and qualifiers: Make base classes and member functions public by default, data members private by default, and the destructor virtual by default.
Apply requirements: Check and enforce that a polymorphic type doesn’t have a public copy/move constructor, doesn’t have assignment operators, and that the destructor is either public and virtual or protected and nonvirtual. Note that these are accurate compile-time errors, the best kind.
Generate functions: Generate a public virtual destructor if the user doesn’t provide one. Generate a protected copy constructor if the user doesn’t provide one. Generate a default constructor if all bases and members are default constructible.

Now the same user code is:

Simple and clean. As far as I can tell, it literally could not be significantly shorter — we have encapsulated a whole set of opt-in defaults, requirements, and generated functions under the single word "clonable" library name that a class author can opt into by uttering that single Word of Power.
Correct by default.
Great error messages if the user writes a mistake.

Live example

class(clonable) B {
virtual void print() const { std::cout << "B"; }
int bdata;
};

class(clonable) C : B {
void print() const override { std::cout << "C"; }
int cdata;
};

class(clonable) D : C {
void print() const override { std::cout << "D"; }
int ddata;
};

That’s it. (And, I’ll add: This is "simplifying C++.")

How did we do it?

In my consteval library, I added the following polymorphic metaclass function, which is invoked by clonable (i.e., a clonable is-a polymorphic). I made it a separate function for just the usual good code factoring reasons: polymorphic offers nicely reusable behavior even for non-clonable types. Here is the code, in addition to the above cloneable which adds the computed clone at the end — and remember, we only need to write the following library code once, and then class authors can enjoy the above simplicity forever:

// Library code: implementing the metaclass functions (once)

consteval void polymorphic(meta::info source) {
using namespace meta;

// For each base class...
bool base_has_virtual_dtor = false;
for (auto mem : base_spec_range(source)) {

// Remember whether we found a virtual destructor in a base class
for (auto base_mem : member_fn_range(mem))
if (is_destructor(base_mem) && is_virtual(base_mem)) {
base_has_virtual_dtor = true;
break;
}

// Apply default: base classes are public by default
if (has_default_access(mem))
make_public(mem);

// And inject it
-> mem;
}

// For each data member...
for (auto mem : data_member_range(source)) {

// Apply default: data is private by default
if (has_default_access(mem))
make_private(mem);

// Apply requirement: and the programmer must not have made it explicitly public
compiler.require(!is_public(mem),
"polymorphic classes' data members must be nonpublic");

// And inject it
-> mem;
}

// Remember whether the user declared these SMFs we will otherwise generate
bool has_dtor = false;
bool has_default_ctor = false;
bool has_copy_ctor = false;

// For each member function...
for (auto mem : member_fn_range(source)) {
has_default_ctor |= is_default_constructor(mem);

// If this is a copy or move constructor...
if ((has_copy_ctor |= is_copy_constructor(mem)) || is_move_constructor(mem)) {
// Apply default: copy/move construction is protected by default in polymorphic types
if (has_default_access(mem))
make_protected(mem);

// Apply requirement: and the programmer must not have made it explicitly public
compiler.require(!is_public(mem),
"polymorphic classes' copy/move constructors must be nonpublic");
}

// Apply requirement: polymorphic types must not have assignment
compiler.require(!is_copy_assignment_operator(mem) && !is_move_assignment_operator(mem),
"polymorphic classes must not have assignment operators");

// Apply default: other functions are public by default
if (has_default_access(mem))
make_public(mem);

// Apply requirement: polymorphic class destructors must be
// either public and virtual, or protected and nonvirtual
if (is_destructor(mem)) {
has_dtor = true;
compiler.require((is_protected(mem) && !is_virtual(mem)) ||
(is_public(mem) && is_virtual(mem)),
"polymorphic classes' destructors must be public and virtual, or protected and nonvirtual");
}

// And inject it
-> mem;
}

// Apply generated function: provide default for destructor if the user did not
if (!has_dtor) {
if (base_has_virtual_dtor)
-> __fragment class Z { public: ~Z() noexcept override { } };
else
-> __fragment class Z { public: virtual ~Z() noexcept { } };
}

// Apply generated function: provide defaults for constructors if the user did not
if (!has_default_ctor)
-> __fragment class Z { public: Z() =default; };
if (!has_copy_ctor)
-> __fragment class Z { protected: Z(const Z&) =default; };

}

GotW-ish Solution: The ‘clonable’ patter

This is the solution to GotW-ish: The ‘clonable’ pattern.

In summary, a distinguished C++ ISO C++ committee expert emailed me to ask:

[To avoid slicing], for each derived class, [I could] write something like

class D: public B {
public:
shared_ptr clone() const {
return make_shared(*this); // not make_shared
}
// ...
};

and then I can write

shared_ptr b1 = /* as before */;
shared_ptr b2 = b1->clone();

and b2 will now point to a shared_ptr that is bound to an object with the same dynamic type as *b1.

However, this technique requires me to insert a member function into every class derived from B, with ugly bugs resulting from failure to do so.

So my question is whether there some way of accomplishing this automatically that I’ve missed?

Let’s take a look.

JG Question

Describe as many approaches as you can think of that could let us semi- or fully-automate this pattern, over just writing it by hand every time as recommended in C++ Core Guidelines #C.130. What are each approach’s advantages and drawbacks?

There are two basic approaches in today’s C++: the Curiously Recurring Template Pattern (a.k.a. "CRTP"), and macros (a.k.a. "ick").

But first let’s consider a class of alternatives that is similar, even though it doesn’t answer this specific question or achieve the basic goal.

Nonintrusive solutions

There are nonintrusive solutions such as using type erasure, which don’t require the class to actually have a clone function. One example currently in the standardization proposal pipeline is P0201: A polymorphic value-type for C++. P0201 leads to code like this:

// The class hierarchy is unaffected

class B {
};

class C : public B {
};

class D : public C {
};

// Wrappers enable writing code that's similar to the question...

polymorphic_value b1(D()); // similar to the target use case
polymorphic_value b2 = poly;

The nonintrusive approaches are interesting too, but they don’t satisfy this particular question about how to automate the intrusive clone pattern. They also generally don’t satisfy the original motivation of the question which was to prevent slicing, because with nonintrusive approaches users can still create objects of the types directly and slice them:

D d;
B b = d; // oops, still works

Only an intrusive solution can make the copy constructor nonpublic or suppressed as part of automating the clone pattern, and all of the intrusive solutions can be extended to do this, with varying degrees of robustness and usability.

So, how can we automate the pattern in the question?

CRTP: The Curiously Recurring Template Pattern

Since C++98, the main recommended method is to use a variation of CRTP, the Curiously Recurring Template Pattern. The idea is that we instantiate a base class with our own type, and the base class provides the boilerplate we want. CRTP leads to code like this (live example — note that all the live examples use reflection to show the code that gets generated):

// User code: using the library to write our own types (many times)

class B : public clonable_base {
};

class C : public clonable {
};

class D : public clonable {
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation typically looks something like this:

// Library code: implementing the CRTP helpers (once)

template
class clonable_base {
public:
virtual std::unique_ptr clone() const {
return std::make_unique(static_cast(*this));
}
};

template
class clonable : public Intermediate {
public:
std::unique_ptr clone() const override {
return std::make_unique(static_cast(*this));
}
};

Advantages include:

It’s standard C++: Works on all compilers.
It semi-automates the pattern.
It’s extensible: It can be directly extended to require nonpublic copying.

Drawbacks include:

It’s incomplete and repetitive: It requires cooperation from the code that uses it to supply the right types. It also violates the DRY principle (don’t repeat yourself). If we have to repeat the types, we can get them wrong, and I did make that mistake while writing the samples.
It makes it harder to diagnose mistakes: If the supplied types are wrong, the error messages can be subtle. For example, as I was writing the live example, sometimes I wrote the template arguments incorrectly (because cut-and-paste), and it took me longer than I’d like to admit to diagnose the bug because the error message was related to the static_cast downcast inside the clonable implementation which wasn’t the root cause.

Macros

And there are, well, macros. They lead to code like this (live example):

// User code: using the macros to write our own types (many times)

class B {
CLONABLE_BASE(B);
};

class C : public B {
CLONABLE(B,C);
};

class D : public C {
CLONABLE(B,D);
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation typically looks something like this:

// Library code: implementing the macros (once)

#define CLONABLE_BASE(Base) \
virtual std::unique_ptr clone() const { \
return std::unique_ptr(new Base(*this)); \
}

#define CLONABLE(Base, Self) \
std::unique_ptr clone() const override { \
return std::unique_ptr(new Self(*this)); \
}

Advantages include:

It’s standard C++: Works on all compilers.
It semi-automates the pattern: Though less so than CRTP did.
It’s extensible: It can be directly extended to require nonpublic copying.
It’s easier than CRTP to diagnose mistakes, if you have a modern compiler: If the supplied types are wrong, the error messages are more obvious, at least with a compiler that has good diagnostics for macros.

Drawbacks include:

It’s brittle: Macros are outside the language and can also alter other code in the same file. We hates macroses. Sneaky little macroses. Wicked. Tricksy. False.
It’s incomplete and repetitive: Like CRTP, we have to supply information and repeat things.

Summary so far

You can find more examples and variations of these proposed by a number of people on the original post’s comments and on the Reddit thread.

Both CRTP and macros have drawbacks. And perhaps the most fundamental is this point from the original question (emphasis added):

However, [writing clone manually] requires me to insert a member function into every class derived from B, with ugly bugs resulting from failure to do so.

Can we do better?

Guru Question

Show a working Godbolt.org link that shows how class authors can write as close as possible to this code with the minimum possible additional boilerplate code:

class B {
};

class C : public B {
};

class D : public C {
};

and that still permits the class’ users to write exactly the following:

shared_ptr b1 = make_shared();
shared_ptr b2 = b1->clone();

Reflection and metaclasses: Basic "starter" solution

Future compile-time reflection will give us an opportunity to do better. The following is based on the active reflection proposals currently in the standardization proposal pipeline, and the syntactic sugar of writing a compile-time consteval metaclass function I am proposing in P0707. Note that draft C++20 already contains part of the first round of reflection-related work to land in the standard: consteval functions that are guaranteed to run at compile time, which came from the reflection work and are designed specifically to be used to manipulate reflection information.

The idea is that we use reflection to actually look at the class and compute and generate what we need. Three common things it lets us do are to express:

Requirements: We can check for mistakes in the users’ code, and report them with clean and readable compile-time diagnostics.
Defaults: We can apply defaults, such as to make member functions public by default.
Generated functions: We can generate functions, such as clone.

Let’s start with a simple direct example that does just answers the immediate question, and leads to code like this live example):

// User code: using the library to write our own types (many times)

class(clonable) B {
};

class(clonable) C : public B {
};

class(clonable) D : public C {
};

shared_ptr b1 = make_shared(); // target use case works
shared_ptr b2 = b1->clone();

The implementation is a compile-time consteval function that takes the reflection of the class and inspects it:

consteval void clonable(meta::info source) {
using namespace meta;

// 1. Repeat bases and members

for (auto mem : base_spec_range(source)) -> mem;
for (auto mem : member_range(source)) -> mem;

// 2. Now apply the clonable-specific default/requirements/generations:

auto clone_type = type_of(source); // if no base has a clone() we'll use our own type
bool base_has_clone = false; // remember whether we found a base clone already

// For each base class...
for (auto mem : base_spec_range(source)) {
// Compute clone() return type: Traverse this base class's member
// functions to find any clone() and remember its return type.
// If more than one is found, make sure the return types agree.
for (auto base_mem : member_fn_range(mem)) {
if (strcmp(name_of(base_mem), "clone") == 0) {
meta::compiler.require(!base_has_clone || clone_type == type_of(base_mem),
"incompatible clone() types found: if more than one base class introduces a clone() function, "
"they must have the same return type");
clone_type = return_type_of(base_mem);
base_has_clone = true;
}
}
}

// Apply generated function: provide polymorphic clone() function using computed clone_type
if (base_has_clone) { // then inject a virtual overrider
-> __fragment struct Z {
typename(clone_type) clone() const override {
return std::unique_ptr(new Z(*this)); // invoke nonpublic copy ctor
}
};
}
else { // else inject a new virtual function
-> __fragment struct Z {
virtual std::unique_ptr clone() const {
return std::unique_ptr(new Z(*this)); // invoke nonpublic copy ctor
}
};
}
};

Advantages include:

It fully automates the pattern.
It’s extensible: It can be directly extended to require nonpublic copying. (See next section.)
It’s complete and nonrepetitive: The code that uses clonable only has to say that one word. It doesn’t have to supply the right types or repeat names; reflection lets the metafunction discover and compute exactly what it needs, accurately every time.
It’s easy to diagnose mistakes: We can’t make the mistakes we made with CRTP and macros, plus we get as many additional new high-quality diagnostics we might want. In this example, we already get a clear compile-time error if we create a class hierarchy that introduces clone() twice with two different types.

Drawbacks include:

It’s not yet standard C++: The reflection proposals are progressing not but yet ready to be adopted.

But wait… all of the solutions so far are flawed

It turns out that by focusing on clone and showing empty-class examples, we have missed a set of usability and correctness problems. Fortunately, we will solve those too in just a moment.

Consider this slightly more complete example of the above code to show what it’s like to write a non-empty class, and a print test function that lets us make sure we really are doing a deep clone:

class(clonable) B {
public:
virtual void print() const { std::cout << "B"; }
private:
int bdata;
};

class(clonable) C : public B {
public:
void print() const override { std::cout << "C"; }
private:
int cdata;
};

class(clonable) D : public C {
public:
void print() const override { std::cout << "D"; }
private:
int ddata;
};

This "works" fine. But did you notice it has pitfalls?

Take a moment to think about it: If you encountered this code in a code review, would you approve it?

OK, for starters, all of these classes are polymorphic, but all of them have public non-virtual destructors and public copy constructors and copy assignment operators. That’s not good. Remember one of the problems of a nonintrusive solution was that it doesn’t actually prevent slicing because you can still write this:

D d;
B b = d; // oops, still works

So what we should actually be writing using all of the solutions so far is something like this:

class(clonable) B {
public:
virtual void print() const { std::cout << "B"; }
virtual ~B() noexcept { }
B() = default;
protected:
B(const B &) = default;
B& operator=(const B&) = delete;
private:
int bdata;
};

class(clonable) C : public B {
public:
void print() const override { std::cout << "C"; }
~C() noexcept override { }
C() = default;
protected:
C(const C &) = default;
C& operator=(const C&) = delete;
private:
int cdata;
};

class(clonable) D : public C {
public:
void print() const override { std::cout << "D"; }
~D() noexcept override { }
D() = default;
protected:
D(const D &) = default;
D& operator=(const D&) = delete;
private:
int ddata;
};

That’s a lot of boilerplate.

In fact, it turns out that even though the original question was about the boilerplate code of the clone function, most of the boilerplate is in other functions assumed and needed by clone pattern that weren’t even mentioned in the original question, but come up as soon as you try to use the proposed patterns in even simple real code.

Metaclasses: Fuller "real" solution

Fortunately, as I hinted above, we can do even better. The metaclass function can take care of all of this for us:

Apply default accessibilities: Make base classes and member functions public by default, data members private by default, and the destructor virtual by default.
Apply requirements: Check and enforce that a polymorphic type doesn’t have a public copy/move constructor, doesn’t have assignment operators, and that the destructor is either public and virtual or protected and nonvirtual. Note that these are accurate compile-time errors, the best kind.
Generate functions: Generate a public virtual destructor if the user doesn’t provide one. Generate a protected copy constructor if the user doesn’t provide one. Generate a default constructor if all bases and members are default constructible.

Now the same user code is:

Simple and clean. As far as I can tell, it literally could not be significantly shorter — we have encapsulated a whole set of opt-in defaults, requirements, and generated functions under the single word "clonable" library name that a class author can opt into by uttering that single Word of Power.
Correct by default.
Great error messages if the user writes a mistake.

Live example

class(clonable) B {
virtual void print() const { std::cout << "B"; }
int bdata;
};

class(clonable) C : B {
void print() const override { std::cout << "C"; }
int cdata;
};

class(clonable) D : C {
void print() const override { std::cout << "D"; }
int ddata;
};

That’s it. (And, I’ll add: This is "simplifying C++.")

How did we do it?

In my consteval library, I added the following polymorphic metaclass function, which is invoked by clonable (i.e., a clonable is-a polymorphic). I made it a separate function for just the usual good code factoring reasons: polymorphic offers nicely reusable behavior even for non-clonable types. Here is the code, in addition to the above cloneable which adds the computed clone at the end — and remember, we only need to write the following library code once, and then class authors can enjoy the above simplicity forever:

// Library code: implementing the metaclass functions (once)

consteval void polymorphic(meta::info source) {
using namespace meta;

// For each base class...
bool base_has_virtual_dtor = false;
for (auto mem : base_spec_range(source)) {

// Remember whether we found a virtual destructor in a base class
for (auto base_mem : member_fn_range(mem))
if (is_destructor(base_mem) && is_virtual(base_mem)) {
base_has_virtual_dtor = true;
break;
}

// Apply default: base classes are public by default
if (has_default_access(mem))
make_public(mem);

// And inject it
-> mem;
}

// For each data member...
for (auto mem : data_member_range(source)) {

// Apply default: data is private by default
if (has_default_access(mem))
make_private(mem);

// Apply requirement: and the programmer must not have made it explicitly public
meta::compiler.require(!is_public(mem),
"polymorphic classes' data members must be nonpublic");

// And inject it
-> mem;
}

// Remember whether the user declared these SMFs we will otherwise generate
bool has_dtor = false;
bool has_default_ctor = false;
bool has_copy_ctor = false;

// For each member function...
for (auto mem : member_fn_range(source)) {
has_default_ctor |= is_default_constructor(mem);

// If this is a copy or move constructor...
if ((has_copy_ctor |= is_copy_constructor(mem)) || is_move_constructor(mem)) {
// Apply default: copy/move construction is private by default in polymorphic types
if (has_default_access(mem))
make_protected(mem);

// Apply requirement: and the programmer must not have made it explicitly non-private
meta::compiler.require(!is_public(mem),
"polymorphic classes' copy/move constructors must be nonpublic");
}

// Apply requirement: polymorphic types must not have assignment
meta::compiler.require(!is_copy_assignment_operator(mem) && !is_move_assignment_operator(mem),
"polymorphic classes must not have assignment operators");

// Apply default: other functions are public by default
if (has_default_access(mem))
make_public(mem);

// Apply requirement: polymorphic class destructors must be
// either public and virtual, or protected and nonvirtual
if (is_destructor(mem)) {
has_dtor = true;
meta::compiler.require ((is_protected(mem) && !is_virtual(mem)) ||
(is_public(mem) && is_virtual(mem)),
"polymorphic classes' destructors must be public and virtual, or protected and nonvirtual");
}

// And inject it
-> mem;
}

// Apply generated function: provide default for destructor if the user did not
if (!has_dtor) {
if (base_has_virtual_dtor)
-> __fragment class Z { public: ~Z() noexcept override { } };
else
-> __fragment class Z { public: virtual ~Z() noexcept { } };
}

// Apply generated function: provide defaults for constructors if the user did not
if (!has_default_ctor)
-> __fragment class Z { public: Z() =default; };
if (!has_copy_ctor)
-> __fragment class Z { protected: Z(const Z&) =default; };

}

My CppCon 2019 talk video is online

My CppCon 2019 talk is now available on YouTube, and the slides will soon be available here. I hope you enjoy it.

GotW-ish: The ‘clonable’ pattern

Yesterday, I received this question from a distinguished C++ expert who served on the ISO C++ committee for many years. The email poses a decades-old question that still has the classic roll-your-own answer in C++ Core Guidelines #C.130, and basically asks whether we’ve made significant progress toward automating this pattern in modern C++ compared to what we had back in the 1990s and 2000s.

Before I present my own answer, I thought I would share just the question and give all of you readers the opportunity to propose your own candidate answers first — kind of retro GotW-style, except that I didn’t write the question myself. Here is the email, unedited except to fix one typo…

[image error]

In trying to wrap my mind around all the new stuff for C++20, I see that there is one idea that has been around for quite a while but I still don’t see implemented. I’m wondering whether I’m missing something obvious, or whether it’s still not there.

Suppose B is a base class and I have a shared_ptr that might point to a B or to any class that might be derived from B. Assuming that B and all of its derived classes are CopyConstructible, I would like to create a new shared_ptr bound to a newly created object that is a copy of the object to which my original shared_ptr points. I cannot write this:

shared_ptr b1 = /* whatever */;
shared_ptr b2 = make_shared(*b1); //NO

because that will bind b2 to a copy of the B part of *b1.

I think I can make this work by putting a member function in B along the following lines:

class B {
shared_ptr clone() const { return make_shared(*this); }
// ...
};

and then for each derived class, write something like

class D: public B {
public:
shared_ptr clone() const {
return make_shared(*this); // not make_shared
}
// ...
};

and then I can write

shared_ptr b1 = /* as before */;
shared_ptr b2 = b1->clone();

and b2 will now point to a shared_ptr that is bound to an object with the same dynamic type as *b1.

However, this technique requires me to insert a member function into every class derived from B, with ugly bugs resulting from failure to do so.

So my question is whether there some way of accomplishing this automatically that I’ve missed?

[image error]

So here’s your challenge:

JG Question

1. Describe as many approaches as you can think of that could let us semi- or fully-automate this pattern, over just writing it by hand every time as recommended in C++ Core Guidelines #C.130. What are each approach’s advantages and drawbacks?

Guru Question

2. Show a working Godbolt.org link that shows how class authors can write as close as possible to this code with the minimum possible additional boilerplate code:

class B {
};

class C : public B {
};

class D : public C {
};

and that still permits the class’ users to write exactly the following:

shared_ptr b1 = make_shared();
shared_ptr b2 = b1->clone();

Hints: Feel free to use any/all mechanism that some production or experimental compiler supports (if it’s somewhere on Godbolt, it’s legal to try), including but not limited to:

the very old (e.g., macros),C++11/14/17 features (e.g., decltype),draft C++20 features (e.g., concepts),proposals currently before the committee for future C++ (e.g., detection idiom),

or any combination thereof.

Q&A: Does string::data() return a pointer valid for size() elements, or capacity() elements?

A reader asked:

In C++17, for std::string::data(), is the returned buffer valid for the range [data(), data() + size()), or is it valid for [data(), data + capacity())?

The latter seems more intuitive and what I think most people would expect reserve() to create given the non-const version of data() since C++17.

… and then helpfully included the answer, but in fairness clearly they were wondering whether cppreference.com was correct:

Relevant quote from cppreference.com: … “Returns a pointer to the underlying array serving as character storage. The pointer is such that the range [data(); data() + size()) is valid and the values in it correspond to the values stored in the string.”

Yes, cppreference.com is correct. Here’s the quote from the current draft standard:

2 A specialization of basic_string is a contiguous container (22.2.1).3 In all cases, [data(), data() + size()] is a valid range, data() + size() points at an object with value charT() (a “null terminator”), and size()

Regarding this potential alternative:

or is it valid for [data(), data + capacity())?

No, that would be strange, because it would mean intentionally supporting reading uninitialized characters in any extra raw memory at the end of the string’s current memory block.

Note that the first part of the above quote from the standard hints at the consistency issue: A string is a container, and we want containers to be consistent. We certainly wouldn’t want vector::data() to behave that way and let callers see raw memory with unconstructed objects.

The latter [… is …] what I think most people would expect reserve() to create

c/reserve/resize/ and I’ll agree :)

Any container’s size()/resize() is about the data you stored in it and that it’s holding for you. Any container’s capacity()/reserve() is about the underlying raw memory buffer just to let you help the container optimize its raw memory management, but it isn’t intended to give you access to the allocated-but-unused memory.

My favorite work-week of 2019

I just can’t get enough of this short video, combining interviews shot at last year’s CppCon with shots of our new “home” location that we’ll be enjoying for the first time two weeks from now.

Please enjoy it — this is an excellent representation of what CppCon is like.

Those of you who know me personally know that I’m a homebody, and that two of my least-liked things are airplanes and hotel rooms. But even so, I wish it was already the afternoon of 15 September, and that I was sitting in a cramped plane seat on the way to Colorado.

If you’ve been waffling on whether to register for this year, you know what to do. I’m looking forward to seeing many of you there soon.