BP Privacy: History and Lessons Learned from Developing a Major BuddyPress Component

Coding great-quality, open source software, while often rewarding, can also be a thankless, difficult task. As many have been asking for an update on BP Privacy–also known as the BuddyPress Privacy Component–I thought I would take the time to write up an exhaustive history of the project and share some lessons learned.

It is important to state up front that there are many wonderful, helpful, supportive, knowledgeable, community-minded members in the greater WordPress community. If you are an active participant within this community, you already understand that fact.

Of course, a great community of supportive, fun-loving people does not guarantee that you will face few challenges with your WordPress or BuddyPress projects—whether that is starting and running a community, designing themes, or developing plugins.

This is the story about the challenges I have faced in bringing BP Privacy to fruition. It is just one developer's journey and, as such, should not be construed as anything more than my perspective.

I hope that those who manage to read through this entire, long article walk away with not only a better understanding of some of the difficulties BP Privacy has faced, but also a feel for how they might want to approach taking on similar open source projects in the future.

Genesis of the Idea

In the beginning, there was an idea that BuddyPress needed privacy. Well, that idea was not present at the genesis of BuddyPress as it does not offer core privacy, but the idea was hatched in the early pre-RC2 release days of the BuddyPress project by two very active community leaders—one of whom was me.

At the inception of this project, BP Privacy had two developers. That's right. I had a project partner. This partner was a key BuddyPress member and very interested in coding his first BP plugin. We teamed up on this project as we realized the complexity of the task at hand and that it would be beneficial to have a project partner.

We had a number of discussions about how we should tackle this project. I set up a subversion repository on my dedicated server for the project and gave him access. I started the long, tedious process of learning, really understanding, the inner workings of BuddyPress. After all, BP Privacy would not be a typical plugin. It had to interact with all the core BuddyPress components. It had to monitor and take control of output based on an individual's desires. We both realized that BP Privacy was going to be a major, foundational component in its own right—even though it would be a third-party plugin.

However, as weeks passed into months, my project partner's schedule did not allow him to participate. So, I told him that I was just going to get started and that he could join in at any time.

So that is the humble, less than exciting beginning of BP Privacy. It started with a two-person project team but ended up becoming a solo effort.

BP Privacy Timeline

On the BP Privacy site, I state in a blog post that this journey has been 16-months long. Of course, that was posted basically October 1, 2010. So as of the date on this post, the process is nearing 20 months. The reality, however, is that this project had its inception even earlier, almost two years ago.

Here is a blow-by-blow timetable for BP Privacy and some of the key factors and issues at each point along the way:

Project idea inception: Early April 2009. My project partner and I began discussing BP Privacy (what was at that time called BPAz or BP-Authz)

First code written: June 23, 2009. This was two months after hatching the concept. It was the point when my project partner determined his schedule would not allow him to participate. So, I started coding the project on my own.

First public beta release: December 5, 2009. Only four months and two weeks after the first code block was written, I released a very solid public beta version to the community. Note that before that public beta release, there was a small, select group of private alpha testers.

This was a very solid beta version with only a few minor bugs. It worked perfectly with BuddyPress v1.1.3, offering privacy filtering for four of BuddyPress' then core components. But the rug was about to be pulled out from underneath the project.

Codebase and platform concerns arise: January 2, 2010. As BuddyPress 1.2 was fast approaching release, it became clear that a major BP Privacy code refactoring would be required. A good portion of the previous 4 months of work would need to be reevaluated and much rewritten. As I looked at the time commitment involved, I realized I needed to try a new approach.

Not surprisingly, this approach failed. It only raised about $175 dollars. Without a big financial boost to help me focus on BP Privacy, I had to turn my attention elsewhere for awhile.

Late spring through early fall of 2010: The BuddyPress project experienced critical uncertainties in my opinion. These uncertainties made me question its long-term health. During this time, the development of BP Privacy progressively slowed down, practically grinding to a halt in late summer of 2010 as I awaited a few, final core patches I had submitted months before to be accepted.

Due to these factors, nine months passed with very little development time being invested.

Announcement of Public Release (v1.0): September 30, 2010. I was privy to some promising developments in the world of BuddyPress that gave me hope that BuddyPress might actually weather the storm. So, after almost nine months of greatly reduced activity on my part, I went out on a limb, venturing back into the BP Privacy project on a more serious level once again.

I created the BP Privacy site and made an announcement on that new site that BP Privacy would be released on November 8, 2010. This is the first officially-advertised date given for the release of the public, production-ready version (v1.0).

A few weeks later came the news for which I had been waiting. The BuddyPress community had a shot of adrenaline and renewed hope. We welcomed the announcement that Paul Gibbs and Boone Gorges had been "promoted" to core committers.

Of course, November 8, 2010 came and went. I continued working on BP Privacy as time permitted as I patiently awaited the release of BuddyPress 1.2.7 which was finally released on December 22, 2010.

BP Privacy's Future: See the end of this article.

Time Invested and Anticipated Returns

Projects of the magnitude of BP Privacy require a considerable time commitment. Whereas it is difficult to be absolutely precise, I have a pretty accurate estimate as to the number of hours I've invested in BP Privacy. My total time spent to date working on BP Privacy is 1450 hours.

What kinds of activities go into a project that would require such a time commitment? A great number of essential activities such as: emails, forum and IRC discussions, support of alpha and beta1 testers, writing and submitting core patches required to bring privacy services to BuddyPress, debating a number of these patches, studying and thoroughly understanding the inner workings of BP, keeping up to date with codebase changes in BP Trac, writing tools that were necessary in figuring out some unexpected behaviors with BuddyPress' action and filter hooks, continuous and exhaustive testing of BP Privacy, and writing detailed documentation. Of course, all of this is on top of the actual coding of the component itself which has required (so far) two major refactorings of the codebase.

What will I earn for all of this effort? Zero. Okay, I had a total of about $225 in donations to help support development ($175 as mentioned above plus $50 received before that post). I am very grateful to all who donated, to my select testers, and to everyone who offered support in other ways.

This means that I will have earned just shy of 16 cents per hour working on BP Privacy. So, the next time you question the commitment and contribution of those who actively volunteer in the open source world, remember that number. Of course, if all the additional hours of time that I've donated on the BP support forums, IRC, via email, Twitter, and Skype are included, that total would undoubtably be about half of that.

Financially, I would have been better off spending that time working at McDonalds. It is ironic that the vast majority of people who will benefit from my work will not even contribute enough for me to buy a Big Mac. By the way, I do not eat at McDonalds so please don't send coupons. In fact, I am not interested in any more donations.

Why do I have a section emphasizing the monetary aspects of BP Privacy? Because like the vast majority of people, I need to pay bills, put food on my family's table, and save for the future. How many of you can donate 1450 hours of time creating free products or services for others to use?

I am a vocal advocate of the open source model, as anyone who reads my blog and tweets would know. I have volunteered a thousand hours plus of my time answering questions on the BuddyPress community support forums, via email, in IRC, on the phone, and via Skype. None of those hours are included in my total time spent on BP Privacy. Like many active members of the community, I give in more ways than just creating plugins.

The reality for me is that this community and its open source model does not make it possible to earn even a small part of my living in a way that I prefer—coding great-quality GPLed plugins that provide needed services to others.

As I do not take on client work–I've discussed this fact with people many times before–I need another means with which to recoup some of the time I have invested in coding open source software for the community. If you really want to learn more about this point, please read this post about this issue from my perspective—and read the comments for a fascinating discussion.

Lessons Learned

Here are a few lessons learned that may help other WordPress and BuddyPress developers have a better experience with offering GPLed software to the greater community.

Work on Projects that Give You Energy, Not Sap Your Strength: By and large, I have lost more energy working on the BP Privacy project than I have gained. It has been exceedingly frustrating at times. To be honest, if this were not something desperately needed for the BuddyPress platform, I would have dropped this project a year ago.

At the time I started coding BP Privacy, I was planning on using BuddyPress as the foundation of my startup, and privacy was key to that vision. So it made sense to continue BP Privacy and then release the component to the greater community once it was ready. Had I any idea how vocal the negative minority would be as they impatiently waited for me to provide them with high-quality, free-as-in-cost software, I would have canned the community release a long time ago and just worked on it for my private use.

The Vocal, Negative Minority: It is important to realize that more likely than not, the vast majority of users will be happy about your work, or at least indifferent. Unfortunately, human nature makes us more vocal when we're displeased than when we are pleased. It is a minority of users that will be anywhere from disappointed to obsessively outraged. It will be this minority that will be most vocal. If you release your work to the community, expect to have a greater volume of "I hate you" than "I love you" feedback from your user base.

Whereas community members may appreciate your volunteer help on various support forums, and paying clients may love you, when it comes to freely-contributed plugins, don't expect the same rosy reception.

Don't Expect Donations: Based on the vast majority of all plugin developers' experience, ninety-nine percent (and realistically more) of all users will never donate to your efforts. There are many plugin developers who have written about this. Here are just a few articles to shed some light on this issue. Again, read the comments to get a more balanced perspective on this issue as there are good points on both sides:

Have you made a donation to your WordPress Plugin Developer?
Do we do enough to support WordPress Plugin Developers?
Open Source Motivations

The donation model is not broken, for the vast majority of creators, it never worked to begin with. I have tried many tactics to increase donation conversions. My plugins and appropriate blog posts all had obvious donate badges. But that has not made a difference. Donating to something that is freely available apparently also goes against human nature.

Plugin support: Unless you clearly and explicitly state that there will be zero support offered for your plugin (at a minimum that should be communicated in the readme.txt file) then it is your moral obligation to offer some level of support if you release a plugin to the community—which includes forking an existing plugin.

Therefore, expect there to be questions that must be answered, that user issues will take away time from your other projects, and possibly impact your paid work and family obligations. There will be users who claim they are having a problem with your plugin when in actuality it will be caused by something other than your plugin. No matter how hard you try to communicate that it is not an issue with your plugin, in these people's minds, you will still be the party at "fault".

It is for this reason that some plugin developers fully exercise their GPL rights. Please note that if you plan to charge for support, you should be aware of a potential issue.

Since all WordPress plugins and themes need to be licensed under then same GPL version as WordPress itself–GPL version 2–you do not technically have the freedom or right to charge support fees. That freedom and right was added later in GPL version 3. (Compare the last paragraph of Section 1 of GPL version 2 to the last paragraph of Section 4 of GPL version 3).

Therefore, if you are planning on charging for support, you are operating outside the freedoms of the GPL version 2. You would be wise to seek legal counsel.

With Plugin Popularity Comes Possible Trouble: I do not envy plugin developers with high download counts. I know that that means one of two things: they are at the first stage of the plugin's support lifecycle where they are spending an inordinate amount of their time supporting the plugin (probably for free), or they will soon be entering the final stage of the plugin's support lifecycle where they discontinue support and future development as their time commitment to the project cannot be sustained.

It is for these two reasons that I always donate to plugin developers whose software I use and only use plugins that I am sufficiently interested in as I expect that one day I will have to maintain them myself.

Plugin development should not be a popularity contest, he or she who has the highest plugin download count often does not win. Do not release a plugin for praise and glory. That rarely happens. What really happens is the more popular your plugin becomes, the greater the potential for you to lose control over your time. This can lead to a rather unpleasant, overall experience with your project.

Alpha & Beta testing: If you have limited time to work on your project, then it is best to make the alpha and maybe first beta version private releases. Provide copies only to those people who you believe will genuinely test it and provide you with useful feedback. It is better to have a small, focused group of testers than a horde of quasi-interested and knowledgeable testers.

The exception to this lesson would be if you have a team of developers who can share the responsibilities of managing a public alpha and beta test. But, if you are a solo developer, you could be in for a world of hurt if you set your pre-release software free for any and all to test.

Bugs will continue to be found even after you've released the first public version. You have to go no farther than WordPress or BuddyPress Trac to see how many bugs still exist in those stable, public products. That is the nature of all software. No matter how mature a software product, there will always be bugs, some of them serious.

Develop on a Developer-stable Version: Although BuddyPress v1.0 was the first official public release deemed suitable for general use, it was far from stable from a developer's standpoint. This is evidenced by the fact that significant changes occurred between BP 1.0 and BP 1.1 that caused developers some grief and then even greater changes occurred between BP 1.1 and BP 1.2.

In my opinion, BP 1.2 should have been then first public release. In other words, BP 1.2 is really v1.0 in my mind. Now, with BP 1.3 close at hand, I'm concerned that developers (and possibly even users) will be faced with difficult upgrade challenges. Although, Paul and Boone have been working hard to make the transition to BP 1.3 as painless as possible. So, perhaps my concerns are not valid. Whatever the reality, when the dust settles, BP 1.3 will become the first developer-stable version in my opinion.

Group or Solo effort: As should be obvious from the start of this article, you need to carefully vet your project partners. Although I had little data with which to make an honest assessment of my project partner's suitability–the BuddyPress community was very new at the time–I nevertheless made a mistake at the start of this project. I should have quietly started by myself and only asked for interested project partners once I had some code to share and knew more about the skill sets of the various BP developers with whom I associated.

Communicate Less, Not More: This may come across as a hypocritical suggestion in light of some of the communication issues BuddyPress had last year. However, you need to differentiate BuddyPress as a developer platform and community from that of developing a BP plugin. With the former, the community is what makes the project a success. With the later, only a few key people need to be kept apprised. Communication is essential to the former, whereas to the latter it is not necessary until the plugin is released.

When it comes to plugin development, it is better to surprise the community with a new release (especially the initial release) than it is to build up their expectations. Although there is a thrill with getting validation for your efforts at the start of a project, there is no way to know what challenges lie ahead and how difficult the task may be. Interest and attention in any project can quickly turn negative if there are seemingly few results to share. Blame will always go to you, whether the issues holding up your project are beyond your control or not. This is especially true for a project that is deemed very important or possibly even vital—such as BP Privacy.

Once a plugin is released to the world, then proactive communication and vigilant project management are crucial to the project's continued success. But before the public release, communicating less might actually help the project succeed as you won't be distracted by the negative vocal minority.

Promised Release Dates: As a follow up to the point above, it is best to never put a date on a release. You are working on a plugin, not the core foundation of WordPress or BuddyPress where it makes sense to have project deadlines and development freeze dates.

If you do communicate a release date, do not be overly concerned if you fail to meet it. You are generously working on GPLed software that will more than likely earn you little if any for your efforts invested. You are not beholden to anyone.

Even the BuddyPress project has difficulties meeting its promised release dates. As this article can witness, there are many factors that contribute to a missed release date. Some are beyond your control. From a community standpoint, it is best if people remain patient and remember that they are getting GPLed software that provides them many freedoms of use. The software will be released when its released.

Use of the WordPress Plugin Repository: The WP Plugin Repo is a great service to developers and the greater community. You should use this service if you are planning not to exercise your full GPL rights. However, do not use the Repo for releasing alpha, beta, or RC versions. Most users will have no clue what an alpha or beta version truly means. More importantly, most will not care. If it's on the Repo they'll expect it to work. You should make your plugins available on the Repo only when they are ready for full public release. Until then, use another service, or your own server, to make pre-release versions available to those whom you wish to have access.

When Will BP Privacy Be Released?

Over the past two months, I have been reassessing my role in this project. As you have found out from the above history, my time commitment and investment into this project have been substantial. I've decided that the time required to support and maintain this project, and the energy required to do it properly, is incompatible with me earning a semblance of a living. It has also taken too much focus away from my current startup.

This project started out as a team effort but unfortunately became a solo effort. I believe that this project needs to become a team effort again—as in a team of developers, not a team of testers.

To be clear, BP Privacy was never intended to be a core BuddyPress component—even though some of you think that was the case. I am not and have never been part of BuddyPress' core development team. I was simply an active community volunteer, support forum moderator, and plugin developer.

As most of you know, I am a staunch privacy advocate. Since my first days with the BuddyPress project, I have believed that privacy was a necessary core feature. That has yet to be realized. Perhaps part of the BP Privacy codebase can serve that purpose in the future. Although it might make more sense to refactor BuddyPress, offering true core privacy as a component.

What does this all mean?

I will be releasing the fully-functioning BP Privacy codebase over the next several days, along with a very extensive manual (35+ pages). At that point, I will end my official involvement with the project, and as such, I will not be providing any support.

The project will be in the hands of the community. It will be available for anyone to use as is, expand upon, fork, or even merge into BuddyPress core. Perhaps a group of developers will adopt BP Privacy and maintain it as a community-based project.

Because of my decision to end my official involvement with the project, I've decide to back tag the version I'll be releasing, making it a release candidate instead of a public, ready-for-production version. Therefore, it will be v1.0-RC1 instead of V1.0. It also means that I will not be placing it on the WordPress Plugin Repository per the reasons I mentioned at the end of the last section. It will be available for a short while on the BP Privacy site before that site is taken down. The link will go to some yet-to-be-determined public repository. I will also be placing the link within a BP support forum thread.

By the way, for any group of developers interested, I have registered the bp-privacy distribution name with the WP Repo. I would be more than willing to assign that over to another group, if that is possible, or at the bare minimum add other committers. But be advised that I will not be participating in the project anymore.

Once BP Privacy v1.0-RC1 is out, it will be up to each person to fully evaluate the plugin and decide for themselves whether or not to run it on a production site. Although in my exhaustive testing BP Privacy works very will under WP 3.0.4 and BP 1.2.7, you must decide for yourself the viability of its use in a production environment. Thus, please be advised, no matter what you do, you are on your own until (and if) a new group of developers takes the reigns of BP Privacy and assumes support and maintenance responsibilities.

As far as the upcoming release of BP 1.3, I have not fully tested the most recent BP trunk version in Trac. Therefore, I cannot say how much refactoring may be required. I may put some effort into that, but do not wait for me. You should take the initiative and bring it up to date on your own volition.

As far as the few people that have pre-purchased BuddyPress Privacy Support Plans, I will be refunding all the monies received over the next week. But first I will focus on getting BP Privacy out the door. I will also be refunding my two, wonderful advertising partners. Yes, your ads have been up on BP Privacy going on three months (I have only charged them for the first month), but you have not received the type of exposure that you had expected. It is only fair that you get full refunds as well.

I hope that BP Privacy finds a useful life going forward!