Coding Horror: Civilized Discourse Construction Kit

February 5, 2013

Occasionally, startups will ask me for advice. That's a shame, because I am a terrible person to ask for advice. The conversation usually goes something like this:

Like I said, I don't get asked for advice too often. But for what it's worth, it is serious advice. And the next question they ask always strikes fear into my heart.

This is the part where I start playing sad trombone in my head. Because all your software options for online community are, quite frankly, terrible. Stack Exchange? We only do strict, focused Q&A there and you'd have to marshal your proposal through Area 51. Get Satisfaction, UserVoice, Desk, etcetera? Sorry, customer support isn't the same as community. Mailing lists? Just awful.

Forum software? Maybe. Let's see, it's 2013, has forum software advanced at all in the last ten years?

I'm thinking no.

Forums are the dark matter of the web, the B-movies of the Internet. But they matter. To this day I regularly get excellent search results on forum pages for stuff I'm interested in. Rarely a day goes by that I don't end up on some forum, somewhere, looking for some obscure bit of information. And more often than not, I find it there.

There's an amazing depth of information on forums.

• A 12 year old girl who finds a forum community of rabid enthusiasts willing to help her rebuild a Fiero from scratch? Check.
• The most obsessive breakdown of Lego collectible minifig kits you'll find anywhere on the Internet? Check.
• Some of the most practical information on stunt kiting in the world? Check.
• The only place I could find with scarily powerful squirt gun instructions and advice? Check.
• The underlying research for a New Yorker article outing a potential serial marathon cheater? Check.

I could go on and on. As much as existing forum software is inexplicably and terrifyingly awful after all these years, it is still the ongoing basis for a huge chunk of deeply interesting information on the Internet. These communities are incredibly passionate about incredibly obscure things. They aren't afraid to let their freak flag fly, and the world is a better place for it.

At Stack Exchange, one of the tricky things we learned about Q&A is that if your goal is to have an excellent signal to noise ratio, you must suppress discussion. Stack Exchange only supports the absolute minimum amount of discussion necessary to produce great questions and great answers. That's why answers get constantly re-ordered by votes, that's why comments have limited formatting and length and only a few display, and so forth. Almost every design decision we made was informed by our desire to push discussion down, to inhibit it in every way we could. Spare us the long-winded diatribe, just answer the damn question already.

After spending four solid years thinking of discussion as the established corrupt empire, and Stack Exchange as the scrappy rebel alliance, I began to wonder – what would it feel like to change sides? What if I became a champion of random, arbitrary discussion, of the very kind that I'd spent four years designing against and constantly lecturing users on the evil of?

I already built an X-Wing; could I build a better Tie Fighter?

If you're wondering what all those sly references to Tie Fighters were about in my previous blog posts and tweets, now you know. All hail the Emperor, and by the way, what's your favorite programming food?

Today we announce the launch of Discourse, a next-generation, 100% open source discussion platform built for the next decade of the Internet.

The goal of the company we formed, Civilized Discourse Construction Kit, Inc., is exactly that – to raise the standard of civilized discourse on the Internet through seeding it with better discussion software:

• 100% open source and free to the world, now and forever.
• Feels great to use. It's fun.
• Designed for hi-resolution tablets and advanced web browsers.
• Built in moderation and governance systems that let discussion communities protect themselves from trolls, spammers, and bad actors – even without official moderators.

Our amazingly talented team has been working on Discourse for almost a year now, and although like any open source software it's never entirely done, we believe it is already a generation ahead of any other forum software we've used.

I greatly admire what WordPress did for the web; to say that we want to be the WordPress of forums is not a stretch at all. We're also serious about this eventually being a viable open-source business, in the mold of WordPress. And we're not the only people who believe in the mission: I'm proud to announce that we have initial venture capital funding from First Round, Greylock, and SV Angel. We're embarking on a five year mission to improve the fabric of the Internet, and we're just getting started. Let a million discussions bloom!

So now, when someone says to me …

I can reply without hesitation.

And hopefully, so can you.

Posted by Jeff Atwood

I'll certainly be checking this out. I agree that forum software sucks. The best I've ever found, even to this day, is not open source or even downloadable. Delphi Forums. When you need ideas for innovation... look there ;-)

I know they're completely separate ecosystems and should probably stay that way but it is tempting to think of a "Migrate to Discourse" button for certain questions on stackexchange sites like arqade that would actually be suitable for continued discussion.

The web... a new frontier. These are the voyages of the startup Discourse. Its five year mission: to explore strange new customers... to seek out new thoughts and new deliberations... to boldly go where no forum dared go before!

(When you said "five year mission", I couldn't resist.)

Ruby on Rails and Postgress? *shudders* Why, Jeff, why?

Very cool. I'll definitely take a look. As far as I'm concerned, integration will be key. Forums, as such, are *part* of the answer even in a lot of places where they suck today. Even the chatrooms connected to SE serve this purpose from time to time. I'd like to see a way for the chaos of forums to seek nice, dark, quiet corners, while the good stuff -- the "sticky posts" or whatever -- can float to the top and be promoted to something more dignified than a "post".

I'd bet you could come up with a pretty good list of requirements just by listing "forum antipatterns" that you want to break:

• The troll.


• The "haven't we done this seventeen times already?" thread.


• The "you're not following the rules for this category" thread.

Should be interesting. Thanks for the contribution.

"it is tempting to think of a "Migrate to Discourse" button for certain questions on stackexchange sites like arqade that would actually be suitable for continued discussion."

I dislike this as some actual signal might escape to the other thread, and similarly it'd encourage people to start noisy discussions where noise is less tolerated, as they know it would be a jumping off point.

You mentioned frequently finding useful information on forums when doing web searches. What does it mean for search engines that the client is entirely JavaScript (and has such high browser requirements)?

Another question: I feel like the high success of WordPress is largely due to the ubiquity of its platform, i.e. PHP/MySQL. What effect do you think the relative scarcity of affordable Ruby/Postgres hosting will do to the adoption rate of Discourse?

This isn't 2007, guys. Hosting Rails is cheap and easy now.

What I'm surprised about it that it's not .NET, given Jeff's frequent comments about how amazing it is.

Coupla things. First, "mailing list" - if you mean listservs - are NOT awful. Best forum/list I was ever on was for ultramarathoners. It was amazing - you could post a question and receive an answer from a top South African ultrunner. Of course, the group was tightly focused around a single topic - had highly respected participants - and a population of about 1000.

That said, I'll be looking at Discourse very seriously as a platform for our organization with 40,000-plus members worldwide.

And THAT said, you could have mentioned at the start that it'll be TWO YEARS before you release it. And that "open source, completely free" doesn't mean, well, free as in beer.

It feels awesome to use, I guess the only gripe I have is with the way it has been released, i.e. as a big code dump (similarly to Android, which is infamous for it). I hope to see a more collective effort in the future development of this platform.

I would love to see Persona/BrowserID support built in here.

Just found out the source for your "let a million discussions bloom:" http://en.wikipedia.org/wiki/Hundred_Flowers_Campaign

The way it looks now, you're not engaging in open-source in the same way as WordPress does.
The GPL is enough, you don't need a contributor license agreement. What it actually means is that you are keeping an exit door if you ever want to close the source again in the future: contributors agree to license you their copyright for whatever future happens.

Probably lots of open-source projects use this method, but it sure is not reassuring. Please don't compare yourself again to WordPress, though. Nuance matters.

Andrew: I'ld also like to see that. Perhaps you could add it, and submit a pull request? :)

(I would, but I'm not really a Ruby/Ember kinda guy. I'm more Node/Backbone. ;)

That's so absolutely awesome, I'm exicted as I was last time when Google Chrome came out.

20 years of suffering is over!

You have to do something about SEO. I read the article in Zite on the way home on my iPad...came home and googled "discource" and got...nothing remotely resembling the site. Had to come to codinghorror to find this article and the URL...

Also: Where in discourse is the refresh button to load new comments while you're reading? Or does it automagically update? If it does, I haven't noticed it..

I'm looking forward to this!

I've cursed forum software for years and a recognized Stackexchange and Quora as important, thoughtful, successful attempts to do something about the problem. I am glad to see that you are continuing to iterate on the problem, and even happier that it is open source.

I think though that Wan raises an excellent point about the licensing though. I can appreciate the desire to be able to re-license, but I hope you can appreciate why potential contributors would be reluctant to participate. I'll also point out that there is nothing to stop people from creating a fork right now and accepting commits without the encumbrance of the licensing agreement.

Wordpress/Automattic managed to pull-off its achievement without having a paid staff and a paid product offering in place until well after the open source project took off. So, you are in many ways starting off way ahead of where Automattic started in being able to capture value from Discourse. You could quickly loose that leading position and favored momentum if the contributor agreement drives an early fork.

Actually, the more I think about it, the more obvious it is to me that the contributor agreement is almost a moot point. You'll never be able to incorporate contributions from a fork without nullifying the flexibility of the contributor agreement, but any fork will be able to pull anything you publish to your repo.

I've been waiting for a long time for an advance in forum collaboration software. If you have done with this as you did to Q&A with the stackexchange platform, this is worth getting exicted about.

@Ckincincy You should check out Project Beehive (http://www.beehiveforum.net/) an open source project that's been running for years and was started shortly after the old Delphi forums went purely commercial. You may find it familiar.

(I'll still be checking Discourse out, of course.)

Heartily second the recommendation of Shacknews chatty. I used to post frequently at the Shack, and was even a moderator at one point. It is far and away the best internet commenting/discussion system I have ever used, and proof that threaded > flat.

Very sad to see Discourse perpetuating the fundamentally broken flat discussion model.

Wow, this came out of nowhere. Cant wait to see how people take to it.

I second Brad Westness' question about how this integrates with search engines.

Wow. I was so excited as I read, what you were doing sounds so great, I could already see using it at NE1UP.com.

Then I read the comments. Ruby?

Sigh.

looks like a less intuitive, slower version of google wave. How is this supposed to overtake forums with an unindexable front end and awkvard ui? Endless scrolling is totally inappropriate in the context. Honestly, you would've done better investing all that time to the now open sourced Wave

I originally started my Forj forum project ( http://forj.heroku.com/ ) after getting really fed up with how terrible forum software is, but I've not had the time/motivation to keep working on it lately, so I'm glad Discourse is there to hopefully give the likes of phpBB a kick into the 21st century.

Users don't go to forums because they look pretty. They go there and stay because there is action, people willing to answer your questions. (many must be unemployed cause they are online all day).
A lot of forums look like shit but they are focused on content.

Fantastic article. Very good points and awesome logo.

Look forward to seeing what you do and inspiring similar visions with our team for open discussions at http://www.studyhall.com

Best,
StudyHall Team!

thanks jef!!! exactly what alot of open source projects need

we've just moved a large forum over to google plus communities and its working pretty well

just 1 thing - please provide free hosting - say via your sub-domain ie. myforum.discourse.org - just like wordpress does at wordpress.com !!

Cheers

Justin

Jeff, I've been waiting to see what you had up your metaphorical sleeve and I can't tell you how excited I am to see this. Really excited to try it out.

That's really interesting! We've actually been working on a similar project (also open source) called Telescope:

http://telesc.pe/

It's closer to Reddit or Hacker News, but I think we have very similar goals. So I would be very interested in knowing what you think.

I'm off to try Discourse to give you more feedback and steal some ideas :)

Will it have an IMAP and NNTP interface, for those of use who prefer local clients?

Will it have true threaded discussions?

I swear, we are slowly reinventing USENET, poorly.

I read the logo as "iscourse", which I guess it's a neat name.

I hope, there'll be an easy way to migrate data from old forum software to be used with this new engine.

I had essentially the exact same idea! Cheers to you for having it and making it too!

Now I don't have to! xD

Reddit does a great job at promoting discussion with their design. Why not create a mini-reddit for your site? All but the anti-spam bit is open sourced: https://github.com/reddit/reddit

This is awesome! :)

I can't wait to play with it more. Beautiful work and congrats on the big launch!

I am no expert but the goal here is discussion. That means the discussion and content there-of is king.

So why all the extra chrome? Avatars do nothing for discussion. (Same thing with signatures but you already knew that). There is a lot of extra white spacing that doesn't make sense.

And finally discussion means back and forth. To me, that means you need a nested / threaded system. A 'flat' mode is fine for two people going back and forth but as soon as there is another subthread, the flat mode falls quickly into chaos.

I like the presentation of the threads but the threads themselves are still a mess.

I want this to succeed because I too often find myself digging through forums for information and it has to get better. But I would encourage you to see how well sites like Shacknews and Reddit get conversation going without all the chrome.

PS - Sticky threads that are used as a "FAQ" that have 100+ replies instead of a wiki page are another forum annoyance. Figure that one out and you will be the richest person alive.

I get a warm feeling when any area of stagnation gets some TLC through innovation. I think this is a ripe are for change and I'm looking forward to what Jeff and the team come out with.

My only problem with it is that the threading is abjectly terrible. When people reply to a post, it gives nothing but the poster's name, with no hint of which post or what content until you click on it. Even the first line of post would be a massive improvement and job the memory more than enough. For people who quote a whole post unmodified, I'd argue that should get the same treatment: Collapse to the first line, allow click to full post. Only edited quote should be shown.

I've come to appreciate the way my email client collapses all lines from a previous poster, but if you're doing more than skimming, that's not good. This is a point where everyone is going to want different things; I like things collapsed, others will like to see point-by-point arguments and rebuttals. So it goes; design sucks.

Great job! I've always thinked that web forums needed a technological review! I will absolutely recommend this to my customers!

@Sean - Avatars are something users demand, not developers. Tell users they can't have something they want, and watch your killer app be ignored. All forum software that has avatars has ways for cantankerous individual users to disable seeing avatars (and images, markup, and other "chrome"), which is far preferable to telling the majority that want it to go screw off.

Why can't I log in with OpenID? I thought you were supposed to be a big fan?

There's two aspects to what you're doing here.

The first is the software itself. Discussion software may have peaked in the 1980s with dial-up BBS forums, which had many useful features and even more importantly integration of those features into a discernible process, and much of this was lost in the 1990s transition to web software. Remember Matt's Scripts?

The second is community management. Having watched Facebook, Digg, Reddit and Hacker News, my conclusion is that most people imitate the successful acts of others from the outside-in. That is, someone has a reason to make a post; others see this post is liked, and so they imitate its form and do not take into account its content and the choices made based on that content that determine its form.

Thus you get threads where 5% of the responses are significant, and the rest are people behaving like monkeys yammering out repeated memes, conventions, stylistic flourishes, demands for attention, etc.

I guess my golden rule is that anything I can script should not be included in the forum. That is, if we all must repeat some line from Seinfeld every time someone makes a grammar correction, I can probably code up a Perl script to watch for grammar-correction-style language and have it post the appropriate gag in response. People shouldn't be doing that for me; it's inefficient. :)

I hope "Discourse" succeeds. I am skeptical of the voting element however. What makes Stack Overflow succeed, and this seems unacknowledged in your post, is that it is based on a technical topic and on finding a clear answer. That separates it from, say, Slashdot, where the goal is "discussion" (a means, now serving as a goal) on that topic. By putting the clear answer requirement into discussion, you impose a goal, and thus discussion again becomes a means and not an end in itself.

Finding out how to impose that requirement on discussion will lead you to a better form of computer-mediated communication (CMC). I used to think voting systems were the answer, but having watched Reddit turn into a self-censorship circle based on community self-imitation, I don't trust that. I'm even skeptical of Hacker News upvoting because crowd "knowledge" is bad with low-commitment activities like voting, and it encourages imitative behavior as well. In the case of Stack Overflow, I think the success is not the voting but the fact that the answers can be verified by whether they work or not.

Thanks im going to look at Discourse.

There is a fantastic community building site thats massively well regarded by those in the know. But most people dont know it. www.meetup.com was created as a response to 9/11 to build communities. Using the internet to get people off the internet and meeting up. Its NOT a dating site in any way.

I have run meetup.com/shy-london for the last three years and have 800 members. There are groups for people sharing an interest in almost anything if you live in a major city. Groups are free to join usually.

Just spreading the word.

Does Discourse have features designed to stop a discussion forum being taken over by pompous middlebrow naysayers - something that's known as the Hacker News effect?

CIX has been doing something like this for 26 years.

First of all, I love it!

Over the past few weeks I've been re-building an application in MVC3 which hasn't been updated in 8 years.

When it came to the forum I had a look around at open-source solutions, but nothing was any improvement over the forum I had built almost a decade ago!

Discourse is definitely a huge improvement, and I for one will be including this in my application as soon as I can get hold of the code :)

Jeff Atwood,

With StackOverflow you have a lot of experience building with .NET.

I'm curious to know why you decided to use Ruby on Rails for Discourse.

Also, if you were to start StackOverflow from the beginning would you consider Rails?

FYI: I'm a ruby dev

So, are you going to improve the support for threaded discussions? That (along with keyboard bindings) seems to be the big thing missing. Indeed, it's something missing from many forum sites, yet it is a key aspect of making old-style discussion software (such as Mozilla Thunderbird, or the USENET readers of old) highly usable.

A discussion is like a tree, not a list. Linearizing it just sucks. (Note that Thunderbird isn't actually that good at handling discussion trees; it just happens to beat every discussion website I've ever seen.)

No OpenID login option... This is the part where I start playing sad trombone in my head.

So, from a user point of view. How is Discourse different from everything else out there? It seems a lot like your run-of-the-mill forum to me?
Is it only in the inner workings and for the administrators that this is a step forward?

When will codinghorror comments be powered by Discourse?

@John re SEO - the project's been public for 12 hours now. You need to Google some time to pick up on it, it's a new site.

> Linearizing it just sucks.

@Donal - it actually makes it much more readable.

http://notthetalk.com is modelled on the old UK Guardian newspaper discussion forum. The folks there think it's pretty much the perfect forum format despite being something like 15 years old. Readability not chrome is the key.

@Tim Sullivan - It may be possible to find cheap Rails hosting, but if it's any harder to set this up than it is to set up phpBB or Simple Machines Forum, it's probably not going to get WordPress level traction.

I Admin a very large community for writers; Absolutewrite.com/forums.

The software is actually decent; it's not cheap. vBulletin from Jelsoft.

If you can FTP and read, you can install vBulletin. We're running an older version; I have the current version on a test forum so members can get used to it before we change their world.

The non-forum parts of vBulletin are ridiculous, for the most part, but the boards are quite flexible.

But what makes the forum work, much as what made forums I've adminned for instruction, is active moderation.

Good moderation encourages conversation. Good moderation tools help with that.

@ Lisa - I agree about active moderation but on a fast moving current affairs forum this presents a problem, in the UK at least. If a site is actively moderated and something defamatory is missed then the complainant has a case for defamation against the site. Active moderation also generates a huge workload.

It's therefore important here to only moderated based on reports from the community although, of course, the admins are part of the community so can report things themselves.

Um, I'd like to play along, but your create a log in thingy won't accept my custom domain as a valid eMail address.

Eh?