wyDay blog

Introducing Just the Forum (JTF), one trillionth forum software!

This is just a quick note to let you know that we've released it and why we made it in the first place. It's being dog-fooded in-house as our customer support forum. Then, in several months (after more polishing), we're releasing the source code. This means you can use it to replace your existing forum software.

Why create another piece of forum software?

We want to have control over the public conversation about our software. Everyone's positive and negative comments (as well as support requests) have a public place to live right on our website. Of course people can still blog, post elsewhere, etc., but customers have the option of doing it publicly here.

This rules out using 3rd party social networks (e.g. Facebook) and 3rd party issue / CRM trackers.

The good news is that there are about a trillion pieces of software that offer "forum" or "discussion" software that can be self-hosted. Many of them are toys (weekend projects, forks with minimal contributions, etc.). But there are a fair number of popular options:

1. phpBB (and too many clones to count).

2. Discourse (and clones like NodeBB)

3. A handful of commercial options (vBulletin, Invision, etc.)

phpBB is a well-intentioned dumpster fire

We've used phpBB for more than a decade and the nicest thing I can say about it is that it's an adequate piece of software that wastes a huge amount of our time. For example:

1. Upgrading the *minor* bugfix versions of phpBB takes multiple hours to do. No one-click upgrades.

2. Spam protection is non-existent, so we have wasted many many days clearing our Russian spam.

3. Upgrading any piece of our backend (e.g. php or MySQL) breaks phpBB every single time. Which requires us to dig into the spaghetti code and manually fix their mess of code and/or comment out the "feature" altogether.

4. Code execution is slooooooooooow. Code is poorly written and database queries are poorly written. Add them together and the rendered forum post takes forever to appear.

And each of those point has negative side-effects. For example, #2 (phpBB has no spam protection) has the side effect of us needing to disable BBCode so that links and pornographic images aren't rendered. Of course, this means phpBB becomes less useful for legitimate customers.

That's just the tip of the iceberg. Overall the quality of phpBB is poor, and the time-cost is very high. Free software that drains your time ain't so free.

Poorly designed software, implemented poorly

Here's an example from just the other day:

Here's what this picture shows:

1. The current version of our phpBB forum (before the JTF switch).

2. An error at the top that says: [phpBB Debug] PHP Warning: in file [ROOT]/includes/session.php on line 611: sizeof(): Parameter must be an array or an object that implements Countable

Now, you may be asking: "Why did you enable "phpBB Debug" statements on your production servers?" We didn't.

Then, you might assume that we enabled warnings or errors in our php config. Again, we didn't.

Instead what you're seeing is a bad idea that's been implemented poorly. A developer decided it was a good idea to capture any warning (even if we've explicitly disabled them) and then instead of logging them discretely in the background, they doubled-down on this terrible idea and decided to plaster them right at the top of the page.

Is there a way to disable these errors? Not in phpBB. Not without digging into their spaghetti code and ripping out this functionality.

This is just a simple illustration of why we call phpBB poorly-written. There are a slew of others (for instance, why does every single post in a topic have its own title?), but I don't want to turn this into a shit-on-phpBB article. It's obviously written by well-intentioned amateurs that might one day turn into skilled developers. They're just not there yet.

Brief aside for amateurs who want to become skilled: you should always be asking yourself "how is this actionable?". Showing a warning with a line number and a reference to a file is not actionable to...

• Random people visiting the forum.
• Most administrators running forum software on their servers.

The only group of people this error is actionable for is the only group of people that will never see the error: the developers of the software.

phpBB isn't the only forum software that does this wrong. Discourse, a popular new-ish forum, has similar obvious design flaws. Setting aside the obvious downsides of doing the rendering client-side (thus slowing things down and losing the obvious benefits of rendering things server-side), why should a static web page ever show this message?

It's rhetorical. There's no reason that should ever be shown. It's bad design.

Why we built JTF from scratch

There are 5 big things that are different:

1. Just the Forum (JTF) is fast

Why do expensive slow work over and over again? Just the Forum (JTF) doesn't waste PHP's or the database's time by requesting the same thing over and over again. It gets it once, then it delivers that static page until something has actually changed. This saves CPU time and, more importantly, the customers time.

2. Secure by default

In building LimeLM and LicenseChest we have a ton of experience designing and maintaining secure web software. There are many aspects to security, but a quick and dirty way to organize those aspects are:

1. Human: that is, who in the organization has access to the data and ensuring customers' data is secure. Up until phpBB version 3.1 (released a couple of years ago) they stored passwords using MD5. Even still storing passwords as MD5 is an option! That's nuts (and a huge legal and financial liability for anyone using phpBB).

2. Software stack: the things one which the software depends. For example, php version, OS version, database version, etc. By default we require at least php 7.4.x and MySQL 8.0.x (and MySQL equivalents like Percona Server). This allows us to use more secure methods of storing and transmitting data.

3. Software itself: data/view separation, we force secure defaults, and we have secure SQL queries. phpBB *still* does not have data/view separation (they slop all of their data calls intermingled with code that displays the data and it is a nightmare to debug). They also do not parameterize SQL statements. Instead they quote and append strings (which is dangerous and has led to security bugs in the past and will likely lead to new as-yet-undiscovered security bugs).

In short, we have a responsibility to secure our data and our user's data. We could've gone about this 2 ways: work patiently and slowly to turn the ocean-liner that is the phpBB development community to security and modern design. Or we could wash our hands of the mess and start with our fast and secure code. We chose option 2.

3. Spam killers built in

The pre-release version of Just the Forum (JTF) has Akismet built-in. In subsequent versions we're going to build in some simple Bayesian spam filtering in case you don't want to outsource the work to Akismet (which is itself basically just one big centralized Bayesian spam filter).

We're not going to waste customers time with crappy CAPTCHA images that spammers can solve in 3 milliseconds, but it takes a normal human several minutes to solve. No, CAPTCHA is idiotic. It was good while it worked (when computers couldn't read it and human could). But this hasn't been the case for several years.

For the past 2 to 3 years I woke up every morning to look at the hundred of spam messages posted overnight. I've wasted more than an hour each day clearing this spam from our forums. It got so bad that I raised this point to the phpBB developers (politely, of course).

I suggested they take a machine learning approach to spam filtering (i.e. technology that has existed for more than a decade) because human-readable CAPTCHAs were no longer effective.

Their response? Add busy-work for the user (solve math problems, answer pop-culture questions, and other nonsense).

Computers exist to remove this busy-work from humans lives! CAPTCHAs are bad enough (useless, now that they've been completely broken by spammers). I'm not going to add busy-work for our customers just to work around lazy developers.

4. There are barely any options

That's right, we pick the defaults and you'll like them. phpBB and the 999,999,999,998 other forums go for the "kitchen sink" approach to user configuration. (Good luck finding an option when you need it).

We don't do that. We're picking good defaults that will work for almost every user. And for those customers that don't like our options — well, there are nearly a trillion alternatives out there. Or, to put it another way, if you don't like the way we do it...

5. No BBCode!

BBCode is junk. It's a buggy, ugly, wrapper that ultimately gets converted to HTML. So why even bother using BBCode when it's going to be HTML in the end anyway?! BBCode is the lazy engineer's way of solving the "HTML is hard to clean" problem. Yes, HTML is hard to clean, but it's also a solved problem (google your favorite programming language for the open source library of choice).

And Markdown is just BBCode for hipsters (just as ill-defined as BBCode, and ultimately gets rendered as HTML anyway).

Just the Forum (JTF) uses a WYSIWYG ("what you see is what you get") editor that outputs to HTML. The HTML output is cleaned in the back-end when a new post is submitted.

In other words, you don't even know how to use HTML to make nice looking posts. You just type what you want to say, format it how you want it to look, and click "Post". That's it.

The only "downside" is that you need JavaScript enabled in your browser. Every normal person will have it enabled by default. If you're getting a warning about JavaScript being disabled then call your grandson and threaten to take them out of your will if he doesn't fix his "optimization" that he made to your computer over Thanksgiving.

Bonus Feature: Least original name

Ok, Just the Forum (JTF) is a bad name. We tried about 50 other names but they were either taken by a domain squatter or taken by one of the trillion crummy competitors to Just the Forum (JTF).

So you're stuck with Just the Forum (JTF). Unoriginal name, fantastic software.

More to come...

There's much more to come (including the public release — when you can actually use Just the Forum (JTF) on your own servers). Stay tuned.