Solving the WordPress Traffic Overload Problem

Anyone who’s been around WordPress for a “Digg effect” or other massive influx of traffic knows that it can be a real problem. From a technical standpoint, the problem is that PHP is entirely loaded into memory for every pageload. That includes the 99% of PHP that is not being used to actually render the page.

On low traffic sites, this problem is not necessarily noticed. It doesn’t have a huge impact. However when there are hundreds of requests hitting a server in a single second, that kind of overhead builds up very fast.

There are solutions to this sort of thing and depending on what the scale of the environment is, some might be more excessive than necessary. The WP Super Cache plugin is a quick solution that will cause pages loaded on WordPress to be cached meaning that if subsequent page loads can pul the HTML from the cache without having to load the overhead of PHP as well, everyone wins. On the more extreme end, server configurations can be made to send requests for different types of content (for instance, images) to specialized servers optimized for that content type.

Very geeky stuff. It’s important to note that WordPress gets a black eye all the time for it’s caching mechanisms and ability to handle the load of a “Digg effect”, etc. In fact, Instapundit Glenn Reynolds is the latest to take a stinging swipe at WordPress and trust me when I say, we heard it loud and clear.

At b5media (where I’ll be leaving as the Director of Technology soon), we’ve had to deal with this as well and have managed to develop really sound solutions to some of these problems. However, for WordPress as a whole, it is a well recognized problem that not everyone can solve by following in our footsteps (or footsteps).

We’re going to do what we can to help solve this problem once and for all as two of our developers, Mark Jaquith and Brian Layman will be mentoring a Google Summer of Code intern to develop a robust caching engine for WordPress. We hope that this exercise will result in a more reliable (and sane!) caching mechanism.

Integrated Caching Solutions will improve WordPress’s speed and reliablity out of the box and allow people to “Digg Proof” their sites without the struggle of installing plugins on a site that is virtually unreachable. (Source: WordPress Google Summer of Code 2008)

Glenn, I hope that the work that Mark, Brian and our intern will be doing will improve the WordPress problem. In the meantime, let me know if I can help you with anything (though I believe you are using Movable Type). It is a known issue and it’s one that needs to be solved and hopefully some steps can be made toward that this summer.

Technosailor Maintenance Today

To give everyone a head’s up, at some point today, this blog will go into “maintenance mode” while I move to a different server. While Technosailor has been a part of the b5media network, I also announced my departure from b5 last week and so the blog is moving to a colocated server at Defender Hosting. They have been gracious enough to work out a fantastic arrangement with me, and I’m happy to have my own box to be able to fiddle around on.

So comments will be going off at some point to prevent data loss and mail might bounce sporadically. I hope to do this as smoothly and as transparently as possible though. Probably this evening.

Departing b5media

About two and a half years ago, I embarked on a journey of a lifetime. It was a breakthrough journey that took me from working in a cubicle at Northrop Grumman to living my dream building an internet startup. At some point, we took funding. I recruited Brian Layman and Mark Jaquith and the rest of the tech team. We moved from hosting a blog network on someone’s shared hosting provider to developing hard plans to encompass hundreds of servers in grid format.

We went from WordPress 1.5 and basic installs to over 350 installs of WordPress working in tandem with robust data sharing via APIs. We developed techniques to deploy advertising campaigns quickly and efficiently. We worked hard to build reporting tools, financial tools, management tools, and more – all in an effort to make this company work.

And it has and continues to grow at an astounding rate.

It’s been a great ride, but now it’s time to look forward and explore new territory. Recently, through my interactions with so many wonderful people in the DC technology community, the PodCamp community, the larger social media, business and technology communities, my appetite for something new and challenging has been overwhelming. I’ve had several conversations with Jeremy Wright, our CEO, over the past months exploring this stuff and he has been amazingly supportive, as has the rest of the b5media team.

An exact date has not been set, as I want to make sure a replacement is found and brought up to speed, however it will probably be within the next couple of months.

In the meantime, I’m looking forward to the next steps. At this point, I’m not sure what those next steps are but I have options. I’d like to develop Mokonji a bit more. I also have a few other venturistic ideas floating around in my head and I want to spend some time focusing more on the DC community that has sprung up around here. Obviously, I’ll continue to be blogging here at Technosailor, though in the short term, I’ll be exploring effective ways to monetize it – gotta pay for the server costs now (whew, forgot how that worked! ;-) )

I don’t know. The eager anticipation of the “what’s next” is crazy for me right now. It’s sort of like going out on a date for the first time. You don’t know what to expect but you know you really, really like the girl a lot.

Thanks for all your support for these years that I’ve been blogging. Looking forward to see what comes next! And if you want to be me (erm, the b5media Director of Technology), the company is actively looking for my replacement.

Funny WordPress Plugin Sneaks In

One of the annoyances we have at b5media is when people spell our company name wrong. We’ve even had our own people spell the name wrong. So annoying. :-)

Somewhere along the line last year, this universal plugin snuck into our build. I don’t know where it came from or who wrote it. (whistles) It solves the problem though. :)


b5media Hiring Support Tech

I am reposting this from the b5media posting that went up earlier today.

b5media is currently looking for a passionate individual who is looking to work with our technology team. This position is a support role, and you would be supporting the day to day needs of our bloggers, channel editors and staff.

While most positions are Toronto-based, we are explicitly fielding candidates in the eastern hemisphere – Australia, New Zealand, Japan, the Phillipines, etc. You must have a working familiarity with WordPress, be able to demonstrate your problem solving ability and have a desire to learn.

This is a part time job, and you will work virtually. If this interests you, forward me your resume. Thanks.

WordCamp Dallas: What do you want to know?

At the end of March, I’ll be speaking at WordCamp Dallas. I’ve been asked to speak about WordPress in an environment, but have been given broad latitude to shape that discussion in any way I like.

So I’m turning it over to you. I’ll be making my slide deck available here, and maybe I’ll stream it live. So what do you want to know about? Subversion? Mass upgrades? Server caching? Something less technical?

If you want to send in questions by Youtube, mayne I’ll feature you during the talk. Leave your comments, links to videos or whatever else in comments. :)

More of the Same in 2008; Or: We ain't no Seesmic

It’s Monday morning and I’m sitting here at Reagan National Airport awaiting a flight to Toronto. This is my last business trip of 2007 and… when I return home on Wednesday, I’m only looking at another two business days before I entirely knock off for the year. I’ll probably blog, but no b5media (if I can help it), none of my “mini-gigs”, and generally, no social media. I say that now, of course.

I figured it was a good time to look at what you can expect from this blog, and more specifically me, in 2008.

More Travel

I traveled more than I ever did before in 2007. Met some great people from the social media community. Reacquainted myself with others. Engaged in my first public speaking engagements. In essence, 2007 for me was about a definite maturing in my professional profile. I haven’t always executed well. Some people may not like me. Others might think I don’t like them. But, I’m happy with where 2007 has taken me.

In 2008, I expect more travel and that means more of an opportunity to meet you somewhere. Though nothing is definite yet, I’m hoping to make it to Future of Web Apps Miami and New Media Expo as well as near definite appearances at SXSWi, WordCamp Dallas, WordCamp San Francisco, Gnomedex 8 and Blog World Expo 2008.

There’s also rumor of b5media doing a cross country tour, but I can neither confirm nor deny that possibility. ;-)


One of the new buzzwords getting thrown around the Web 2.0-a-sphere is “hyperlocal” – the focus on local/regional services, community and communications. While 2007 has been critical for me in developing my reach internationally and nationally, I have neglected my profile here at home. In the words of Jesus, “A prophet is not without honor save in his own country,” and while I don’t claim to be a prophet, I did predict the Ravens loss to the Miami Dolphins this past Sunday.

In 2008, I plan to cultivate the relationships that I have begun to develop in the Baltimore/Washington region more throughly. For instance, Geoff Livingston and I will be launching a Blog Talk Radio show surrounding the social media and communications environment in the Washington, D.C. area entitled “District of Corruption”. This will begin at 2pm on Tuesday, January 8.

Other potential alliances exist between myself and Nick O’Neill of Social Times and All Facebook, Mike Brenner who is looking to launch Refresh Baltimore, Ann Bernard and Keith Casey at Why Go Solo, NewMediaJim, Frank Gruber of AOL and co-founder of TECH Cocktail, Greg Cangialosi of Blue Sky Factory… and others. In the new year, I’ll be focusing a lot of my time and energy in these areas and with these people and maybe something cool will come of it.

More Original Non-English Content

Carlos Granier-Phelps has been doing a smash up job producing original Spanish language content for I’ve learned from early mistakes and provided a separate Spanish feed for this content and I expect to learn more from the experiment. I say experiment because I did this not knowing what to expect. A month and a half in and I’m seeing definite signs of traction. It’s always hard to build a new audience, so I’m grateful to Carlos for helping to spearhead this under the Technosailor banner. Social media and business is not exclusive to English speakers and so I don’t want to ignore that demographic.

In an ideal world, I’d love to see the new year bring original French and, I don’t know, Japanese content as well. We’ll see. Certainly, let me know if you’re interested in reading or writing here.

We Ain’t Seesmic

Finally, you can expect more of the same from me. In the past year, I’ve recognized that it dilutes content to force a quota on myself. I used to force myself to write once a day at least and now I only write when inspired. As a result, my content is better and more original. Traffic has shot through the roof and my subscriber count has more than doubled. Unlike Seesmic, I’m not too concerned with what critics say. ;-)

Time to get going, the plane boards soon.

Google PageRank Penalties For Network Blogs

It appears this morning that Google has issued pagerank penalties on network blogs. This was first brought to my attention by Darren over on his blog who saw his blog drop from a pagerank 7 to a pagerank 4. Interestingly, is a solid pagerank 6 and it redirects to, so I don’t know entirely what to make of that.

I’ve seen Technosailor drop from a solid pagerank 6 to a pagerank 3 in most cases. Engadget was dropped from a 7 to a 5. Copyblogger also was dropped from a 7 to a 4.

A number of people have emailed, Skyped or Twittered looking for an explanation of this. I am not Google so I can only offer speculation. If I had to guess, it comes down to nofollow not being applied to “permanent links”. Last month, we saw Google penalize people selling Text links without nofollow added. This month, we are seeing networks who links among themselves penalized.

This is where I find tremendous fault with this Google action. If you remember back to six months ago, all our b5 blogs linked to all other b5 blogs. It was a tremendously lengthy and unwieldy blogroll. We recognized at that time that for practical reasons, as well as search engine purposes, we needed to keep the blogroll limited to relevant links. Thus entered our second version blogroll which now presents relevant blogs within our network based on the channels they are in. I think it’s safe to assume that people interested in Lindsay Lohan, might very well be interested in Brad Pitt or Britney Spears. Likewise, people who like First Person shooters are probably gamers interested in breaking video games news from one of the worlds leading sources. Folks wanting to know about Apple products might also be interested in iPhone discussions over at Cellphone9.

Makes sense right?

Google doesn’t like it. But here’s my beef. Google’s algorithm, as tremendous as it is, doesn’t consider common sense like this. Either that or there was some anti-spam vigilante assuming that blogroll links are spam regardless of the topic and manually culling from the index.

At b5media, we are weighing how we want to respond to this. Either we give in to Google and let them dictate what we do and have the unenviable position of losing pagerank and possibly advertising dollars, or we take the stand that quality content is quality content regardless of Google and that our content will speak for itself. We still produce millions of pages of content per month. We still have respect in the community. We still have advertisers recognizing that these sites are valuable assets to leverage to get their campaigns out on.

I’m interested in your take on this blood bath. Please weigh in.

Update: Duncan Riley weighs in at TechCrunch

The move by Google could well cause many smaller blog networks, including a number with funding, to close given their heavy reliance on text link ads and related sales that depend on strong Google page ranks for each site. Although traffic alone can and does sell ads on bigger sites, a drop from say PR7 to PR4 in one example makes the ad sell that much more difficult, particularly on blogs with little traffic. I’d suggest that the Deadpool will soon see a number of new entrants.

Deadpool is a little extreme but he makes a good point.

Update: Video comments!

Music Channel Launch

I’m about 3 days late on jumping on the music channel launch bandwagon at b5media. I’ve been late because we’ve been working behind the scenes on the launch and there has been a lot to do. In fact, we’ve had about two months since we first hired Mike Laba, the music channel editor and it has been head down, plow ahead since.

The process for this has been pretty intensive since there’s several parts of a new blog launch that the tech team is responsible for. We setup the domain, deploy our standard build of WordPress, have our designer create logos, setup the standard b5 theme, make sure everything is in FeedBurner properly, make sure channel blogrolls are functioning as they should, etc.

There’s a lot to do and I’ve personally been going crazy getting this thing out the door. For most of the past 10 days, I haven’t gotten to bed before 3am.

But the music channel is now here and doing awesome. These bloggers rock, plain and simple, and I’m loving the content – specifically loving Jam Band News and The Good Musician.

This is the first major expansion that we’ve made into a new vertical since the Business Channel was launched last year. Pretty intense. Thanks for the traffic!

Guide to Disaster: How The Tech Team Handled WordPress Security Flaw

By now, the news has spread rapidly in security circles and on mailing lists about an exploit to the WordPress software less than or equal to version 2.1.3. To give you some background, we had held off on upgrading to version 2.2 that came out last week due to bugs in the software that we felt were unacceptable to our company. Nothing critical, but as we are nearly finished rolling out new themes that are all widgetized to the network, I felt that lingering widget bugs were pretty critical to our platform. The decision was made on release day that we would not upgrade until WP 2.2.1 was released next month sometime.

That was the plan as recently as Monday evening. But something changed quickly and I want to give you a window into how the team worked together to avert a crisis. As this timeline is fairly raw, I hope it gives some perspective on how we are able to react and triage situations quickly and put issues to rest all the time. We don’t always have critical security flaws, but we do work together to problem solve on a daily basis. This is how we roll.

Monday, May 21 – 10:52PM EDT
An email is sent to the WordPress hackers mailing list alerting the community of a posted exploit to all versions of WordPress under version 2.1.3.

Monday, May 21 – 11:17PM EDT
Exercising caution as with all security alerts, I carefully setup a test and run proof of concept script against one of our blogs. Threat confirmed.

Tuesday, May 22 – 12:07AM EDT
I forward the notice to the tech team for them to digest in the morning.

Tuesday, May 22 – 8:21AM EDT
Brian Layman confirms threat and indicates that our upgrade timeframe decision has been made. I agree.

Tuesday, May 22 – 9:22AM EDT
Sean Walberg, our systems administrator, suggests we delay upgrade until peak traffic time is passed. Already, we were under a Digg storm and we did not need to exacerbate issues with an upgrade.

Tuesday, May 22 – 9:40AM EDT
Channel Editors notified of the problem and the impending upgrades and are given instructions to change passwords after the upgrade.

Tuesday, May 22 – 2:30PM EDT
Brian Layman and I work up more verification of the exploit by analyzing and executing the code against further targets on our next work. Re-confirmed.

Tuesday, May 22 – 4:30PM EDT
Upgrade script and subversion repositories prepped for switch to WordPress 2.2. We chose revision 5505 as most of the widget issues we were initially concerned with were addressed prior to this revision. Core plugin set re-evaluated by team. Eliminated one plugin due to security.

Tuesday, May 22 – 6:00PM EDT
Upgraded Tech channel and verified functionality of widgets, in particular.

Tuesday, May 22 – 9:00PM EDT
Upgraded entire network to r5505.

Tuesday, May 22 – 9:30PM EDT
Support, support, support. Reports roll in regarding broken this and that – mostly having to do with plugins and widgets. Solve almost all except a weird database error on one blog.

Tuesday, May 22 – 10:40PM EDT
Major bug discovered – well, not major for WordPress, but certainly for us from a user experience perspective.

Wednesday, May 23 – 12:35AM EDT
Reupgraded network to r5520 which included further fixes for widgets.

All in all, because we have created tools and standardized everything we do, we are able to avert problems before they become problems. We do it all the time for big problems and small. Folks who run networks, whether blog networks like b5media or simply groups of blogs that are maintained by the same person or group can choose to upgrade blogs by hand, one by one, or sit on the problem hoping to not be attacked “until the weekend”, or they can take attacks seriously, use tools that assist in upgrading (Brian’s upgrade script is very good too) and be done very quickly and efficiently.

Our upgrade of over 200 blogs was completed in 30 minutes and 6 seconds – a slowdown from earlier reported times based on instituting a pause between each upgrade. Our time of execution from problem introduction to problem solution? Less than 24 hours.