Chapter 16

Select a Content Management System

Summary

Selecting a content management system (CMS) is a combination of research and vendor engagement. You need to identify prospective systems, investigate their capabilities, engage with the vendors for demonstrations or questions, and finally distill and synthesize all that information and come to a decision.

Does This Apply?

If you need to select a new CMS, then yes, absolutely. However, you might not need all of this, and we’ll talk about how to abbreviate the process. If you’re not picking a new CMS, or have already picked one, then feel free to skip this chapter.

Narrative

Everyone knows an audiophile. One of those people who elevate their sound systems to wild levels. They buy things like stereo wires that cost thousands of dollars, then claim a difference in sound quality that no one in the world could probably identify.

Then, at the other end of the spectrum, you have someone who plays music from a tinny AM radio powered by dying batteries that they’ve dropped in the bathtub one too many times.

This is juxtaposition of message and medium. The message is the music. The medium is the physical thing that makes the sound.

While you can go overboard on hardware, not paying any attention to it is just as bad. No matter the quality of music, it needs to be played on something that will allow it to be received to its full potential.

Likewise, good content is wildly important to your project, but that content needs to be managed and delivered in such a way that does it justice. You’re most likely going to deliver your content via some type of CMS. This entire book is predicated on the assumption you would be using a CMS for your website.

You need to make sure you pick the right CMS. Not too much, but not too little either. It needs to be enough to manage and deliver your content to the level it deserves, but not so much that it breaks your budget or is too complex to work with.

To be fair, you don’t have to use a CMS. Whether they’re generated by a CMS or hand-crafted, web pages are still just HTML, and there have been some fun advances in static site generation in the last few years. But for most organizations, usage of a CMS is a non-debatable point. Once your content gets over a certain volume, and especially if you have multiple editors working with it, then a CMS usually becomes a requirement.

At some point, you’re going to need to select which platform to use, and this is where things can get difficult. When you consider that the CMS as we know has been around for only about 25 years, you can imagine the level of volatility still in the industry. What’s more, for some on your team, this is the first time they’ve evaluated a CMS, or it’s been over a decade. A lot has changed.

CMSs are complicated things, and so are projects. They both have a lot of moving parts.

You know those inflatable dancers that car dealerships use to promote sales? The ones with the arms that whip around non-stop? Try getting two of them to dance in perfect synchronization. Now throw software vendors and their sales teams into the mix, and we’ve got a real party.

The best we can do is try to wrap some structure around the process so it doesn’t devolve into a confusing free-for-all.

(Disclaimer: you may read through this and say, “Wow, this is a lot.” And it is. But we thought it was better to show you the long-form method, and then let you dial it back if you like. We’ll address how much of this might apply to you in a section at the end. Hang on – we’ll put it all in context.)

Episode 16: Select a Content Management System (w/ Cathy McKnight)

Corey and Deane talk about a Donald Rumsfeld quote, and about the concept of “unknown unknowns.” Then, Cathy McKnight, Chief Problem Solver and lead analyst for The Content Advisory, joins us to talk about selecting a content management system — the process for choosing a system, how a CMS selection analyst can help keep vendors honest, and what to do if you’re selecting on your own.

37:21 | Cathy McKnight

Past Episodes

The Selection Funnel

A funnel is a handy metaphor. A funnel can represent anything that starts out with a lot of something, then slowly whittles the number down over time.

The reality show Survivor is like a funnel. There are lots of people on the island, and we slowly vote them off as we move from the big end of the funnel to the small end. Finally, a single winner emerges from the wild.

With CMS selection, this funnel is just two or three steps. You’re going to start wide, asking vendors to provide general information about their systems, then you’ll eliminate some of them and narrow the group by asking the remaining vendors to relate their systems specifically to your requirements, then you’ll eliminate more, then you might go an extra step and ask very few of them to actually do some integration work for you.

Finally, you’ll end up with a selection.

The Long List

This is the widest end of the funnel. This is where you create a “Long List” of potential vendors and toss them into the mix.

How do you come up with the Long List? There’s no great answer to this. The longest list is every possible vendor on the planet. But clearly, that’s unmanageable and not really fair because you should only include vendors that have at least some chance of satisfying your requirements. In general, you should try to keep the “long” list as short as reasonably possible – roughly five to eight vendors long.

Two reasons for this:

  • Selection processes take work: You need to manage requests and responses and do the evaluation, usually spread among multiple members of a team. Every extra vendor means more work for your team, and can add to extra confusion as they present features not central to your project.
  • Vendors are people too: When a vendor responds to your request, they’re investing time that they could be spending on other opportunities. Sometimes, they’ll work nights and weekends to get your request out the door (especially towards the end of a quarter). You’re in a position to create work for someone, so don’t do that unless they have a reasonable chance of seeing a reward. Don’t waste people’s time.

When developing the Long List, we will lean heavily on the decisions made when we determined system requirements. Your Long List will depend on:

  1. The nature — or “gestalt” — of your project: Are you simply managing content, like technical documentation or other non-marketing content, or do you really need a full-blown content marketing platform that happens to also manage content?
  2. Your technical restrictions: This might be a specific language, or a hosting requirement, or some other rigid requirement that will eliminate a chunk of vendors right off the top.
  1. Your price point: Is your project budget $10,000? $100,000? $1 million? And how much of that can you spend on software?
  2. Your need for services: There’s a big difference between selecting a system you can implement, and selecting a system that you need someone else to implement.

Simply accounting for these four points will eliminate a majority of vendors. You can compare your answers and requirements to the information vendors provide on their websites and from other online information – it’s not hard to understand the basic nature of a system and figure out if it even remotely overlays on what you want to do.

Bringing in Help

However, if you still have no idea who to include on your list, you can employ outside help.

Analyst firms exist to help organizations with technology questions – Gartner and Forrester are big ones, but there are hundreds ranging from enterprise to boutique.

Companies pay a monthly subscription fee to these firms for a set number of consulting hours – or even full “advisory days” – and access to the firm’s published research. Your organization might have one of these subscriptions, and you might be able to get an analyst on the phone, describe your requirements, and ask them to suggest a long list of firms for you.

Alternatively, these analyst firms often publish some of their research reports. Generally, they publish their flashiest report, which is usually well-branded and well-known in the technology industry.

For example, Gartner publishes what it calls the “Magic Quadrant” (known as “the MQ” or just “the quadrant”) and Forrester publishes the “Forrester Wave.” These reports provide a very top-level ranking of vendors in a particular space, such as “Business Intelligence and Analytics Platforms” or “Mobile App Development Platforms.”

If one of the analyst firms isn’t an option, consider a limited engagement with a consultant. Several exist in the CMS space (Deane used to be one of them, before he went to work for a vendor). For a set fee, they can advise you on your selection process. A small engagement might be reviewing your requirements and helping you form a long list. A more complete engagement would be to help you write the Request for Information (RFI), the Request for Proposal (RFP), manage the demos and manage a Proof of Concept (POC), and finally assist in analysis and decision-making.

If you work with analyst firms like Gartner or Forrester, they can help here. And there are smaller, content-focused firms like The Real Story Group and The Content Advisory.

The Request for Information (RFI)

Now let’s get some more detailed information, which we do through a process creatively called a Request for Information (RFI). This is exactly what the title says – we’re asking the vendor to answer some questions so we can decide whether or not to move them along in the process.

There is no set format for this, so know that when someone says “RFI,” they could mean almost anything. Generally speaking, you’re sending a multi-section document that both provides some information and requests some information.

First, you’ll provide some information around the following:

  • Your organization: What do you do? Why do you exist?
  • This project’s background: Why did it come about? What are the major problems you want to fix?
  • The selection process: What are the steps in the process? What is the timeframe for selection? Who is the primary contact person? When is the response to the RFI due?
  • The technical environment: What is your plan for implementation – internal, partner, or vendor? Where do you plan to host the software?
  • The scope of your implementation: How many websites do you need to run? How much content, in terms of objects or pages? How many editors will work with content? How many page views do you think you’ll need to handle?

You’re explaining this to give the vendor some context. Remember that vendors get a lot of these requests, and many vendors need to decide if the project fits their product.

Vendors might respond to say they won’t participate, or they might just not respond at all. This is a good thing. Very rarely do we see a vendor withdraw that should have stayed in the process. If a vendor bows out, it’s very unlikely they were the right choice for your project anyway.

In addition to providing information, you’ll also request information. The heart of the RFI is a series of questions you want answered about the vendor’s product. You can ask about anything, but these are some common questions:

  • What are the technical specifications of your product? What is its hosting disposition? What supporting software is required?
  • What is a rough price range for your product, given the sizing information we have provided, and how is it structured?
  • What does the product ecosystem look like? Do you offer training or certification programs for customers using your product, or do you have a customer extranet or community?
  • How many customers do you have in our industry (or organizational size, or project size, etc.)?
  • What integrations with other software does your product offer around a specific functionality? What would our relationship be with your company after purchase? Will we have a dedicated account representative?

The goal is to find out enough information about the company and the product to decide if this is a vendor you want to move down the funnel to the next round. You’re not getting too specific here – this is a general, high-level summary to get some questions answered and establish contact and rapport with a vendor.

Give vendors enough time to return this – at least two weeks, and perhaps more, depending on the number or depth of your questions. 

Some vendors will ask for a phone call for “discovery.” They likely want a chance to ask you some further questions, though some of them might be looking for an opportunity to make an early sales pitch. You can handle this any way you wish. We know some customers who refuse all contact requests, with the goal of treating all vendors equally, but so long as all vendors have the opportunity for further discovery, then let them ask for it. More communication and transparency can do nothing but help you.

The evaluation of the RFI response is how your short list is developed. The goal is to cut the vendor pool from the aforementioned five to eight down to two to three for the next round.

The Short List and Request for Proposal (RFP)

Once you’ve notified a handful of vendors that you’ve decided not to move forward with them, you need to prepare an RFP for the remaining vendors.

This document has the goal of fact-finding, like the RFI, but it gets more in-depth, particularly in two respects:

  1. You will provide specific usage scenarios you want demonstrated.
  2. You will ask for a binding price quote.

The response to the RFP will usually come in two forms. First, vendors will provide a written response with a price quote. Second, vendors will perform a demo of their product, either remote or occasionally onsite.

Usage Scenarios

The bulk of the RFP should be a set of scenarios – examples of your expected usage that you’d like to see in action. Here’s a very simple example:

Mary wants to create a new press release. She logs into the system and selects the Press Release content type. She enters a title and a summary. She selects an image from her local machine and uploads it to be added to the content. She enters several paragraphs of text and formats it using a formatting toolbar. She previews the content as it will be displayed on the website, and schedules it to be published tomorrow at 8 a.m. When she submits the content, it appears as an approval task for Brenda, who reviews and approves the content for publication. At 8 a.m. the next morning, the content is published and automatically appears in the index of press releases.

This is a very simple, mainstream editorial workflow scenario. It’s helpful for something like this to be the first scenario demonstrated, because in showing the functionality to complete the tasks described above, the vendor gives you a summary tour of the CMS itself.

Further scenarios can get more complicated. Things you might want to see:

  • Marketing functionality, like A/B testing and personalization
  • Localization support (“Mary also creates a Spanish translation of the press release.”)
  • Content quality tools, like spell checking, link checking, and reporting
  • Collaboration support
  • Advanced workflows, like content annotation and processing multiple changes as a unit (“changesets”)
  • User management and permissions

Keep your scenarios to seven or eight, at most. Remember that each one will take fifteen to twenty minutes to work through when you account for questions and follow-ups from a group. You could be in for hours of watching a screen if you’re not careful, and that’s just for one vendor!

Also, be realistic about how much time vendors have to respond. If your scenarios require customization work, three to four weeks is a reasonable amount of time for a vendor to respond as well as prepare and schedule a demo.

Pricing

For the pricing section, you will need to provide whatever information that vendor needs in order to price. What is this information? Well, it could be anything.

As we discussed last chapter, software is priced on an infinitely variable list of factors, and each vendor is going to need different information. Some common ways pricing is handled include:

  • Number of servers the software will be deployed on
  • Number of editors who will be able to use the software (either the number of editor user accounts, or the number of editors who will use the software concurrently)
  • Number of content objects under management
  • Number of websites the CMS will manage and serve (note: exactly what constitutes a “website” is very much up for debate)
  • Number of aggregate page views the websites will deliver during a particular timespan (by month or year)

Just be prepared that a vendor might ask for anything in order to come up with a price, depending on what their business model is. Several vendors will likely ask for another discovery call, which is fine. We believe that more transparency is better, and it’s tough to predict what information a vendor might need to price your solution.

Again, remember this is pricing for the software itself. Implementation is often handled by another firm, which might require an implementation RFP, or at least for the implementation and software RFPs to be combined.

We never said this would be a fast process.

Software Demos

As part of the RFP, you’re asking for the vendor to demonstrate parts of their system for you. You can do this remotely over a desktop-sharing system, or on-site and in-person. Vendors will gladly get on a plane for the right opportunity, which normally means that it’s (1) large enough in pure dollars, and (2) one that they have a reasonable chance of winning (described as “well-qualified”).

Either way, the vendor is likely going to need to build out functionality for your scenarios. Be understanding of their ability to do this for your demo. They will likely customize their system, but know that this isn’t production-quality code, and some of the functionality you want demonstrated is too in-depth to build in advance. In these cases, vendors might show you a combination of actual demo and then talk through some of the more in-depth points, explaining how it would work in a production deployment.

You need to interpret and discern here. While it’s unreasonable for a vendor to build everything out, you can get a general feel for the level of effort the vendor is putting into it. You want to gauge how well they’ve actually read your scenarios and how well they understand them. We’ve seen situations where it’s obvious the vendor first looked at the scenarios on the flight in.

For your part, you need to have an evaluation team prepared for any demo. If the vendor is putting forth the effort, you need to make sure your team is assembled without anything else to do. There’s little more disheartening for a vendor to spend thousands of dollars preparing and traveling, only to find that six of the eight people canceled, and one of the remaining two spends the day with their head buried in a laptop.

Timing will vary, but plan on three to four hours for each demo. You’ll spend a couple hours getting through the scenarios, then another thirty to sixty minutes discussing pricing and follow-up questions.

Sometimes it’s also helpful to send the vendor out of the room for thirty minutes to discuss their presentation as a private group, then invite them back in for any questions that came up during that discussion.

The demo is your chance to see the system in action and ask questions about what you see. This demo is for you, not them. Some vendors might try to gloss over issues or redirect your questions and concerns to an area of functionality where they’re stronger. This is not a naturally adversarial engagement, but be prepared to be assertive in what you want to see.

Verifying the Decision with a Proof of Concept (POC)

For many organizations, an RFI and RFP are enough to come to a decision, but some organizations want to go one more step further into a proof of concept. In this step, the vendor works with the actual customer team to implement some solutions to project requirements. It’s a deeper dive, where the customer can get hands on with the system and interact with the vendor’s team.

Sometimes, two vendors do a competitive POC (usually in consecutive weeks). Other times, the customer has chosen a vendor after the RFP round, and the POC is seen as a verification step for that choice. If the chosen vendor fails completely at the POC, then the customer backs up to their second choice and has them do a POC.

Here’s the thing about POCs – you need to be very serious about the project, and you need to have a big enough budget to warrant this.

POCs should be paid for.

We’ll say it again: POCs should be paid for.

Vendors can only invest so much in an opportunity without any return, so you need to be prepared to invest in your POC. Often, POC costs are proposed as a “kill fee” arrangement, where the vendor is paid if the POC doesn’t result in a sale. (Other times, the customer asks that the POC fee is credited toward future services. Whatever, same thing.)

Another caveat with POCs is that you need to be sure your team has the time to invest in it. If you’re having the vendor come on-site for a week, then you need your team to be liberated from anything else they have to do that week. In fact, it helps to have POCs off-site if possible, so your team doesn’t get dragged back into other work.

Think of a POC as a “super demo.” Refer back to our scenario example above. If we were to expand this to a POC, then the vendor would actually work with your development, editorial, and marketing teams to implement the press releases functionality, including:

  • Modeling the content with all validation rules and editorial interface support
  • Setting up the users and permissions
  • Defining and creating the approval workflow
  • Creating the templates
  • Training the customer on how the system works
  • Some hands-on content operations work (e.g. each person on the customer team enters two complete press releases in a workshop/ classroom environment)

The goal is to get end-to-end on some segment of functionality. Ideally, the hands-on work would even reveal some shortcomings which the vendor can then address, which would illustrate how the system changes and evolves post-launch and the attentiveness and responsiveness of the vendor.

However, don’t confuse a POC for actual production code. You might solve some logical questions and come up with some technical design and implementation patterns and ideas, but the code generated during a POC should always be considered disposable.

Decision Time

We’re calling this section “decision time,” but that’s something of a misnomer. By this point in the process, you’ve likely made multiple decisions (remember: a funnel gets narrower with each step), and the team has probably come to some informal consensus.

Nevertheless, at some point you have to call the process to a close with a final decision. When you do finally reach a decision, you’ll need to enter a formal negotiation with the vendor. Remember, everything is negotiable. If you need a particular price or pricing structure to make your project work, ask for it. Vendors can be flexible. They want to make a sale, and they’ll deal if they need to.

And, be prepared for the paperwork to take some time. It’s not uncommon for vendor vetting and due diligence at larger organizations to take thirty to sixty days, so be sure you understand their policies and know when you can actually take possession of a license or account to get started on your project.

So … Do I Really Need to Do All That?

Yes, yes, we know – this is a lot.

Rest assured, not every project needs all of this. Larger projects with a wide range of CMS options will need something resembling this, but other projects can cut this process way down.

Some options:

  • POCs are probably the exception, not the rule. There are reasons not to do one. As we note, POCs should be paid, they take a lot of effort from your own team, and they push out your decision timeframe considerably. Think very carefully before you embark on one.
  • You might skip the RFI. If you start with a relatively short list of vendors, you might jump right to a combined RFI/RFP, which is an RFP with some additional questions. You might send these to a slightly larger group of vendors (call it a “medium list”), and explicitly state that you will invite vendors to demo scenarios depending on their response. You might only have one vendor demo.
  • If you have a consultant you trust, let them guide you. If you find a consultant who you think has enough experience with your type of project, then perhaps let them make a recommendation? Have them tell you what system they would use for your project, then have that vendor demo. You might find yourself comfortable enough to make a decision solely from that.

Note too that, for some projects, you might not even be able to do all this even if you wanted to. The process detailed above assumes you have a willing set of vendors with sales teams ready to engage in a personal selling relationship. This isn’t always the case — some systems simply don’t do individualized sales, and open-source software often has no seller.

There’s just no process that fits every project, every team, and every mix of potential vendors. Pick and choose, and come up with a process that works for you.

Above all, keep an open mind. Seek to learn as much as possible. Even a system that ends up not being the one for you can teach important lessons about what’s available on the market, what works, and what doesn’t. Sometimes, seeing something you don’t like helps you clarify what you’re looking for.

Finally, know that there’s no one system for everyone. The truth is that your project will likely be equally successful with more than one system. Remember this when and if you get overwhelmed by the decision.

Do your best. Good luck.

Inputs and Outputs

You need the requirements that we discussed in Chapter 15: Determine System Requirements, and the integration stuff from Chapter 14: Know Your Integrations. Also handy – a site map (Chapter 10: Organize Your Content), a content model (Chapter 11: Model Your Content), and any designs and wireframes you have (Chapter 13: Develop the Graphic and Interface Design). The output of this phase is a binding CMS platform decision, and a purchase agreement for that system.

The Big Picture

Clearly, you need a decision on CMS before you can start building your site, or sometimes even before you start looking for an integration partner if they aren’t helping with your CMS selection. However, you can’t rush into this right away – you need significant site planning in place before you have enough information to make a solid decision. You need to thread the needle between those two things.

Staffing

As we discussed in this chapter, a consultant is pretty helpful here. You can hire one for the entire selection process, or perhaps find someone in the web content management space who would give you just a few hours to review your requirements and make some recommendations for your Long List.

Besides that, the core project team should be enough, so long as everyone is involved. Everyone needs to see the CMS demonstrated and provide their feedback on it. Considering the typical expense here, you don’t want to surprise anyone.

Resources

Articles

Books