Episode 17: Plan for Hosting (w/ Elias Lundmark)

Hello, this is the Web Project Guide podcast and this is episode 17: Plan For Hosting. I'm Corey Vilhauer, Director of Strategy at Blend Interactive, and co-author of the Web Project Guide. Later, we'll be joined by Elias Lundmark, Senior Project Manager overseeing cloud services at Optimizely. But first I'm joined by my co-host, Deane Barker. Do you get it, Deane?

Oh, host. Oh, I do get it.

It's about hosting.

Yeah. So I've been co-author, co-anchor, and co-host, I get it. I get it, that's great.

That's what I did. I want to ask you about two things. Number one, did you know that I was in New York yesterday and we walked by the Friends fountain?

Oh, did you really? I took my family to New York once and we actually went to the building, which was the exterior shot of the apartment building, and my daughter who was 15 at the time, was furious that it wasn't the actual building because she went around the side where the balcony is supposed to be. It wasn't there, and it couldn't possibly be. "That doesn't look anything like it," she's like. I'm like, "Well yeah, they film this on a sound stage in Los Angeles." She was furious about this, but we got a picture taken out there so.

My other question is, Deane, our episode's about hosting and I don't know anything about it. Why should I?

We are going to be talking later to Elias Lundmark, who is a very, very smart guy and knows a lot about hosting, and Elias will be able to tell us more. But in general, it's safe to say that your website has to live somewhere. It has to be somewhere. At any given time, you as a human have to be somewhere, your website actually has to be somewhere that's accessible. And the question of how much should you care about hosting, and again, we'll talk to Elias about this, but it depends on your website. It depends how important it is to your business. It depends on a lot of different things. And hosting is a weird thing because you can start out with very simple, and you can get extremely complex and extravagant. And the point is figuring out where along that continuum you should stop. Because when you talk about hosting an infrastructure, it is easy to spend a lot of money, and you just got to figure out where is it appropriate for me to stop?

(02:19)
How much is enough? And there are people that have a hardware fetish and they're just like, "It's never enough. We need more and more servers," and we need all these other things. And then there are people who are like, "You know what? If my website's down for two, three days a year, that's not the end of the world." I always remember talking to a university that wanted ridiculous uptime constraints, and explained to them the difference between what they wanted and what was reasonable was $25,000 a year and the difference was three hours a year in downtime. I'm like, "If someone came to you today and said, 'We're going to take your website down for three hours once a year from 2:00 to 5:00 AM,' and in exchange for two, that we'll give you a check for $25,000... In the country would probably do that."

Yeah.

So it's one of those things where it's just, it's a trade-off between how much you want to spend and what you need in terms of reliability.

Okay, well now I'm convinced, we should listen to the rest of this episode. We'll talk to Elias here in a bit. But first we need to remind you that the Web Project Guide podcast is sponsored by Blend Interactive, a web strategy design and development shop dedicated to guiding teams through complicated web and content projects. I read this from a script and I forgot how to read there for a second. Blend's been building great websites for when this goes out, 18 years. Our anniversary is on March 15th, 2023.

We would always at Blend, we were on downtown, Blend still to this day has just a phenomenal office location at downtown Sioux Falls, right on the route of the St. Patrick's Day parade. And so we had the legend inside Blend that it really wasn't a St. Patrick's Day parade, it was a birthday parade for Blend, and we would go celebrate it every year because they were throwing us a parade for our birthday.

It's the sweetest thing the city's ever done for us.

It is, it's lovely.

We will be old enough to smoke, and we'll be old enough to vote and which means we're essentially old enough to be your partner. So we're always looking for new partners, visit us at blendinteractive.com.

(04:26)
All right, let's welcome our guest, Elias Lundmark. Hi Elias.

Hi, Corey. Thanks for having me.

Thanks for being on here. Deane knows you because you actually work for the same company as him.

Yes. Yes, we are in the same company. So I'm a product manager at Optimizely. I'm responsible for what we call DXP cloud services, which is the platform as a service that powers our CMS and customized commerce offerings. Now, we just recently changed the name of that commerce offering and I'm still trying to get it to roll off the tongue, but.

That's a good introduction because I know you, Elias, as Elias the guy who keeps our stuff running.

I do cloud things, that's what I do.

Elias, if anyone listening is like me, they know that hosting is a thing that a website needs and that's about it. That's about where it is. Can you explain at a real high level what we mean when we're talking about hosting for a website?

Right, so in its most rudimentary form, it's taking a website that you've created, operationalizing it in a way that your target audience can actually consume it. And say you've worked on a WordPress site locally on your laptop, and once you expose that to the internet somehow, it's most likely not a good idea to expose your laptop to the internet and running the sites off of your laptop. So you have to figure out a way to host it in a way, you have to have some sort of server, a web server, and expose that to the internet so that someone can actually reach your website over the public internet. So of course there's more nuance to it, but in its very simplest form, it is that.

Yeah, I think that what people don't, especially, I still struggle with this I think myself, but people often forget that a website, which they see as a thing that is just living out there is just a bunch of files and those files have to live someplace.

Yes, and that place has to be available so that someone can actually consume it and it needs an amount of resources to actually handle those requests and so on. But hosting can be as simple as running something on your laptop or on a Raspberry Pi, or at a scale where you're serving hundreds and thousands or millions of users on a daily basis. And at that point you're most likely running something on the public cloud with huge amounts of resources of course, depending on how complex of an application you're trying to host.

So I remember back in the day when, I mean my first hosting account, I paid $50 a month for a raw Unix account somewhere. But when I started working in corporate IT doing content management in the mid to late '90s, when we wanted to host something, we had to actually order a physical server and unbox it. I mean, literally we wanted get excited because the server arrived, we had to unbox it. It was in a pallet in the hallway, and we would unbox it and put it in a rack.

(07:39)
And then I would have to go into this freezing cold data center and install a content management system from a CD, that you would pop out the little unpowered drive, you would hit it and have to pull the little drawer out and you would install it. And I stood there and I was working in this freezing cold data center on this seven inch monitor. And Elias, I don't know how far you go back in the history of the web, but let's talk about how that's evolved from server sitting on pallets in the hallway to you can flip a switch and get an enterprise CMS in 30 seconds.

Yeah, that's definitely an interesting evolution. And I mean, it's not that long ago. I did my bachelor's degree between 2014 and 2017 and we worked a lot with rack mounted stuff, installing VMware on fiscal servers and dealing with that redundancy and storage attached networks and so on. And we still do on-premise hosting as well as at Optimizely, we call it everywhere, but we have two data centers running in Stockholm. But we've definitely come a long way, especially with how easy it's to start running something in a public cloud. And you can just get a huge amount of resources at a moment's notice.

(08:55)
Of course, the easiest way if you're coming from that on-premise world is infrastructure as a service where you essentially do the same thing, you get access to one or more virtual machines and run your workloads as you would on your on-premise stuff. But then you also have the platform as a service offerings, which we offer at Optimizely, which takes all of that operating systems and that daily running and patching out of the equation as well. And then you have something that's really easy or really fast to stand up.

Yeah, I think there's been an evolution away from when you look at hosting, it used to be that you would take control of the entire machine, right? I remember renting for $150 a month when I was at Blend Interactive, our first server we had, we got an entire Linux machine. It was running cPanel, that's how we interacted with it. And we can get a command line if we had to do, but we paid $150 a month for it, and you don't do that so much anymore. I mean for an end customer, you don't get a whole machine, you just get the part of the machine you need, just the computational resources you need. And I think that's been a shift in hosting. Does that reflect what you're saying?

Definitely. And I mean I think the first evolution, if you will was virtualization. Right, so you had one big server or multiple big servers that you could effectively split up in bite size chunks and you could provision something to an application in the sense that it only got what he needed. So you needed four cores and you needed X amount of RAMs, that's what you got. And you were able to quite efficiently split up a big server into manageable chunks and be really flexible with it.

Let's talk about what keeps a guy like you up at night. What aspects of hosting? If someone who manages hosting and designs hosting and manages the product that really underlies all of our hosting, what are the things that your type of people are concerned about these days?

Well, when we first started our journey on Azure and public cloud, it was definitely not as stable as it is today. And what would keep one up at nights then would be a complete failure in an Azure region, say North Europe would go down for just a couple of hours, we would have massive outages.

So this is not somebody's website is broken, this is the whole thing underlying hundreds of websites breaks and it's nothing we can do anything about.

Exactly. But we also have redundancy between data centers and so on. But just standing up, that would be a massive effort. But of course redundancy and availability and nowadays security as well.

Elias, can you help me understand the benefit of one or the other between hosting something yourself on premises and doing something through the cloud?

It's quite easy to, as Deane said, order a server, stand it up in a rack, install some kind of server operating system on it, but then that expands quite quickly as your needs as a business expands. So from that server, at some point you're going to have to replace that server because your warranty runs out. You don't have any guarantees on the hardware, so at that point you have to start thinking, all right, how do I decommission this server and stand up the new one? You also have to manage things like patching, and maintenance, and running cost of that. And of course as the needs of your business scale, you need have more traffic coming to your sites, you have more robust security needs and so on. That single server will most likely have to scale to more servers and firewalls and so on.

(13:13)
And there's a non-trivial amount of complexity and ongoing costs to operate that, right? But whereas something with a public cloud provider, all of that is just abstracted away from you. You don't need to care for that. You'll just pay a operational expenditure it's called on a monthly basis say, and Microsoft or any other public cloud cloud provider would just sort that bit out for you. And of course, that also leads into the cost aspect of it because with buying service, you have something called capital expenditure where you'll buy everything upfront and you'll use it for the next three or four years or however long your warranty is on for. But with the public cloud, you'll instead have operational expenditure where you'll just pay by the month instead and you can just stop using those resources whenever you want to. It's not something that you physically own that you have to care for.

Hey, let's talk about site uptime because I think it's very sexy and glamorous to ask for an inordinate number of nines. And I wrote about this in the book is that we would get, when I was at Blend, we would get RFPs all the time, and clearly the RFP had been passed around to all of the stakeholders in the organization and the CIS admin infrastructure person added, "Well, we'd like five nines uptime." And so we would dutifully scope that out, what it would take to keep a website up for five nines. And five nines, if you haven't read the book, but certainly hope you're listening this, you've read the book, five nines would be 9.9999%, right? The nine, it's on both. It's all the nines on either side of the decimal point. So this is something like 30 seconds of downtime a year, and whenever we'd price this out, I mean we would do it.

(15:14)
But the numbers would just be stunning. I mean, to keep five nines of uptime, you have to have multiple redundancy across multiple different regions on different sites of the world, it's insane. And so we would always sit them down and say, "Okay, for one 20th of this price, you can get three nines," and three nines is four hours of downtime a year. And so what we would find is we'd always roll our eyes a little bit, I can say this now because I'm not at Blend anymore, but we would always roll our eyes a little bit when people would come in and say they want five nines, because we're like, "Yeah, you really don't." When you talk to people about site uptime, do you think that there is too much lip service paid to things without being cognizant of how much this is all going to cost? It's very sexy and glamorous to ask for crazy hardware, it's also very expensive.

Very much so, and I start to think back around the time I was studying computer engineering and just solving for something like storage and that sort of availability, because spinning hard disks tend to fail quite a bit. Just handling that, both redundancy and backup strategy to, and the disaster recovery if something really goes wrong is that you come into setups like active-active, which are really expensive to build on premise. But yeah, that's definitely where the cost of something starts to escalate. And when you have that requirement of, all right, I need five nines, then it's certainly going to get expensive, it really is. But then it's also why public cloud providers are so good at that because while they have so much redundancy built in just because of the sheer amount of hardware that they have. So it's really easy for public cloud providers to provide that, whereas if you're building something that is tailor-made for a specific application and doing that on premise, it's going to be much more expensive.

I think I wrote about this in the book too, is that disaster recovery, DR environments is the requirement as many IT requirements. I don't know if it's required for an industry, but many organizations have DR requirements, they have disaster recovery environments, you have to have a DR environment somewhere else. I was actually meeting with a company recently, and they mentioned that they had a DR environment exactly 10 miles away because that was the minimum distance they were allowed to have it in their IT policy, but they didn't want to have to drive too far if they ever did anything there. So they had it exactly 10 miles away.

(17:58)
I have never in my 15 years of working services at Blend, I have never seen a customer fail over to a DR environment, because bringing up a DR environment is really non-trivial I think. I mean there's some complexity there to bring up a DR environment. And additionally, once somebody fails over to a second environment, if they start editing content there, then it gets really weird because now you got to sync it back. All sorts of things can result. And so you have to cross a threshold of disaster I think before you're willing to actually go over to a DR environment. What's your experience? In your years in IT, have you seen these environments used or are people just checking a regulatory box?

Very much, as you say, a tick box activity. To a large extent, of course, acts of God happen and we do have to be prepared for a worst case scenario. This is often called part of a business continuity plan, disaster recovery being a part of that plan like, "All right, if things go wrong, how do we continue to operate as a business?" And of course, depending on your company and your situation, those regulations will vary of course. But yeah, I've seen disaster recovery happen. It's never fun when you have to do it, but for us, it's quite inexpensive to do it because, well, that is already parts of the public cloud infrastructure. But trying to do that on-prem with on on-prem resources are much more different.

What's hard is the disaster recovery is hard enough, but I always think there's a disaster recovery-recovery, which is now failing back to the original environment. That's almost even harder because again, if data was changed on the first environment or in the second environment, the DR environment, now you have a synchronization issue. In general, Elias, if I have a website and I'm looking to host it, what should I be concerned about? What should I be looking for in hosting providers?

I usually break it down into four different areas, of course, depending on the scale that you're trying to achieve. But if you're a mid-size company, you should really consider four different areas. The first one being performance and scalability. How many users do you actually need to serve on a regular basis? What do you think your traffic patterns will look like? How much resources are you going to need to meet that demand? And of course, you also need the performance sites, especially in today's day and age. You need something that is quick, right? Because users just don't have that time to focus, and they'll be bored really quickly. And that's where hosting can actually play a large part of it, especially if you start looking at things like content delivery networks and so on to speed up your side. So performance and scalability is one aspect of it.

(20:51)
And then I usually try to talk about developers as well because, well, you are going to look to launch your site, but launching your site is only the first step of your new exciting journey, right. After you launch your site, you're going to look at, all right, how are users actually interacting with my site? Where can I improve that experience, and what new features do I want build on my new site? And that's going to take more and more iterations, and your hosting platform plays a big part in continuous integration and continuous delivery. And then security and operations as we've discussed a bit where with availability, but also confidentiality and integrity, what kind of information are you gathering on our site. We have form submissions, do you handle personal data in any type of way, and what kind type of security controls do you have for your website and so on. But yeah, no, I'd say those are the three main things. Performance, scalability, security and privacy, and empowering developers if you will, to continue to develop that digital experience.

I always think you cross a threshold. The question I would always ask to customers when I was in services was, do you need your website to stay in business? Is it a line of business thing for you? If you're like, I don't know, an architecture firm and you have a website, maybe list your case studies, how to contact you, et cetera, that's great and it's very important, and I'm not trying to minimize that, but if that goes down, you're still in business, you're still generating home plans.

(22:34)
If you're Amazon and the website goes down, you are literally out of business until that website comes back up. And so I think that was the line you crossed, and I know at Blend, Corey, I don't know if it's changed, but we did not love hosting things that were line of business mission-critical stuff. We did not love that, because if it were to go down and we had a great record of reliability, but somebody was out of business until it came back up. And so you really cross a threshold there where it gets much more serious in terms of what you do for hosting and reliability and uptime and such.

I feel as though a lot of the organizations, I might may not understand this as well as I think I do, but the organizations that rely on it for their entire business model also often seem to keep it in-house. Does that sound right in that they have some level of control over it versus a university? Again, if their site goes down, their site goes down, and kids can't access the portal for a bit, but it isn't a case of everything falling apart and they have to close on a building.

Yeah, I think that's true, especially if you're a company who has a large in-house IT infrastructure and track record in hosting other application, then it just feels good to continue on that path. So that's not uncommon of course, and also, why we sell licensed software as well as our cloud services. But overall, to your point in insecurity, there's usually this saying if you will, about protecting assets. Say you buy a lawnmower, but your backyard is completely unshielded from the rest of the world and your neighborhood, how much are you going to spend in protecting your new lawnmower? You might build a shed, you might build a fence around your yard, you might put barbed wire around your fence, you might put up security cameras, but where is that sweet spot? Where do you actually have to pay too much for controls to safeguard your lawnmower?

It's a pretty nice lawnmower if it's got.

Right.

Security cameras.

Let's just talk briefly about, I don't know how to phrase this, territorialism when it comes to hosting. We worked with some organizations where people would get weirdly territorial about giving a website to somebody else. Higher education, colleges were notorious for this because colleges would usually have some IT administrator that'd been there for 40 years. And I remember going to a college once and asking where the web server was, and they literally told me it's that machine over there with a cup of coffee sitting on it, they keep everything in-house. But now with the cloud, everything is distributed. And do you still see, Elias, organizations getting weirdly territorial about where their stuff is? And do you think that those concerns are rational or are people just being weird?

Well, it's definitely a shift, especially as we've started containerizing applications and trying to get out of the business of directly configuring operating systems, scaling horizontally quite far and so on. And I guess [inaudible 00:26:03], a client called today about a customer who made the transition from on-prem to our cloud services who wanted to get into the operating system, because they've done that in the past with their on-premise stuff and trying to explain that there is actually a lot of abstractions to the operating system, and we're not supposed to work in that fashion working with public cloud resources. But it ended up being more of an observability issue rather than anything at the heart of it because, and I think that's probably the biggest shift just because, well, you don't have that bare metal access anymore. So how do we solve for things like observability when we have abstractions in place?

The biggest thing I think when people move our particular CMS software from on-prem to the cloud is that our cloud systems don't really have a native file system anymore. And I know from integrating our CMS for 15 years, I used to write things to the file system all the time. Well, you don't really have a file system in the cloud, and it really has changed. And that can be a little off-putting and disorienting at first, but you realize you gave up things in order to get other advantages. And if you wanted access to the core OS, and if you want to have access to the file system, well, then you shouldn't have gone to the cloud. You don't get all the benefits that the cloud offers, so it's definitely a balancing.

Yeah, exactly. Things like writing to the file system at run time, and then you realize, oh, right, this actually scales out automatically and it scales in automatically. So I lost my file star, I wrote [inaudible 00:27:48] to the file system.

You wrote a file to a server, just not the one that's currently serving your website.

Exactly.

Well, Elias, you will continue to be the guy that keeps all of our stuff running. I will continue to be the guy who attempts to break it all the time and.

And I'll continue to be the guy who doesn't know anything about it.

That's been the hallmark of our relationship. Elias, thank you very much for joining us. We appreciate your insight.

Thanks for having me.

Deane, we're back.

We are, and you know what? I don't feel like I set up Elias enough, so let me just explain a bit, and I asked Elias in the interview what keeps him up at night. And to give you the idea, the scope of what all Elias deals with, we are at Optimizely, really a cloud and platform vendor. We host pretty much all of our customers, web solutions, digital solutions, and we have thousands of customers. And so Elias manages the platform that keeps thousands and thousands of customers on the internet 24 hours a day, 365 days a year. And so when I asked him what keeps you up at night? And that wasn't just an aphorism. Literally if I was him, I would sit up at night and stare at the wall and how bad things don't happen.

I don't. It feels with that many sites and that many, when you're thinking about this, it's not just that many sites, it's that many businesses who have gone with Optimizely to the level that they treat the content as a business asset, and their business asset is hopefully safe and secure, but it's at risk if that hosting solution doesn't work whatsoever. My understanding of the idea of posting on the cloud is it's more efficient, this redundancy that obviously it's helpful, it's scalable. I didn't think I got an answer from Elias, why would anybody want to host on-prem, is it just a control issue?

Yeah, there are some organizations that are still, the biggest ones we've seen that are slow to move are healthcare and financial services, still really concerned. And I think a lot of organizations just really haven't updated their policies. They're still working with IT policies that were back in the day when on-prem was a big deal. And so if they take a fresh look at them, they would realize that they're really behind the curve by not moving their stuff to the cloud.

(30:13)
The other thing, and I touched on this with the interview in Elias is that there's territorialism, and every once in a while you'll run into a person at a client who's just like, "Nope, I got to own it and it's got to be in this data center." Right now, what we're seeing as a vendor is a huge wave of these territorial customers now saying, "Okay, I'm done. I don't need to put this in my own data center." And so moving off, but I met with a customer, and I'm not implying this customer's territorial, but I met with a customer in the Midwest last week who literally, we were sitting in a conference room talking and their website, which is a brand name you would absolutely know, was running about 20 feet behind us.

That's incredible.

So yeah, it happens. And now we look back on that and we think, "Oh, that's amazing that they're doing that," but back in the day, that's what you did. I mean, the stories I was telling in the interview about installing CMSs on servers. And freezing cold data centers, that's what we did. And just everything evolves, and I think it's going to be rare for people to stay on prem. Although I did just note, there's a very interesting article from David Heinemeier Hansson from Basecamp, the company that runs the Basecamp app. And they are famously right now moving off the cloud back to on-prem. They spent 3.2 million last year on cloud services, and they can believe that they can cut that down to something a million dollars. And apparently their biggest service or server cloud cost is storage. They have something like 16 petabytes of storage, and they believe that they can cut that.

(31:46)
And so I really want to thank Basecamp for doing this, because it's going to be interesting to watch, and I don't wish any poor things on Basecamp, but thankfully, they're going to allow everybody to watch what happens when a large vendor moves from the cloud back to on-prem. And either it's going to go great or it's going to go poorly, and we're all going to learn from it. It's going to be an interesting thing to watch and play out. But that has happened with some other vendors as well. I believe Dropbox was on AWS on Amazon, and they were obviously, Dropbox storage costs are through the roof, it's literally what they do. And they built their own data centers and built their own infrastructure, moved back on prem. Now, in no way am I implying that Basecamp and Dropbox are analogous to the average companies.

Right. These are companies large enough that they can afford to essentially create what is their own-

Right.

Cloud offering.

Basically their own cloud.

Yeah.

And so it's just going to be interesting to watch and I think this is a discussion and a decision point that's still playing out and we'll probably be arguing about this 10 years from now.

Well, this has all been illuminating and I know a lot more than I did. I edited this chapter and I feel like I understand it more now after this podcast than I did after editing the chapter.

That's terrible, because I wrote most of this chapter, so thank you for that.

Yeah, perfect. Well, a huge thanks to our guest, Elias Lundmark over at Optimizely, great to have him on. The Web Project Guide is a product of Blend Interactive. Again, our web strategy design and development shop 18 years old, that builds websites and guides teams through complicated web and content problems. We're dedicated to making great things for the web, and this podcast is one of those things. This is episode 17 of the Web Project Guide, which also corresponds with chapter 17 of the book, Plan For Hosting.

(33:32)
You can read the full text of this chapter at webproject.guide/hosting. I joke about it, it's actually a pretty concise and great view of what hosting actually is. And it goes into a few things that we didn't talk about today that will pop up. Listen, if you have to choose a hosting solution, it's worth reading it just to get understand the terms that are used. Anyway, as always, you can order your own copy of the book and you can read it on your own. You can get either a physical copy of the book, or a digital PDF by visiting order.webproject.guide, and you can also pick up a gift set that includes festive Web Project Guide socks. I don't know why I said festive. What holiday do you think Web Project Guide best fits with, Deane?

I don't know, but I'm looking around my floor right now because my dog had one of those socks in her mouth, and she puts great value on things that she finds. If you give her something that's cool, but if she finds it and can grab it and run away with it, it's the most valuable thing in the world.

Absolutely.

And she found one of those socks and she was running away with it, and you know not to try to take it back from her because then she thinks it's a game and she'll destroy it. But if I had to pick a holiday, I don't know, Flag Day.

Great. Oh, perfect. So we're going to have Web Project Guide flags for Flag Day, which I believe is in May. As always, leave a review on Amazon for the book, leave a note or five star review on your chosen podcast network, and thank you for joining us for another month. We will talk next month about, let me see here. Oh, we're going to wrap up discussion around the technical foundation of a project by helping you select a implementation team. And until then, go do amazing things.

Good luck.