Episode 2 – Alll The Stuff Underneath

It’s episode 2 of the Rent, Buy, Build Cloud Native podcast.  This time, we’re talking about infrastructure, all the bits and pieces that make your Cloud Native platforms actually go.

James Hunt
Welcome to Rent, Buy, Build. I’m James Hunt.
Brian Seguin
And I’m Brian Seguin.
James Hunt
Each week we’ll discuss the pieces and parts of cloud native platforms and answer the question, “should you rent this, buy this or build it yourself?”. This week, we’re talking about infrastructure, all the stuff that sits underneath your cloud platform servers, networking, gear, power storage arrays, and more. So Brian, when we talk about infrastructure, we’re talking about all that stuff that you don’t want to have to worry about when people talk about containers and cloud platform, all the stuff that’s not you know, cool and sexy, right? servers, switches, routers, and all that stuff. And as far as I can see, we’ve we’ve got three options. As far as renting, we’re talking, what do you think we’re doing for rent.
Brian Seguin
So rent, I think this is going to be your on-demand cloud providers, this is going to be like your big three, your, your Amazon Web Services, or your Google Cloud. And your Microsoft as your platforms are actually they’re not really platforms, they’re, they’re really IaaSes, and then they have platforms alongside of them. But we’re not gonna we already talked about platforms. And, you know, on the last podcast, we’re gonna talk specifically about the actual infrastructure today.
James Hunt
Right, the infrastructure as a service, aptly named the IaaS layer, right, AWS, GCP, Azure, there’s a couple of other players in the market, right? Linode, DigitalOcean are all is is I define an IaaS as an on-demand API driven bit of infrastructure, usually, we call this virtualization as a service. And it’s, you know, given your credit card, and I’ll give you a server that you will be billed hourly for. So if you run the server for, you know, three hours, I might charge in nine cents,
Brian Seguin
right, and the price of the room, the price of the rooms per se, right, because just to make the analogy back to the housing example, the price of the rooms, it depends on the length of the lease, and the number of rooms are renting and the type of rooms you’re renting. And actually, it’s kind of confusing for most for everyone, actually, between all the different sizes, because everything is named differently. They all have different type of pricing plans. It’s not a one for one,
James Hunt
well, and even in an individual vendor. Right. Corey Quinn runs a consultancy called The Duckbill Group, their entire purpose in life is to help you make sense of your Amazon bill. And either get it lower or understand why it is what it is, when you’ve got a cottage market, or cottage industry, of explaining how to pay for another service, you know, you’ve got a lot of confusion there.
Brian Seguin
Not to mention that. But there’s a whole FinOps movement, which is part of the Linux Foundation, that’s gaining a lot of traction, a lot of Fortune 2000 companies are starting these fin ops departments. And their sole purpose is to figure out cost of cloud,
James Hunt
Right to tie back billing to show where the money is being spent, for whom, etc. So the chief advantage of renting your infrastructure, really is that you get to focus on what you are trying to build that from from the work aspect of it the the delivery aspect, your responsibility stops at the server, right? You don’t have to worry about networking, patching, you know, Juniper Networks systems, or F5 load balancers or, or IDS’s and firewall management and all that stuff. You’re basically just here’s your servers, you wire them up together correctly, and then you deploy your code to them.
Brian Seguin
Right? I like that. There’s no upfront cost.
James Hunt
Yeah, no. The pay-as-you-go model is extremely good for small- to medium-, and even to large-scale businesses, as you get into reserved instances and other things where you can kind of offset that, that ballooning cost of more and more rental, which is actually a benefit, right, the scaling model of rented infrastructure is far superior to the other two options, we’re going to talk about the buy and the build. Because in the case of, for example, Google Cloud compute, Google has these data centers with idle time that they’re willing to sell you when you need it. Right. When you need to scale your offering, let’s say you get the front page of whatever blog is important to your industry and you get an influx of traffic. All you have to do is place an order for more services, and they spring into existence.
Brian Seguin
I do want to touch on that real quick, because there’s a big misconception in non-technical folks that just because you can have the flexibility to procure more services doesn’t mean your actual application can scale the way it does. So that that’s kind of one cautionary tale that, you know, just because you’re in the cloud, just because you’re on the is your application might prevent you from actually scaling
James Hunt
Yeah, oh 100%. So when we say that your your the scaling is built in the scaling is built into the infrastructure, what you do with the scaling as you get more, more capacity is up to you. And all kinds of factors in your application design are going to enter into this. Namely, things like persistent storage, if you have, for example, I’ll take WordPress, a very common blogging engine. WordPress has a persistent volume that it uses for uploaded files, right? So if you scale WordPress, if you get you know, you run a blog, like Huffington Post, right, Huffington Post is a WordPress installation, I’m assuming it’s been heavily modified, because you cannot scale one WordPress instance into two, because what will happen is you won’t have the same uploaded files on both instances. So these are things that have to be built into the application. The the advantage of the public cloud infrastructure and renting your infrastructure is you don’t have to solve both problems, right, you only have to solve the application scaling, because the infrastructure scales as an intrinsic feature of the offering.
Brian Seguin
I also like that you can have almost a physical scaling as well, because these big cloud providers, you can co locate where your runtimes are or where your applications are to avoid, you know, latency or to have giant disaster recovery zones. If I’m operating a company, and I’m deciding to expand into an Asian market, for example, in the rest of my markets in the United States, I don’t have to go build physical servers out in Asia, I can actually just say, Oh, well, we’re already on Microsoft Azure, they have they already have all that figured out over there, I’m just going to move my workload over and just rent some more services from Microsoft.
James Hunt
Yep, yeah. And in most cases, I would, I would say a lot of people that are currently able to take advantage of data centers that are run by Google and Azure, or Microsoft, and even Linode. In places like Singapore, those businesses would not be able to navigate the the very complicated bureaucracy, paperwork, and language barriers to even find a colo facility in an Asia Pacific or in the Middle East, or, or in anywhere. That’s not where they’re currently doing business.
Brian Seguin
That’s right. Now, one of the disadvantages of this, I will say is actually the variable cost. Now, I like the variable cost. But it’s also a complete disadvantage, because it’s a lot to consider. You know, from a financial planning standpoint, accountants and finance people are used to fixed costs amortization. They’re used to depreciation, they’re not used to their cloud spend doubling and tripling it within months, right. And, and the biggest problem really goes into the the tie back between the cost of cloud and the actual business value added, there’s no way there’s no direct way without a lot of financial planning and financial work to actually tie those two together in the public cloud space as their as a cloud spend expands,
James Hunt
Right? If you implement naively on top of public cloud, you’re, you’re basically cutting off your economy of scale, right, the more stuff, you do the same price, right, you’re gonna pay the same hourly rate for an m3.xlarge on Amazon if you have one versus 1000 of them. Whereas in our other options for building buy, you get to start actually taking advantage of the economies of scale, by buying more in bulk and getting a better discount. Now, there are things like reserved instances and other you know, pre paying for stuff that will get you some some reduction in price or some reduction in that bet variable costs. But the very nature of the pays yo plus the easy scaling, makes it very possible that at factors outside of your control, right, market movements, or increased, like take a look at Zoom. For example, in in, say, January of last year, Zoom was a video conferencing platform used by a lot of the enterprises. But come March, April, May, Zoom just took off because of the pandemic. And because people needed to work from home who hadn’t previously been working from home. Now, Zoom is — they’re a very large company. They’ve been doing this a while. So they knew what they were getting into. But had they naively implemented on public cloud, their costs would have skyrocketed almost overnight, due to nothing that they did. Right? All these factors are outside of their control.
Brian Seguin
Right. One of the other disadvantages is the potential lock-in–
James Hunt
you’re gonna you’re gonna bring up The L Word.
Brian Seguin
Well, I mean, a lot of developers and a lot of applications are built directly to run natively on top of one of these infrastructures. They also might be natively consuming some of the services provided by these infrastructures. So the switching cost of moving to another cloud vendor is actually going to be significantly higher than if They were to say have a platform and their own layer in between them. So that that’s one of the things that I would worry about is the opening up the regulation of how much I can how much volume I can actually give for my, you know, developers or my engine engineering teams to rent these spaces as well. Right? So there’s a lot of flexibility out there. And it’s, it’s actually extremely difficult to regulate it from a from a top down perspective,
James Hunt
right. Yeah, and as we were talking about, you know, your application isn’t just going to natively scale, you have to kind of insinuate it into the fabric of whatever is you’re dealing with, right? If you’re going to use all the the panoply of services that these IaaSes is offer between, whether it’s cloud sequel or RDS that they’re different enough that you have actual engineering work to do to either isolate yourself from those differences, right with, as you mentioned, another layer in between you and the service provider, or as you move, make those changes to the new thing, old old thing goes away, a new thing comes in, you switch off of RDS onto cloud sequel, your SQL queries may not change but how you connect to it does. So there is a there’s a friction and impotence to get over as you’re looking to move, which is why I’ve honestly not seen a lot of people who aren’t, who don’t go into public cloud with this concern actually surface this concern. If you start on AWS, and you’re fine with that, I think most of the people I’ve talked to have have just stayed on AWS or stayed on Google,
Brian Seguin
a lot of people are okay with being locked into these vendors, there’s there’s a ton of actual benefits with all the other services that are provided with it that they can just consume now is, is that going to be the same thing for you know, a lot of Fortune 2000? organizations? No, because they’re going to have advantages actually switching clouds, because they can, they can get a lot of scale
James Hunt
and volume to play one off of one
Brian Seguin
Exactly. Now, that kind of goes into the, that kind of goes into the buy, because by you have a lot
James Hunt
Speaking of segues.
Brian Seguin
Yeah, that kind of goes into–
James Hunt
When we say buying infrastructure, there’s there’s really the buy build on this one is very intertwined. Because if you’re not talking public cloud, you’re talking on-premises. The question and where we’re gonna draw the line today is, we’re gonna say buy is somebody else’s premises, and your servers and build is everything is yours. So, so buy is really about colocation, and if you’ve been in the industry, as long as I have you remember, colo being like the gold standard of internet hosting in the ’90s, you know, you could run a server over an ISDN connection from your house. Sure. But if you were serious and professional, if you were serious and professional you ran on a colo facility and a colo facility is or a data hotel or whatever name you want to use for it, it’s a big building that somebody else runs that has redundant power generator backup, usually three or more ISP hookups and connections on fiber, very fast connections into the backbone of the internet. And what they do is they rent you volume or space, they will rent you a cabinet. That is a place where you can bring your own servers, slot them in, power them up. And and they give you a couple of things in the offering. So the most important being remote hands, right? Somebody will somebody who is on on-site 24/7 can go in and you know, and again, I’m going to show my age here, can pop a CD or a floppy disk into the server or plug a USB drive in or or hit the power button and physically interact with the equipment so that you don’t have to. So this is somebody else runs the building, you bring bring the server so you’re still purchasing equipment, but you’re not worrying about networking, or power or cooling or any of that stuff somebody else is handling that this part of the industry has been in decline because of the rise of public cloud providers that are much easier faster and, and generally less hassle to deal with. But if you have concerns about you know, servers, and we’ll get into the why in a little bit, but you can still use these people and people like Rackspace or sunguard or equinix. Those are the big national players that have these these large colo facilities. But there’s also local ones. There’s a lot of regional providers who have you know, maybe 30 cabinets of space and you can go rent from them.
Brian Seguin
I guess one of the biggest advantages and disadvantages from you know, my finance brain is you actually have a fixed cost and you have a fixed amount of space, right? Which means I can amortize this, I know what my spend is I can I can make a lot of strategic planning but I also don’t have the flexibility to scale. So this is really good. If I’m a organic That knows or has, or has workloads that are going to be consistently expected over the next, you know, five to seven years, right. I also like that I can typically have multiple different vendors in this space, I can, I can easily go and pick from different colo facilities, I can easily go and maybe even have the same colo facility but or same colo vendor and have them have colo locations in my different regions that I write that I
James Hunt
might have an East Coast data hotel on a West Coast data hotel all build through the same vendor relationship,
Brian Seguin
right, which that makes things easier from a management perspective. Also, with the with the restrictions, it makes it easier for me to not have to worry so much about my engineers, you know, going off and doing what massive spends, although I do I have to worry about things spinning up and potentially reaching the capacity of my physical infrastructure, right?
James Hunt
You’ve really got two concerns on that front one is saturation of workload, right, you’ve got too much stuff hitting the limited and capped number of servers that you have in the colo facility. And at some point, either performance suffers, you know, the latency goes up, processing times prolong, or you just stop accepting new stuff, right, the server, the website, spins forever, because it just can’t handle anymore. The other concern is the more insidious one. And it’s the engineering cost of not being able to just reach into the public cloud is toolkit and find something that works, right. So let’s say you’re building out a new piece of functionality into your offering, and you’re a colo, or solely 100%. colo, you’ve bought your infrastructure, if you need a key value store, somebody has to spin that up now for you, right, and we not to get too much into the containerization Kubernetes. side, if you don’t have a platform like that, you’re going to be either adding more processes to existing servers, which is going to increase the load on your systems administration staff. Or you’re gonna have to bring new servers into the mix, which might impact your budget, right. So if somebody says, Hey, we need a new database server, and the one we have isn’t going to work, I need another for you of cabinet space that increases your cost. But it also is time that the engineering team has to build into their schedules and their sprints, to focus on solving the problems that are already solved for you in a public cloud scenario.
Brian Seguin
Right. And that’s when we see both in the buy and build scenario here is a lot of over provisioning over purchasing over hardwearing. whatever term you want to set to it, one of the one of the other concerns that I have is it’s non trivial to move out of your data center. Right? It’s, it takes a lot of work. And with the decline of these colo facilities, should I be concerned about you know, my colo facility, going out of business and having to move to somewhere else, or even move to the cloud
James Hunt
is a very real concern, and definitely one that should enter into the calculus of whether you should buy your infrastructure or rented. And that’s why you see a lot of people are going down the hybrid model, because they have some of the need for the advantages, you know, a good baseline for cost. As we get into the build, which we’ll talk about very shortly, the security and performance concerns enter into it. But they want to fail over to public cloud for less critical things.
Brian Seguin
Yeah, and just to give an example, I know of a we had a customer that had 30 days to vacate their facility and move to a completely new facility because they were just going to shut everything down.
James Hunt
And and listener, I want you to know when Brian says they had 30 days, in 30 days, they were cutting the fiber cables. And I don’t mean like metaphorically cutting, I mean, they had a backhoe ready to dig them out of the trenches. This was not a soft deadline. This was a hard 30 days out. proposition that–
Brian Seguin
We joked about physically going there and picking up some of the hardware and driving it to Texas.
James Hunt
Right how much is a U-Haul gonna cost? What’s the get like this is all stuff that entered into our decision and our way forward on that one. And it’s a very real thing. Because sometimes your leases expire on these colo facilities. And even if the colo facility is still in business, they might want to renegotiate, you might not like the new terms. And that’s that.
Brian Seguin
And maybe that’s why you start to build it. Maybe that’s why you want to put the big investment into building your own infrastructure, building your own data centers, getting everything so you can you don’t have these big buildings for at least 20, 30, 50 years, right?
James Hunt
Yeah, no. And that’s that’s what we mean when we say building our infrastructure is you’re gonna take an empty lot somewhere and you are going to sink footers into the ground. And you’re going to literally build your infrastructure from the the footings on up to the roof, right and you’re going to put in, you’re going to have to put in cooling, redundant power generators, transfer switches, all the things that make the computers run so that you can make the cloud on top of it run is your responsibility in a build proposition.
Brian Seguin
I like it from a fixed cost standpoint, but–
James Hunt
It’s the ultimate fixed cost
Brian Seguin
It’s the ultimate fixed cost. But I also have a lot of liability, I have to be responsible, I have to increase my insurance, I have to have my own physical security, I actually have to pay people to go to the location if need be, I have to also, now I have to replace my hardware and both the colo facility and the end the data center that I’m building myself, regardless, so that’s right, that’s gonna be the same, I also have to worry about my fiber lines.
James Hunt
And when you’re when you’re built, when you have built a data center, your hardware refresh schedule now includes not just your servers, but your network switches, your cabinets themselves, your storage arrays, your cracks, your AC units, your
Brian Seguin
parking lot, outside all of that stuff.
James Hunt
Because you’re on the hook for all of that you have full and total ownership,
Brian Seguin
you also probably have to build two since you want a DR facility unless you’re going to do a DR offsite on a on a colo or a public cloud.
James Hunt
And you can’t just say that this half of the building is the A availability zone. And that half is the B; I have seen that it doesn’t actually work when somebody cuts power to the neighborhood. So yeah, build is the ultimate “not invented here” by the way, of cloud, it’s I’m going to do all of this myself. And really the only advantage is that known pricing profile, or, and the full control of being able to spec everything you want. But in reality, Brian, I don’t see anybody building new data centers in 2021.
Brian Seguin
So when do we when do we use what and why they when do we rent? When do we buy? When do we build and why?
James Hunt
That’s a great question. I’m glad this podcast is going to answer that once and for all. For my money, we’ll work backwards, right? Well, so the build is the only time I think you should build a data center is if you have already built a data center,
Brian Seguin
you have the sunk cost of the facility, you have another 20 to 50 years on the facility with some minor upgrades. Yep. You have the scalability to do massive procurement purchases for reduction of hardware costs. Is that what you’re saying? Right?
James Hunt
Yeah, because if we look at it, the most of the most companies that are running their own data centers are of a certain age and size. Most of them are at least 30 years old. And most of them are above 2,000 to 5,000 employees, and all of their data center is that they’re driven by two things, either, this is what they had to do back then, because AWS didn’t exist until 2000, and something. So in the ’80s, ’90s, etc, you built a data center, because you had to have some place to put the machines. In fact, oftentimes they were called the machine room. And that’s just you have that and you’ve built your applications to take advantage of that, right, you’ve probably got sand storage, you’ve got fiber connected drives, you’ve got all this stuff, you have a second data center, and you’ve already dealt with, you know, you probably replaced your frame relay networks with wagons, all this stuff is already there. So it would be silly to not use it while you have it. But if you’re starting out greenfield, I think public cloud is the best option for you. Because with a greenfield project of building out a whole new cloud native platform or an application that you want to put on top of cloud native platform, you have the ultimate freedom to implement however you wish. And you’re going to be able to start small in the pay as you go model with public cloud and build out. So if you have bursty workloads, if you have an application that has non regular demand for it, it’s maybe it’s seasonal. Maybe it’s event based. I look at somebody like Ticketmaster, who has a large burst of traffic around event, occurrence and ticket sales opening. And then it kind of dies off. Those types of workloads benefit from that public cloud because they’re the pay as you go model is an excellent use of idle time because you’re not paying for idle time because you’re spinning down as you scale down.
Brian Seguin
Yeah. So the build today for the infrastructure is really the edge cases. These are people who have specific hardware performance requirement needs where they they need to have specific hardware, they, you know, and that’s not available for them on public cloud or even in colos, maybe it also might be a location thing, they may need to be right next to the New York Stock Exchange, they might just happen to have a facility there that they could turn into a data center. And, you know, take advantage of that proximity. It might be a security thing, right, you might need to have air gapped environments that need to run offline. And it may need to be located where your operators are, or where your developers are.
James Hunt
Yeah, usually, it’s performance security, and very unusual or highly exotic hardware requirements, things like hasps, or specific peripherals that you just cannot get in, in cloud or, and, and colos, you can usually bring all your hardware with you for that. So you can use either the buyer or the build, if you have specific like GSM modems that you need for doing things network wise. But yeah, really the security in the performance if and I want to very, very, I want to couch that performance option in a whole lot of, but only ifs, right, and you have to really understand that the scaling of virtualized cloud based systems is not going to work for your performance requirements. Most of the hardware that’s available on the market today and has been for the last 10 years, is all geared towards high core count, high ram virtualization applications. The days of having a single server that only does one thing are basically gone. Because it’s so much more efficient to just pack the ram into the box and pack cores onto the CPU die. And and let the virtualization make more effective use of that raw compute capacity. Most applications unless they’ve specifically been designed for high memory, high CPU systems are going to benefit performance wise, from a horizontal scale, not a vertical scale.
Brian Seguin
Well, what about legacy systems? How do they fit in here?
James Hunt
That’s an excellent question, Brian. And it really depends on what what we mean by legacy. Because in the, in the cloud native space, we throw the term legacy around as a as a way of saying, We don’t like that that’s legacy. But if we’re saying legacy is something like an AS400, that’s running COBOL jobs for your accounts reconciliation, then I’m not aware of any public cloud that’s offering AS400 time via API. So you’re, you’re probably looking at a buy build proposition. If you’ve got legacy systems that run on on stock hardware, and I’m looking at things like Oracle CRM, or ERP in the CRM stuff, or — yeah, really, it’s usually ERPs. If that’s what you mean by legacy systems on top of stock hardware, you can usually run those virtualized because the virtual Nic two instance is still a Linux box at the end of the day. And if your legacy system can run on a Linux box, or on a stock windows kernel, there are public cloud offerings that will work for you. So legacy systems is as a nebulous term is a bit of a problem to unpack. But yeah, again, it goes back to do you have specific hardware requirements for that legacy system specific performance requirements. And let’s face it, most legacy systems don’t have performance requirements in this day and age, because the compute capacity has far surpassed what was even dreamt of when the legacy system wasn’t legacy.
Brian Seguin
Interesting. I mean, I like to buy scenario, you know, it gives me a fixed cost. And I can have a modest footprint, I could also have flexibility to to go into the cloud if I need to. I actually also have that with a build scenario, if my applications architected correctly. You know, I like the idea of owning my own physical hardware, but also not being responsible for the security of it.
James Hunt
Yes, I think personally, at least, for the stuff that I run, I fall into a quasi by rent. And by that, I mean, I’m using public cloud EC to instances linode, VMs, etc. And I have an upper limit, so I know roughly how much I’m going to spend. But that’s because I’m not availing myself of all the scale capacity. Right. If my websites get more traffic, they just get slower. And I’m okay with that. That’s a decision I made. So you can do rent without the the uncapped usage, the runaway usage, you just have to be careful about it.
Brian Seguin
I mean, rent is really good for greenfield projects, you know, bursty workloads. It it’s really good for especially company just starting out from scratch, I don’t know any company that started from scratch and wants to put 10s of 1000s of dollars into, you know, purchasing space in a colo facility and purchasing hardware on the colo facility.
James Hunt
Right. And I don’t know of any investors who are like, yeah, you’re gonna need all this money so that you can build out what already exists over here in Amazon.
Brian Seguin
Yeah, I mean, I think for most organizations, the limitation that might keep them in buy versus rent might be their actual physical applications that, you know, may have difficulty running on one of the is is may have difficulty running inside of some type of virtualization. You know, and for those there, those companies are going to have to buy some type of colo facility or, you know, I wouldn’t, I would shy away from recommending build in their scenario, they really want to start to migrate to an application that can run in the cloud, because that gives them a lot of options, a lot of flexibility. I think there’s also a nuance between buy and build in general. And I think it really comes down to a, you know, a CFO or an accounting standpoint, decision where, you know, they’re going to want us to think of if they want to build an actual physical data center, what are all the reasons that going into it, and, you know, versus the the buy scenario, since those two are fairly closely aligned, but I think most companies are going to want to not have that liability. But if it’s part of a long term investment plan, or a long term plan for the organization, maybe it makes sense. Mm hmm.
James Hunt
So to summarize, in general, I think you should rent your infrastructure. If you have specific performance, security, or hardware concerns, whether that’s from legacy systems, or just a specialized application that needs a specific peripheral, that you just cannot get in the public cloud, then you’re looking at a buy versus build. And really, I can only recommend build, if you’ve already built, I don’t think too many people are going out and building their own data centers from scratch, unless they have a very compelling reason. Like Wall Street, investment firms or the military or etc. Most people are going to end up on at the colo is going to be probably the most on premises that they ever get. And everything else is going to be in the public cloud being rented. Now you can rent from multiple places, if to avoid to mitigate some of those disadvantages. We talked about the vendor lock in, and some of that portability concerns, you can go multi cloud, it just takes time and effort and energy. And you have to know why you’re doing that.
Brian Seguin
I will say that that many companies take a hybrid approach and actually rent buy and build. A lot of companies have colo facilities, a lot of companies have built data centers, a lot of those data centers were built previously though, and every company wants to be renting in the cloud if they can, and that’s just bend the trend. So a lot of the Fortune 2000s are even doing multi IaaS, right like multi, you know, they’re they’re doing Amazon, Google and GCP, all at the same time, just so they could have the flexibility of porting from one to the other.
James Hunt
Cool. So join us next time on rent buy build as we discuss CI/CD, continuous integration and continuous delivery. We’re gonna talk about testing your code, testing, integration, testing for pull requests and all the way through to deployment.
James Hunt
You can find all episodes of Rent, Buy, Build online at https://rbb.starkandwayne.com or wherever podcasts are sold.