Hey, it’s Bryan from Oxide, and I’m back with another FAQ Friday. One of the questions we get from time to time is:
Can’t I just do this myself? You know, couldn’t I just like, I don’t know, buy a couple commodity machines and run Kubernetes on bare metal? I mean, wouldn’t I be effectively getting the same thing?
And you know, in answer to this question, I kind of feel as a parent of teenagers, the way I do when a teenager has an idea that is a bad idea, but is not otherwise unsafe. That is to say, something that they’re not gonna hurt themselves executing on, and you kind of think to yourself, "Well, that’s obviously a bad idea."
But it’s gonna be actually much easier than explaining to you why this is a bad idea. It’s gonna be easier to just encourage you to act on that. Go ahead, give it a try, and you’re gonna learn a lot of things. What would you learn by, say, running Kubernetes on bare metal on a bunch of commodity servers?
You would learn why this is deceptively painful, and it’s really basic stuff, or starts with the basic stuff. Because remember, you need to not just actually get this thing working once, you need to be able to update it. You’ve got vulnerabilities that come out. You need to be able to update that infrastructure.
But it gets a lot deeper. How do you deal with actual faults in the system? How are those coherently managed? And Kubernetes is not really designed to function as a control plane of bare metal, and it really shows. Then that leaves aside all of the bare metal historic commodity junk that you have to deal with, that if you’ve dealt with, you know what I’m talking about.
If you haven’t, you may need to find it out the hard way. Things like the baseboard management controller. That’s gotta be on its own separate network. You’ve gotta manage all those machines. You’ve gotta be able to go deal with the BIOS updates. I know it feels like we talk about the BIOS a lot, but that’s because that level of firmware is really, really painful.
You’ve got drive firmware that you’ve gotta go deal with. That’s gotta be updated. All of these things, all the different hardware components in the system have software on them, and when you’re running Kubernetes on bare metal, you are not managing that software at all. And unfortunately, the reality is that software bites you right in the butt.
And when you deploy the Kubernetes on commodity servers, unfortunately it will be its own punishment. But hey, maybe it’ll be a good learning experience for you. Don’t take my word for it. Stand it up for yourself. Thanks, and see you next time.