The Bucket Problem – Part IPosted: October 12, 2011
There is a problem that I like to think of as The Bucket Problem. I run into it perhaps most often at work, but the more I think about it I realize that this particular problem manifests itself in all different areas of life. It’s a pain in the ass, it takes up way more of my time than I feel it should, and I’d really like to be done with it once and for all.
So…what is it?
Well, The Bucket Problem can be stated (very) informally thus: Any time you split your resources into two (or more) buckets (“pools” or “piles”, if you prefer…I like “buckets”) you’ll wish you hadn’t. Even if you had very good reasons for doing so. In fact, even if you really had no choice other than to do so.
The (classic?) example that I’m thinking of: the splitting of resources between development environments and production environments – dev “buckets” and prod “buckets”. Now, if you’re not a technology person then this bears some amount of explanation. I’ll stick to Internet services, because presumably if you’re reading this you have some grasp of them – at the very least, you know they exist.
You have a service – say, WordPress – which you host at wordpress.com. End users of this service have all the usual expectations for this service – it has to be up all the time, it has to be easy to use, it has to have this-or-that feature, etc. As a guy on the systems side of the house, the first of those criteria is really the critical one. It can’t go down. Ever. (Or, if you prefer, it can’t go down unexpectedly.) That is what Production means. It is a “product” in the sense that you are making it available to customers (whether they’re directly paying for it or not), and where the Internet is concerned it’s simply unacceptable for that product to be unavailable.
…but in order to remain relevant and continue to be a great product (or to become a great product if it’s not to begin with) work probably has to be done on it over time. Code has to be written and modified as a part of the natural course of things. Now, you don’t want to do this work on the version of the service that’s running in production – I mean, what if you fuck it up? So where do you do it? Development, of course. It’s easy, right: We just split things into Development and Production buckets and put in place some kind of a promotion-to-production process/policy. Hell, maybe we even do QA! <gasp> We could even set up another bucket just for QA. Problem solved. Right?
[This discussion is continued at The Bucket Problem – Part II]
[I didn’t mean to digress this deeply into dev vs. prod, but I think it will end up being valuable in the end. Bonus points if you can see where I’m headed with this. (Oh…and I welcome comments, but no fair giving it away if you work with me!)]