The Bucket Problem – Part IIPosted: October 13, 2011
For the first part of this disussion, see The Bucket Problem – Part I.
So having multiple buckets fixes the problem beautifully…if only problem you’re trying to solve is “Don’t Fuck Up Production”. (Incidentally, what I’ve described generally only mitigates this…but that’s a whole different discussion.) What if you have other goals? Like, say, not spending One Billion Dollars on replicating every server you build in every data center you manage in every country in which you have a presence? (Note: Seperate data centers actually is a form of the Bucket Problem, which should become apparent shortly).
Okay, so you start to delve into the problem a bit more thoroughly and say “Development will be set up thus and Production will be set up so“, and you feel totally satisfied that you’ve arranged it just perfectly – all of your resources are allocated in perfect harmony. …until they’re not. So you’ve got two buckets – dev and prod – and inevitably one of them fills up first. Now you’ve got to go to the people who write the checks for the equipment and say something like “We’re only using 70% of our capacity, but we’ve got to buy more.” Why? Because the resources in the particular bucket you need are used up, and you’ve set up your buckets such that Ne’er the Twain Shall Meet.
Think of this as the Shampoo/Conditioner problem, which is a form of the Bucket Problem stated thus: When you purchase two bottles of hair product – one of shampoo, one of conditioner – even if they are precisely the same size at purchase time, you will run out of one of the other (generally shampoo) significantly earlier than the other. The interesting thing about this particular problem is that it’s typically cyclical. Run out of shampoo? Buy just shampoo the next time you’re at the store. …then you run out of conditioner, since you only started out with a partial bottle to begin with. And so on.
Alternately, maybe you prefer to spend staff time playing the “Shell Game”, shuffling resources around between storage subsystems, networks, etc. to try to meet the requirements of this-or-that bucket. This could involve an arbitrary amount of difficulty, from just changing some configuration parameters to having to haul physical hardware to another data center. This problem is a little more like having multiple bank accounts, or perhaps a more-accessible way to state it is having one or more gift cards. I think everyone has had the following interaction with a cashier at least once: “Okay, so there’s $3.17 on this card, $7.49 on that one, put $20 of it on my Visa, and I’ll pay the rest in cash.” How fun is that?
There are a number of Bucket Problems and corrolaries I can come up with off the top of my head – multiple email addresses/address books (the particular email/contact information you’re looking for is never in the account where you’re looking), booze & mixers (you never run out of both at the same time), chips and salsa (classic problem!). I think you get the idea.
Note that most of the above examples involve exactly 2 buckets. As you might expect, this problem tends to get (exponentially?) worse as the number of buckets increases.
So, that’s a brief introduction to the Bucket Problem. Where have you seen this in your life?