Dungeon Masters, like anyone else, can fall prey to the allure of min-maxing. And just like anyone else, it can happen unintentionally. However, it comes with many down sides. It takes focus away from the collaborative storytelling, encourages players to Min-max and metagame, as well as often punishing players for trying out new things. It is for this reason I hope to get my thoughts on the matter in text. That way I can reflect on the situation. Hopefully it helps someone out there too.
What Do We Min-Max?
What I find typically happens is that when I feel the pull of min-maxing, it's in a combat encounter. Often the problem is very similar to when we railroad. We want the combat encounter to be interesting and challenging so we set it up to be so. The problem is that we, as the Dungeon Master, have more information than players will. Making a combat encounter that is optimized to be difficult when approached correctly can and probably will be frustrating when approached in another way. Often times characters in a combat encounter aren't outside of it. This also means we often don't have the same checks and balances a player does. A player will need to use their character out of combat as well to role-play. Many of the things we put our players against won't be seen outside of combat, allowing us to min-max purely for combat. This is especially true if you use the player character rules as inspiration but also comes up when making stats for creatures.
I almost think of this as a particular kind of railroading. Only in this case, we are restricting a combat encounter unintentionally. We can also design an encounter to hit our players right in their weaknesses. Every party will typically have something about it that can be exploited. Now, an occasional encounter that reminds players of their weaknesses and forces them to overcome them or tough through isn't bad. However, we also don't want to only hit them in their weaknesses either. Sometimes we'll want to play to their strengths, or both at the same time in creative ways.
The Long View
It's also important that often, even when I min-max, I do it for a single encounter. The further issue is that a gaming session tends to be more than one encounter. If you prefer to have a single set piece battle for your games, it'll be easier. It'll be a hard fought battle that will end and allow a chance to rest. However, if there are multiple encounters such as occur when going on a dungeon delve, it might not make sense. We want to think about the entire period before the players get a chance to rest. If we don't, we can't be surprised if the players want to try and rest after every room. After all, with our encounter building there aren't too many other options (excluding unreliable accidents such as players getting lucky or out thinking the Dungeon Master). Thinking of the entire session or campaign from time to time is a good idea to try to avoid this kind of optimization on our part.
Players Make Trouble Themselves
Another issue comes up quickly when we move beyond theory and throw some players into the mix. Play a tabletop role-playing game long enough, though it won't take very long in my opinion, and you'll run across a situation where player ingenuity can make the impossible trivial. However, there is also the other side of the coin. Players can, and will in some cases, turn a trivial situation into a serious one. Clever players can do this too by over-thinking things or things just not going according to plan. If we try our best to give our players hell and min-max our encounters, we run the risk that they'll dig their grave even deeper instead of climbing out of it.
Min-Max Arms Race
Where this gets tricky is who did it first. There is a bit of a feedback loop that takes place here. I think I've mentioned it briefly before. If our players think we are being unfair, they very well might start min-maxing. Then we will raise the difficulty in order to challenge them and everything can spiral out of control. Things get a bit more complicated because some players get a real sense of enjoyment from min-maxing. However, I'd be weary of doing this in kind. The other players who don't like to are unlikely to be happy. Instead, though it isn't ideal either, finding a separate way to challenge the min-maxed player is usually a better way from my experience. Also making sure that the other players have a role to play goes a long way too.
Rolling is part of the game and combat in general. This means we also need to consider the impact of bad luck. Of course, there's the small chance everyone rolls 1s for the entire session. However, that's unreasonable to expect. Instead, we need to remember to expect some failures. Some attacks will miss, even if their expected value is above the AC of a creature. This little feature of statistics, combined with min-maxing an encounter, can lead to massive issues that I've seen too many times. Instead, when looking at how much damage your party can dish out, look at the average damage weighed by their chance of success. This gives a more realistic expectation of damage. Don't forget to factor in the possibility of certain actions doing half damage as well. I've seen many encounters that turned out easier than expected because of that overlooked detail.
Consider the background of your characters before throwing them into an encounter. Just because we can move stats to Dex, Con and Str for a new creature/character doesn't mean we should, even if we are outside of combat. I find it helps to realize these things tell an implicit story. If the guards of a town in the middle of nowhere outclass your player characters, the implication is different than if they come across veterans coming back from a war or the body guards of a rich noble. Again, I prefer to design a scenario for my players to encounter. I usually don't want to force my players into a particular solution. I do make exceptions for situations that they created for themselves through unreasonable actions (if you try to hunt down and fight a dragon at level 1, you better run unless that was the point of the adventure from the start). Don't forget about the compounding effect that encounter after encounter has as well.