Because it shortens the game by a big amount. It's almost impossible to reach tier 6 and have fun with those really big scalers.
Does it though? Maybe I'm playing at a low MMR (7.3k) but it's hardly "impossible" to reach Tier 6 with 7-player lobbies. In fact, you get the chance to upgrade tavern tiers more aggressively whenever you face the ghost.
Why do you care? One person leaving means your chances of winning the lobby have increased. The removal of the 15-damage cap is neutral (it just means players get eliminated more quickly, which could work to your advantage), and isn't a reason to resent people for leaving.
Yes, the "flow" of the game changes, but the whole point of Battlegrounds is that you should adapt to the lobby you're given, instead of following a standard playbook each time.