As Satoshi correctly pointed out, silencing one of the first minions doesn't do anything. Silence removes that minion's card text, but it's still that minion. When the final one dies, its Deathrattle triggers, which checks the graveyard and the game sees that all other Horsemen have died. It's rather frustrating, but it's not a bug.
And yes, this is why players tend to kill off their final Horseman in their own turn. To prevent that one getting silenced.