Ruleswise it is allowed but personally I'd recommend not to do such a thing since it would probably be a very boring game for the threatened player. And because I have a strong dislike for bullies in general.
Pax Magnifica Bellum Gloriosum
I was not thinking about a single bully who threatens all the other players, more of a situation in which one player became too powerful and is now threatening a players homesystem or last system and all the other players know that they will be next. So they could threaten the bully that if he proceeds to be aggressive, he will be excluded from the council. The other way I around a protectorate would be much better, like the bully stops to attack the player but he always has to vote in favor of the bully and some other things.