Ability to associate templates with an audience

It would be great to be able to associate Templates with a given audience, and them have them only shown to users with the permissions to access that Audience.