You really can't though. Do you want the guy you drafted being mentored from the guy he's replacing? Also, when the rookie struggles, because he will, you can't bench him for the former number 1 guy.
The only way to do it sadly would be to cut ties completely, bring in someone who could be a bridge QB for the first 4-6 games, then take his role as in game mentor. Anything else is just too damn messy.