A Celtics game I went to this season was stopped twice in the final minute of play for extended periods of time as the referees all huddled around a monitor trying to figure out which team the ball went out of bounds off of. On one occasion the scoreboard big screen zoomed in on them, and it was clear that they were just watching ABC's regular TV coverage of the game, with no control over the replays themselves.
I have a solution. It's something that will work for all pro sports using video review. In fact, I actually got the idea last NFL season from that atrocious Seahawks touchdown call against the Packers which brought an end to the replacement refs. There is no reason why the actual referees who are on on the field/court/ice have to be the ones who watch the replays and make these decisions. In fact, the people doing this don't even need to be at the game. Each league should have it's own "replay office."
It would consist of five, ten, maybe even twenty (however many it takes) people sitting in a room somewhere. They work for every game going on all at once. They're only job is to watch replays and make correct rulings; so they have access to all the best available technology, and know exactly how to use it. The replay office would have somebody watching each game as it happens, and in turn could already be working on getting the controversial calls right before the officials at the game even decide to check with them. Not only would the accuracy improve, but this might even speed up the process as well.
5/13 UPDATE: It has been brought to my attention that the NHL already has this. So I guess my "idea" isn't that revolutionary after all. Anyway, well done NHL, and it's definitely time for the other 3 leagues to get with the program.