A few decades ago, it seems like actors weren't traveling across the Atlantic as much. There were, yes, a number of Britons who had relocated to Hollywood, where they'd heard the money was. The inverse, American actors staying long-term in Britain, was rarer by far. And the cost of getting someone to make a special trip was expensive, Like, major motion picture expensive.
The upshot is that if you saw an American character on a British show, the accents were very faux. I've noticed this in, among other things, reruns of Danger Man. This show, Patrick McGoohan's pre-Prisoner outlet, was eventually marketed as Secret Agent in the US, with Johnny Rivers's "Secret Agent Man" tacked onto the credits in place of the original spritely instrumental. It was a mostly high quality program, making good use of McGoohan's moody anti-charisma, but it wasn't a big budget affair by any means.
The tell isn't the accent itself. In terms of phrasing, these players could often pass for people from the larger Midwestern cities, say. But sometimes the scenarist doesn't know the right word to use, so someone says "rubbish" when they'd really say "garbage." Or the stress is on the wrong syllable. Hearing a Yank soldier say la-BORE-a-tree instead of LAB-ra-tore-e is enough to break the illusion.
It need hardly be said that this era saw a lot of painful British accents on American TV as well.