Well, there'd be moments where the time-looped one would grab the coffee because the POV character keeps spilling it.

And yeah, probably more, and then it'd the end it would be utterly jarring.

But it's definitely tricky.