From the OP. what I "see" is this:
1) Pitcher in the stretch
2) Pitcher stands up, shortens his stride, and stops
At that point, he's SET. if he does anything other than pitch, throw to a base, or step off, you have to call the balk.
From OP, it sounds like after 2), pitcher brought his hands together and stopped again. that's a balk, and you have to call it.
One poster said, (paraphrasing) that it's hard to see any intent or any deception there, so you shouldn't be too picky. That's a bad habit to fall into -- you don't call it and the next umpire does because it's a balk and now the next umpire has to clean up after you because F1 and HC want to know why it's a balk now when "that's been his motion all year, blue."
The next umpire gets all the grief because you didn't enforce the rule...