In this situation, it was a good pitch..belt level middle of the plate pitch. In any case, whether the batter swung at it or not, it will be called a strike. On that basis, it would be easier to defend your non-CI call, on the late swing.
However, if it was a bad pitch, one likely to be called ball...but, if the batter offerered a late swing and you're inclined to call it a strike, then I would award the CI to be consistent with the strike call....if not, I would call it ball (ignore the late swing) and not give the CI. This would probably get us out of a jam with coaches.