Clock drift correction
It seems the millisecond accuracy of the clock is very unstable over time and that corrections must be repeated regularly if not constantly. Perhaps something to add as an idle timer job?
It is also possible that initial estimate of latency is incorrect.
Edited by Nicolas Pope