Optimizing HTML5 Video Scrubbing

I’ve recently been working on improving the experience of the video time selector inside the Vibbio platform. It’s a little component that allows you to select a range of time from a video.

Previously, it worked like this: you move one handle to a point along the timeline, and the video updates its position.

This isn’t ideal. The user has to rely on that clunky thumbstrip on the timeline to guess where they want the marker to land. Before they release the handle, they have very little indication of where they are actually scrubbing to.

I wanted to change this so that the video scrubs along with the handle. Move the handle, move the video. In that way, the user has accurate feedback about where they are and what time they are selecting.

Under the hood, we’re using a <video> element to display the video, and an rc-slider component for the slider. My first naïve attempt at scrubbing was simply to update the video element’s currentTime property when the handle moved:

function onHandlePositionChange(time) {
  videoElement.currentTime = time
}

In theory, I would have expected this to work, we’re simply moving the time of the video with the handle. In practice, it chokes the video:

I’m not 100% certain of the technical reason behind this. I am guessing the video element is constantly trying to load data, but never loading enough to actually display before the time is updated again. Given that hypothesis, we want to wait until the video element has loaded some data and can display it before attempting to start loading more data.

My first solution to this problem was to throttle updates to currentTime. This immediately made scrubbing more performant and responsive, but it was hit and miss based on the user’s network speed. On a slow network, I was often seeing the same locking of the video that was happening to start with. Even on a fast network, large videos would become unresponsive when attempting to scrub through large periods of the video. What we need is to actually wait until a frame has loaded before trying to load a new one.

Luckily for us, there is an event that indicates exactly that! The seeked event fires when a seek operation completes. That is, the video element’s appearance reflects its currentTime property.

So, lets use that event to only seek when the video is ready. We track when the handle moves position, maintain a targetSeekTime of what the user wants to be seeing, and seek to that point as soon as the video element has displayed the last thing we requested.

let isSeeking = false, targetSeekTime
function onHandlePositionChange(time) {
  targetSeekTime = time
  if (!isSeeking) videoElement.currentTime = targetSeekTime
}
function onSeeked() {
  if (targetSeekTime == null || videoElement.currentTime === targetSeekTime) return
  videoElement.currentTime = targetSeekTime
}

Lo and behold, it works like a charm:

In addition, we added a timeout which checks if the video is taking a long time to seek. In that case we dim the video and show a little loading spinner. This is a kind of backup for users with a slow network, so that they know the video is doing something and hasn’t just fallen apart. The video element does have a stalled event for this, but I found it to be a bit unreliable—this method was giving much more consistent feedback.

let isSeeking = false, targetSeekTime, stalledTimeout
function onHandlePositionChange(time) {
  targetSeekTime = time
  if (!isSeeking) videoElement.currentTime = targetSeekTime
}
function onSeeking() {
  stalledTimeout = setTimeout(() => {
    videoElement.classList.add('stalled')
  }, 2000) // mark as "stalled" after 2 seconds 
}
function onSeeked() {
  if (stalledTimeout) clearTimeout(stalledTimeout)
  videoElement.classList.remove('stalled')

  if (targetSeekTime == null || videoElement.currentTime === targetSeekTime) return
  videoElement.currentTime = targetSeekTime
}

Which results in this, here using a simulated slow network and a very long video:

With these two small changes, suddenly the time selector was far more enjoyable to use. We ended up removing the thumbstrip (seen in the first gif) altogether, with the newly responsive scrubbing it wasn’t even needed!