Adaptive video with FFmpeg, Vite and a custom player
How I turn recordings into cached AV1 and H.264 streams with FFmpeg and Vite, then play them with native HLS or Mattebox behind custom React controls.

The debug demo video in my table of contents post is one of 18 videos on this site as of writing this post.
I first hosted the TOC demo on Mux in August 2025, then moved it to YouTube that December. I wasn’t using Mux’s analytics, and YouTube was a simpler way to host it without worrying about future costs. Having YouTube’s branding and related videos in the middle of a blog post felt excessive for a clip showing a few seconds of UI.
The My Next Filing demo was my first self-hosted video. I put an MP4 on the site and used the browser’s controls, which I’d still do for a short clip. For longer recordings, I wanted a sharp version for reading small text and smaller versions for slower connections. Each file also had to fit within Cloudflare’s 25 MiB limit for static assets.
I could have gone back to Mux for the encoding and quality switching. I’d still have uploads and playback IDs to manage separately from the site. I wanted videos to work like images already do here. Put the file in the repo, reference it, and let the build handle the rest.
Codecs, containers and HLS
An MP4 file holds compressed video, usually audio, and timing information. MP4 is the container format. H.264 and AV1 are video codecs, which define how the picture data is compressed and decoded. AAC does the same for audio. Either video codec can go inside an MP4.
HTTP Live Streaming (HLS) uses playlists to tell the player which files to request. A master playlist lists the available versions of a video, each with its own playlist of short segments. With several resolutions available, the player can switch as the connection changes. The playlists and segments are ordinary files served over HTTP.
Those segments need a container too. My pipeline puts H.264 in MPEG-TS (.ts) files and AV1 in fragmented MP4 (.m4s) files. The AV1 playlist also points to an init.mp4 file, which the player reads to initialize playback before loading the segments.
From a recording to public URLs
A Vite plugin handles the preparation and makes the files available in dev and production. Adding a video now looks like this:
src/data/videos/my-post/
walkthrough.mp4 the recording
walkthrough.vtt reviewed English captions, optional
walkthrough.poster.webp a poster I picked, optional
walkthrough.json {"posterAt": 3}, optionalThe path becomes the video ID. That recording gets a master playlist at /videos/my-post/walkthrough/index.m3u8 and a poster beside it. I reference those URLs in the post. There isn’t a separate packaging command to remember.
Vite starts preparation
Vite already knows when I’m starting dev or making a build, so I let it start video preparation too.
The videoAssets plugin lives in scripts/build/plugins/videos.ts. With the rest of the app setup left out, this is how I register it:
import { defineConfig } from 'vite';
import { videoAssets } from './scripts/build/plugins/videos.ts';
export default defineConfig(({ isPreview }) => ({
plugins: isPreview ? [] : [videoAssets(process.cwd())],
}));Preview serves the completed build and can skip preparation. The app config also passes a separate metadata directory for dev and each build target, so they can run together.
Inside the plugin, prepareVideos finds the recordings and asks packageVideo to prepare anything missing from the cache. It returns a map from public URLs to files on disk. That map is how dev and the build find the finished packages.
The work starts in Vite’s config hook, with the returned URL map kept in assets:
config: {
order: 'pre',
async handler(_config, { command }) {
assets = await prepareVideos(root, generatedDir, {
background: command === 'serve',
});
},
},The source path becomes the video ID, so walkthrough.mp4 and walkthrough.mov would collide. Discovery rejects that pair before starting an encode. It also picks up each recording’s matching VTT captions, poster image and JSON poster setting.
A build waits for every video. Dev can use what’s already cached while missing packages encode in the background. A new video’s URLs won’t work until its encode finishes. Then its files join the map, and the terminal prints the playlist URL. Posters are prepared the same way but join on their own, so a cached package is served while its poster is still being chosen.
When I change a recording, captions or poster, the content watcher restarts dev. An encode already running in the same process survives that restart. Only the current dev instance adds its results to the map.
FFmpeg builds the streaming package
Most of the work happens in packageVideo, in scripts/build/plugins/video-package.ts. It uses ffprobe to read the recording’s dimensions and rotation, then works out which sizes to encode.
The targets are 1080p, 720p and 480p, preserving the source’s shape and never upscaling it. Each size gets both an H.264 and an AV1 version, so browsers without AV1 support can use H.264.
I use x264 at CRF 20 for H.264 and SVT-AV1 at CRF 30 for AV1. CRF is the encoder’s quality setting. These settings assume SDR recordings, and the machine running the build needs FFmpeg with both encoders, plus ffprobe.
The cuts between segments have to line up for quality switching to work. I use six-second segments at 30 fps and force a keyframe at each boundary. A keyframe can be decoded without earlier frames, so the player can pick up the next segment from another resolution at the same moment in the recording. FFmpeg writes a media playlist for each version with -hls_time 6 and -hls_playlist_type vod.
The player also needs to know how much data each version requires. The packager gets those bandwidth figures from the encoded segment sizes and durations, then writes them into the master playlist alongside the resolution, codec and media-playlist URL. I list AV1 first, though decoder support and the player’s adaptation logic determine what actually plays.
Captions need an extra adjustment. I review the transcription and word-level timestamps before saving the VTT file beside the recording. Its cues count from zero, while FFmpeg gives my H.264 segments a later starting timestamp. The build offsets AV1 to match H.264 and adds X-TIMESTAMP-MAP to the HLS copy of the captions. With that mapping, captions stay in sync after seeking or switching quality.
The build also keeps a clean captions.vtt without that mapping for the player’s HTML caption track.
Choosing a poster
The poster used to come from the same encode, at frame zero unless the JSON file named a time. Frame zero is often a blank screen or the start of a fade. Drool’s demo reel opened on black and My Next Filing on an empty paper texture. Altpack’s stop-motion poster was a 984-byte WebP of a plain white frame.
Before changing the build, I tried a few ways of choosing a frame in a throwaway experiment. It sampled four recordings into small frames, measured each one, and ran five methods over the same samples.
- The current poster, at frame zero or the configured time.
- FFmpeg’s
thumbnailfilter, which picks the frame closest to the video’s average color histogram. - A quality score on one frame every two seconds.
- The same score on dense samples, every 0.2 or 0.5 seconds.
- Stable scenes, which favors good frames from shots the video holds for a while.
Each method proposed up to three frames, and a review page put them side by side so I could pick one per video. The experiment also checked that the sampled timestamps matched ffprobe and that rankings came out the same on every run.
Three of the four current posters were blank. WeKeep’s was already set to five seconds, and I kept it. For the other three, I picked a frame from dense sampling each time, though only Drool’s was its first choice.
Dense sampling became the build’s default. FFmpeg decodes the recording at 160 pixels on its longest side, keeping a frame every 0.2 seconds for clips under 20 seconds and every 0.5 seconds for longer ones. Each frame gets a score from its sharpness, contrast, brightness, color and edge density. Frames in the first and last 4% of the video are skipped, since that’s usually where title cards and fades sit, and so are flat, blank or blurry ones. The winner is extracted from the original recording at its exact timestamp and scaled to fit within 720p.
The scoring uses plain pixel measurements, with no model or paid service. The same recording and settings always produce the same poster.
My picks for Altpack and My Next Filing weren’t its first choice, so a picked image still wins. A walkthrough.poster.webp beside the recording is published as-is, without recompression. Without one, a posterAt in the JSON file is honored, including zero. Only videos with neither get the automatic choice. Four videos on the site have a picked image, one has a timestamp, and the other 13 use the frame the build chose.
Extracted posters have their own cache under .cache/video-posters/, keyed by the recording, the poster choice, the poster code and FFmpeg’s version.
Seek previews
Hovering over the track now shows a thumbnail and the time. FFmpeg generates the thumbnails from the smallest H.264 rendition and puts them in one WebP sprite. It starts with a five-second interval, increasing it for longer videos to keep the sprite to at most 100 frames:
const interval = Math.max(5, Math.ceil(duration / 100));
const frames = Math.ceil(duration / interval);
const columns = Math.min(10, frames);
const filter = [
'setpts=PTS-STARTPTS',
`fps=1/${interval}:start_time=0:round=up`,
'scale=160:90:force_original_aspect_ratio=decrease',
'pad=160:90:(ow-iw)/2:(oh-ih)/2',
`tile=${columns}x${Math.ceil(frames / columns)}`,
].join(',');That filter goes to FFmpeg’s -vf option, with -frames:v 1 -c:v libwebp -quality 60. The player gets the interval and grid dimensions from generated metadata, then crops the sprite to the tile for the pointer’s time. The sprite loads when someone first opens a preview.
Previews have their own cache too, so changing their size or sampling interval doesn’t re-encode the HLS video.
Reusing completed packages
Encoding 20 minutes of video in two codecs takes long enough that I want the next build to reuse the work. Each finished package gets a cache key made from the recording, the tools that encoded it and the caption text:
const fingerprint = createHash('sha256')
.update([inputHash, tools, captions].join('\n'))
.digest('hex');The tools part includes the packager and rendition code hashes, plus the output of ffmpeg -version. That catches changes to encoder settings even when the recording hasn’t changed. If all those inputs match, the package under .cache/video-packages/files/<fingerprint>/ is ready to use again.
The poster time used to be part of that key, so moving a poster by a second re-encoded every rendition of that video. Now each recording is hashed once, and that hash feeds both the package and poster caches. Changing a poster only touches the poster.
A fresh encode goes into a staging directory first. The packager checks that it decodes, that the segments line up, and that every file fits the host’s size limit. Only then does it rename the directory into the cache. A failed encode leaves no completed package for the site to pick up.
Serving files in dev and production
During dev, the files can stay in the cache. Middleware added through Vite’s configureServer hook looks up each requested video URL and serves the matching file with the correct media type. There’s no extra copy sitting in public.
Deployment needs those files in the build output. The plugin’s writeBundle hook copies them there, preserving the paths used in dev:
writeBundle: {
order: 'pre',
sequential: true,
handler() {
if (this.environment.name !== 'client') return;
const output = path.resolve(root, this.environment.config.build.outDir);
for (const [url, file] of assets) {
const target = path.join(output, url.slice(1));
fs.mkdirSync(path.dirname(target), { recursive: true });
fs.copyFileSync(file, target);
}
},
},The environment check keeps the copy in the client build, where the public files belong. A post can use the same /videos/… URLs while I’m editing it and after deployment.
Only packages for current source files enter that map. Removing a recording leaves its old cache entry alone, but excludes it from the next build.
Cloudflare’s static assets ignored byte-range requests, which players use to ask for part of a file, often when seeking. I added a Worker handler to return those bytes. The plugin writes the file-size metadata it needs, and dev uses the same handler so seeking works the same way in both places.
Playing the generated package
I first used Mux’s open-source player with a src pointing at the generated playlist. It can play self-hosted HLS, so I got its controls without uploading the video to Mux.
The player chunk was 287.2 KB gzipped, even with tracking disabled and its analytics SDK removed from the build. I wanted to keep playback, seeking, captions, speed and fullscreen, with more control over what I was loading. I replaced it with custom React controls, native HLS where available, and Mattebox as the fallback engine.
The player lives in src/features/video-player. Its controls use the site’s shadcn-style buttons, sliders and popovers, with Lucide icons. Speed, quality and captions share a settings panel. Desktop keeps volume beside it, while phones move volume into the panel too.
The video, poster and captions are now in the initial HTML, with the browser’s own controls. On a browser with native HLS support, someone can press Play before JavaScript loads, or with it turned off:
<video src={nativeSrc} poster={poster} controls playsInline preload="none">
<track kind="captions" src={captions} srcLang="en" label="English" />
</video>The wrapper waits until the video is close to the viewport before importing the custom controls. They attach to the same video element, keeping its current time, volume, speed and caption selection. The native source is left alone if it’s already set, since assigning it again would restart loading. Autoplay demos still wait until the custom player loads, so videos further down a post don’t all start downloading at once.
With quality set to Auto, the initial source choice looks like this:
const native = Boolean(
video.canPlayType('application/vnd.apple.mpegurl') ||
video.canPlayType('application/x-mpegURL'),
);
if (native) {
if (video.getAttribute('src') !== nativeSrc) video.src = nativeSrc;
} else {
video.removeAttribute('src');
video.load();
const { createEngine } = await import('./mattebox-engine');
const engine = createEngine();
const attached = engine.attach(video);
if (caption) video.append(caption);
await attached;
engine.load(new URL(src, document.baseURI).href, { start: 0 });
}Native HLS lets the browser load the playlists, buffer segments and choose a quality. The controls still use the video element’s APIs for seeking, volume and speed. This path doesn’t download a JavaScript streaming engine.
The nativeSrc URL points to native.m3u8, an H.264-only version of the master playlist. It reuses the existing segments and leaves captions to the HTML track, avoiding duplicate caption choices in the browser’s menu. I added it after a native Chromium path accepted the mixed-codec playlist but reported a roughly 130-second video as only six seconds long. The adapter also falls back if native playback errors, reports an incomplete duration or stays stalled for eight seconds.
The fallback module uses Mattebox 0.17.0 with the stages needed by these packages:
import { mattebox } from 'mattebox';
import hlsCmaf from 'mattebox/protocols/hls-cmaf';
import abr from 'mattebox/stages/abr';
import abrCapSize from 'mattebox/stages/abr-cap-size';
import recovery from 'mattebox/stages/recovery';
import cmafTiming from 'mattebox/stages/cmaf-timing';
import codecProbe from 'mattebox/stages/codec-probe';
import mp4Box from 'mattebox/stages/mp4-box';
import tsTransmux from 'mattebox/containers/ts-transmux';
export function createEngine() {
const engine = mattebox({
stages: [
hlsCmaf(), abr(), abrCapSize(), recovery(),
cmafTiming(), mp4Box(), codecProbe(), tsTransmux(),
],
});
engine.on('tracks:changed', () => {
const codec = engine.quality.selectable.some((rendition) =>
rendition.codecs?.startsWith('av01'),
) ? 'av01' : 'avc1';
engine.quality.constrain('video-codec', { codecs: [codec] });
});
return engine;
}The codec constraint keeps quality switching within AV1 or H.264. Switching between AV1/fMP4 and H.264/TS stalled the buffer transition during development, and each codec already has all the resolutions it needs.
Choosing a manual resolution also loads Mattebox, since the native video API doesn’t expose a way to pin an HLS rendition. Once attached, the engine applies the selection:
if (quality === 'auto') {
engine.quality.auto();
} else {
const rendition = engine.quality.selectable.find(
(item) => item.height === quality,
);
if (rendition) engine.quality.pin(rendition.id);
}The adapter preserves the current time and playback rate during that handoff, and checks whether the reader still wants playback before resuming. Returning to Auto keeps the engine attached. The source-choice snippet leaves out that handoff, error handling and cleanup.
Both paths use the same clean caption track. Mattebox clears the video’s children when it attaches, so caption in the snippet is the existing track node, restored immediately afterwards. React reads its active cues to render captions, with native captions used when the browser opens its own video fullscreen view. The player also keeps the seek previews and keyboard controls. Controls and cursor hide together after two seconds of inactivity during playback, unless focus, an open menu or a drag keeps them visible.
Player size
The replacement loads in parts. At the end of the Mux replacement, these were the additional gzipped player assets compared with the old 287.2 KB Mux Player chunk:
The native path is 94.9% smaller before settings load. Settings add another 53.2 KB, fetched on hover, focus or a press of the settings button. The last row counts both the H.264 worker and its main-thread alternative, so it is a conservative total. Extra copies used to retry failed imports aren’t part of these successful playback paths.
Uncompressed, the old player chunk was 1,018.0 KB. The native path adds 41.2 KB, and the total with all fallback modules and settings is 342.9 KB.
These figures use gzip level 9 and decimal KB on saved optimized Vite builds, comparing Mux Player 3.14.0 in the production output with the replacement’s local-target output. They count player JS and CSS added after the shared code for /work/my-next-filing is loaded, excluding media and fonts. The shared vendor chunk was identical. The surrounding wrapper grew from 1.5 KB to 2.9 KB gzipped, outside these totals. They’re measurements of this site’s integration, not a standalone package size. They predate the server-rendered video described above, whose bundle size hasn’t been remeasured.
What the pipeline produces
The whole library is 18 videos, about 20 minutes of footage. That’s 152 MiB of recordings, turned into roughly 500 MiB across 1,291 files, including the preview sprites and native playlists. The largest single file is 4.7 MiB, well under Cloudflare’s 25 MiB cap.
The AV1 media came out 36% smaller than H.264 at my settings. Individual videos ranged from 18% to 47% smaller. Those measurements cover every prepared size, including audio and container overhead.
That compares file sizes only. The CRF quality settings aren’t comparable across encoders. I haven’t measured whether the two outputs look identical. Check small text in your own recordings before trusting the savings.
AV1 also takes noticeably longer to encode in my builds. Keeping both versions makes the library on disk bigger. What a viewer downloads still depends on the codec and quality selected, buffering, and how long they watch.
What I still have to maintain
There’s no playback dashboard. Server logs tell me a segment was requested, not whether someone’s video kept buffering.
Earlier versions included a separate MP4 fallback. Keeping those files under Cloudflare’s 25 MiB cap meant cutting some recordings to 480p. I removed that fallback and kept HLS, which splits the higher-resolution versions into smaller files.
Native HLS playback and the poster work without JavaScript. The custom controls and Mattebox fallback still need it. If the custom controls fail to load, the browser’s controls remain available, with a retry option below the video.
I’ve done my best to test every scenario I could think of on the devices I have access to, but I’m sure I’ve missed some edge cases. That’s a reliability tradeoff I was willing to make when I switched from Mux Player to my own player.
Every video request also runs through the Worker that handles byte ranges, so delivery isn’t free compute. Public video URLs can change their contents when I replace a recording, even though the local cache uses hashes. Those URLs can’t safely get a long-lived immutable cache policy.
If you need user uploads, live streams, private or paid videos, or real playback analytics, use a platform. That’s a lot of work this pipeline doesn’t do.
For a portfolio with 20 minutes of video, this is more machinery than the problem strictly needed. But adding a video is now a file in a folder, and everything a viewer sees is mine to change. I’m happy to maintain it for that.