Generate Subtitles with Whisper and FFmpeg in Python

Whisper can transcribe speech in an audio or video file, while FFmpeg can extract and convert the audio and help prepare the finished subtitle file. This Python workflow produces timestamped subtitles in formats such as SRT; you’ll need Python, FFmpeg, and a Whisper transcription package installed before starting.

What You’ll Need

Python 3 installed and available from your terminal.
FFmpeg installed, with its executable accessible on your system PATH. Verify by running ffmpeg -version.
A supported audio or video file, such as WAV, MP3, MP4, or MOV, that you have permission to process.
A Whisper implementation. This guide uses the open-source openai-whisper Python package, which runs transcription locally after installation. It is separate from hosted transcription APIs.
Enough disk space and memory for the audio and selected model. Larger models generally need more resources and take longer.

Step-by-Step

Check your Python and FFmpeg installations. Open a terminal or command prompt and run python --version and ffmpeg -version. On some systems, use python3 instead of python. If either command is not found, install the missing program and reopen the terminal so updated PATH settings are loaded.
Create a project environment. Make a folder for the script and input media, then create and activate a virtual environment there. For example, run python -m venv .venv; activate it with .venv\Scripts\activate on Windows or source .venv/bin/activate on macOS or Linux. A virtual environment keeps project packages separate from other Python projects.
Install Whisper. With the environment active, run python -m pip install -U openai-whisper. The package may require a compatible PyTorch installation, and installation details can vary by operating system and hardware. If installation reports that Torch is missing or incompatible, follow the current PyTorch installation instructions for your system, then retry. FFmpeg is a separate system dependency; installing the Python package does not install the FFmpeg executable.
Optionally convert the source with FFmpeg. Whisper can read many common media formats, so conversion is not always necessary. If a file fails to load, create a standard mono audio file with ffmpeg -i input.mp4 -vn -ac 1 -ar 16000 audio.wav. Replace input.mp4 with your file’s name. This removes video, converts to one audio channel, and sets a 16 kHz sample rate. Keep the original file; this command writes a new WAV file.
Write a transcription script. Save the following as make_subtitles.py in the project folder. Change the input filename and, if needed, the language code. Whisper’s segment timestamps are in seconds; the package’s write_srt helper formats them as SRT timestamps.

import whisper

input_file = "input.mp4"
model = whisper.load_model("small")

result = model.transcribe(
    input_file,
    language="en",       # Set the spoken language; omit to detect it
    task="transcribe",
    verbose=False,
)

with open("subtitles.srt", "w", encoding="utf-8") as f:
    writer = whisper.utils.WriteSRT("subtitles.srt")
    writer.write_result(result, f)

print("Saved subtitles.srt")

Run the script and allow the model to download. Execute python make_subtitles.py. The first run may download the selected model, so an internet connection is needed then; transcription itself can run locally afterward. The small model is a practical starting point, not a guarantee of a particular speed or accuracy. Try base for lower resource use or a larger model when you can afford more processing time.
Review and correct the SRT. Open subtitles.srt in a text editor or subtitle editor. Each cue has a sequence number, a start and end time, and text. Listen while following the captions, correct names and punctuation, and check that cue timing matches the speech. Automatic speech recognition can mishear words, especially with noise, overlapping speakers, or unfamiliar terms.
Use the subtitles with your video. Many players can load an SRT file alongside a video when both files have matching base names and are in the same folder. To create a new video with subtitles burned into the picture, use FFmpeg’s subtitles filter, for example: ffmpeg -i input.mp4 -vf subtitles=subtitles.srt -c:a copy output.mp4. This renders the captions into the image, so viewers cannot turn them off. For selectable captions inside a compatible container, use a subtitle stream instead; supported formats and options depend on the container and playback device.

Common Mistakes

Assuming pip installed FFmpeg: Whisper relies on the FFmpeg executable for media handling. Install it separately and confirm that ffmpeg -version works in the same terminal used to run Python.
Using the wrong input path: A script looks for a relative filename in its current working directory, which may differ from the script’s folder. Use an absolute path or run the command from the project directory.
Leaving language detection to chance: Automatic detection can choose incorrectly for short clips or similar-sounding languages. Set language to the spoken language’s code when you know it; omit the setting to let Whisper detect it.
Expecting perfect punctuation and timing: Model output needs review. Do not publish without checking names, numbers, technical vocabulary, and the beginning and end of every cue.
Overwriting the wrong file: Choose a distinct output name and verify the input and output paths before running FFmpeg, especially when making a burned-in copy.

Tips

Use clear audio where possible. Reducing background noise or separating speech from music can improve recognition, but aggressive processing can also distort voices.
Set a language explicitly for recordings with one known spoken language. For multilingual speech, test the output carefully; language handling can vary across segments.
Keep the original media and save edited subtitles as a separate file. SRT is plain text, so it is easy to revise and move between many subtitle tools.
For a long recording, test a short representative section first. This helps you judge model speed and accuracy before processing the whole file.
To make an SRT in another language, Whisper can translate speech into English with task="translate"; it does not translate into arbitrary target languages through that task.

Troubleshooting

“ffmpeg” is not recognized or not found: Install FFmpeg for your operating system, add its executable location to PATH, then start a new terminal. Confirm the command works before rerunning the script.

Python cannot import whisper: Activate the same virtual environment where you installed the package and run python -m pip show openai-whisper. If it is absent, install it with that environment’s Python. Avoid confusing this package with other packages that use a similar import name.

The audio cannot be opened: Check the spelling and path, and try converting the source to WAV with FFmpeg. Ensure the file is not incomplete or damaged and that your account can read it.

The script runs out of memory or is very slow: Try a smaller model such as base, close resource-heavy applications, or process shorter sections. Hardware acceleration is optional and requires a compatible setup; CPU processing may work but take longer.

Captions are inaccurate or oddly timed: Verify the selected language, listen to the source, and try a clearer audio conversion. Edit the SRT manually where needed. If burned-in captions are missing, check that the subtitles filter can locate the SRT file; using an absolute subtitle path can help.

Frequently Asked Questions

Can Whisper create an SRT file directly?
Yes. The script uses Whisper’s WriteSRT helper to save timestamped segments as an SRT file.

Do I need to extract audio from every video first?
No. Whisper can process many common video formats through FFmpeg. Convert to WAV only if the input causes problems or you want to standardize the audio.

Can I use a GPU?
Yes, if your PyTorch and hardware setup supports it. The example uses automatic device selection; a GPU is not required, though CPU transcription may be slower.

Can I create subtitles in a language other than English?
Transcription can retain the spoken language, including when you specify its language code. Whisper’s built-in translation task translates speech to English, not to any chosen target language.

Can I edit an SRT file?
Yes. It is plain text, but preserve the cue numbering and timestamp format. A subtitle editor can make timing changes easier to check.

Are generated subtitles accurate enough to publish unchanged?
Not reliably. Review the transcript and timing against the recording, particularly for names, numbers, specialized terms, and unclear speech.

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this bed

The whole Tech How-To bed →