Whisper Transcription (Audio & Video)

Generate transcripts for local audio and video files using local Whisper engines. All processing happens on your device. On macOS, the Social Archiver Desktop app can use the bundled WhisperKit backend; CLI backends remain available for Obsidian and fallback setups.

Desktop App Setup (macOS WhisperKit)

For the standalone Desktop app on macOS, WhisperKit is the recommended first backend. It avoids asking users to install Python, Homebrew Whisper tools, or separate model files before the first transcription.

1. Open Settings → AI → Transcription executor in the Desktop app.
2. Set Whisper backend to Auto or WhisperKit. Auto uses WhisperKit first when the bundled helper and a model are ready, then falls back to installed CLI backends when needed.
3. In the WhisperKit model banner, choose a model from the dropdown and click the download icon.
4. Keep the app open until the circular progress indicator completes. Installed models can be deleted from the same banner if you need to free disk space or reinstall.
5. Open a Timeline item with saved local media and click Transcribe. If the item already has a transcript, the modal becomes an Additional transcription flow so you can add another language or model result.

WhisperKit model choices

Model	Best for	Notes
Tiny	Quick previews	Smallest and fastest, lowest accuracy
Small	Daily use	Recommended balance for most archived speech
Large v3 turbo	Quality	Quantized WhisperKit model, about the 632 MB tier in the current Desktop build

Language and model selection

The transcription modal lets you choose the WhisperKit model and language for each run. Match original is the default language option. If auto-detection chooses the wrong language, select a specific language such as Korean, English, or Japanese before starting the job. When multiple transcripts exist, the Timeline player shows a transcript language dropdown near the player controls.

Video and downloader note

WhisperKit handles supported audio locally on macOS. Video files still need audio extraction, and remote video jobs still need a downloader. Keep the managed yt-dlp copy and ffmpeg/ffprobe ready if you transcribe YouTube, TikTok, or other video posts.

Mobile Request Setup Checklist

If the mobile app says transcription is unavailable, the mobile request flow still needs a capable desktop executor. Use the Desktop App Setup for the standalone app. If you use the Obsidian plugin executor, check these items first:

1. Install the Social Archiver Obsidian plugin on macOS, Windows, or Linux and sign in with the same account as the mobile app. See Plugin Settings for where to check the desktop-side setup.
2. Keep desktop Obsidian open and connected so it can receive transcription jobs from mobile. The request flow is summarized in How It Works.
3. In desktop Obsidian, open Settings → Social Archiver → Transcription Settings and turn on Enable Whisper transcription. The setting is covered in Plugin Settings.
4. Install a Whisper-compatible tool. openai-whisper is easiest on Windows, whisper.cpp is best on Apple Silicon, and faster-whisper is fastest for many Linux/CPU setups.
5. For video transcription, install ffmpeg. If the desktop client must download video before transcribing, also install yt-dlp. See Requirements and Supported Workflows.

After the desktop plugin reports that transcription is ready, mobile requests can run immediately over the live connection. Once the initial setup is complete, mobile can still create a transcription request while desktop Obsidian is closed; that job may stay queued until desktop Obsidian is opened again and reconnects over WebSocket. Actual transcription, live progress, and fastest result delivery require the desktop client to be online.

Obsidian Shell Execution Warning

Obsidian security scans may show a Shell Execution warning for Social Archiver. For transcription, desktop Obsidian runs the local Whisper-compatible CLI and may call ffmpeg/ffprobe for media processing. If download-and-transcribe is enabled for video posts, it can also run yt-dlp before transcription. The standalone Desktop app uses its bundled WhisperKit helper for the native macOS backend, but still uses media tools for video extraction and remote video downloads. Commands run only for transcription work you enable or request. Mobile requests can queue jobs, but mobile Obsidian does not run these local shell commands; the connected desktop executor does.

Overview

After archiving media in Social Archiver, you can generate full transcripts using Whisper speech recognition models running locally on your computer. This covers podcast audio, saved local files, and downloaded video files, including YouTube/TikTok workflows in Timeline.

Why Local Processing?

This feature is designed with two considerations in mind:

1. Privacy: Your media files never leave your device. All transcription happens locally using open-source tools, ensuring complete privacy for sensitive content.

2. No API Costs: Unlike cloud-based transcription services that charge per minute, local Whisper is completely free to use once installed.

The trade-off is that you need to install additional tools and transcription speed depends on your computer's performance.

Requirements

You need one of the following local speech recognition backends:

Tool	CLI Included	Speed	Auto Model Download
WhisperKit (Desktop app on macOS)	Bundled helper	Fast	✓ In-app model download
faster-whisper	No (requires wrapper)	Fastest	✓ Yes
openai-whisper	Yes	Moderate	✓ Yes
whisper.cpp	Yes	Fast	✗ Manual

Desktop-only + FFmpeg note

• Transcription is available on desktop only.
• The standalone Desktop app can use WhisperKit on macOS when the bundled helper is present and at least one model is downloaded.
• For video transcription (.mp4, .webm, .mov, .avi, .mkv, .m4v), keep ffmpeg available. Social Archiver extracts audio from video before running Whisper when native audio input is not enough.

Recommendation

• Social Archiver Desktop on macOS: Use WhisperKit first. It has the simplest model install and is the default release path for native local transcription.
• Obsidian plugin on Apple Silicon Mac (M1/M2/M3/M4): Use whisper.cpp for best CLI performance (Metal GPU acceleration)
• Windows: Use openai-whisper for easiest setup (CLI included, no wrapper needed)
• Linux: Use faster-whisper for best CPU performance (auto model download)

Option 1: faster-whisper (Recommended)

CTranslate2-based implementation. Up to 4x faster than openai-whisper with lower memory usage. Models are automatically downloaded on first use.

• Repository: github.com/SYSTRAN/faster-whisper
• Models: Auto-downloaded to ~/.cache/huggingface/

CLI Wrapper Required

faster-whisper is a Python library without a built-in CLI. You need to install our CLI wrapper script.

Windows Users

For Windows, we recommend using openai-whisper instead (see Option 2 below). It includes a CLI out of the box and requires no additional setup. If you still want to use faster-whisper on Windows, follow the Windows-specific instructions below.

macOS / Linux Installation

Step 1: Install the library

Linux

pip install faster-whisper

macOS (Homebrew)

# macOS Homebrew blocks system-wide pip installs (PEP 668)
# Create a dedicated virtual environment instead:

# Create venv and install faster-whisper
python3 -m venv ~/.local/share/faster-whisper-venv
~/.local/share/faster-whisper-venv/bin/pip install faster-whisper

Step 2: Install CLI wrapper

Download the wrapper script and save it to your PATH:

# Create bin directory if it doesn't exist
mkdir -p ~/.local/bin

# Download wrapper script
curl -o ~/.local/bin/faster-whisper \
  https://raw.githubusercontent.com/hyungyunlim/obsidian-social-archiver-releases/main/faster-whisper-cli.py

# Make it executable
chmod +x ~/.local/bin/faster-whisper

macOS Users: Update the shebang

If you installed faster-whisper in a venv (Step 1), you need to update the script's shebang to use the venv Python:

# Replace the first line of the script
sed -i '' '1s|.*|#!/Users/'$USER'/.local/share/faster-whisper-venv/bin/python|' ~/.local/bin/faster-whisper

Step 3: Add to PATH (if not already)

# Add to your shell config (~/.zshrc or ~/.bashrc)
export PATH="$HOME/.local/bin:$PATH"

# Reload shell
source ~/.zshrc  # or ~/.bashrc

Verify installation:

faster-whisper --version

Windows Installation

Step 1: Install the library

pip install faster-whisper

Step 2: Install CLI wrapper

Open PowerShell and run:

# Download wrapper script to Python Scripts folder
# First, find your Python Scripts path
python -c "import sys; print(sys.prefix + '\\Scripts')"

# Download the wrapper (adjust path if needed)
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/hyungyunlim/obsidian-social-archiver-releases/main/faster-whisper-cli.py" -OutFile "$env:LOCALAPPDATA\Programs\Python\Python311\Scripts\faster-whisper.py"

Step 3: Create a batch wrapper

Create a file named faster-whisper.bat in the same Scripts folder:

@echo off
python "%~dp0faster-whisper.py" %*

Or run this PowerShell command:

$scriptsPath = python -c "import sys; print(sys.prefix + '\\Scripts')"
Set-Content -Path "$scriptsPath\faster-whisper.bat" -Value '@echo off`r`npython "%~dp0faster-whisper.py" %*'

Verify installation:

faster-whisper --version

Option 2: openai-whisper

The original Python implementation by OpenAI. Easy to install, works out of the box. Models are automatically downloaded on first use.

• Repository: github.com/openai/whisper
• Models: Auto-downloaded to ~/.cache/whisper/

macOS / Linux Installation

# Requires Python 3.8+
pip install openai-whisper

# Or with pipx (recommended for isolation)
pipx install openai-whisper

Verify installation:

whisper --help

Windows Installation

Prerequisites Required

Windows users must install FFmpeg and ensure PyTorch is properly configured before using openai-whisper.

Step 1: Install FFmpeg

FFmpeg is required for media processing. Choose one method:

winget (Recommended)

winget install ffmpeg

Chocolatey

choco install ffmpeg

Manual

# 1. Download from https://ffmpeg.org/download.html (Windows builds)
# 2. Extract to C:\ffmpeg
# 3. Add C:\ffmpeg\bin to system PATH

After installation, verify FFmpeg is in PATH:

ffmpeg -version

Step 2: Install openai-whisper

# Requires Python 3.8+
pip install openai-whisper

This will automatically install PyTorch. For NVIDIA GPU acceleration, install the CUDA version first:

# Optional: Install PyTorch with CUDA support (for NVIDIA GPUs)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

# Then install whisper
pip install openai-whisper

Step 3: Add Python Scripts to PATH (if not already)

# Find your Python Scripts path
python -c "import sys; print(sys.prefix + '\\Scripts')"

# Add this path to your system PATH environment variable
# Usually: C:\Users\<username>\AppData\Local\Programs\Python\Python3xx\Scripts

Verify installation:

whisper --help

Troubleshooting

If whisper command is not found:

1. Restart your terminal/PowerShell after PATH changes
2. Try running as python -m whisper instead
3. Ensure the Python Scripts folder is in your PATH

Option 3: whisper.cpp

High-performance C++ implementation. Best performance on Apple Silicon Macs with Metal GPU acceleration.

• Repository: github.com/ggerganov/whisper.cpp
• Models: huggingface.co/ggerganov/whisper.cpp

Manual Model Download Required

whisper.cpp requires you to manually download GGML model files before use.

Installation:

# macOS (using Homebrew)
brew install whisper-cpp

# Or build from source
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
cmake -B build && cmake --build build --config Release

Download models:

# Create models directory
mkdir -p ~/whisper-models

# Download small model (recommended, 465MB)
curl -L -o ~/whisper-models/ggml-small.bin \
  https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin

# Other models available:
# ggml-tiny.bin (74MB), ggml-base.bin (142MB)
# ggml-medium.bin (1.5GB), ggml-large-v3.bin (2.9GB)

The plugin searches for models in these locations:

• ~/whisper-models/
• ~/.cache/whisper-cpp/
• ~/whisper.cpp/models/

Verify installation:

whisper-cli --help

Plugin Settings

Configure transcription in Settings → Social Archiver → Transcription Settings:

Setting	Description
Enable Whisper transcription	Toggle transcription feature on/off
Preferred Whisper variant	Choose which Whisper implementation to use
Preferred model	Select model size (tiny to large)
Default language	Auto-detect or specify language
Custom Whisper path	Override automatic binary detection
Force enable custom path	Skip strict binary validation for custom path edge cases
Batch transcription mode	Transcribe only or Download & transcribe for video batch jobs
Batch transcribe videos in notes	Start / pause / resume / cancel batch transcription

Choosing a Variant

If you have multiple Whisper variants installed, you can select which one to use:

• Auto-detect: On Apple Silicon Mac, tries whisper.cpp first; on other systems, tries faster-whisper first
• faster-whisper: Easiest setup, auto model download, great CPU performance
• openai-whisper: Original implementation, easy setup, auto model download
• whisper.cpp: Best for Apple Silicon Macs (Metal GPU acceleration), requires manual model download

Models Are Not Shared

Each variant uses a different model format. Models downloaded for one variant cannot be used by another:

Variant	Model Format	Storage Location
faster-whisper	CTranslate2	~/.cache/huggingface/
openai-whisper	PyTorch (.pt)	~/.cache/whisper/
whisper.cpp	GGML (.bin)	Manual location

Supported Workflows

Workflow	Trigger	Notes
Desktop app local transcription	Timeline item has saved local audio/video	Choose model and language in the transcription modal
Additional language transcript	Timeline item already has a transcript	Reopen the transcription modal and choose another language/model
Podcast audio	Local audio is present in the archived note	Timeline shows transcription banner for downloaded/local audio
YouTube/TikTok video	Local video is present (downloaded first)	Timeline shows transcription banner for local video
Existing local videos in archive folder	Run batch transcription in Settings	Uses batch mode (transcribe-only or download-and-transcribe)

How It Works

1. Archive + download media: Ensure the item has a saved local audio/video file, or choose a download-and-transcribe flow for remote video.
2. Open in Timeline: View the archived post in Timeline view.
3. Click transcribe: Choose the available local-media or download-and-transcribe action.
4. Choose model and language: Desktop app WhisperKit runs can choose the installed model and language per transcription. Match original is the default.
5. Wait for processing: Progress is shown while Whisper runs locally.
6. View transcript: Transcript is saved with the archive item and shown in Timeline. If more than one transcript exists, switch languages from the player transcript dropdown.

Video transcription pipeline

For video files, Social Archiver may first extract mono 16kHz WAV audio with ffmpeg, then run Whisper on that extracted audio. Audio-only files can use the native backend directly when supported.

Transcript Features

Interactive Timestamps

Click any timestamp in the transcript to jump to that point in the media player. The current segment is highlighted during playback.

Search (Desktop)

Use the search box to find specific words or phrases in the transcript.

Collapse/Expand

Transcripts start collapsed to save space. Click to expand and view the full content.

Transcript Storage

The Desktop app stores transcripts with the archived item and exposes them in Timeline. When the Obsidian plugin writes back to a vault, transcripts are saved directly in markdown files under a ## Transcript section:

## Transcript

[00:00] Welcome everyone...

[00:15] Today we're discussing...

Metadata is stored in the YAML frontmatter:

transcriptionModel: small
transcriptionLanguage: en
transcriptionDuration: 1847.5
transcriptionTime: 2024-12-12T05:30:00.000Z

Model Selection

Choose a model based on your backend and needs.

For the Desktop app WhisperKit backend:

Model	Speed	Accuracy	Best For
Tiny	Fastest	Low	Quick checks and short clips
Small	Fast	Good	Default, balanced
Large v3 turbo	Slower	High	Quality-focused Korean, English, and mixed-language content

For CLI Whisper backends:

Model	Size	Speed	Accuracy	Best For
tiny	74MB	~32x	Low	Quick previews
base	142MB	~16x	Fair	Short clips
small	466MB	~6x	Good	Default, balanced
medium	1.5GB	~2x	High	Important content
large	2.9GB	~1x	Best	Maximum accuracy

Speed is relative to media duration (e.g., ~6x means 10 min audio/video takes ~1.5 min to transcribe).

Recommendation

Start with the small model. It offers a good balance of speed and accuracy for most spoken-content archives. Use Large v3 turbo when the content is important, noisy, or mixed-language and you can accept a slower run.

Troubleshooting

Transcribe Button Doesn't Appear

Common causes:

• No Whisper tool detected in system PATH
• WhisperKit is selected but no model is downloaded yet
• The Desktop app build does not include the WhisperKit helper
• Transcription is disabled in plugin settings
• Media is not downloaded to a local path yet (especially for YouTube/TikTok videos)

Solution:

1. In the Desktop app, choose WhisperKit, download a model, and confirm the readiness row turns Ready
2. For CLI backends, verify installation with whisper --version or faster-whisper --version
3. Enable transcription in Settings → Social Archiver → Transcription
4. Download media locally first, then reopen Timeline
5. Restart Obsidian after installing tools
6. Check that the CLI tool is in your system PATH

Transcription Fails or Times Out

Common causes:

• Insufficient memory for the selected model
• Corrupted audio/video file
• Very long media (2+ hours)
• ffmpeg missing for video transcription

Solutions:

• Try a smaller model (e.g., tiny or base)
• Verify the media file plays correctly
• Install ffmpeg and verify with ffmpeg -version (required for video)
• Check available disk space and memory
• If WhisperKit fails, open Installation status details in the Desktop app and check the helper/model status

Poor Transcription Quality

Common causes:

• Background noise in audio
• Multiple speakers talking over each other
• Non-standard accents or technical jargon
• Low-quality source audio in compressed video files

Solutions:

• Use a larger model (medium or large)
• Specify the language in the transcription modal or settings if auto-detection fails

Slow Transcription

Transcription speed depends on:

• Your CPU/GPU performance
• Selected model size
• Media length

Performance Tips

• Desktop app on macOS: Use WhisperKit first
• Obsidian plugin on Apple Silicon Macs: Use whisper.cpp for best CLI performance
• NVIDIA GPU: Use faster-whisper with CUDA support
• CPU only: Use tiny or base models for reasonable speed

Language Support

Whisper supports 99+ languages with automatic detection. For best results with non-English content:

1. Leave the language as Match original for the first run.
2. If detection fails, manually specify the language in the transcription modal or plugin settings.
3. To keep both versions, run an additional transcription in the target language and switch between transcript languages in Timeline.

Privacy & Storage

Local Processing Only

All transcription happens locally on your device. Media files are never uploaded to a transcription server. The Desktop app stores transcripts with the archived item, while the Obsidian plugin stores them in your vault.

Disk Space

Transcripts are text-only and very small (typically under 100KB even for long audio/video content). They won't significantly impact your archive or vault size.