Miscoshorts downloads a video, transcribes it with Whisper, asks an LLM for the 30–60 second segments with the most viral potential, reframes them to 9:16 following the speaker's face and burns in word-by-word subtitles. You approve the proposals (or type your own timestamps) before anything is rendered, and you get a title, description and hashtags ready to paste.

It works out of the box with Google Gemini + local Whisper, and can run fully on your own machine with any OpenAI-compatible server (Ollama, LM Studio, llama.cpp) and Whisper or a whisper.cpp server.

_{Made 100% locally from the tutorial with Whisper + qwen3:8b on Ollama.}

Features

• 🎥 YouTube (or any yt-dlp site) in the best available quality, or a local video file
• 🔍 Transcription with Whisper, faster-whisper or a whisper.cpp server, cached so you only transcribe once
• 🤖 Segment selection with Gemini or any OpenAI-compatible LLM, as validated JSON, with a retry when the answer is wrong
• 🎬 Several Shorts per video (-n 3), without overlaps, ranked by viral score
• 🎯 Smart crop that follows the speaker's face, with centre crop as fallback
• 📝 "Shorts style" subtitles: a few words at a time, the current word highlighted, bundled font on every OS
• 📐 Exported at 1080×1920 and the source frame rate, ready to upload
• 🏷️ Title, description and hashtags for YouTube / TikTok / Reels next to every video
• 📚 Long videos are split into chunks so local models with small contexts can handle them

How it works

flowchart LR
    A[URL or file] -->|yt-dlp| B[Video]
    B -->|Whisper / faster-whisper<br>/ whisper-server| C[Timestamped<br>transcript]
    C -->|Gemini or<br>OpenAI-compatible LLM| D[Clips as JSON:<br>times, score, metadata]
    D --> E{You approve,<br>pick or edit}
    E -->|face tracking +<br>MoviePy| F[1080×1920 +<br>word subtitles]
    F --> G[short.mp4<br>+ short.json]

Requirements

• Python 3.10+
• FFmpeg in your PATH
• One of:
  • a Google Gemini API key (default), or
  • an OpenAI-compatible LLM server (Ollama, LM Studio, llama.cpp…)

Install

git clone https://github.com/JoaquinRuiz/miscoshorts-ai.git
cd miscoshorts-ai
python -m venv .venv && source .venv/bin/activate
pip install -e ".[all]"        # or just `pip install -e .` without smart crop / faster-whisper

Optional extras: smartcrop (OpenCV face tracking), faster (faster-whisper), all (both). pip install -r requirements.txt still works and installs the base package.

Install FFmpeg if you don't have it:

brew install ffmpeg                               # macOS
sudo apt update && sudo apt install ffmpeg        # Ubuntu / Debian
# Windows: https://ffmpeg.org/download.html

For the default setup, put your Gemini key in a .env file in the folder where you run the command (or export it):

echo "GEMINI_API_KEY=your-key" > .env

Usage

miscoshorts "https://www.youtube.com/watch?v=VIDEO_ID"

python maker.py URL and python -m miscoshorts URL do the same. The tool downloads and transcribes the video, then shows the AI's proposals:

🤖 PROPUESTAS DE SHORT (2):
 [1] ⭐ 9  📌 El truco que nadie te cuenta
     ⏱️  2:00.5 → 2:35.0  (34.5s)
     💡 Gancho fuerte y una idea completa
 [2] ⭐ 7  📌 ...
¿Te mola? 's' (o Enter) para crearlos todos, números para elegir (ej: 1,3), nuevos tiempos (ej: 120-140 o 2:00-2:20), o 'q' para salir:

Each Short is saved as an .mp4 with a .json next to it holding the title, description, hashtags and times.

Examples

miscoshorts video.mp4                                  # local file
miscoshorts URL -n 3                                   # three Shorts from the same video
miscoshorts URL -n 3 --yes -o out/short.mp4            # no questions: out/short_1.mp4 … _3.mp4
miscoshorts URL --start 2:00 --end 2:35                # your own segment, no AI needed
miscoshorts URL --lang es --whisper-model small        # better transcription for Spanish
miscoshorts URL --min-seconds 15 --max-seconds 30      # shorter clips
miscoshorts URL --crop center --words-per-caption 1    # fixed crop, one word at a time
miscoshorts --help                                     # every option

Configuration

Every setting can be an environment variable or a line in .env; command-line options win over both. The defaults reproduce the original setup (Gemini + local Whisper).

Reasoning models are supported: <think>…</think> blocks are stripped before the answer is parsed.

Subtitles

Run it 100% local

No API keys and nothing leaves your network:

# 1. An LLM server, e.g. Ollama
ollama pull qwen3:8b

# 2. Run Miscoshorts with local Whisper and the local LLM
CEREBRO_BACKEND=openai OPENAI_MODEL=qwen3:8b \
  miscoshorts "https://www.youtube.com/watch?v=VIDEO_ID"

To transcribe with whisper.cpp instead, start ./whisper-server -m models/ggml-base.bin --port 9191 and add --whisper-api. The audio is converted to 16 kHz mono WAV first, so the server does not need --convert; it must support response_format=srt. whisper-server gives no word timestamps, so word timings are estimated within each subtitle line.

Output files

The downloaded video goes to a temporary folder that is always deleted, even if something fails.

Project layout

miscoshorts/
├── cli.py          Command line and the pipeline
├── config.py       Settings from env / .env
├── download.py     yt-dlp download or local file
├── transcribe.py   Whisper, faster-whisper, whisper-server + cache
├── llm.py          Prompt, Gemini / OpenAI backends, chunking, retries
├── clips.py        Parse, validate and rank the LLM's clips
├── captions.py     Group words into captions
├── subtitles.py    Draw captions with Pillow
├── smartcrop.py    Face tracking with OpenCV
├── render.py       Reframe, resize and export with MoviePy
└── assets/fonts/   Anton (SIL Open Font License)
tests/              pytest
maker.py            `python maker.py URL`, kept for the tutorial

Troubleshooting

pip uninstall whisper
pip install openai-whisper

Development

pip install -e ".[all,dev]"
pytest          # unit tests + a small render test (needs FFmpeg)
ruff check . && ruff format --check .

CI runs the same on every pull request.

Contributing

Issues and pull requests are welcome: read CONTRIBUTING.md first. Thanks to @Jay4242 for the OpenAI-compatible and whisper-server backends.

Author

Made by Joaquín Ruiz: YouTube @jokioki · LinkedIn

If this code helped you, understanding the logic behind the AI is what will set you apart as an engineer. My books on AI (in Spanish):

License

MIT. The bundled Anton font is under the SIL Open Font License.