Skip to content
Projects

gBillal/xav has 82 GitHub stars and 5 forks. GitHot measured 0 stars in 24 hours and 0 in 7 days. Last checked 2026-11-15 20:27 UTC. Last push 2026-10-28.

How these counts are measured

Measured star growth

in 24h
0
in 7 days
0
in 15 days
Update delayed
in 1 month
Update delayed

Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.

About

Elixir wrapper over FFmpeg for reading and decoding audio and video data

Creative & Media

README.md

Original

Xav

Hex.pm API Docs CI codecov

Elixir wrapper over FFmpeg for reading audio and video files.

See an interview with FFmpeg enthusiast: https://youtu.be/9kaIXkImCAM

Installation

Make sure you have installed FFMpeg (ver. 4.x - 7.x) development packages on your system (see here for installation one-liners) and add Xav to the list of your dependencies:

def deps do
  [
    {:xav, "~> 0.12.0"},
    # Add Nx if you want to have Xav.Frame.to_nx/1
    {:nx, ">= 0.0.0"}
  ]
end

Controlling FFmpeg log output

FFmpeg's underlying libraries (libavcodec, libswscale, ...) print to stderr at the AV_LOG_INFO level by default. This is usually fine but can produce informational noise such as

[swscaler @ 0x1490a0000] No accelerated colorspace conversion found from yuv420p to rgb24.

when libswscale falls back to a generic colorspace conversion path. These are not errors — decoded frames are bit-exact — but they can clutter test output and logs.

You can raise the threshold from Elixir:

Xav.set_log_level(:error)

Or set it once at application start by configuring your application env:

# config/runtime.exs
config :xav, ffmpeg_log_level: :error

Xav.Application reads this on boot and applies it before your supervision tree starts. Valid atoms are :quiet, :panic, :fatal, :error, :warning, :info, :verbose, :debug, and :trace. An integer FFmpeg level is also accepted.

Note that av_log_set_level/1 is process-global — changing the level affects every libav* call made from the current OS process, not just the reader that triggered the change.

Usage

Decode

decoder = Xav.Decoder.new(:vp8, out_format: :rgb24)
{:ok, %Xav.Frame{} = frame} = Xav.Decoder.decode(decoder, <<"somebinary">>)

Decode with audio resampling

decoder = Xav.Decoder.new(:opus, out_format: :flt, out_sample_rate: 16_000)
{:ok, %Xav.Frame{} = frame} = Xav.Decoder.decode(decoder, <<"somebinary">>)

Read from a file:

r = Xav.Reader.new!("./some_mp4_file.mp4")
{:ok, %Xav.Frame{} = frame} = Xav.Reader.next_frame(r)
tensor = Xav.Frame.to_nx(frame)
Kino.Image.new(tensor)

Read from a camera:

r = Xav.Reader.new!("/dev/video0", device?: true, out_format: :rgb24)
{:ok, %Xav.Frame{} = frame} = Xav.Reader.next_frame(r)
tensor = Xav.Frame.to_nx(frame)
Kino.Image.new(tensor)

Speech to text:

{:ok, whisper} = Bumblebee.load_model({:hf, "openai/whisper-tiny"})
{:ok, featurizer} = Bumblebee.load_featurizer({:hf, "openai/whisper-tiny"})
{:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "openai/whisper-tiny"})
{:ok, generation_config} = Bumblebee.load_generation_config({:hf, "openai/whisper-tiny"})

serving =
  Bumblebee.Audio.speech_to_text_whisper(whisper, featurizer, tokenizer, generation_config,
    defn_options: [compiler: EXLA]
  )

# Read a couple of frames.
# See https://hexdocs.pm/bumblebee/Bumblebee.Audio.WhisperFeaturizer.html for default sampling rate.
frames =
    Xav.Reader.stream!("sample.mp3", read: :audio, out_format: :flt, out_channels: 1, out_sample_rate: 16_000)
    |> Stream.take(200)
    |> Enum.map(fn frame -> Xav.Frame.to_nx(frame) end)

batch = Nx.Batch.concatenate(frames)
batch = Nx.Defn.jit_apply(&Function.identity/1, [batch])
Nx.Serving.run(serving, batch) 

Development

To make clangd aware of the header files used in your project, you can create a compile_commands.json file. clangd uses this file to know the compiler flags, include paths, and other compilation options for each source file.

Install bear

The easiest way to generate compile_commands.json from a Makefile is to use the bear tool. bear is a tool that records the compiler calls during a build and creates the compile_commands.json file.

You can install bear with your package manager:

  • macOS: brew install bear
  • Ubuntu/Debian: sudo apt install bear
  • Fedora: sudo dnf install bear

Generate compile_commands.json

After installing bear, you can run it alongside your make command to capture the necessary information.

bear -- mix compile