servius/songbird
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Driver/Input: Migrate audio backend to Symphonia (#89)

Browse Source 
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
README.md
View File

@@ -1,4 +1,4 @@
[![docs-badge][]][docs] [![build badge]][build] [![guild-badge][]][guild] [![crates.io version]][crates.io link] [![rust 1.60.0+ badge]][rust 1.60.0+ link]
[![docs-badge][]][docs] [![build badge]][build] [![guild-badge][]][guild] [![crates.io version]][crates.io link] [![rust 1.61.0+ badge]][rust 1.61.0+ link]
# Songbird
@@ -19,27 +19,48 @@ The library offers:
## Intents
Songbird's gateway functionality requires you to specify the `GUILD_VOICE_STATES` intent.
## Codec support
Songbird supports all [codecs and formats provided by Symphonia] (pure-Rust), with Opus support
provided by [audiopus] (an FFI wrapper for libopus).
**By default, *Songbird will not request any codecs from Symphonia*.** To change this, in your own
project you will need to depend on Symphonia as well.
```toml
# Including songbird alone gives you support for Opus via the DCA file format.
[dependencies.songbird]
version = "0.4"
features = ["builtin-queue"]
# To get additional codecs, you *must* add Symphonia yourself.
# This includes the default formats (MKV/WebM, Ogg, Wave) and codecs (FLAC, PCM, Vorbis)...
[dependencies.symphonia]
# version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"] # ...as well as any extras you need!
# **NOTE**: For now, please use this fork in pre-releases for a key fix to
# seeking on streamed MKV/WebM files.
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
```
## Dependencies
Songbird needs a few system dependencies before you can use it.
- Opus - Audio codec that Discord uses.
If you are on Windows and you are using the MSVC toolchain, a prebuilt DLL is provided for you, you do not have to do anything.
On other platforms, you will have to install it. You can install the library with `apt install libopus-dev` on Ubuntu or `pacman -S opus` on Arch Linux.
[audiopus] will use installed libopus binaries if available via pkgconf on Linux/MacOS, otherwise you will need to install cmake to build opus from source.
This is always the case on Windows.
For Unix systems, you can install the library with `apt install libopus-dev` on Ubuntu or `pacman -S opus` on Arch Linux.
If you do not have it installed it will be built for you. However, you will need a C compiler and the GNU autotools installed.
Again, these can be installed with `apt install build-essential autoconf automake libtool m4` on Ubuntu or `pacman -S base-devel` on Arch Linux.
This is a required dependency. Songbird cannot work without it.
- FFmpeg - Audio/Video conversion tool.
You can install the tool with `apt install ffmpeg` on Ubuntu or `pacman -S ffmpeg` on Arch Linux.
This is an optional, but recommended dependency. It allows Songbird to convert from, for instance, .mp4 files to the audio format Discord uses.
- youtube-dl - Audio/Video download tool.
You can install the tool with Python's package manager, pip, which we recommend for youtube-dl. You can do it with the command `pip install youtube_dl`.
- yt-dlp / youtube-dl / (similar forks) - Audio/Video download tool.
yt-dlp can be installed [according to the installation instructions on the main repo].
You can install youtube-dl with Python's package manager, pip, which we recommend for youtube-dl. You can do it with the command `pip install youtube_dl`.
Alternatively, you can install it with your system's package manager, `apt install youtube-dl` on Ubuntu or `pacman -S youtube-dl` on Arch Linux.
This is an optional dependency. It allows Songbird to download an audio source from the Internet, which will be converted to the audio format Discord uses.
This is an optional dependency for users, but is required as a dev-dependency. It allows Songbird to download audio/video sources from the Internet from a variety of webpages, which it will convert to the Opus audio format Discord uses.
## Examples
Full examples showing various types of functionality and integrations can be found in [this crate's examples directory].
@@ -56,6 +77,9 @@ Songbird's logo is based upon the copyright-free image ["Black-Capped Chickadee"
[lavalink]: https://github.com/freyacodes/Lavalink
[this crate's examples directory]: https://github.com/serenity-rs/songbird/tree/current/examples
[our contributor guidelines]: CONTRIBUTING.md
[codecs and formats provided by Symphonia]: https://github.com/pdeljanov/Symphonia#formats-demuxers
[audiopus]: https://github.com/lakelezz/audiopus
[according to the installation instructions on the main repo]: https://github.com/yt-dlp/yt-dlp#installation
[build badge]: https://img.shields.io/github/workflow/status/serenity-rs/songbird/CI?style=flat-square
[build]: https://github.com/serenity-rs/songbird/actions
@@ -69,5 +93,5 @@ Songbird's logo is based upon the copyright-free image ["Black-Capped Chickadee"
[crates.io link]: https://crates.io/crates/songbird
[crates.io version]: https://img.shields.io/crates/v/songbird.svg?style=flat-square
[rust 1.60.0+ badge]: https://img.shields.io/badge/rust-1.60.0+-93450a.svg?style=flat-square
[rust 1.60.0+ link]: https://blog.rust-lang.org/2022/04/07/Rust-1.60.0.html
[rust 1.61.0+ badge]: https://img.shields.io/badge/rust-1.61.0+-93450a.svg?style=flat-square
[rust 1.61.0+ link]: https://blog.rust-lang.org/2022/05/19/Rust-1.61.0.html