Driver/Input: Migrate audio backend to Symphonia (#89)

This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions

View File

@@ -3,7 +3,7 @@
html_favicon_url = "https://raw.githubusercontent.com/serenity-rs/songbird/current/songbird-ico.png"
)]
#![deny(missing_docs)]
#![deny(broken_intra_doc_links)]
#![deny(rustdoc::broken_intra_doc_links)]
//! ![project logo][logo]
//!
//! Songbird is an async, cross-library compatible voice system for Discord, written in Rust.
@@ -16,7 +16,7 @@
//! can run the songbird voice driver.
//! * And, by default, a fully featured voice system featuring events, queues, RT(C)P packet
//! handling, seeking on compatible streams, shared multithreaded audio stream caches,
//! and direct Opus data passthrough from DCA files.
//! and direct Opus data passthrough.
//!
//! ## Intents
//! Songbird's gateway functionality requires you to specify the `GUILD_VOICE_STATES` intent.
@@ -25,6 +25,26 @@
//! Full examples showing various types of functionality and integrations can be found
//! in [this crate's examples directory].
//!
//! ## Codec support
//! Songbird supports all [codecs and formats provided by Symphonia] (pure-Rust), with Opus support
//! provided by [audiopus] (an FFI wrapper for libopus).
//!
//! **By default, *Songbird will not request any codecs from Symphonia*.** To change this, in your own
//! project you will need to depend on Symphonia as well.
//!
//! ```toml
//! ## Including songbird alone gives you support for Opus via the DCA file format.
//! [dependencies.songbird]
//! version = "0.4"
//! features = ["builtin-queue"]
//!
//! ## To get additional codecs, you *must* add Symphonia yourself.
//! ## This includes the default formats (MKV/WebM, Ogg, Wave) and codecs (FLAC, PCM, Vorbis)...
//! [dependencies.symphonia]
//! version = "0.5"
//! features = ["aac", "mp3", "isomp4", "alac"] # ...as well as any extras you need!
//! ```
//!
//! ## Attribution
//!
//! Songbird's logo is based upon the copyright-free image ["Black-Capped Chickadee"] by George Gorgas White.
@@ -36,55 +56,64 @@
//! ["Black-Capped Chickadee"]: https://www.oldbookillustrations.com/illustrations/black-capped-chickadee/
//! [`ConnectionInfo`]: struct@ConnectionInfo
//! [lavalink]: https://github.com/freyacodes/Lavalink
//! [codecs and formats provided by Symphonia]: https://github.com/pdeljanov/Symphonia#formats-demuxers
//! [audiopus]: https://github.com/lakelezz/audiopus
#[cfg(all(feature = "youtube-dlc", feature = "yt-dlp"))]
compile_error!("feature \"youtube-dlc\" and feature \"yt-dlp\" cannot be enabled at the same time");
#![warn(clippy::pedantic)]
#![allow(
// Allowed as they are too pedantic
clippy::module_name_repetitions,
clippy::wildcard_imports,
clippy::too_many_lines,
clippy::cast_lossless,
clippy::cast_sign_loss,
clippy::cast_possible_wrap,
clippy::cast_precision_loss,
clippy::cast_possible_truncation,
// TODO: would require significant rewriting of all existing docs
clippy::missing_errors_doc,
)]
mod config;
pub mod constants;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub mod driver;
pub mod error;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub mod events;
#[cfg(feature = "gateway-core")]
#[cfg(feature = "gateway")]
mod handler;
pub mod id;
pub(crate) mod info;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub mod input;
#[cfg(feature = "gateway-core")]
#[cfg(feature = "gateway")]
pub mod join;
#[cfg(feature = "gateway-core")]
#[cfg(feature = "gateway")]
mod manager;
#[cfg(feature = "serenity")]
pub mod serenity;
#[cfg(feature = "gateway-core")]
#[cfg(feature = "gateway")]
pub mod shards;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub mod tracks;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
mod ws;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub use discortp as packet;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub use serenity_voice_model as model;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub use typemap_rev as typemap;
#[cfg(test)]
use utils as test_utils;
#[cfg(feature = "driver-core")]
#[cfg(feature = "driver")]
pub use crate::{
driver::Driver,
events::{CoreEvent, Event, EventContext, EventHandler, TrackEvent},
input::{ffmpeg, ytdl},
tracks::create_player,
};
#[cfg(feature = "gateway-core")]
#[cfg(feature = "gateway")]
pub use crate::{handler::*, manager::*};
#[cfg(feature = "serenity")]