Driver/Input: Migrate audio backend to Symphonia (#89)

This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions

158
src/input/sources/ytdl.rs Normal file
View File

@@ -0,0 +1,158 @@
use crate::input::{
metadata::ytdl::Output,
AudioStream,
AudioStreamError,
AuxMetadata,
Compose,
HttpRequest,
Input,
};
use async_trait::async_trait;
use reqwest::{
header::{HeaderMap, HeaderName, HeaderValue},
Client,
};
use std::error::Error;
use symphonia_core::io::MediaSource;
use tokio::process::Command;
const YOUTUBE_DL_COMMAND: &str = "yt-dlp";
/// A lazily instantiated call to download a file, finding its URL via youtube-dl.
///
/// By default, this uses yt-dlp and is backed by an [`HttpRequest`]. This handler
/// attempts to find the best audio-only source (typically `WebM`, enabling low-cost
/// Opus frame passthrough).
///
/// [`HttpRequest`]: super::HttpRequest
#[derive(Clone, Debug)]
pub struct YoutubeDl {
program: &'static str,
client: Client,
metadata: Option<AuxMetadata>,
url: String,
}
impl YoutubeDl {
/// Creates a lazy request to select an audio stream from `url`, using "yt-dlp".
///
/// This requires a reqwest client: ideally, one should be created and shared between
/// all requests.
#[must_use]
pub fn new(client: Client, url: String) -> Self {
Self::new_ytdl_like(YOUTUBE_DL_COMMAND, client, url)
}
/// Creates a lazy request to select an audio stream from `url` as in [`new`], using `program`.
///
/// [`new`]: Self::new
#[must_use]
pub fn new_ytdl_like(program: &'static str, client: Client, url: String) -> Self {
Self {
program,
client,
metadata: None,
url,
}
}
async fn query(&mut self) -> Result<Output, AudioStreamError> {
let ytdl_args = ["-j", &self.url, "-f", "ba[abr>0][vcodec=none]/best"];
let output = Command::new(self.program)
.args(&ytdl_args)
.output()
.await
.map_err(|e| AudioStreamError::Fail(Box::new(e)))?;
let stdout: Output = serde_json::from_slice(&output.stdout[..])
.map_err(|e| AudioStreamError::Fail(Box::new(e)))?;
self.metadata = Some(stdout.as_aux_metadata());
Ok(stdout)
}
}
impl From<YoutubeDl> for Input {
fn from(val: YoutubeDl) -> Self {
Input::Lazy(Box::new(val))
}
}
#[async_trait]
impl Compose for YoutubeDl {
fn create(&mut self) -> Result<AudioStream<Box<dyn MediaSource>>, AudioStreamError> {
Err(AudioStreamError::Unsupported)
}
async fn create_async(
&mut self,
) -> Result<AudioStream<Box<dyn MediaSource>>, AudioStreamError> {
let stdout = self.query().await?;
let mut headers = HeaderMap::default();
if let Some(map) = stdout.http_headers {
headers.extend(map.iter().filter_map(|(k, v)| {
Some((
HeaderName::from_bytes(k.as_bytes()).ok()?,
HeaderValue::from_str(v).ok()?,
))
}));
}
let mut req = HttpRequest {
client: self.client.clone(),
request: stdout.url,
headers,
content_length: stdout.filesize,
};
req.create_async().await
}
fn should_create_async(&self) -> bool {
true
}
async fn aux_metadata(&mut self) -> Result<AuxMetadata, AudioStreamError> {
if let Some(meta) = self.metadata.as_ref() {
return Ok(meta.clone());
}
self.query().await?;
self.metadata.clone().ok_or_else(|| {
let msg: Box<dyn Error + Send + Sync + 'static> =
"Failed to instansiate any metadata... Should be unreachable.".into();
AudioStreamError::Fail(msg)
})
}
}
#[cfg(test)]
mod tests {
use reqwest::Client;
use super::*;
use crate::{constants::test_data::YTDL_TARGET, input::input_tests::*};
#[tokio::test]
#[ntest::timeout(20_000)]
async fn ytdl_track_plays() {
track_plays_mixed(|| YoutubeDl::new(Client::new(), YTDL_TARGET.into())).await;
}
#[tokio::test]
#[ntest::timeout(20_000)]
async fn ytdl_forward_seek_correct() {
forward_seek_correct(|| YoutubeDl::new(Client::new(), YTDL_TARGET.into())).await;
}
#[tokio::test]
#[ntest::timeout(20_000)]
async fn ytdl_backward_seek_correct() {
backward_seek_correct(|| YoutubeDl::new(Client::new(), YTDL_TARGET.into())).await;
}
}