servius/songbird
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Driver/Input: Migrate audio backend to Symphonia (#89)

Browse Source 
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions
Download Patch File Download Diff File Expand all files Collapse all files

76
src/driver/crypto.rs
View File

@@ -12,7 +12,7 @@ use xsalsa20poly1305::{
TAG_SIZE,
};
/// Variants of the XSalsa20Poly1305 encryption scheme.
/// Variants of the `XSalsa20Poly1305` encryption scheme.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive]
pub enum CryptoMode {
@@ -35,57 +35,58 @@ pub enum CryptoMode {
impl From<CryptoState> for CryptoMode {
fn from(val: CryptoState) -> Self {
use CryptoState::*;
match val {
Normal => CryptoMode::Normal,
Suffix => CryptoMode::Suffix,
Lite(_) => CryptoMode::Lite,
CryptoState::Normal => Self::Normal,
CryptoState::Suffix => Self::Suffix,
CryptoState::Lite(_) => Self::Lite,
}
}
}
impl CryptoMode {
/// Returns the name of a mode as it will appear during negotiation.
#[must_use]
pub fn to_request_str(self) -> &'static str {
use CryptoMode::*;
match self {
Normal => "xsalsa20_poly1305",
Suffix => "xsalsa20_poly1305_suffix",
Lite => "xsalsa20_poly1305_lite",
Self::Normal => "xsalsa20_poly1305",
Self::Suffix => "xsalsa20_poly1305_suffix",
Self::Lite => "xsalsa20_poly1305_lite",
}
}
/// Returns the number of bytes each nonce is stored as within
/// a packet.
#[must_use]
pub fn nonce_size(self) -> usize {
use CryptoMode::*;
match self {
Normal => RtpPacket::minimum_packet_size(),
Suffix => NONCE_SIZE,
Lite => 4,
Self::Normal => RtpPacket::minimum_packet_size(),
Self::Suffix => NONCE_SIZE,
Self::Lite => 4,
}
}
/// Returns the number of bytes occupied by the encryption scheme
/// which fall before the payload.
pub fn payload_prefix_len(self) -> usize {
#[must_use]
pub fn payload_prefix_len() -> usize {
TAG_SIZE
}
/// Returns the number of bytes occupied by the encryption scheme
/// which fall after the payload.
#[must_use]
pub fn payload_suffix_len(self) -> usize {
use CryptoMode::*;
match self {
Normal => 0,
Suffix | Lite => self.nonce_size(),
Self::Normal => 0,
Self::Suffix | Self::Lite => self.nonce_size(),
}
}
/// Calculates the number of additional bytes required compared
/// to an unencrypted payload.
#[must_use]
pub fn payload_overhead(self) -> usize {
self.payload_prefix_len() + self.payload_suffix_len()
Self::payload_prefix_len() + self.payload_suffix_len()
}
/// Extracts the byte slice in a packet used as the nonce, and the remaining mutable
@@ -95,10 +96,9 @@ impl CryptoMode {
header: &'a [u8],
body: &'a mut [u8],
) -> Result<(&'a [u8], &'a mut [u8]), CryptoError> {
use CryptoMode::*;
match self {
Normal => Ok((header, body)),
Suffix | Lite => {
Self::Normal => Ok((header, body)),
Self::Suffix | Self::Lite => {
let len = body.len();
if len < self.payload_suffix_len() {
Err(CryptoError)
@@ -135,7 +135,7 @@ impl CryptoMode {
&nonce
};
let body_start = self.payload_prefix_len();
let body_start = Self::payload_prefix_len();
let body_tail = self.payload_suffix_len();
if body_start > body_remaining.len() {
@@ -183,22 +183,33 @@ impl CryptoMode {
}
}
#[allow(missing_docs)]
/// State used in nonce generation for the `XSalsa20Poly1305` encryption variants
/// in [`CryptoMode`].
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive]
pub enum CryptoState {
/// The RTP header is used as the source of nonce bytes for the packet.
///
/// No state is required.
Normal,
/// An additional random 24B suffix is used as the source of nonce bytes for the packet.
/// This is regenerated randomly for each packet.
///
/// No state is required.
Suffix,
/// An additional random 4B suffix is used as the source of nonce bytes for the packet.
/// This nonce value increments by `1` with each packet.
///
/// The last used nonce is stored.
Lite(Wrapping<u32>),
}
impl From<CryptoMode> for CryptoState {
fn from(val: CryptoMode) -> Self {
use CryptoMode::*;
match val {
Normal => CryptoState::Normal,
Suffix => CryptoState::Suffix,
Lite => CryptoState::Lite(Wrapping(rand::random::<u32>())),
CryptoMode::Normal => CryptoState::Normal,
CryptoMode::Suffix => CryptoState::Suffix,
CryptoMode::Lite => CryptoState::Lite(Wrapping(rand::random::<u32>())),
}
}
}
@@ -213,12 +224,11 @@ impl CryptoState {
let mode = self.kind();
let endpoint = payload_end + mode.payload_suffix_len();
use CryptoState::*;
match self {
Suffix => {
Self::Suffix => {
rand::thread_rng().fill(&mut packet.payload_mut()[payload_end..endpoint]);
},
Lite(mut i) => {
Self::Lite(mut i) => {
(&mut packet.payload_mut()[payload_end..endpoint])
.write_u32::<NetworkEndian>(i.0)
.expect(
@@ -233,8 +243,8 @@ impl CryptoState {
}
/// Returns the underlying (stateless) type of the active crypto mode.
pub fn kind(&self) -> CryptoMode {
CryptoMode::from(*self)
pub fn kind(self) -> CryptoMode {
CryptoMode::from(self)
}
}
@@ -246,7 +256,7 @@ mod test {
#[test]
fn small_packet_decrypts_error() {
let mut buf = [0u8; MutableRtpPacket::minimum_packet_size() + 0];
let mut buf = [0u8; MutableRtpPacket::minimum_packet_size()];
let modes = [CryptoMode::Normal, CryptoMode::Suffix, CryptoMode::Lite];
let mut pkt = MutableRtpPacket::new(&mut buf[..]).unwrap();