servius/songbird
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Driver/Input: Migrate audio backend to Symphonia (#89)

Browse Source 
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.

`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.

`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.

Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.

Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.

Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
This commit is contained in:
Kyle Simpson
2022-07-23 23:29:02 +01:00
parent 6c6ffa7ca8
commit 8cc7a22b0b
136 changed files with 9761 additions and 4891 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
.github/workflows/ci.yml vendored
View File

@@ -39,7 +39,6 @@ jobs:
- Windows - Windows
- driver only - driver only
- gateway only - gateway only
- legacy tokio
include: include:
- name: beta - name: beta
@@ -75,6 +74,16 @@ jobs:
sudo apt-get update sudo apt-get update
sudo apt-get install -y libopus-dev sudo apt-get install -y libopus-dev
- name: Install yt-dlp (Unix)
if: runner.os != 'Windows'
run: |
sudo wget https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -O /usr/local/bin/yt-dlp
sudo chmod a+rx /usr/local/bin/yt-dlp
- name: Install yt-dlp (Windows)
if: runner.os == 'Windows'
run: choco install yt-dlp
- name: Setup cache - name: Setup cache
if: runner.os != 'macOS' if: runner.os != 'macOS'
uses: actions/cache@v2 uses: actions/cache@v2
@@ -175,9 +184,9 @@ jobs:
- name: 'Build serenity/voice_receive' - name: 'Build serenity/voice_receive'
working-directory: examples working-directory: examples
run: cargo build -p voice_receive run: cargo build -p voice_receive
- name: 'Build serenity/voice_storage' - name: 'Build serenity/voice_cached_audio'
working-directory: examples working-directory: examples
run: cargo build -p voice_storage run: cargo build -p voice_cached_audio
- name: 'Build twilight' - name: 'Build twilight'
working-directory: examples working-directory: examples
run: cargo build -p twilight run: cargo build -p twilight

17
ARCHITECTURE.md
View File

@@ -29,12 +29,14 @@ Songbird's **driver** is a mixed sync/async system for running voice connections
Audio processing remains synchronous for the following reasons: Audio processing remains synchronous for the following reasons:
* Encryption, encoding, and mixing are compute bound tasks which cannot be subdivided cleanly by the Tokio executor. Having these block the scheduler's finite thread count has a significant impact on servicing other tasks. * Encryption, encoding, and mixing are compute bound tasks which cannot be subdivided cleanly by the Tokio executor. Having these block the scheduler's finite thread count has a significant impact on servicing other tasks.
* `Read` and `Seek` are considerably more user-friendly to use, implement, and integrate than `AsyncRead`, `AsyncBufRead`, and `AsyncSeek`. * `Read` and `Seek` are considerably more user-friendly to use, implement, and integrate than `AsyncRead`, `AsyncBufRead`, and `AsyncSeek`.
* Symphonia implements all of its functionality based on synchronous I/O.
## Tasks ## Tasks
Songbird subdivides voice connection handling into several long- and short-lived tasks. Songbird subdivides voice connection handling into several long- and short-lived tasks.
* **Core**: Handles and directs commands received from the driver. Responsible for connection/reconnection, and creates network tasks. * **Core**: Handles and directs commands received from the driver. Responsible for connection/reconnection, and creates network tasks.
* **Mixer**: Combines audio sources together, Opus encodes the result, and encrypts the built packets every 20ms. Responsible for handling track commands/state. ***Synchronous***. * **Mixer**: Combines audio sources together, Opus encodes the result, and encrypts the built packets every 20ms. Responsible for handling track commands/state. ***Synchronous***.
* **Thread Pool**: A dynamically sized thread-pool for I/O tasks. Creates lazy tracks using `Compose` if sync creation is needed, otherwise spawns a tokio task. Seek operations always go to the thread pool. ***Synchronous***.
* **Disposer**: Used by mixer thread to dispose of data with potentially long/blocking `Drop` implementations (i.e., audio sources). ***Synchronous***. * **Disposer**: Used by mixer thread to dispose of data with potentially long/blocking `Drop` implementations (i.e., audio sources). ***Synchronous***.
* **Events**: Stores and runs event handlers, tracks event timing, and handles * **Events**: Stores and runs event handlers, tracks event timing, and handles
* **Websocket**: *Network task.* Sends speaking status updates and keepalives to Discord, and receives client (dis)connect events. * **Websocket**: *Network task.* Sends speaking status updates and keepalives to Discord, and receives client (dis)connect events.
@@ -52,23 +54,22 @@ src/driver/*
## Audio handling ## Audio handling
### Input ### Input
Inputs are raw audio sources: composed of a `Reader` (which can be `Read`-only or `Read + Seek`), a framing mechanism, and a codec. Inputs are audio sources supporting lazy initialisation, being either:
Several wrappers exist to add `Seek` capabilities to one-way streams via storage or explicitly recreating the struct. * **lazy inputs**—a trait object which allows an instructions to create an audio source to be cheaply stored. This will be initialised when needed either synchronously or asynchronously based on what which methods the trait object supports.
* **live inputs**—a usable audio object implementing `MediaSource: Read + Seek`. `Seek` support may be dummied in, as seek use and support is gated by `MediaSource`. These can be passed in at various stages of processing by symphonia.
Framing is not always needed (`Raw`), but makes it possible to consume the correct number of bytes needed to decode one audio packet (and/or simplify skipping through the stream). Several wrappers exist to add `Seek` capabilities to one-way streams via storage or explicitly recreating the struct, `AsyncRead` adapters, and raw audio input adapters.
Currently, Opus and raw (`i16`/`f32`) audio sources are supported, though only the DCA framing for Opus is implemented.
At present, the use of the FFmpeg executable allows us to receive raw input, but at heavy memory cost.
Further implementations are possible in the present framework (e.g., WebM/MKV and Ogg containers, MP3 and linked FFI FFmpeg as codecs).
Internally, the mixer uses floating-point audio to prevent clipping and allow more granular volume control. Internally, the mixer uses floating-point audio to prevent clipping and allow more granular volume control.
If a source is known to use the Opus codec (and is the only source), then it can bypass mixing altogether. Symphonia is used to demux and decode input files in a variety of formats into this floating-point buffer: songbird supports all codecs and containers which are part of the symphonia project, while adding support for Opus decoding and DCA1 container files.
If a source uses the Opus codec (and is the only source), then it can bypass mixing and re-encoding altogether, saving CPU cycles per server.
``` ```
src/input/* src/input/*
``` ```
### Tracks ### Tracks
Tracks hold additional state which is expected to change over the lifetime of a track: position, play state, and modifiers like volume. Tracks hold additional state which is expected to change over the lifetime of a track: position, play state, and modifiers like volume.
Tracks (and their handles) also allow per-source events to be inserted. Tracks (and their handles) also allow per-source events to be inserted.
Tracks are defined in user code, where they are fully modifiable, before being passed into the driver. Tracks are defined in user code, where they are fully modifiable, before being passed into the driver.

116
Cargo.toml
View File

@@ -2,7 +2,7 @@
authors = ["Kyle Simpson <kyleandrew.simpson@gmail.com>"] authors = ["Kyle Simpson <kyleandrew.simpson@gmail.com>"]
description = "An async Rust library for the Discord voice API." description = "An async Rust library for the Discord voice API."
documentation = "https://docs.rs/songbird" documentation = "https://docs.rs/songbird"
edition = "2018" edition = "2021"
homepage = "https://github.com/serenity-rs/songbird" homepage = "https://github.com/serenity-rs/songbird"
include = ["src/**/*.rs", "Cargo.toml", "build.rs"] include = ["src/**/*.rs", "Cargo.toml", "build.rs"]
keywords = ["discord", "api", "rtp", "audio"] keywords = ["discord", "api", "rtp", "audio"]
@@ -10,15 +10,20 @@ license = "ISC"
name = "songbird" name = "songbird"
readme = "README.md" readme = "README.md"
repository = "https://github.com/serenity-rs/songbird.git" repository = "https://github.com/serenity-rs/songbird.git"
version = "0.3.0" version = "0.2.2"
rust-version = "1.61"
[dependencies] [dependencies]
derivative = "2" derivative = "2"
pin-project = "1"
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
tracing = { version = "0.1", features = ["log"] } tracing = { version = "0.1", features = ["log"] }
tracing-futures = "0.2" tracing-futures = "0.2"
symphonia-core = "0.5"
[dependencies.once_cell]
version = "1"
optional = true
[dependencies.async-trait] [dependencies.async-trait]
optional = true optional = true
@@ -45,9 +50,8 @@ version = "5"
[dependencies.discortp] [dependencies.discortp]
features = ["discord-full"] features = ["discord-full"]
optional = true optional = true
version = "0.4" version = "0.5"
# Temporary hack to pin MSRV.
[dependencies.flume] [dependencies.flume]
optional = true optional = true
version = "0.10" version = "0.10"
@@ -55,18 +59,41 @@ version = "0.10"
[dependencies.futures] [dependencies.futures]
version = "0.3" version = "0.3"
[dependencies.lazy_static]
optional = true
version = "1"
[dependencies.parking_lot] [dependencies.parking_lot]
optional = true optional = true
version = "0.12" version = "0.12"
[dependencies.pin-project]
optional = true
version = "1"
[dependencies.rand] [dependencies.rand]
optional = true optional = true
version = "0.8" version = "0.8"
[dependencies.reqwest]
optional = true
default-features = false
features = ["stream"]
version = "0.11"
[dependencies.ringbuf]
optional = true
version = "0.2"
[dependencies.rubato]
optional = true
version = "0.12"
[dependencies.rusty_pool]
optional = true
version = "0.7"
[dependencies.serde-aux]
default-features = false
optional = true
version = "3"
[dependencies.serenity] [dependencies.serenity]
optional = true optional = true
version = "0.11" version = "0.11"
@@ -81,11 +108,29 @@ version = "0.1"
optional = true optional = true
version = "1" version = "1"
[dependencies.symphonia]
optional = true
default-features = false
version = "0.5"
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
[dependencies.symphonia-core]
optional = true
version = "0.5"
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
[dependencies.tokio] [dependencies.tokio]
optional = true optional = true
version = "1.0" version = "1.0"
default-features = false default-features = false
[dependencies.tokio-util]
optional = true
version = "0.7"
features = ["io"]
[dependencies.twilight-gateway] [dependencies.twilight-gateway]
optional = true optional = true
version = "0.12.0" version = "0.12.0"
@@ -106,7 +151,7 @@ version = "2"
[dependencies.uuid] [dependencies.uuid]
optional = true optional = true
version = "0.8" version = "1"
features = ["v4"] features = ["v4"]
[dependencies.xsalsa20poly1305] [dependencies.xsalsa20poly1305]
@@ -116,7 +161,10 @@ features = ["std"]
[dev-dependencies] [dev-dependencies]
criterion = "0.3" criterion = "0.3"
ntest = "0.8"
symphonia = { version = "0.5", features = ["mp3"], git = "https://github.com/FelixMcFelix/Symphonia", branch = "songbird-fixes" }
utils = { path = "utils" } utils = { path = "utils" }
tokio = { version = "1", features = ["rt", "rt-multi-thread"] }
[features] [features]
# Core features # Core features
@@ -126,19 +174,33 @@ default = [
"gateway", "gateway",
] ]
gateway = [ gateway = [
"gateway-core", "dashmap",
"flume",
"once_cell",
"parking_lot",
"tokio/sync", "tokio/sync",
"tokio/time", "tokio/time",
] ]
gateway-core = [
"dashmap",
"flume",
"parking_lot",
"pin-project",
]
driver = [ driver = [
"async-trait",
"async-tungstenite", "async-tungstenite",
"driver-core", "audiopus",
"byteorder",
"discortp",
"reqwest",
"flume",
"lazy_static",
"parking_lot",
"rand",
"ringbuf",
"rubato",
"serde-aux",
"serenity-voice-model",
"streamcatcher",
"symphonia",
"symphonia-core",
"rusty_pool",
"tokio-util",
"tokio/fs", "tokio/fs",
"tokio/io-util", "tokio/io-util",
"tokio/macros", "tokio/macros",
@@ -147,24 +209,13 @@ driver = [
"tokio/rt", "tokio/rt",
"tokio/sync", "tokio/sync",
"tokio/time", "tokio/time",
]
driver-core = [
"async-trait",
"audiopus",
"byteorder",
"discortp",
"flume",
"parking_lot",
"rand",
"serenity-voice-model",
"streamcatcher",
"typemap_rev", "typemap_rev",
"url", "url",
"uuid", "uuid",
"xsalsa20poly1305", "xsalsa20poly1305",
] ]
rustls = ["async-tungstenite/tokio-rustls-webpki-roots", "rustls-marker"] rustls = ["async-tungstenite/tokio-rustls-webpki-roots", "reqwest/rustls-tls", "rustls-marker"]
native = ["async-tungstenite/tokio-native-tls", "native-marker"] native = ["async-tungstenite/tokio-native-tls", "native-marker", "reqwest/native-tls"]
serenity-rustls = ["serenity/rustls_backend", "rustls", "gateway", "serenity-deps"] serenity-rustls = ["serenity/rustls_backend", "rustls", "gateway", "serenity-deps"]
serenity-native = ["serenity/native_tls_backend", "native", "gateway", "serenity-deps"] serenity-native = ["serenity/native_tls_backend", "native", "gateway", "serenity-deps"]
twilight-rustls = ["twilight", "twilight-gateway/rustls-native-roots", "rustls", "gateway"] twilight-rustls = ["twilight", "twilight-gateway/rustls-native-roots", "rustls", "gateway"]
@@ -178,8 +229,6 @@ rustls-marker = []
native-marker = [] native-marker = []
# Behaviour altering features. # Behaviour altering features.
youtube-dlc = []
yt-dlp = []
builtin-queue = [] builtin-queue = []
# Used for docgen/testing/benchmarking. # Used for docgen/testing/benchmarking.
@@ -189,6 +238,7 @@ internals = []
[[bench]] [[bench]]
name = "base-mixing" name = "base-mixing"
path = "benches/base-mixing.rs" path = "benches/base-mixing.rs"
required-features = ["internals"]
harness = false harness = false
[[bench]] [[bench]]

4
Makefile.toml
View File

@@ -26,6 +26,10 @@ dependencies = ["format"]
[tasks.build-variants] [tasks.build-variants]
dependencies = ["build", "build-gateway", "build-driver"] dependencies = ["build", "build-gateway", "build-driver"]
[tasks.check]
args = ["check", "--features", "full-doc"]
dependencies = ["format"]
[tasks.clippy] [tasks.clippy]
args = ["clippy", "--features", "full-doc", "--", "-D", "warnings"] args = ["clippy", "--features", "full-doc", "--", "-D", "warnings"]
dependencies = ["format"] dependencies = ["format"]

50
README.md
View File

@@ -1,4 +1,4 @@
[![docs-badge][]][docs] [![build badge]][build] [![guild-badge][]][guild] [![crates.io version]][crates.io link] [![rust 1.60.0+ badge]][rust 1.60.0+ link] [![docs-badge][]][docs] [![build badge]][build] [![guild-badge][]][guild] [![crates.io version]][crates.io link] [![rust 1.61.0+ badge]][rust 1.61.0+ link]
# Songbird # Songbird
@@ -19,27 +19,48 @@ The library offers:
## Intents ## Intents
Songbird's gateway functionality requires you to specify the `GUILD_VOICE_STATES` intent. Songbird's gateway functionality requires you to specify the `GUILD_VOICE_STATES` intent.
## Codec support
Songbird supports all [codecs and formats provided by Symphonia] (pure-Rust), with Opus support
provided by [audiopus] (an FFI wrapper for libopus).
**By default, *Songbird will not request any codecs from Symphonia*.** To change this, in your own
project you will need to depend on Symphonia as well.
```toml
# Including songbird alone gives you support for Opus via the DCA file format.
[dependencies.songbird]
version = "0.4"
features = ["builtin-queue"]
# To get additional codecs, you *must* add Symphonia yourself.
# This includes the default formats (MKV/WebM, Ogg, Wave) and codecs (FLAC, PCM, Vorbis)...
[dependencies.symphonia]
# version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"] # ...as well as any extras you need!
# **NOTE**: For now, please use this fork in pre-releases for a key fix to
# seeking on streamed MKV/WebM files.
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
```
## Dependencies ## Dependencies
Songbird needs a few system dependencies before you can use it. Songbird needs a few system dependencies before you can use it.
- Opus - Audio codec that Discord uses. - Opus - Audio codec that Discord uses.
If you are on Windows and you are using the MSVC toolchain, a prebuilt DLL is provided for you, you do not have to do anything. [audiopus] will use installed libopus binaries if available via pkgconf on Linux/MacOS, otherwise you will need to install cmake to build opus from source.
On other platforms, you will have to install it. You can install the library with `apt install libopus-dev` on Ubuntu or `pacman -S opus` on Arch Linux. This is always the case on Windows.
For Unix systems, you can install the library with `apt install libopus-dev` on Ubuntu or `pacman -S opus` on Arch Linux.
If you do not have it installed it will be built for you. However, you will need a C compiler and the GNU autotools installed. If you do not have it installed it will be built for you. However, you will need a C compiler and the GNU autotools installed.
Again, these can be installed with `apt install build-essential autoconf automake libtool m4` on Ubuntu or `pacman -S base-devel` on Arch Linux. Again, these can be installed with `apt install build-essential autoconf automake libtool m4` on Ubuntu or `pacman -S base-devel` on Arch Linux.
This is a required dependency. Songbird cannot work without it. This is a required dependency. Songbird cannot work without it.
- FFmpeg - Audio/Video conversion tool. - yt-dlp / youtube-dl / (similar forks) - Audio/Video download tool.
You can install the tool with `apt install ffmpeg` on Ubuntu or `pacman -S ffmpeg` on Arch Linux. yt-dlp can be installed [according to the installation instructions on the main repo].
You can install youtube-dl with Python's package manager, pip, which we recommend for youtube-dl. You can do it with the command `pip install youtube_dl`.
This is an optional, but recommended dependency. It allows Songbird to convert from, for instance, .mp4 files to the audio format Discord uses.
- youtube-dl - Audio/Video download tool.
You can install the tool with Python's package manager, pip, which we recommend for youtube-dl. You can do it with the command `pip install youtube_dl`.
Alternatively, you can install it with your system's package manager, `apt install youtube-dl` on Ubuntu or `pacman -S youtube-dl` on Arch Linux. Alternatively, you can install it with your system's package manager, `apt install youtube-dl` on Ubuntu or `pacman -S youtube-dl` on Arch Linux.
This is an optional dependency. It allows Songbird to download an audio source from the Internet, which will be converted to the audio format Discord uses. This is an optional dependency for users, but is required as a dev-dependency. It allows Songbird to download audio/video sources from the Internet from a variety of webpages, which it will convert to the Opus audio format Discord uses.
## Examples ## Examples
Full examples showing various types of functionality and integrations can be found in [this crate's examples directory]. Full examples showing various types of functionality and integrations can be found in [this crate's examples directory].
@@ -56,6 +77,9 @@ Songbird's logo is based upon the copyright-free image ["Black-Capped Chickadee"
[lavalink]: https://github.com/freyacodes/Lavalink [lavalink]: https://github.com/freyacodes/Lavalink
[this crate's examples directory]: https://github.com/serenity-rs/songbird/tree/current/examples [this crate's examples directory]: https://github.com/serenity-rs/songbird/tree/current/examples
[our contributor guidelines]: CONTRIBUTING.md [our contributor guidelines]: CONTRIBUTING.md
[codecs and formats provided by Symphonia]: https://github.com/pdeljanov/Symphonia#formats-demuxers
[audiopus]: https://github.com/lakelezz/audiopus
[according to the installation instructions on the main repo]: https://github.com/yt-dlp/yt-dlp#installation
[build badge]: https://img.shields.io/github/workflow/status/serenity-rs/songbird/CI?style=flat-square [build badge]: https://img.shields.io/github/workflow/status/serenity-rs/songbird/CI?style=flat-square
[build]: https://github.com/serenity-rs/songbird/actions [build]: https://github.com/serenity-rs/songbird/actions
@@ -69,5 +93,5 @@ Songbird's logo is based upon the copyright-free image ["Black-Capped Chickadee"
[crates.io link]: https://crates.io/crates/songbird [crates.io link]: https://crates.io/crates/songbird
[crates.io version]: https://img.shields.io/crates/v/songbird.svg?style=flat-square [crates.io version]: https://img.shields.io/crates/v/songbird.svg?style=flat-square
[rust 1.60.0+ badge]: https://img.shields.io/badge/rust-1.60.0+-93450a.svg?style=flat-square [rust 1.61.0+ badge]: https://img.shields.io/badge/rust-1.61.0+-93450a.svg?style=flat-square
[rust 1.60.0+ link]: https://blog.rust-lang.org/2022/04/07/Rust-1.60.0.html [rust 1.61.0+ link]: https://blog.rust-lang.org/2022/05/19/Rust-1.61.0.html

115
benches/base-mixing.rs
View File

@@ -1,29 +1,102 @@
use criterion::{black_box, criterion_group, criterion_main, BatchSize, Criterion}; use criterion::{black_box, criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion};
use songbird::{constants::*, input::Input}; use songbird::{
constants::*,
driver::{
bench_internals::mixer::{mix_logic, state::DecodeState},
MixMode,
},
input::{codecs::*, Input, LiveInput, Parsed},
};
use std::io::Cursor;
use symphonia_core::audio::{AudioBuffer, Layout, SampleBuffer, Signal, SignalSpec};
pub fn mix_one_frame(c: &mut Criterion) { pub fn mix_one_frame(c: &mut Criterion) {
let floats = utils::make_sine(STEREO_FRAME_SIZE, true); let floats = utils::make_sine(1 * STEREO_FRAME_SIZE, true);
let mut raw_buf = [0f32; STEREO_FRAME_SIZE];
c.bench_function("Mix stereo source", |b| { let symph_layout = MixMode::Stereo.into();
b.iter_batched_ref(
|| black_box(Input::float_pcm(true, floats.clone().into())),
|input| {
input.mix(black_box(&mut raw_buf), black_box(1.0));
},
BatchSize::SmallInput,
)
});
c.bench_function("Mix mono source", |b| { let mut symph_mix = AudioBuffer::<f32>::new(
b.iter_batched_ref( MONO_FRAME_SIZE as u64,
|| black_box(Input::float_pcm(false, floats.clone().into())), symphonia_core::audio::SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, symph_layout),
|input| { );
input.mix(black_box(&mut raw_buf), black_box(1.0)); let mut resample_scratch = AudioBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, Layout::Stereo),
);
let mut group = c.benchmark_group("Stereo Target");
for (pres, hz) in [("", 48_000), (" (Resample)", 44_100)] {
group.bench_with_input(
BenchmarkId::new(format!("Stereo Source{}", pres), hz),
&hz,
|b, i| {
b.iter_batched_ref(
|| black_box(make_src(&floats, 2, *i)),
|(ref mut input, ref mut local_input)| {
symph_mix.clear();
symph_mix.render_reserved(Some(MONO_FRAME_SIZE));
resample_scratch.clear();
black_box(mix_logic::mix_symph_indiv(
&mut symph_mix,
&mut resample_scratch,
input,
local_input,
black_box(1.0),
None,
));
},
BatchSize::SmallInput,
)
}, },
BatchSize::SmallInput, );
)
}); group.bench_with_input(
BenchmarkId::new(format!("Mono Source{}", pres), hz),
&hz,
|b, i| {
b.iter_batched_ref(
|| black_box(make_src(&floats, 1, *i)),
|(ref mut input, ref mut local_input)| {
symph_mix.clear();
symph_mix.render_reserved(Some(MONO_FRAME_SIZE));
resample_scratch.clear();
black_box(mix_logic::mix_symph_indiv(
&mut symph_mix,
&mut resample_scratch,
input,
local_input,
black_box(1.0),
None,
));
},
BatchSize::SmallInput,
)
},
);
}
group.finish();
}
fn make_src(src: &Vec<u8>, chans: u32, hz: u32) -> (Parsed, DecodeState) {
let local_input = Default::default();
let adapted: Input =
songbird::input::RawAdapter::new(Cursor::new(src.clone()), hz, chans).into();
let promoted = match adapted {
Input::Live(l, _) => l.promote(&CODEC_REGISTRY, &PROBE),
_ => panic!("Failed to create a guaranteed source."),
};
let parsed = match promoted {
Ok(LiveInput::Parsed(parsed)) => parsed,
Err(e) => panic!("AR {:?}", e),
_ => panic!("Failed to create a guaranteed source."),
};
(parsed, local_input)
} }
criterion_group!(benches, mix_one_frame); criterion_group!(benches, mix_one_frame);

62
benches/mixing-task.rs
View File

@@ -1,3 +1,5 @@
use std::error::Error;
use criterion::{ use criterion::{
black_box, black_box,
criterion_group, criterion_group,
@@ -11,12 +13,18 @@ use flume::{Receiver, Sender, TryRecvError};
use songbird::{ use songbird::{
constants::*, constants::*,
driver::{ driver::{
bench_internals::{mixer::Mixer, task_message::*, CryptoState}, bench_internals::{
self,
mixer::{state::InputState, Mixer},
task_message::*,
CryptoState,
},
Bitrate, Bitrate,
}, },
input::{cached::Compressed, Input}, input::{cached::Compressed, codecs::*, Input, RawAdapter},
tracks, tracks,
}; };
use std::io::Cursor;
use tokio::runtime::{Handle, Runtime}; use tokio::runtime::{Handle, Runtime};
use xsalsa20poly1305::{aead::NewAead, XSalsa20Poly1305 as Cipher, KEY_SIZE}; use xsalsa20poly1305::{aead::NewAead, XSalsa20Poly1305 as Cipher, KEY_SIZE};
@@ -79,14 +87,17 @@ fn mixer_float(
let floats = utils::make_sine(10 * STEREO_FRAME_SIZE, true); let floats = utils::make_sine(10 * STEREO_FRAME_SIZE, true);
let mut tracks = vec![];
for i in 0..num_tracks { for i in 0..num_tracks {
let input = Input::float_pcm(true, floats.clone().into()); let input: Input = RawAdapter::new(Cursor::new(floats.clone()), 48_000, 2).into();
tracks.push(tracks::create_player(input).0.into()); let promoted = match input {
Input::Live(l, _) => l.promote(&CODEC_REGISTRY, &PROBE),
_ => panic!("Failed to create a guaranteed source."),
};
let (handle, mut ctx) =
bench_internals::track_context(Input::Live(promoted.unwrap(), None).into());
out.0.add_track(ctx);
} }
out.0.tracks = tracks;
out out
} }
@@ -104,15 +115,18 @@ fn mixer_float_drop(
) { ) {
let mut out = dummied_mixer(handle); let mut out = dummied_mixer(handle);
let mut tracks = vec![];
for i in 0..num_tracks { for i in 0..num_tracks {
let floats = utils::make_sine((i / 5) * STEREO_FRAME_SIZE, true); let floats = utils::make_sine((i / 5) * STEREO_FRAME_SIZE, true);
let input = Input::float_pcm(true, floats.clone().into()); let input: Input = RawAdapter::new(Cursor::new(floats.clone()), 48_000, 2).into();
tracks.push(tracks::create_player(input).0.into()); let promoted = match input {
Input::Live(l, _) => l.promote(&CODEC_REGISTRY, &PROBE),
_ => panic!("Failed to create a guaranteed source."),
};
let (handle, mut ctx) =
bench_internals::track_context(Input::Live(promoted.unwrap(), None).into());
out.0.add_track(ctx);
} }
out.0.tracks = tracks;
out out
} }
@@ -129,22 +143,28 @@ fn mixer_opus(
) { ) {
// should add a single opus-based track. // should add a single opus-based track.
// make this fully loaded to prevent any perf cost there. // make this fully loaded to prevent any perf cost there.
let mut out = dummied_mixer(handle); let mut out = dummied_mixer(handle.clone());
let floats = utils::make_sine(6 * STEREO_FRAME_SIZE, true); let floats = utils::make_sine(6 * STEREO_FRAME_SIZE, true);
let mut tracks = vec![]; let input: Input = RawAdapter::new(Cursor::new(floats), 48_000, 2).into();
let mut src = handle.block_on(async move {
Compressed::new(input, Bitrate::BitsPerSecond(128_000))
.await
.expect("These parameters are well-defined.")
});
let mut src = Compressed::new(
Input::float_pcm(true, floats.clone().into()),
Bitrate::BitsPerSecond(128_000),
)
.expect("These parameters are well-defined.");
src.raw.load_all(); src.raw.load_all();
tracks.push(tracks::create_player(src.into()).0.into()); let promoted = match src.into() {
Input::Live(l, _) => l.promote(&CODEC_REGISTRY, &PROBE),
_ => panic!("Failed to create a guaranteed source."),
};
let (handle, mut ctx) =
bench_internals::track_context(Input::Live(promoted.unwrap(), None).into());
out.0.tracks = tracks; out.0.add_track(ctx);
out out
} }

2
examples/Cargo.toml
View File

@@ -1,8 +1,8 @@
[workspace] [workspace]
members = [ members = [
"serenity/voice", "serenity/voice",
"serenity/voice_cached_audio",
"serenity/voice_events_queue", "serenity/voice_events_queue",
"serenity/voice_receive", "serenity/voice_receive",
"serenity/voice_storage",
"twilight", "twilight",
] ]

9
examples/serenity/voice/Cargo.toml
View File

@@ -2,12 +2,19 @@
name = "voice" name = "voice"
version = "0.1.0" version = "0.1.0"
authors = ["my name <my@email.address>"] authors = ["my name <my@email.address>"]
edition = "2018" edition = "2021"
[dependencies] [dependencies]
tracing = "0.1" tracing = "0.1"
tracing-subscriber = "0.2" tracing-subscriber = "0.2"
tracing-futures = "0.2" tracing-futures = "0.2"
reqwest = "0.11"
[dependencies.symphonia]
version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"]
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
[dependencies.songbird] [dependencies.songbird]
path = "../../../" path = "../../../"

417
examples/serenity/voice/src/main-skip.rs Normal file
View File

@@ -0,0 +1,417 @@
//! Requires the "client", "standard_framework", and "voice" features be enabled in your
//! Cargo.toml, like so:
//!
//! ```toml
//! [dependencies.serenity]
//! git = "https://github.com/serenity-rs/serenity.git"
//! features = ["client", "standard_framework", "voice"]
//! ```
use std::env;
// This trait adds the `register_songbird` and `register_songbird_with` methods
// to the client builder below, making it easy to install this voice client.
// The voice client can be retrieved in any command using `songbird::get(ctx).await`.
use songbird::SerenityInit;
// Event related imports to detect track creation failures.
use songbird::events::{Event, EventContext, EventHandler as VoiceEventHandler, TrackEvent};
use std::time::Duration;
// Import the `Context` to handle commands.
use serenity::client::Context;
use serenity::{
async_trait,
client::{Client, EventHandler},
framework::{
standard::{
macros::{command, group},
Args, CommandResult,
},
StandardFramework,
},
model::{channel::Message, gateway::Ready},
prelude::GatewayIntents,
Result as SerenityResult,
};
struct Handler;
#[async_trait]
impl EventHandler for Handler {
async fn ready(&self, _: Context, ready: Ready) {
println!("{} is connected!", ready.user.name);
}
}
#[group]
#[commands(deafen, join, leave, mute, play, ping, undeafen, unmute)]
struct General;
#[tokio::main]
async fn main() {
tracing_subscriber::fmt::init();
// Configure the client with your Discord bot token in the environment.
let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment");
let mut framework = StandardFramework::new()
.group(&GENERAL_GROUP);
framework.configure(|c| c.prefix("~"));
let intents = GatewayIntents::non_privileged() | GatewayIntents::MESSAGE_CONTENT;
let mut client = Client::builder(&token, intents)
.event_handler(Handler)
.framework(framework)
.register_songbird()
.await
.expect("Err creating client");
tokio::spawn(async move {
let _ = client
.start()
.await
.map_err(|why| println!("Client ended: {:?}", why));
});
tokio::signal::ctrl_c().await;
println!("Received Ctrl-C, shutting down.");
}
#[command]
#[only_in(guilds)]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) {
Some(handler) => handler,
None => {
check_msg(msg.reply(ctx, "Not in a voice channel").await);
return Ok(());
},
};
let mut handler = handler_lock.lock().await;
if handler.is_deaf() {
check_msg(msg.channel_id.say(&ctx.http, "Already deafened").await);
} else {
if let Err(e) = handler.deafen(true).await {
check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
}
check_msg(msg.channel_id.say(&ctx.http, "Deafened").await);
}
Ok(())
}
#[command]
#[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let channel_id = guild
.voice_states
.get(&msg.author.id)
.and_then(|voice_state| voice_state.channel_id);
let connect_to = match channel_id {
Some(channel) => channel,
None => {
check_msg(msg.reply(ctx, "Not in a voice channel").await);
return Ok(());
},
};
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let (handler_lock, _success) = manager.join(guild_id, connect_to).await;
// Attach an event handler to see notifications of all track errors.
let mut handler = handler_lock.lock().await;
handler.add_global_event(TrackEvent::Error.into(), TrackErrorNotifier);
Ok(())
}
struct TrackErrorNotifier;
#[async_trait]
impl VoiceEventHandler for TrackErrorNotifier {
async fn act(&self, ctx: &EventContext<'_>) -> Option<Event> {
if let EventContext::Track(track_list) = ctx {
for (state, handle) in *track_list {
println!(
"Track {:?} encountered an error: {:?}",
handle.uuid(),
state.playing
);
}
}
None
}
}
#[command]
#[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let has_handler = manager.get(guild_id).is_some();
if has_handler {
if let Err(e) = manager.remove(guild_id).await {
check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
}
check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await);
} else {
check_msg(msg.reply(ctx, "Not in a voice channel").await);
}
Ok(())
}
#[command]
#[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) {
Some(handler) => handler,
None => {
check_msg(msg.reply(ctx, "Not in a voice channel").await);
return Ok(());
},
};
let mut handler = handler_lock.lock().await;
if handler.is_mute() {
check_msg(msg.channel_id.say(&ctx.http, "Already muted").await);
} else {
if let Err(e) = handler.mute(true).await {
check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
}
check_msg(msg.channel_id.say(&ctx.http, "Now muted").await);
}
Ok(())
}
#[command]
async fn ping(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http, "Pong!").await);
Ok(())
}
#[command]
#[only_in(guilds)]
async fn play(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
let url = match args.single::<String>() {
Ok(url) => url,
Err(_) => {
check_msg(
msg.channel_id
.say(&ctx.http, "Must provide a URL to a video or audio")
.await,
);
return Ok(());
},
};
let time = match args.single::<f64>() {
Ok(t) => t,
Err(_) => {
check_msg(
msg.channel_id
.say(&ctx.http, "Must provide a valid f64 timestamp to jump to.")
.await,
);
return Ok(());
},
};
if !url.starts_with("http") {
check_msg(
msg.channel_id
.say(&ctx.http, "Must provide a valid URL")
.await,
);
return Ok(());
}
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await;
let jar = reqwest::cookie::Jar::default();
jar.add_cookie_str("CONSENT=YES+; Path=/; Domain=youtube.com; Secure; Expires=Fri, 01 Jan 2038 00:00:00 GMT;", &"https://youtube.com".parse().unwrap());
let src = songbird::input::YoutubeDl::new_ytdl_like("yt-dlp", reqwest::Client::builder().cookie_provider(std::sync::Arc::new(jar)).build().unwrap(), url);
let h = handler.play_input(src.into());
h.add_event(
Event::Delayed(Duration::from_secs(10)),
SkipHandler {
skip_to: Duration::from_secs_f64(time),
},
);
check_msg(msg.channel_id.say(&ctx.http, "Playing song").await);
} else {
check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to play in")
.await,
);
}
Ok(())
}
struct SkipHandler {
skip_to: Duration,
}
#[async_trait]
impl VoiceEventHandler for SkipHandler {
async fn act(&self, ctx: &EventContext<'_>) -> Option<Event> {
if let EventContext::Track(&[(state, handle)]) = ctx {
println!("Current position is {:?}", state.position);
println!("Seeking to {:?}", self.skip_to);
let resp = handle.seek_time(self.skip_to);
println!("Seek response {:?}", resp);
return None;
}
println!("This wasn't supposed to happen.");
None
}
}
#[command]
#[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await;
if let Err(e) = handler.deafen(false).await {
check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
}
check_msg(msg.channel_id.say(&ctx.http, "Undeafened").await);
} else {
check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to undeafen in")
.await,
);
}
Ok(())
}
#[command]
#[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await;
if let Err(e) = handler.mute(false).await {
check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
}
check_msg(msg.channel_id.say(&ctx.http, "Unmuted").await);
} else {
check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to unmute in")
.await,
);
}
Ok(())
}
/// Checks that a message successfully sent; if not, then logs why to stdout.
fn check_msg(result: SerenityResult<Message>) {
if let Err(why) = result {
println!("Error sending message: {:?}", why);
}
}

230
examples/serenity/voice/src/main.rs
View File

@@ -13,6 +13,15 @@ use std::env;
// The voice client can be retrieved in any command using `songbird::get(ctx).await`. // The voice client can be retrieved in any command using `songbird::get(ctx).await`.
use songbird::SerenityInit; use songbird::SerenityInit;
// Event related imports to detect track creation failures.
use songbird::events::{Event, EventContext, EventHandler as VoiceEventHandler, TrackEvent};
// To turn user URLs into playable audio, we'll use yt-dlp.
use songbird::input::YoutubeDl;
// YtDl requests need an HTTP client to operate -- we'll create and store our own.
use reqwest::Client as HttpClient;
// Import the `Context` to handle commands. // Import the `Context` to handle commands.
use serenity::client::Context; use serenity::client::Context;
@@ -20,17 +29,24 @@ use serenity::{
async_trait, async_trait,
client::{Client, EventHandler}, client::{Client, EventHandler},
framework::{ framework::{
StandardFramework,
standard::{ standard::{
Args, CommandResult,
macros::{command, group}, macros::{command, group},
Args,
CommandResult,
}, },
StandardFramework,
}, },
model::{channel::Message, gateway::Ready}, model::{channel::Message, gateway::Ready},
prelude::GatewayIntents, prelude::{GatewayIntents, TypeMapKey},
Result as SerenityResult, Result as SerenityResult,
}; };
struct HttpKey;
impl TypeMapKey for HttpKey {
type Value = HttpClient;
}
struct Handler; struct Handler;
#[async_trait] #[async_trait]
@@ -49,40 +65,47 @@ async fn main() {
tracing_subscriber::fmt::init(); tracing_subscriber::fmt::init();
// Configure the client with your Discord bot token in the environment. // Configure the client with your Discord bot token in the environment.
let token = env::var("DISCORD_TOKEN") let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment");
.expect("Expected a token in the environment");
let framework = StandardFramework::new() let framework = StandardFramework::new()
.configure(|c| c
.prefix("~"))
.group(&GENERAL_GROUP); .group(&GENERAL_GROUP);
framework.configure(|c| c.prefix("~"));
let intents = GatewayIntents::non_privileged() let intents = GatewayIntents::non_privileged() | GatewayIntents::MESSAGE_CONTENT;
| GatewayIntents::MESSAGE_CONTENT;
let mut client = Client::builder(&token, intents) let mut client = Client::builder(&token, intents)
.event_handler(Handler) .event_handler(Handler)
.framework(framework) .framework(framework)
.register_songbird() .register_songbird()
// We insert our own HTTP client here to make use of in
// `~play`. If we wanted, we could supply cookies and auth
// details ahead of time.
//
// Generally, we don't want to make a new Client for every request!
.type_map_insert::<HttpKey>(HttpClient::new())
.await .await
.expect("Err creating client"); .expect("Err creating client");
tokio::spawn(async move { tokio::spawn(async move {
let _ = client.start().await.map_err(|why| println!("Client ended: {:?}", why)); let _ = client
.start()
.await
.map_err(|why| println!("Client ended: {:?}", why));
}); });
tokio::signal::ctrl_c().await; let _signal_err = tokio::signal::ctrl_c().await;
println!("Received Ctrl-C, shutting down."); println!("Received Ctrl-C, shutting down.");
} }
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult { async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) { let handler_lock = match manager.get(guild_id) {
Some(handler) => handler, Some(handler) => handler,
@@ -99,7 +122,11 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http, "Already deafened").await); check_msg(msg.channel_id.say(&ctx.http, "Already deafened").await);
} else { } else {
if let Err(e) = handler.deafen(true).await { if let Err(e) = handler.deafen(true).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Deafened").await); check_msg(msg.channel_id.say(&ctx.http, "Deafened").await);
@@ -111,12 +138,15 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult { async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let (guild_id, channel_id) = {
let guild_id = guild.id; let guild = msg.guild(&ctx.cache).unwrap();
let channel_id = guild
.voice_states
.get(&msg.author.id)
.and_then(|voice_state| voice_state.channel_id);
let channel_id = guild (guild.id, channel_id)
.voice_states.get(&msg.author.id) };
.and_then(|voice_state| voice_state.channel_id);
let connect_to = match channel_id { let connect_to = match channel_id {
Some(channel) => channel, Some(channel) => channel,
@@ -124,30 +154,60 @@ async fn join(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.reply(ctx, "Not in a voice channel").await); check_msg(msg.reply(ctx, "Not in a voice channel").await);
return Ok(()); return Ok(());
} },
}; };
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let _handler = manager.join(guild_id, connect_to).await; let (handler_lock, _success) = manager.join(guild_id, connect_to).await;
// Attach an event handler to see notifications of all track errors.
let mut handler = handler_lock.lock().await;
handler.add_global_event(TrackEvent::Error.into(), TrackErrorNotifier);
Ok(()) Ok(())
} }
struct TrackErrorNotifier;
#[async_trait]
impl VoiceEventHandler for TrackErrorNotifier {
async fn act(&self, ctx: &EventContext<'_>) -> Option<Event> {
if let EventContext::Track(track_list) = ctx {
for (state, handle) in *track_list {
println!(
"Track {:?} encountered an error: {:?}",
handle.uuid(),
state.playing
);
}
}
None
}
}
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult { async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let has_handler = manager.get(guild_id).is_some(); let has_handler = manager.get(guild_id).is_some();
if has_handler { if has_handler {
if let Err(e) = manager.remove(guild_id).await { if let Err(e) = manager.remove(guild_id).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await); check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await);
@@ -161,11 +221,12 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult { async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) { let handler_lock = match manager.get(guild_id) {
Some(handler) => handler, Some(handler) => handler,
@@ -182,7 +243,11 @@ async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http, "Already muted").await); check_msg(msg.channel_id.say(&ctx.http, "Already muted").await);
} else { } else {
if let Err(e) = handler.mute(true).await { if let Err(e) = handler.mute(true).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Now muted").await); check_msg(msg.channel_id.say(&ctx.http, "Now muted").await);
@@ -192,9 +257,8 @@ async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
} }
#[command] #[command]
async fn ping(context: &Context, msg: &Message) -> CommandResult { async fn ping(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&context.http, "Pong!").await); check_msg(msg.channel_id.say(&ctx.http, "Pong!").await);
Ok(()) Ok(())
} }
@@ -204,43 +268,53 @@ async fn play(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
let url = match args.single::<String>() { let url = match args.single::<String>() {
Ok(url) => url, Ok(url) => url,
Err(_) => { Err(_) => {
check_msg(msg.channel_id.say(&ctx.http, "Must provide a URL to a video or audio").await); check_msg(
msg.channel_id
.say(&ctx.http, "Must provide a URL to a video or audio")
.await,
);
return Ok(()); return Ok(());
}, },
}; };
if !url.starts_with("http") { if !url.starts_with("http") {
check_msg(msg.channel_id.say(&ctx.http, "Must provide a valid URL").await); check_msg(
msg.channel_id
.say(&ctx.http, "Must provide a valid URL")
.await,
);
return Ok(()); return Ok(());
} }
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let http_client = {
.expect("Songbird Voice client placed in at initialisation.").clone(); let data = ctx.data.read().await;
data.get::<HttpKey>()
.cloned()
.expect("Guaranteed to exist in the typemap.")
};
let manager = songbird::get(ctx)
.await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
let source = match songbird::ytdl(&url).await { let src = YoutubeDl::new(http_client, url);
Ok(source) => source, let _ = handler.play_input(src.into());
Err(why) => {
println!("Err starting source: {:?}", why);
check_msg(msg.channel_id.say(&ctx.http, "Error sourcing ffmpeg").await);
return Ok(());
},
};
handler.play_source(source);
check_msg(msg.channel_id.say(&ctx.http, "Playing song").await); check_msg(msg.channel_id.say(&ctx.http, "Playing song").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to play in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to play in")
.await,
);
} }
Ok(()) Ok(())
@@ -249,21 +323,30 @@ async fn play(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult { async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
if let Err(e) = handler.deafen(false).await { if let Err(e) = handler.deafen(false).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Undeafened").await); check_msg(msg.channel_id.say(&ctx.http, "Undeafened").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to undeafen in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to undeafen in")
.await,
);
} }
Ok(()) Ok(())
@@ -272,21 +355,30 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult { async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
if let Err(e) = handler.mute(false).await { if let Err(e) = handler.mute(false).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Unmuted").await); check_msg(msg.channel_id.say(&ctx.http, "Unmuted").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to unmute in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to unmute in")
.await,
);
} }
Ok(()) Ok(())

1
examples/serenity/voice_cached_audio/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
*.dca

10
examples/serenity/voice_storage/Cargo.toml → examples/serenity/voice_cached_audio/Cargo.toml
View File

@@ -1,8 +1,8 @@
[package] [package]
name = "voice_storage" name = "voice_cached_audio"
version = "0.1.0" version = "0.1.0"
authors = ["my name <my@email.address>"] authors = ["my name <my@email.address>"]
edition = "2018" edition = "2021"
[dependencies] [dependencies]
tracing = "0.1" tracing = "0.1"
@@ -16,6 +16,12 @@ path = "../../../"
version = "0.11" version = "0.11"
features = ["cache", "framework", "standard_framework", "voice", "http", "rustls_backend"] features = ["cache", "framework", "standard_framework", "voice", "http", "rustls_backend"]
[dependencies.symphonia]
version = "0.5"
features = ["mp3"]
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
[dependencies.tokio] [dependencies.tokio]
version = "1.0" version = "1.0"
features = ["macros", "rt-multi-thread"] features = ["macros", "rt-multi-thread"]

235
examples/serenity/voice_storage/src/main.rs → examples/serenity/voice_cached_audio/src/main.rs
View File

@@ -9,17 +9,23 @@
//! git = "https://github.com/serenity-rs/serenity.git" //! git = "https://github.com/serenity-rs/serenity.git"
//! features = ["cache", "framework", "standard_framework", "voice"] //! features = ["cache", "framework", "standard_framework", "voice"]
//! ``` //! ```
use std::{collections::HashMap, convert::TryInto, env, sync::{Arc, Weak}}; use std::{
collections::HashMap,
convert::TryInto,
env,
sync::{Arc, Weak},
};
use serenity::{ use serenity::{
async_trait, async_trait,
client::{Client, Context, EventHandler}, client::{Client, Context, EventHandler},
framework::{ framework::{
StandardFramework,
standard::{ standard::{
Args, CommandResult,
macros::{command, group}, macros::{command, group},
Args,
CommandResult,
}, },
StandardFramework,
}, },
model::{channel::Message, gateway::Ready}, model::{channel::Message, gateway::Ready},
prelude::{GatewayIntents, Mentionable, Mutex}, prelude::{GatewayIntents, Mentionable, Mutex},
@@ -29,8 +35,8 @@ use serenity::{
use songbird::{ use songbird::{
driver::Bitrate, driver::Bitrate,
input::{ input::{
self,
cached::{Compressed, Memory}, cached::{Compressed, Memory},
File,
Input, Input,
}, },
Call, Call,
@@ -62,9 +68,9 @@ impl From<&CachedSound> for Input {
fn from(obj: &CachedSound) -> Self { fn from(obj: &CachedSound) -> Self {
use CachedSound::*; use CachedSound::*;
match obj { match obj {
Compressed(c) => c.new_handle() Compressed(c) => c.new_handle().into(),
.into(), Uncompressed(u) => u
Uncompressed(u) => u.new_handle() .new_handle()
.try_into() .try_into()
.expect("Failed to create decoder for Memory source."), .expect("Failed to create decoder for Memory source."),
} }
@@ -86,16 +92,13 @@ async fn main() {
tracing_subscriber::fmt::init(); tracing_subscriber::fmt::init();
// Configure the client with your Discord bot token in the environment. // Configure the client with your Discord bot token in the environment.
let token = env::var("DISCORD_TOKEN") let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment");
.expect("Expected a token in the environment");
let framework = StandardFramework::new() let framework = StandardFramework::new()
.configure(|c| c
.prefix("~"))
.group(&GENERAL_GROUP); .group(&GENERAL_GROUP);
framework.configure(|c| c.prefix("~"));
let intents = GatewayIntents::non_privileged() let intents = GatewayIntents::non_privileged() | GatewayIntents::MESSAGE_CONTENT;
| GatewayIntents::MESSAGE_CONTENT;
let mut client = Client::builder(&token, intents) let mut client = Client::builder(&token, intents)
.event_handler(Handler) .event_handler(Handler)
@@ -120,16 +123,16 @@ async fn main() {
// `spawn_loader` creates a new thread which works to copy all the audio into memory // `spawn_loader` creates a new thread which works to copy all the audio into memory
// ahead of time. We do this in both cases to ensure optimal performance for the audio // ahead of time. We do this in both cases to ensure optimal performance for the audio
// core. // core.
let ting_src = Memory::new( let ting_src = Memory::new(File::new("../../../resources/ting.wav").into())
input::ffmpeg("ting.wav").await.expect("File should be in root folder."), .await
).expect("These parameters are well-defined."); .expect("These parameters are well-defined.");
let _ = ting_src.raw.spawn_loader(); let _ = ting_src.raw.spawn_loader();
audio_map.insert("ting".into(), CachedSound::Uncompressed(ting_src)); audio_map.insert("ting".into(), CachedSound::Uncompressed(ting_src));
// Another short sting, to show where each loop occurs. // Another short sting, to show where each loop occurs.
let loop_src = Memory::new( let loop_src = Memory::new(File::new("../../../resources/loop.wav").into())
input::ffmpeg("loop.wav").await.expect("File should be in root folder."), .await
).expect("These parameters are well-defined."); .expect("These parameters are well-defined.");
let _ = loop_src.raw.spawn_loader(); let _ = loop_src.raw.spawn_loader();
audio_map.insert("loop".into(), CachedSound::Uncompressed(loop_src)); audio_map.insert("loop".into(), CachedSound::Uncompressed(loop_src));
@@ -139,26 +142,42 @@ async fn main() {
// //
// Music by Cloudkicker, used under CC BY-SC-SA 3.0 (https://creativecommons.org/licenses/by-nc-sa/3.0/). // Music by Cloudkicker, used under CC BY-SC-SA 3.0 (https://creativecommons.org/licenses/by-nc-sa/3.0/).
let song_src = Compressed::new( let song_src = Compressed::new(
input::ffmpeg("Cloudkicker_-_Loops_-_22_2011_07.mp3").await.expect("Link may be dead."), File::new("../../../resources/Cloudkicker - 2011 07.mp3").into(),
Bitrate::BitsPerSecond(128_000), Bitrate::BitsPerSecond(128_000),
).expect("These parameters are well-defined."); )
.await
.expect("These parameters are well-defined.");
let _ = song_src.raw.spawn_loader(); let _ = song_src.raw.spawn_loader();
// Compressed sources are internally stored as DCA1 format files.
// Because `Compressed` implements `std::io::Read`, we can save these
// to disk and use them again later if we want!
let mut creator = song_src.new_handle();
std::thread::spawn(move || {
let mut out_file = std::fs::File::create("ckick-dca1.dca").unwrap();
std::io::copy(&mut creator, &mut out_file).expect("Error writing out song!");
});
audio_map.insert("song".into(), CachedSound::Compressed(song_src)); audio_map.insert("song".into(), CachedSound::Compressed(song_src));
data.insert::<SoundStore>(Arc::new(Mutex::new(audio_map))); data.insert::<SoundStore>(Arc::new(Mutex::new(audio_map)));
} }
let _ = client.start().await.map_err(|why| println!("Client ended: {:?}", why)); let _ = client
.start()
.await
.map_err(|why| println!("Client ended: {:?}", why));
} }
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult { async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) { let handler_lock = match manager.get(guild_id) {
Some(handler) => handler, Some(handler) => handler,
@@ -175,7 +194,11 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http, "Already deafened").await); check_msg(msg.channel_id.say(&ctx.http, "Already deafened").await);
} else { } else {
if let Err(e) = handler.deafen(true).await { if let Err(e) = handler.deafen(true).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Deafened").await); check_msg(msg.channel_id.say(&ctx.http, "Deafened").await);
@@ -187,13 +210,15 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult { async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let (guild_id, channel_id) = {
let guild_id = guild.id; let guild = msg.guild(&ctx.cache).unwrap();
let channel_id = guild
let channel_id = guild .voice_states
.voice_states.get(&msg.author.id) .get(&msg.author.id)
.and_then(|voice_state| voice_state.channel_id); .and_then(|voice_state| voice_state.channel_id);
(guild.id, channel_id)
};
let connect_to = match channel_id { let connect_to = match channel_id {
Some(channel) => channel, Some(channel) => channel,
@@ -201,11 +226,13 @@ async fn join(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.reply(ctx, "Not in a voice channel").await); check_msg(msg.reply(ctx, "Not in a voice channel").await);
return Ok(()); return Ok(());
} },
}; };
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let (handler_lock, success_reader) = manager.join(guild_id, connect_to).await; let (handler_lock, success_reader) = manager.join(guild_id, connect_to).await;
@@ -213,14 +240,26 @@ async fn join(ctx: &Context, msg: &Message) -> CommandResult {
if let Ok(_reader) = success_reader { if let Ok(_reader) = success_reader {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
check_msg(msg.channel_id.say(&ctx.http, &format!("Joined {}", connect_to.mention())).await); check_msg(
msg.channel_id
.say(&ctx.http, &format!("Joined {}", connect_to.mention()))
.await,
);
let sources_lock = ctx.data.read().await.get::<SoundStore>().cloned().expect("Sound cache was installed at startup."); let sources_lock = ctx
.data
.read()
.await
.get::<SoundStore>()
.cloned()
.expect("Sound cache was installed at startup.");
let sources_lock_for_evt = sources_lock.clone(); let sources_lock_for_evt = sources_lock.clone();
let sources = sources_lock.lock().await; let sources = sources_lock.lock().await;
let source = sources.get("song").expect("Handle placed into cache at startup."); let source = sources
.get("song")
.expect("Handle placed into cache at startup.");
let song = handler.play_source(source.into()); let song = handler.play_input(source.into());
let _ = song.set_volume(1.0); let _ = song.set_volume(1.0);
let _ = song.enable_loop(); let _ = song.enable_loop();
@@ -233,7 +272,11 @@ async fn join(ctx: &Context, msg: &Message) -> CommandResult {
}, },
); );
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Error joining the channel").await); check_msg(
msg.channel_id
.say(&ctx.http, "Error joining the channel")
.await,
);
} }
Ok(()) Ok(())
@@ -250,11 +293,14 @@ impl VoiceEventHandler for LoopPlaySound {
if let Some(call_lock) = self.call_lock.upgrade() { if let Some(call_lock) = self.call_lock.upgrade() {
let src = { let src = {
let sources = self.sources.lock().await; let sources = self.sources.lock().await;
sources.get("loop").expect("Handle placed into cache at startup.").into() sources
.get("loop")
.expect("Handle placed into cache at startup.")
.into()
}; };
let mut handler = call_lock.lock().await; let mut handler = call_lock.lock().await;
let sound = handler.play_source(src); let sound = handler.play_input(src);
let _ = sound.set_volume(0.5); let _ = sound.set_volume(0.5);
} }
@@ -265,16 +311,21 @@ impl VoiceEventHandler for LoopPlaySound {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult { async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let has_handler = manager.get(guild_id).is_some(); let has_handler = manager.get(guild_id).is_some();
if has_handler { if has_handler {
if let Err(e) = manager.remove(guild_id).await { if let Err(e) = manager.remove(guild_id).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await); check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await);
@@ -288,11 +339,12 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult { async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let handler_lock = match manager.get(guild_id) { let handler_lock = match manager.get(guild_id) {
Some(handler) => handler, Some(handler) => handler,
@@ -309,7 +361,11 @@ async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http, "Already muted").await); check_msg(msg.channel_id.say(&ctx.http, "Already muted").await);
} else { } else {
if let Err(e) = handler.mute(true).await { if let Err(e) = handler.mute(true).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Now muted").await); check_msg(msg.channel_id.say(&ctx.http, "Now muted").await);
@@ -321,24 +377,37 @@ async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn ting(ctx: &Context, msg: &Message, _args: Args) -> CommandResult { async fn ting(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
let sources_lock = ctx.data.read().await.get::<SoundStore>().cloned().expect("Sound cache was installed at startup."); let sources_lock = ctx
.data
.read()
.await
.get::<SoundStore>()
.cloned()
.expect("Sound cache was installed at startup.");
let sources = sources_lock.lock().await; let sources = sources_lock.lock().await;
let source = sources.get("ting").expect("Handle placed into cache at startup."); let source = sources
.get("ting")
.expect("Handle placed into cache at startup.");
let _sound = handler.play_source(source.into()); let _sound = handler.play_input(source.into());
check_msg(msg.channel_id.say(&ctx.http, "Ting!").await); check_msg(msg.channel_id.say(&ctx.http, "Ting!").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to play in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to play in")
.await,
);
} }
Ok(()) Ok(())
@@ -347,22 +416,31 @@ async fn ting(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult { async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
if let Err(e) = handler.deafen(false).await { if let Err(e) = handler.deafen(false).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Undeafened").await); check_msg(msg.channel_id.say(&ctx.http, "Undeafened").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to undeafen in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to undeafen in")
.await,
);
} }
Ok(()) Ok(())
@@ -371,21 +449,30 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult { async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id; let manager = songbird::get(ctx)
let manager = songbird::get(ctx).await .await
.expect("Songbird Voice client placed in at initialisation.").clone(); .expect("Songbird Voice client placed in at initialisation.")
.clone();
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
if let Err(e) = handler.mute(false).await { if let Err(e) = handler.mute(false).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http, "Unmuted").await); check_msg(msg.channel_id.say(&ctx.http, "Unmuted").await);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Not in a voice channel to unmute in").await); check_msg(
msg.channel_id
.say(&ctx.http, "Not in a voice channel to unmute in")
.await,
);
} }
Ok(()) Ok(())

11
examples/serenity/voice_events_queue/Cargo.toml
View File

@@ -2,9 +2,10 @@
name = "voice_events_queue" name = "voice_events_queue"
version = "0.1.0" version = "0.1.0"
authors = ["my name <my@email.address>"] authors = ["my name <my@email.address>"]
edition = "2018" edition = "2021"
[dependencies] [dependencies]
reqwest = "0.11"
tracing = "0.1" tracing = "0.1"
tracing-subscriber = "0.2" tracing-subscriber = "0.2"
tracing-futures = "0.2" tracing-futures = "0.2"
@@ -17,6 +18,12 @@ path = "../../../"
version = "0.11" version = "0.11"
features = ["cache", "standard_framework", "voice", "rustls_backend"] features = ["cache", "standard_framework", "voice", "rustls_backend"]
[dependencies.symphonia]
version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"]
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"
[dependencies.tokio] [dependencies.tokio]
version = "1.0" version = "1.0"
features = ["macros", "rt-multi-thread"] features = ["macros", "rt-multi-thread", "signal"]

111
examples/serenity/voice_events_queue/src/main.rs
View File

@@ -18,6 +18,8 @@ use std::{
time::Duration, time::Duration,
}; };
use reqwest::Client as HttpClient;
use serenity::{ use serenity::{
async_trait, async_trait,
client::{Client, Context, EventHandler}, client::{Client, Context, EventHandler},
@@ -31,15 +33,12 @@ use serenity::{
}, },
http::Http, http::Http,
model::{channel::Message, gateway::Ready, prelude::ChannelId}, model::{channel::Message, gateway::Ready, prelude::ChannelId},
prelude::{GatewayIntents, Mentionable}, prelude::{GatewayIntents, Mentionable, TypeMapKey},
Result as SerenityResult, Result as SerenityResult,
}; };
use songbird::{ use songbird::{
input::{ input::YoutubeDl,
self,
restartable::Restartable,
},
Event, Event,
EventContext, EventContext,
EventHandler as VoiceEventHandler, EventHandler as VoiceEventHandler,
@@ -47,6 +46,12 @@ use songbird::{
TrackEvent, TrackEvent,
}; };
struct HttpKey;
impl TypeMapKey for HttpKey {
type Value = HttpClient;
}
struct Handler; struct Handler;
#[async_trait] #[async_trait]
@@ -70,16 +75,16 @@ async fn main() {
let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment"); let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment");
let framework = StandardFramework::new() let framework = StandardFramework::new()
.configure(|c| c.prefix("~"))
.group(&GENERAL_GROUP); .group(&GENERAL_GROUP);
framework.configure(|c| c.prefix("~"));
let intents = GatewayIntents::non_privileged() let intents = GatewayIntents::non_privileged() | GatewayIntents::MESSAGE_CONTENT;
| GatewayIntents::MESSAGE_CONTENT;
let mut client = Client::builder(&token, intents) let mut client = Client::builder(&token, intents)
.event_handler(Handler) .event_handler(Handler)
.framework(framework) .framework(framework)
.register_songbird() .register_songbird()
.type_map_insert::<HttpKey>(HttpClient::new())
.await .await
.expect("Err creating client"); .expect("Err creating client");
@@ -87,12 +92,28 @@ async fn main() {
.start() .start()
.await .await
.map_err(|why| println!("Client ended: {:?}", why)); .map_err(|why| println!("Client ended: {:?}", why));
tokio::spawn(async move {
let _ = client
.start()
.await
.map_err(|why| println!("Client ended: {:?}", why));
});
let _signal_err = tokio::signal::ctrl_c().await;
println!("Received Ctrl-C, shutting down.");
}
async fn get_http_client(ctx: &Context) -> HttpClient {
let data = ctx.data.read().await;
data.get::<HttpKey>()
.cloned()
.expect("Guaranteed to exist in the typemap.")
} }
#[command] #[command]
async fn deafen(ctx: &Context, msg: &Message) -> CommandResult { async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild(&ctx.cache).unwrap().id;
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -130,13 +151,15 @@ async fn deafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message) -> CommandResult { async fn join(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let (guild_id, channel_id) = {
let guild_id = guild.id; let guild = msg.guild(&ctx.cache).unwrap();
let channel_id = guild
.voice_states
.get(&msg.author.id)
.and_then(|voice_state| voice_state.channel_id);
let channel_id = guild (guild.id, channel_id)
.voice_states };
.get(&msg.author.id)
.and_then(|voice_state| voice_state.channel_id);
let connect_to = match channel_id { let connect_to = match channel_id {
Some(channel) => channel, Some(channel) => channel,
@@ -245,8 +268,7 @@ impl VoiceEventHandler for ChannelDurationNotifier {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult { async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild(&ctx.cache).unwrap().id;
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -274,8 +296,7 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn mute(ctx: &Context, msg: &Message) -> CommandResult { async fn mute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild(&ctx.cache).unwrap().id;
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -343,8 +364,9 @@ async fn play_fade(ctx: &Context, msg: &Message, mut args: Args) -> CommandResul
return Ok(()); return Ok(());
} }
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let http_client = get_http_client(ctx).await;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -354,20 +376,11 @@ async fn play_fade(ctx: &Context, msg: &Message, mut args: Args) -> CommandResul
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
let source = match input::ytdl(&url).await { let src = YoutubeDl::new(http_client, url);
Ok(source) => source,
Err(why) => {
println!("Err starting source: {:?}", why);
check_msg(msg.channel_id.say(&ctx.http, "Error sourcing ffmpeg").await);
return Ok(());
},
};
// This handler object will allow you to, as needed, // This handler object will allow you to, as needed,
// control the audio track via events and further commands. // control the audio track via events and further commands.
let song = handler.play_source(source); let song = handler.play_input(src.into());
let send_http = ctx.http.clone(); let send_http = ctx.http.clone();
let chan_id = msg.channel_id; let chan_id = msg.channel_id;
@@ -474,8 +487,9 @@ async fn queue(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
return Ok(()); return Ok(());
} }
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let http_client = get_http_client(ctx).await;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -487,18 +501,9 @@ async fn queue(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
// Here, we use lazy restartable sources to make sure that we don't pay // Here, we use lazy restartable sources to make sure that we don't pay
// for decoding, playback on tracks which aren't actually live yet. // for decoding, playback on tracks which aren't actually live yet.
let source = match Restartable::ytdl(url, true).await { let src = YoutubeDl::new(http_client, url);
Ok(source) => source,
Err(why) => {
println!("Err starting source: {:?}", why);
check_msg(msg.channel_id.say(&ctx.http, "Error sourcing ffmpeg").await); handler.enqueue_input(src.into()).await;
return Ok(());
},
};
handler.enqueue_source(source.into());
check_msg( check_msg(
msg.channel_id msg.channel_id
@@ -522,8 +527,7 @@ async fn queue(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn skip(ctx: &Context, msg: &Message, _args: Args) -> CommandResult { async fn skip(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -557,8 +561,7 @@ async fn skip(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult { async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -568,7 +571,7 @@ async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
if let Some(handler_lock) = manager.get(guild_id) { if let Some(handler_lock) = manager.get(guild_id) {
let handler = handler_lock.lock().await; let handler = handler_lock.lock().await;
let queue = handler.queue(); let queue = handler.queue();
let _ = queue.stop(); queue.stop();
check_msg(msg.channel_id.say(&ctx.http, "Queue cleared.").await); check_msg(msg.channel_id.say(&ctx.http, "Queue cleared.").await);
} else { } else {
@@ -585,8 +588,7 @@ async fn stop(ctx: &Context, msg: &Message, _args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult { async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
@@ -618,8 +620,7 @@ async fn undeafen(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn unmute(ctx: &Context, msg: &Message) -> CommandResult { async fn unmute(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx) let manager = songbird::get(ctx)
.await .await
.expect("Songbird Voice client placed in at initialisation.") .expect("Songbird Voice client placed in at initialisation.")

2
examples/serenity/voice_receive/Cargo.toml
View File

@@ -2,7 +2,7 @@
name = "voice_receive" name = "voice_receive"
version = "0.1.0" version = "0.1.0"
authors = ["my name <my@email.address>"] authors = ["my name <my@email.address>"]
edition = "2018" edition = "2021"
[dependencies] [dependencies]
tracing = "0.1" tracing = "0.1"

128
examples/serenity/voice_receive/src/main.rs
View File

@@ -12,17 +12,14 @@ use serenity::{
async_trait, async_trait,
client::{Client, Context, EventHandler}, client::{Client, Context, EventHandler},
framework::{ framework::{
StandardFramework,
standard::{ standard::{
macros::{command, group}, macros::{command, group},
Args, CommandResult, Args,
CommandResult,
}, },
StandardFramework,
}, },
model::{ model::{channel::Message, gateway::Ready, id::ChannelId},
channel::Message,
gateway::Ready,
id::ChannelId,
},
prelude::{GatewayIntents, Mentionable}, prelude::{GatewayIntents, Mentionable},
Result as SerenityResult, Result as SerenityResult,
}; };
@@ -53,7 +50,7 @@ impl Receiver {
pub fn new() -> Self { pub fn new() -> Self {
// You can manage state here, such as a buffer of audio packet bytes so // You can manage state here, such as a buffer of audio packet bytes so
// you can later store them in intervals. // you can later store them in intervals.
Self { } Self {}
} }
} }
@@ -63,9 +60,12 @@ impl VoiceEventHandler for Receiver {
async fn act(&self, ctx: &EventContext<'_>) -> Option<Event> { async fn act(&self, ctx: &EventContext<'_>) -> Option<Event> {
use EventContext as Ctx; use EventContext as Ctx;
match ctx { match ctx {
Ctx::SpeakingStateUpdate( Ctx::SpeakingStateUpdate(Speaking {
Speaking {speaking, ssrc, user_id, ..} speaking,
) => { ssrc,
user_id,
..
}) => {
// Discord voice calls use RTP, where every sender uses a randomly allocated // Discord voice calls use RTP, where every sender uses a randomly allocated
// *Synchronisation Source* (SSRC) to allow receivers to tell which audio // *Synchronisation Source* (SSRC) to allow receivers to tell which audio
// stream a received packet belongs to. As this number is not derived from // stream a received packet belongs to. As this number is not derived from
@@ -79,9 +79,7 @@ impl VoiceEventHandler for Receiver {
// to the user ID and handle their audio packets separately. // to the user ID and handle their audio packets separately.
println!( println!(
"Speaking state update: user {:?} has SSRC {:?}, using {:?}", "Speaking state update: user {:?} has SSRC {:?}, using {:?}",
user_id, user_id, ssrc, speaking,
ssrc,
speaking,
); );
}, },
Ctx::SpeakingUpdate(data) => { Ctx::SpeakingUpdate(data) => {
@@ -90,14 +88,17 @@ impl VoiceEventHandler for Receiver {
println!( println!(
"Source {} has {} speaking.", "Source {} has {} speaking.",
data.ssrc, data.ssrc,
if data.speaking {"started"} else {"stopped"}, if data.speaking { "started" } else { "stopped" },
); );
}, },
Ctx::VoicePacket(data) => { Ctx::VoicePacket(data) => {
// An event which fires for every received audio packet, // An event which fires for every received audio packet,
// containing the decoded data. // containing the decoded data.
if let Some(audio) = data.audio { if let Some(audio) = data.audio {
println!("Audio packet's first 5 samples: {:?}", audio.get(..5.min(audio.len()))); println!(
"Audio packet's first 5 samples: {:?}",
audio.get(..5.min(audio.len()))
);
println!( println!(
"Audio packet sequence {:05} has {:04} bytes (decompressed from {}), SSRC {}", "Audio packet sequence {:05} has {:04} bytes (decompressed from {}), SSRC {}",
data.packet.sequence.0, data.packet.sequence.0,
@@ -114,9 +115,7 @@ impl VoiceEventHandler for Receiver {
// containing the call statistics and reporting information. // containing the call statistics and reporting information.
println!("RTCP packet received: {:?}", data.packet); println!("RTCP packet received: {:?}", data.packet);
}, },
Ctx::ClientDisconnect( Ctx::ClientDisconnect(ClientDisconnect { user_id, .. }) => {
ClientDisconnect {user_id, ..}
) => {
// You can implement your own logic here to handle a user who has left the // You can implement your own logic here to handle a user who has left the
// voice channel e.g., finalise processing of statistics etc. // voice channel e.g., finalise processing of statistics etc.
// You will typically need to map the User ID to their SSRC; observed when // You will typically need to map the User ID to their SSRC; observed when
@@ -127,7 +126,7 @@ impl VoiceEventHandler for Receiver {
_ => { _ => {
// We won't be registering this struct for any more event classes. // We won't be registering this struct for any more event classes.
unimplemented!() unimplemented!()
} },
} }
None None
@@ -143,22 +142,18 @@ async fn main() {
tracing_subscriber::fmt::init(); tracing_subscriber::fmt::init();
// Configure the client with your Discord bot token in the environment. // Configure the client with your Discord bot token in the environment.
let token = env::var("DISCORD_TOKEN") let token = env::var("DISCORD_TOKEN").expect("Expected a token in the environment");
.expect("Expected a token in the environment");
let framework = StandardFramework::new() let framework = StandardFramework::new()
.configure(|c| c
.prefix("~"))
.group(&GENERAL_GROUP); .group(&GENERAL_GROUP);
framework.configure(|c| c.prefix("~"));
let intents = GatewayIntents::non_privileged() let intents = GatewayIntents::non_privileged() | GatewayIntents::MESSAGE_CONTENT;
| GatewayIntents::MESSAGE_CONTENT;
// Here, we need to configure Songbird to decode all incoming voice packets. // Here, we need to configure Songbird to decode all incoming voice packets.
// If you want, you can do this on a per-call basis---here, we need it to // If you want, you can do this on a per-call basis---here, we need it to
// read the audio data that other people are sending us! // read the audio data that other people are sending us!
let songbird_config = Config::default() let songbird_config = Config::default().decode_mode(DecodeMode::Decode);
.decode_mode(DecodeMode::Decode);
let mut client = Client::builder(&token, intents) let mut client = Client::builder(&token, intents)
.event_handler(Handler) .event_handler(Handler)
@@ -167,26 +162,33 @@ async fn main() {
.await .await
.expect("Err creating client"); .expect("Err creating client");
let _ = client.start().await.map_err(|why| println!("Client ended: {:?}", why)); let _ = client
.start()
.await
.map_err(|why| println!("Client ended: {:?}", why));
} }
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult { async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
let connect_to = match args.single::<u64>() { let connect_to = match args.single::<std::num::NonZeroU64>() {
Ok(id) => ChannelId(id), Ok(id) => ChannelId(id),
Err(_) => { Err(_) => {
check_msg(msg.reply(ctx, "Requires a valid voice channel ID be given").await); check_msg(
msg.reply(ctx, "Requires a valid voice channel ID be given")
.await,
);
return Ok(()); return Ok(());
}, },
}; };
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let (handler_lock, conn_result) = manager.join(guild_id, connect_to).await; let (handler_lock, conn_result) = manager.join(guild_id, connect_to).await;
@@ -194,34 +196,27 @@ async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
// NOTE: this skips listening for the actual connection result. // NOTE: this skips listening for the actual connection result.
let mut handler = handler_lock.lock().await; let mut handler = handler_lock.lock().await;
handler.add_global_event( handler.add_global_event(CoreEvent::SpeakingStateUpdate.into(), Receiver::new());
CoreEvent::SpeakingStateUpdate.into(),
Receiver::new(),
);
handler.add_global_event( handler.add_global_event(CoreEvent::SpeakingUpdate.into(), Receiver::new());
CoreEvent::SpeakingUpdate.into(),
Receiver::new(),
);
handler.add_global_event( handler.add_global_event(CoreEvent::VoicePacket.into(), Receiver::new());
CoreEvent::VoicePacket.into(),
Receiver::new(),
);
handler.add_global_event( handler.add_global_event(CoreEvent::RtcpPacket.into(), Receiver::new());
CoreEvent::RtcpPacket.into(),
Receiver::new(),
);
handler.add_global_event( handler.add_global_event(CoreEvent::ClientDisconnect.into(), Receiver::new());
CoreEvent::ClientDisconnect.into(),
Receiver::new(),
);
check_msg(msg.channel_id.say(&ctx.http, &format!("Joined {}", connect_to.mention())).await); check_msg(
msg.channel_id
.say(&ctx.http, &format!("Joined {}", connect_to.mention()))
.await,
);
} else { } else {
check_msg(msg.channel_id.say(&ctx.http, "Error joining the channel").await); check_msg(
msg.channel_id
.say(&ctx.http, "Error joining the channel")
.await,
);
} }
Ok(()) Ok(())
@@ -230,19 +225,24 @@ async fn join(ctx: &Context, msg: &Message, mut args: Args) -> CommandResult {
#[command] #[command]
#[only_in(guilds)] #[only_in(guilds)]
async fn leave(ctx: &Context, msg: &Message) -> CommandResult { async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
let guild = msg.guild(&ctx.cache).unwrap(); let guild_id = msg.guild_id.unwrap();
let guild_id = guild.id;
let manager = songbird::get(ctx).await let manager = songbird::get(ctx)
.expect("Songbird Voice client placed in at initialisation.").clone(); .await
.expect("Songbird Voice client placed in at initialisation.")
.clone();
let has_handler = manager.get(guild_id).is_some(); let has_handler = manager.get(guild_id).is_some();
if has_handler { if has_handler {
if let Err(e) = manager.remove(guild_id).await { if let Err(e) = manager.remove(guild_id).await {
check_msg(msg.channel_id.say(&ctx.http, format!("Failed: {:?}", e)).await); check_msg(
msg.channel_id
.say(&ctx.http, format!("Failed: {:?}", e))
.await,
);
} }
check_msg(msg.channel_id.say(&ctx.http,"Left voice channel").await); check_msg(msg.channel_id.say(&ctx.http, "Left voice channel").await);
} else { } else {
check_msg(msg.reply(ctx, "Not in a voice channel").await); check_msg(msg.reply(ctx, "Not in a voice channel").await);
} }
@@ -252,7 +252,7 @@ async fn leave(ctx: &Context, msg: &Message) -> CommandResult {
#[command] #[command]
async fn ping(ctx: &Context, msg: &Message) -> CommandResult { async fn ping(ctx: &Context, msg: &Message) -> CommandResult {
check_msg(msg.channel_id.say(&ctx.http,"Pong!").await); check_msg(msg.channel_id.say(&ctx.http, "Pong!").await);
Ok(()) Ok(())
} }

9
examples/twilight/Cargo.toml
View File

@@ -2,10 +2,11 @@
name = "twilight" name = "twilight"
version = "0.1.0" version = "0.1.0"
authors = ["Twilight and Serenity Contributors"] authors = ["Twilight and Serenity Contributors"]
edition = "2018" edition = "2021"
[dependencies] [dependencies]
futures = "0.3" futures = "0.3"
reqwest = "0.11"
tracing = "0.1" tracing = "0.1"
tracing-subscriber = "0.2" tracing-subscriber = "0.2"
tokio = { features = ["macros", "rt-multi-thread", "sync"], version = "1" } tokio = { features = ["macros", "rt-multi-thread", "sync"], version = "1" }
@@ -18,3 +19,9 @@ twilight-standby = "0.12"
default-features = false default-features = false
path = "../.." path = "../.."
features = ["driver", "twilight-rustls", "zlib-stock"] features = ["driver", "twilight-rustls", "zlib-stock"]
[dependencies.symphonia]
version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"]
git = "https://github.com/FelixMcFelix/Symphonia"
branch = "songbird-fixes"

43
examples/twilight/src/main.rs
View File

@@ -22,11 +22,11 @@
use futures::StreamExt; use futures::StreamExt;
use songbird::{ use songbird::{
input::{Input, Restartable}, input::{Compose, YoutubeDl},
tracks::{PlayMode, TrackHandle}, tracks::{PlayMode, TrackHandle},
Songbird, Songbird,
}; };
use std::{collections::HashMap, env, error::Error, future::Future, sync::Arc}; use std::{collections::HashMap, env, error::Error, future::Future, num::NonZeroU64, sync::Arc};
use tokio::sync::RwLock; use tokio::sync::RwLock;
use twilight_gateway::{Cluster, Event, Intents}; use twilight_gateway::{Cluster, Event, Intents};
use twilight_http::Client as HttpClient; use twilight_http::Client as HttpClient;
@@ -68,7 +68,7 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
let http = HttpClient::new(token.clone()); let http = HttpClient::new(token.clone());
let user_id = http.current_user().exec().await?.model().await?.id; let user_id = http.current_user().exec().await?.model().await?.id;
let intents = Intents::GUILD_MESSAGES | Intents::GUILD_VOICE_STATES; let intents = Intents::GUILD_MESSAGES | Intents::MESSAGE_CONTENT | Intents::GUILD_VOICE_STATES;
let (cluster, events) = Cluster::new(token, intents).await?; let (cluster, events) = Cluster::new(token, intents).await?;
cluster.up().await; cluster.up().await;
@@ -81,8 +81,8 @@ async fn main() -> Result<(), Box<dyn Error + Send + Sync + 'static>> {
trackdata: Default::default(), trackdata: Default::default(),
songbird, songbird,
standby: Standby::new(), standby: Standby::new(),
}, }),
)) )
}; };
while let Some((_, event)) = events.next().await { while let Some((_, event)) = events.next().await {
@@ -128,6 +128,8 @@ async fn join(msg: Message, state: State) -> Result<(), Box<dyn Error + Send + S
let channel_id = msg.content.parse::<u64>()?; let channel_id = msg.content.parse::<u64>()?;
let guild_id = msg.guild_id.ok_or("Can't join a non-guild channel.")?; let guild_id = msg.guild_id.ok_or("Can't join a non-guild channel.")?;
let channel_id =
NonZeroU64::new(channel_id).ok_or("Joined voice channel must have nonzero ID.")?;
let (_handle, success) = state.songbird.join(guild_id, channel_id).await; let (_handle, success) = state.songbird.join(guild_id, channel_id).await;
@@ -190,21 +192,12 @@ async fn play(msg: Message, state: State) -> Result<(), Box<dyn Error + Send + S
let guild_id = msg.guild_id.unwrap(); let guild_id = msg.guild_id.unwrap();
if let Ok(song) = Restartable::ytdl(msg.content.clone(), false).await { let mut src = YoutubeDl::new(reqwest::Client::new(), msg.content.clone());
let input = Input::from(song); if let Ok(metadata) = src.aux_metadata().await {
let content = format!( let content = format!(
"Playing **{:?}** by **{:?}**", "Playing **{:?}** by **{:?}**",
input metadata.track.as_ref().unwrap_or(&"<UNKNOWN>".to_string()),
.metadata metadata.artist.as_ref().unwrap_or(&"<UNKNOWN>".to_string()),
.track
.as_ref()
.unwrap_or(&"<UNKNOWN>".to_string()),
input
.metadata
.artist
.as_ref()
.unwrap_or(&"<UNKNOWN>".to_string()),
); );
state state
@@ -216,7 +209,7 @@ async fn play(msg: Message, state: State) -> Result<(), Box<dyn Error + Send + S
if let Some(call_lock) = state.songbird.get(guild_id) { if let Some(call_lock) = state.songbird.get(guild_id) {
let mut call = call_lock.lock().await; let mut call = call_lock.lock().await;
let handle = call.play_source(input); let handle = call.play_input(src.into());
let mut store = state.trackdata.write().await; let mut store = state.trackdata.write().await;
store.insert(guild_id, handle); store.insert(guild_id, handle);
@@ -251,11 +244,11 @@ async fn pause(msg: Message, state: State) -> Result<(), Box<dyn Error + Send +
PlayMode::Play => { PlayMode::Play => {
let _success = handle.pause(); let _success = handle.pause();
false false
} },
_ => { _ => {
let _success = handle.play(); let _success = handle.play();
true true
} },
}; };
let action = if paused { "Unpaused" } else { "Paused" }; let action = if paused { "Unpaused" } else { "Paused" };
@@ -301,12 +294,8 @@ async fn seek(msg: Message, state: State) -> Result<(), Box<dyn Error + Send + S
let store = state.trackdata.read().await; let store = state.trackdata.read().await;
let content = if let Some(handle) = store.get(&guild_id) { let content = if let Some(handle) = store.get(&guild_id) {
if handle.is_seekable() { let _success = handle.seek(std::time::Duration::from_secs(position));
let _success = handle.seek_time(std::time::Duration::from_secs(position)); format!("Seeking to {}s", position)
format!("Seeked to {}s", position)
} else {
format!("Track is not compatible with seeking!")
}
} else { } else {
format!("No track to seek over!") format!("No track to seek over!")
}; };

BIN
images/arch.afdesign
View File

Binary file not shown.

BIN
images/driver.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 142 KiB

After

Width:  |  Height:  |  Size: 166 KiB

39
images/driver.svg
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg width="100%" height="100%" viewBox="0 0 1525 1013" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" xmlns:serif="http://www.serif.com/" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:1.5;"> <svg width="100%" height="100%" viewBox="0 0 1525 1071" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xml:space="preserve" xmlns:serif="http://www.serif.com/" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:1.5;">
<g transform="matrix(1,0,0,1,-31,-921)"> <g transform="matrix(1,0,0,1,-31,-921)">
<g transform="matrix(1,0,0,1,0,900)"> <g transform="matrix(1,0,0,1,0,900)">
<g transform="matrix(1,0,0,1,299,-52.947)"> <g transform="matrix(1,0,0,1,299,-52.947)">
@@ -49,6 +49,40 @@
<text x="762.453px" y="1473.58px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">C<tspan x="789.953px 819.153px 837.953px " y="1473.58px 1473.58px 1473.58px ">ore</tspan></text> <text x="762.453px" y="1473.58px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">C<tspan x="789.953px 819.153px 837.953px " y="1473.58px 1473.58px 1473.58px ">ore</tspan></text>
</g> </g>
</g> </g>
<g transform="matrix(1,0,0,1,53.0449,-6.9189)">
<g transform="matrix(1.75342,0,0,1.61288,854.748,-134.087)">
<rect x="101.876" y="1143.29" width="184.054" height="163.884" style="fill:rgb(255,193,74);stroke:rgb(62,62,62);stroke-width:2.31px;"/>
</g>
<g transform="matrix(1,0,0,1,931.504,566.618)">
<text x="126.789px" y="1193.29px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">Thr<tspan x="200.739px " y="1193.29px ">e</tspan>ad P<tspan x="326.639px " y="1193.29px ">o</tspan>ol</text>
</g>
<g transform="matrix(1,0,0,1,-31.0449,9.9189)">
<g transform="matrix(0.506003,0,0,0.506003,744.047,1049.39)">
<g transform="matrix(1.01835,0,0,1.01835,-11.9813,-27.0542)">
<ellipse cx="829.267" cy="1498.79" rx="134.889" ry="73.117" style="fill:white;stroke:rgb(62,62,62);stroke-width:7.56px;"/>
</g>
<g transform="matrix(1,0,0,1,9,40.3406)">
<text x="770.503px" y="1473.58px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">Seek</text>
</g>
</g>
<g transform="matrix(0.506003,0,0,0.506003,804.011,1098.98)">
<g transform="matrix(1.01835,0,0,1.01835,-11.9813,-27.0542)">
<ellipse cx="829.267" cy="1498.79" rx="134.889" ry="73.117" style="fill:white;stroke:rgb(62,62,62);stroke-width:7.56px;"/>
</g>
<g transform="matrix(1,0,0,1,9,40.3406)">
<text x="719.778px" y="1473.58px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">C<tspan x="747.278px " y="1473.58px ">o</tspan>mpose</text>
</g>
</g>
<g transform="matrix(0.506003,0,0,0.506003,864.793,1148.74)">
<g transform="matrix(1.01835,0,0,1.01835,-11.9813,-27.0542)">
<ellipse cx="829.267" cy="1498.79" rx="134.889" ry="73.117" style="fill:white;stroke:rgb(62,62,62);stroke-width:7.56px;"/>
</g>
<g transform="matrix(1,0,0,1,9,40.3406)">
<text x="806.428px" y="1473.58px" style="font-family:'FiraSans-Regular', 'Fira Sans', sans-serif;font-size:50px;">...</text>
</g>
</g>
</g>
</g>
<g transform="matrix(1,0,0,1,-47.9044,-158.841)"> <g transform="matrix(1,0,0,1,-47.9044,-158.841)">
<g transform="matrix(0.724138,0,0,0.724138,223.721,413.46)"> <g transform="matrix(0.724138,0,0,0.724138,223.721,413.46)">
<ellipse cx="829.267" cy="1498.79" rx="134.889" ry="73.117" style="fill:rgb(255,193,74);stroke:rgb(62,62,62);stroke-width:5.38px;"/> <ellipse cx="829.267" cy="1498.79" rx="134.889" ry="73.117" style="fill:rgb(255,193,74);stroke:rgb(62,62,62);stroke-width:5.38px;"/>
@@ -113,6 +147,9 @@
<g transform="matrix(0.675184,-0.370199,0.303584,0.553689,739.208,1743.48)"> <g transform="matrix(0.675184,-0.370199,0.303584,0.553689,739.208,1743.48)">
<path d="M212.72,146.182L212.72,132.063L524.856,132.063L524.856,103.825L540.487,139.123L524.856,174.421L524.856,146.182L212.72,146.182Z" style="fill:rgb(62,62,62);"/> <path d="M212.72,146.182L212.72,132.063L524.856,132.063L524.856,103.825L540.487,139.123L524.856,174.421L524.856,146.182L212.72,146.182Z" style="fill:rgb(62,62,62);"/>
</g> </g>
<g transform="matrix(0.494195,0.135428,-0.12143,0.443115,829.694,1701.34)">
<path d="M230.444,146.182L230.444,174.421L212.72,139.123L230.444,103.825L230.444,132.063L523.396,132.063L523.396,103.825L540.487,139.123L523.396,174.421L523.396,146.182L230.444,146.182Z" style="fill:rgb(62,62,62);"/>
</g>
<g transform="matrix(0.0764172,-0.232117,0.599787,0.197461,745.524,1728.62)"> <g transform="matrix(0.0764172,-0.232117,0.599787,0.197461,745.524,1728.62)">
<path d="M212.72,146.182L212.72,132.063L491.234,132.063L491.234,103.825L540.487,139.123L491.234,174.421L491.234,146.182L212.72,146.182Z" style="fill:rgb(62,62,62);"/> <path d="M212.72,146.182L212.72,132.063L491.234,132.063L491.234,103.825L540.487,139.123L491.234,174.421L491.234,146.182L212.72,146.182Z" style="fill:rgb(62,62,62);"/>
</g> </g>
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 13 KiB

BIN
images/gateway.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 129 KiB

After

Width:  |  Height:  |  Size: 129 KiB

BIN
resources/Cloudkicker - 2011 07.dca1 Normal file
View File

Binary file not shown.

0
examples/serenity/voice_storage/Cloudkicker_-_Loops_-_22_2011_07.mp3 → resources/Cloudkicker - 2011 07.mp3
View File

BIN
resources/Cloudkicker - Making Will Mad.opus Normal file
View File

Binary file not shown.

BIN
resources/Cloudkicker - Making Will Mad.webm Normal file
View File

Binary file not shown.

8
resources/README.md Normal file
View File

@@ -0,0 +1,8 @@
# Resources
This folder contains various audio files used for testing or within examples.
Some songs are used under creative commons licenses ([Ben Sharp/Cloudkicker](https://cloudkicker.bandcamp.com/)):
* `Cloudkicker - 2011 07.[mp3,dca1]` – [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/)
* `Cloudkicker - Making Will Mad.[opus,webm]` – [CC BY 3.0](https://creativecommons.org/licenses/by/3.0/)
All sound files are made by contributors.

BIN
resources/loop-48.mp3 Normal file
View File

Binary file not shown.

0
examples/serenity/voice_storage/loop.wav → resources/loop.wav
View File

BIN
resources/ting.mp3 Normal file
View File

Binary file not shown.

0
examples/serenity/voice_storage/ting.wav → resources/ting.wav
View File

162
src/config.rs
View File

@@ -1,13 +1,24 @@
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use super::driver::{retry::Retry, CryptoMode, DecodeMode}; use crate::{
driver::{retry::Retry, CryptoMode, DecodeMode, MixMode},
input::codecs::*,
};
#[cfg(test)]
use crate::driver::test_config::*;
#[cfg(feature = "driver")]
use symphonia::core::{codecs::CodecRegistry, probe::Probe};
use derivative::Derivative;
use std::time::Duration; use std::time::Duration;
/// Configuration for drivers and calls. /// Configuration for drivers and calls.
#[derive(Clone, Debug)] #[derive(Clone, Derivative)]
#[derivative(Debug)]
#[non_exhaustive] #[non_exhaustive]
pub struct Config { pub struct Config {
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Selected tagging mode for voice packet encryption. /// Selected tagging mode for voice packet encryption.
/// ///
/// Defaults to [`CryptoMode::Normal`]. /// Defaults to [`CryptoMode::Normal`].
@@ -18,7 +29,7 @@ pub struct Config {
/// ///
/// [`CryptoMode::Normal`]: CryptoMode::Normal /// [`CryptoMode::Normal`]: CryptoMode::Normal
pub crypto_mode: CryptoMode, pub crypto_mode: CryptoMode,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Configures whether decoding and decryption occur for all received packets. /// Configures whether decoding and decryption occur for all received packets.
/// ///
/// If voice receiving voice packets, generally you should choose [`DecodeMode::Decode`]. /// If voice receiving voice packets, generally you should choose [`DecodeMode::Decode`].
@@ -34,7 +45,7 @@ pub struct Config {
/// [`DecodeMode::Pass`]: DecodeMode::Pass /// [`DecodeMode::Pass`]: DecodeMode::Pass
/// [user speaking events]: crate::events::CoreEvent::SpeakingUpdate /// [user speaking events]: crate::events::CoreEvent::SpeakingUpdate
pub decode_mode: DecodeMode, pub decode_mode: DecodeMode,
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
/// Configures the amount of time to wait for Discord to reply with connection information /// Configures the amount of time to wait for Discord to reply with connection information
/// if [`Call::join`]/[`join_gateway`] are used. /// if [`Call::join`]/[`join_gateway`] are used.
/// ///
@@ -47,7 +58,15 @@ pub struct Config {
/// [`Call::join`]: crate::Call::join /// [`Call::join`]: crate::Call::join
/// [`join_gateway`]: crate::Call::join_gateway /// [`join_gateway`]: crate::Call::join_gateway
pub gateway_timeout: Option<Duration>, pub gateway_timeout: Option<Duration>,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Configures the maximum amount of time to wait for an attempted voice
/// connection to Discord.
///
/// Defaults to [`Stereo`].
///
/// [`Stereo`]: MixMode::Stereo
pub mix_mode: MixMode,
#[cfg(feature = "driver")]
/// Number of concurrently active tracks to allocate memory for. /// Number of concurrently active tracks to allocate memory for.
/// ///
/// This should be set at, or just above, the maximum number of tracks /// This should be set at, or just above, the maximum number of tracks
@@ -60,7 +79,7 @@ pub struct Config {
/// Changes to this field in a running driver will only ever increase /// Changes to this field in a running driver will only ever increase
/// the capacity of the track store. /// the capacity of the track store.
pub preallocated_tracks: usize, pub preallocated_tracks: usize,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Connection retry logic for the [`Driver`]. /// Connection retry logic for the [`Driver`].
/// ///
/// This controls how many times the [`Driver`] should retry any connections, /// This controls how many times the [`Driver`] should retry any connections,
@@ -68,65 +87,131 @@ pub struct Config {
/// ///
/// [`Driver`]: crate::driver::Driver /// [`Driver`]: crate::driver::Driver
pub driver_retry: Retry, pub driver_retry: Retry,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Configures the maximum amount of time to wait for an attempted voice /// Configures the maximum amount of time to wait for an attempted voice
/// connection to Discord. /// connection to Discord.
/// ///
/// Defaults to 10 seconds. If set to `None`, connections will never time out. /// Defaults to 10 seconds. If set to `None`, connections will never time out.
pub driver_timeout: Option<Duration>, pub driver_timeout: Option<Duration>,
#[cfg(feature = "driver")]
#[derivative(Debug = "ignore")]
/// Registry of the inner codecs supported by the driver, adding audiopus-based
/// Opus codec support to all of Symphonia's default codecs.
///
/// Defaults to [`CODEC_REGISTRY`].
///
/// [`CODEC_REGISTRY`]: static@CODEC_REGISTRY
pub codec_registry: &'static CodecRegistry,
#[cfg(feature = "driver")]
#[derivative(Debug = "ignore")]
/// Registry of the muxers and container formats supported by the driver.
///
/// Defaults to [`PROBE`], which includes all of Symphonia's default format handlers
/// and DCA format support.
///
/// [`PROBE`]: static@PROBE
pub format_registry: &'static Probe,
// Test only attributes
#[cfg(feature = "driver")]
#[cfg(test)]
/// Test config to offer precise control over mixing tick rate/count.
pub(crate) tick_style: TickStyle,
#[cfg(feature = "driver")]
#[cfg(test)]
/// If set, skip connection and encryption steps.
pub(crate) override_connection: Option<OutputMode>,
} }
impl Default for Config { impl Default for Config {
fn default() -> Self { fn default() -> Self {
Self { Self {
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
crypto_mode: CryptoMode::Normal, crypto_mode: CryptoMode::Normal,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
decode_mode: DecodeMode::Decrypt, decode_mode: DecodeMode::Decrypt,
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
gateway_timeout: Some(Duration::from_secs(10)), gateway_timeout: Some(Duration::from_secs(10)),
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
mix_mode: MixMode::Stereo,
#[cfg(feature = "driver")]
preallocated_tracks: 1, preallocated_tracks: 1,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
driver_retry: Default::default(), driver_retry: Retry::default(),
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
driver_timeout: Some(Duration::from_secs(10)), driver_timeout: Some(Duration::from_secs(10)),
#[cfg(feature = "driver")]
codec_registry: &CODEC_REGISTRY,
#[cfg(feature = "driver")]
format_registry: &PROBE,
#[cfg(feature = "driver")]
#[cfg(test)]
tick_style: TickStyle::Timed,
#[cfg(feature = "driver")]
#[cfg(test)]
override_connection: None,
} }
} }
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
impl Config { impl Config {
/// Sets this `Config`'s chosen cryptographic tagging scheme. /// Sets this `Config`'s chosen cryptographic tagging scheme.
#[must_use]
pub fn crypto_mode(mut self, crypto_mode: CryptoMode) -> Self { pub fn crypto_mode(mut self, crypto_mode: CryptoMode) -> Self {
self.crypto_mode = crypto_mode; self.crypto_mode = crypto_mode;
self self
} }
/// Sets this `Config`'s received packet decryption/decoding behaviour. /// Sets this `Config`'s received packet decryption/decoding behaviour.
#[must_use]
pub fn decode_mode(mut self, decode_mode: DecodeMode) -> Self { pub fn decode_mode(mut self, decode_mode: DecodeMode) -> Self {
self.decode_mode = decode_mode; self.decode_mode = decode_mode;
self self
} }
/// Sets this `Config`'s audio mixing channel count.
#[must_use]
pub fn mix_mode(mut self, mix_mode: MixMode) -> Self {
self.mix_mode = mix_mode;
self
}
/// Sets this `Config`'s number of tracks to preallocate. /// Sets this `Config`'s number of tracks to preallocate.
#[must_use]
pub fn preallocated_tracks(mut self, preallocated_tracks: usize) -> Self { pub fn preallocated_tracks(mut self, preallocated_tracks: usize) -> Self {
self.preallocated_tracks = preallocated_tracks; self.preallocated_tracks = preallocated_tracks;
self self
} }
/// Sets this `Config`'s timeout for establishing a voice connection. /// Sets this `Config`'s timeout for establishing a voice connection.
#[must_use]
pub fn driver_timeout(mut self, driver_timeout: Option<Duration>) -> Self { pub fn driver_timeout(mut self, driver_timeout: Option<Duration>) -> Self {
self.driver_timeout = driver_timeout; self.driver_timeout = driver_timeout;
self self
} }
/// Sets this `Config`'s voice connection retry configuration. /// Sets this `Config`'s voice connection retry configuration.
#[must_use]
pub fn driver_retry(mut self, driver_retry: Retry) -> Self { pub fn driver_retry(mut self, driver_retry: Retry) -> Self {
self.driver_retry = driver_retry; self.driver_retry = driver_retry;
self self
} }
/// Sets this `Config`'s symphonia codec registry.
#[must_use]
pub fn codec_registry(mut self, codec_registry: &'static CODEC_REGISTRY) -> Self {
self.codec_registry = codec_registry;
self
}
/// Sets this `Config`'s symphonia format registry/probe set.
#[must_use]
pub fn format_registry(mut self, format_registry: &'static PROBE) -> Self {
self.format_registry = format_registry;
self
}
/// This is used to prevent changes which would invalidate the current session. /// This is used to prevent changes which would invalidate the current session.
pub(crate) fn make_safe(&mut self, previous: &Config, connected: bool) { pub(crate) fn make_safe(&mut self, previous: &Config, connected: bool) {
if connected { if connected {
@@ -135,9 +220,50 @@ impl Config {
} }
} }
#[cfg(feature = "gateway-core")] // Test only attributes
#[cfg(all(test, feature = "driver"))]
impl Config {
#![allow(missing_docs)]
#[must_use]
pub fn tick_style(mut self, tick_style: TickStyle) -> Self {
self.tick_style = tick_style;
self
}
/// Sets this `Config`'s voice connection retry configuration.
#[must_use]
pub fn override_connection(mut self, override_connection: Option<OutputMode>) -> Self {
self.override_connection = override_connection;
self
}
pub fn test_cfg(raw_output: bool) -> (DriverTestHandle, Config) {
let (tick_tx, tick_rx) = flume::unbounded();
let (conn, rx) = if raw_output {
let (pkt_tx, pkt_rx) = flume::unbounded();
(OutputMode::Raw(pkt_tx), OutputReceiver::Raw(pkt_rx))
} else {
let (rtp_tx, rtp_rx) = flume::unbounded();
(OutputMode::Rtp(rtp_tx), OutputReceiver::Rtp(rtp_rx))
};
let config = Config::default()
.tick_style(TickStyle::UntimedWithExecLimit(tick_rx))
.override_connection(Some(conn));
let handle = DriverTestHandle { rx, tx: tick_tx };
(handle, config)
}
}
#[cfg(feature = "gateway")]
impl Config { impl Config {
/// Sets this `Config`'s timeout for joining a voice channel. /// Sets this `Config`'s timeout for joining a voice channel.
#[must_use]
pub fn gateway_timeout(mut self, gateway_timeout: Option<Duration>) -> Self { pub fn gateway_timeout(mut self, gateway_timeout: Option<Duration>) -> Self {
self.gateway_timeout = gateway_timeout; self.gateway_timeout = gateway_timeout;
self self

54
src/constants.rs
View File

@@ -1,16 +1,16 @@
//! Constants affecting driver function and API handling. //! Constants affecting driver function and API handling.
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use audiopus::{Bitrate, SampleRate}; use audiopus::{Bitrate, SampleRate};
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use discortp::rtp::RtpType; use discortp::rtp::RtpType;
use std::time::Duration; use std::time::Duration;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// The voice gateway version used by the library. /// The voice gateway version used by the library.
pub const VOICE_GATEWAY_VERSION: u8 = crate::model::constants::GATEWAY_VERSION; pub const VOICE_GATEWAY_VERSION: u8 = crate::model::constants::GATEWAY_VERSION;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Sample rate of audio to be sent to Discord. /// Sample rate of audio to be sent to Discord.
pub const SAMPLE_RATE: SampleRate = SampleRate::Hz48000; pub const SAMPLE_RATE: SampleRate = SampleRate::Hz48000;
@@ -23,10 +23,16 @@ pub const AUDIO_FRAME_RATE: usize = 50;
/// Length of time between any two audio frames. /// Length of time between any two audio frames.
pub const TIMESTEP_LENGTH: Duration = Duration::from_millis(1000 / AUDIO_FRAME_RATE as u64); pub const TIMESTEP_LENGTH: Duration = Duration::from_millis(1000 / AUDIO_FRAME_RATE as u64);
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Default bitrate for audio. /// Default bitrate for audio.
pub const DEFAULT_BITRATE: Bitrate = Bitrate::BitsPerSecond(128_000); pub const DEFAULT_BITRATE: Bitrate = Bitrate::BitsPerSecond(128_000);
/// Number of output samples at 48kHZ to produced when resampling subframes.
pub(crate) const RESAMPLE_OUTPUT_FRAME_SIZE: usize = MONO_FRAME_SIZE / 2;
/// The maximum number of bad frames to allow in an Opus source before blocking passthrough.
pub(crate) const OPUS_PASSTHROUGH_STRIKE_LIMIT: u8 = 3;
/// Number of samples in one complete frame of audio per channel. /// Number of samples in one complete frame of audio per channel.
/// ///
/// This is equally the number of stereo (joint) samples in an audio frame. /// This is equally the number of stereo (joint) samples in an audio frame.
@@ -70,6 +76,42 @@ pub const SILENT_FRAME: [u8; 3] = [0xf8, 0xff, 0xfe];
/// The one (and only) RTP version. /// The one (and only) RTP version.
pub const RTP_VERSION: u8 = 2; pub const RTP_VERSION: u8 = 2;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Profile type used by Discord's Opus audio traffic. /// Profile type used by Discord's Opus audio traffic.
pub const RTP_PROFILE_TYPE: RtpType = RtpType::Dynamic(120); pub const RTP_PROFILE_TYPE: RtpType = RtpType::Dynamic(120);
#[cfg(test)]
pub mod test_data {
/// URL for a source which YTDL must extract.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const YTDL_TARGET: &str = "https://cloudkicker.bandcamp.com/track/94-days";
/// URL for a source which can be read via an Http Request.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const HTTP_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%202011%2007.mp3";
/// URL for an opus/ogg source which can be read via an Http Request.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const HTTP_OPUS_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%20Making%20Will%20Mad.opus";
/// URL for an opus/webm source which can be read via an Http Request.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const HTTP_WEBM_TARGET: &str = "https://github.com/FelixMcFelix/songbird/raw/symphonia/resources/Cloudkicker%20-%20Making%20Will%20Mad.webm";
/// Path to a DCA source.
///
/// Referenced under CC BY-NC-SA 3.0 -- https://creativecommons.org/licenses/by-nc-sa/3.0/
pub const FILE_DCA_TARGET: &str = "resources/Cloudkicker - 2011 07.dca1";
/// Path to an opus source which can be read via a File.
///
/// Referenced under CC BY 3.0 -- https://creativecommons.org/licenses/by/3.0/
pub const FILE_WEBM_TARGET: &str = "resources/Cloudkicker - Making Will Mad.webm";
/// Path to a Wav source which can be read via a File.
pub const FILE_WAV_TARGET: &str = "resources/loop.wav";
}

13
src/driver/bench_internals.rs
View File

@@ -1,8 +1,19 @@
//! Various driver internals which need to be exported for benchmarking. //! Various driver internals which need to be exported for benchmarking.
//! //!
//! Included if using the `"internals"` feature flag. //! Included if using the `"internals"` feature flag.
//! You should not and/or cannot use these as part of a normal application. //! You should not and/or cannot use these as part of a normal application.
#![allow(missing_docs)]
pub use super::tasks::{message as task_message, mixer}; pub use super::tasks::{message as task_message, mixer};
pub use super::crypto::CryptoState; pub use super::crypto::CryptoState;
use crate::{
driver::tasks::message::TrackContext,
tracks::{Track, TrackHandle},
};
pub fn track_context(t: Track) -> (TrackHandle, TrackContext) {
t.into_context()
}

48
src/driver/connection/error.rs
View File

@@ -94,21 +94,21 @@ impl From<Elapsed> for Error {
impl fmt::Display for Error { impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "failed to connect to Discord RTP server: ")?; write!(f, "failed to connect to Discord RTP server: ")?;
use Error::*;
match self { match self {
AttemptDiscarded => write!(f, "connection attempt was aborted/discarded"), Self::AttemptDiscarded => write!(f, "connection attempt was aborted/discarded"),
Crypto(e) => e.fmt(f), Self::Crypto(e) => e.fmt(f),
CryptoModeInvalid => write!(f, "server changed negotiated encryption mode"), Self::CryptoModeInvalid => write!(f, "server changed negotiated encryption mode"),
CryptoModeUnavailable => write!(f, "server did not offer chosen encryption mode"), Self::CryptoModeUnavailable => write!(f, "server did not offer chosen encryption mode"),
EndpointUrl => write!(f, "endpoint URL received from gateway was invalid"), Self::EndpointUrl => write!(f, "endpoint URL received from gateway was invalid"),
ExpectedHandshake => write!(f, "voice initialisation protocol was violated"), Self::ExpectedHandshake => write!(f, "voice initialisation protocol was violated"),
IllegalDiscoveryResponse => write!(f, "IP discovery/NAT punching response was invalid"), Self::IllegalDiscoveryResponse =>
IllegalIp => write!(f, "IP discovery/NAT punching response had bad IP value"), write!(f, "IP discovery/NAT punching response was invalid"),
Io(e) => e.fmt(f), Self::IllegalIp => write!(f, "IP discovery/NAT punching response had bad IP value"),
Json(e) => e.fmt(f), Self::Io(e) => e.fmt(f),
InterconnectFailure(e) => write!(f, "failed to contact other task ({:?})", e), Self::Json(e) => e.fmt(f),
Ws(e) => write!(f, "websocket issue ({:?}).", e), Self::InterconnectFailure(e) => write!(f, "failed to contact other task ({:?})", e),
TimedOut => write!(f, "connection attempt timed out"), Self::Ws(e) => write!(f, "websocket issue ({:?}).", e),
Self::TimedOut => write!(f, "connection attempt timed out"),
} }
} }
} }
@@ -116,19 +116,19 @@ impl fmt::Display for Error {
impl StdError for Error { impl StdError for Error {
fn source(&self) -> Option<&(dyn StdError + 'static)> { fn source(&self) -> Option<&(dyn StdError + 'static)> {
match self { match self {
Error::AttemptDiscarded => None, Error::AttemptDiscarded
| Error::CryptoModeInvalid
| Error::CryptoModeUnavailable
| Error::EndpointUrl
| Error::ExpectedHandshake
| Error::IllegalDiscoveryResponse
| Error::IllegalIp
| Error::InterconnectFailure(_)
| Error::Ws(_)
| Error::TimedOut => None,
Error::Crypto(e) => e.source(), Error::Crypto(e) => e.source(),
Error::CryptoModeInvalid => None,
Error::CryptoModeUnavailable => None,
Error::EndpointUrl => None,
Error::ExpectedHandshake => None,
Error::IllegalDiscoveryResponse => None,
Error::IllegalIp => None,
Error::Io(e) => e.source(), Error::Io(e) => e.source(),
Error::Json(e) => e.source(), Error::Json(e) => e.source(),
Error::InterconnectFailure(_) => None,
Error::Ws(_) => None,
Error::TimedOut => None,
} }
} }
} }

76
src/driver/crypto.rs
View File

@@ -12,7 +12,7 @@ use xsalsa20poly1305::{
TAG_SIZE, TAG_SIZE,
}; };
/// Variants of the XSalsa20Poly1305 encryption scheme. /// Variants of the `XSalsa20Poly1305` encryption scheme.
#[derive(Clone, Copy, Debug, Eq, PartialEq)] #[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive] #[non_exhaustive]
pub enum CryptoMode { pub enum CryptoMode {
@@ -35,57 +35,58 @@ pub enum CryptoMode {
impl From<CryptoState> for CryptoMode { impl From<CryptoState> for CryptoMode {
fn from(val: CryptoState) -> Self { fn from(val: CryptoState) -> Self {
use CryptoState::*;
match val { match val {
Normal => CryptoMode::Normal, CryptoState::Normal => Self::Normal,
Suffix => CryptoMode::Suffix, CryptoState::Suffix => Self::Suffix,
Lite(_) => CryptoMode::Lite, CryptoState::Lite(_) => Self::Lite,
} }
} }
} }
impl CryptoMode { impl CryptoMode {
/// Returns the name of a mode as it will appear during negotiation. /// Returns the name of a mode as it will appear during negotiation.
#[must_use]
pub fn to_request_str(self) -> &'static str { pub fn to_request_str(self) -> &'static str {
use CryptoMode::*;
match self { match self {
Normal => "xsalsa20_poly1305", Self::Normal => "xsalsa20_poly1305",
Suffix => "xsalsa20_poly1305_suffix", Self::Suffix => "xsalsa20_poly1305_suffix",
Lite => "xsalsa20_poly1305_lite", Self::Lite => "xsalsa20_poly1305_lite",
} }
} }
/// Returns the number of bytes each nonce is stored as within /// Returns the number of bytes each nonce is stored as within
/// a packet. /// a packet.
#[must_use]
pub fn nonce_size(self) -> usize { pub fn nonce_size(self) -> usize {
use CryptoMode::*;
match self { match self {
Normal => RtpPacket::minimum_packet_size(), Self::Normal => RtpPacket::minimum_packet_size(),
Suffix => NONCE_SIZE, Self::Suffix => NONCE_SIZE,
Lite => 4, Self::Lite => 4,
} }
} }
/// Returns the number of bytes occupied by the encryption scheme /// Returns the number of bytes occupied by the encryption scheme
/// which fall before the payload. /// which fall before the payload.
pub fn payload_prefix_len(self) -> usize { #[must_use]
pub fn payload_prefix_len() -> usize {
TAG_SIZE TAG_SIZE
} }
/// Returns the number of bytes occupied by the encryption scheme /// Returns the number of bytes occupied by the encryption scheme
/// which fall after the payload. /// which fall after the payload.
#[must_use]
pub fn payload_suffix_len(self) -> usize { pub fn payload_suffix_len(self) -> usize {
use CryptoMode::*;
match self { match self {
Normal => 0, Self::Normal => 0,
Suffix | Lite => self.nonce_size(), Self::Suffix | Self::Lite => self.nonce_size(),
} }
} }
/// Calculates the number of additional bytes required compared /// Calculates the number of additional bytes required compared
/// to an unencrypted payload. /// to an unencrypted payload.
#[must_use]
pub fn payload_overhead(self) -> usize { pub fn payload_overhead(self) -> usize {
self.payload_prefix_len() + self.payload_suffix_len() Self::payload_prefix_len() + self.payload_suffix_len()
} }
/// Extracts the byte slice in a packet used as the nonce, and the remaining mutable /// Extracts the byte slice in a packet used as the nonce, and the remaining mutable
@@ -95,10 +96,9 @@ impl CryptoMode {
header: &'a [u8], header: &'a [u8],
body: &'a mut [u8], body: &'a mut [u8],
) -> Result<(&'a [u8], &'a mut [u8]), CryptoError> { ) -> Result<(&'a [u8], &'a mut [u8]), CryptoError> {
use CryptoMode::*;
match self { match self {
Normal => Ok((header, body)), Self::Normal => Ok((header, body)),
Suffix | Lite => { Self::Suffix | Self::Lite => {
let len = body.len(); let len = body.len();
if len < self.payload_suffix_len() { if len < self.payload_suffix_len() {
Err(CryptoError) Err(CryptoError)
@@ -135,7 +135,7 @@ impl CryptoMode {
&nonce &nonce
}; };
let body_start = self.payload_prefix_len(); let body_start = Self::payload_prefix_len();
let body_tail = self.payload_suffix_len(); let body_tail = self.payload_suffix_len();
if body_start > body_remaining.len() { if body_start > body_remaining.len() {
@@ -183,22 +183,33 @@ impl CryptoMode {
} }
} }
#[allow(missing_docs)] /// State used in nonce generation for the `XSalsa20Poly1305` encryption variants
/// in [`CryptoMode`].
#[derive(Clone, Copy, Debug, Eq, PartialEq)] #[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive] #[non_exhaustive]
pub enum CryptoState { pub enum CryptoState {
/// The RTP header is used as the source of nonce bytes for the packet.
///
/// No state is required.
Normal, Normal,
/// An additional random 24B suffix is used as the source of nonce bytes for the packet.
/// This is regenerated randomly for each packet.
///
/// No state is required.
Suffix, Suffix,
/// An additional random 4B suffix is used as the source of nonce bytes for the packet.
/// This nonce value increments by `1` with each packet.
///
/// The last used nonce is stored.
Lite(Wrapping<u32>), Lite(Wrapping<u32>),
} }
impl From<CryptoMode> for CryptoState { impl From<CryptoMode> for CryptoState {
fn from(val: CryptoMode) -> Self { fn from(val: CryptoMode) -> Self {
use CryptoMode::*;
match val { match val {
Normal => CryptoState::Normal, CryptoMode::Normal => CryptoState::Normal,
Suffix => CryptoState::Suffix, CryptoMode::Suffix => CryptoState::Suffix,
Lite => CryptoState::Lite(Wrapping(rand::random::<u32>())), CryptoMode::Lite => CryptoState::Lite(Wrapping(rand::random::<u32>())),
} }
} }
} }
@@ -213,12 +224,11 @@ impl CryptoState {
let mode = self.kind(); let mode = self.kind();
let endpoint = payload_end + mode.payload_suffix_len(); let endpoint = payload_end + mode.payload_suffix_len();
use CryptoState::*;
match self { match self {
Suffix => { Self::Suffix => {
rand::thread_rng().fill(&mut packet.payload_mut()[payload_end..endpoint]); rand::thread_rng().fill(&mut packet.payload_mut()[payload_end..endpoint]);
}, },
Lite(mut i) => { Self::Lite(mut i) => {
(&mut packet.payload_mut()[payload_end..endpoint]) (&mut packet.payload_mut()[payload_end..endpoint])
.write_u32::<NetworkEndian>(i.0) .write_u32::<NetworkEndian>(i.0)
.expect( .expect(
@@ -233,8 +243,8 @@ impl CryptoState {
} }
/// Returns the underlying (stateless) type of the active crypto mode. /// Returns the underlying (stateless) type of the active crypto mode.
pub fn kind(&self) -> CryptoMode { pub fn kind(self) -> CryptoMode {
CryptoMode::from(*self) CryptoMode::from(self)
} }
} }
@@ -246,7 +256,7 @@ mod test {
#[test] #[test]
fn small_packet_decrypts_error() { fn small_packet_decrypts_error() {
let mut buf = [0u8; MutableRtpPacket::minimum_packet_size() + 0]; let mut buf = [0u8; MutableRtpPacket::minimum_packet_size()];
let modes = [CryptoMode::Normal, CryptoMode::Suffix, CryptoMode::Lite]; let modes = [CryptoMode::Normal, CryptoMode::Suffix, CryptoMode::Lite];
let mut pkt = MutableRtpPacket::new(&mut buf[..]).unwrap(); let mut pkt = MutableRtpPacket::new(&mut buf[..]).unwrap();

1
src/driver/decode_mode.rs
View File

@@ -26,6 +26,7 @@ pub enum DecodeMode {
impl DecodeMode { impl DecodeMode {
/// Returns whether this mode will decrypt received packets. /// Returns whether this mode will decrypt received packets.
#[must_use]
pub fn should_decrypt(self) -> bool { pub fn should_decrypt(self) -> bool {
self != DecodeMode::Pass self != DecodeMode::Pass
} }

59
src/driver/mix_mode.rs Normal file
View File

@@ -0,0 +1,59 @@
use audiopus::Channels;
use symphonia_core::audio::Layout;
use crate::constants::{MONO_FRAME_SIZE, STEREO_FRAME_SIZE};
/// Mixing behaviour for sent audio sources processed within the driver.
///
/// This has no impact on Opus packet passthrough, which will pass packets
/// irrespective of their channel count.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum MixMode {
/// Audio sources will be downmixed into a mono buffer.
Mono,
/// Audio sources will be mixed into into a stereo buffer, where mono sources
/// will be duplicated into both channels.
Stereo,
}
impl MixMode {
pub(crate) const fn to_opus(self) -> Channels {
match self {
Self::Mono => Channels::Mono,
Self::Stereo => Channels::Stereo,
}
}
pub(crate) const fn sample_count_in_frame(self) -> usize {
match self {
Self::Mono => MONO_FRAME_SIZE,
Self::Stereo => STEREO_FRAME_SIZE,
}
}
pub(crate) const fn channels(self) -> usize {
match self {
Self::Mono => 1,
Self::Stereo => 2,
}
}
pub(crate) const fn symph_layout(self) -> Layout {
match self {
Self::Mono => Layout::Mono,
Self::Stereo => Layout::Stereo,
}
}
}
impl From<MixMode> for Layout {
fn from(val: MixMode) -> Self {
val.symph_layout()
}
}
impl From<MixMode> for Channels {
fn from(val: MixMode) -> Self {
val.to_opus()
}
}

145
src/driver/mod.rs
View File

@@ -14,20 +14,26 @@ pub mod bench_internals;
pub(crate) mod connection; pub(crate) mod connection;
mod crypto; mod crypto;
mod decode_mode; mod decode_mode;
mod mix_mode;
pub mod retry; pub mod retry;
pub(crate) mod tasks; pub(crate) mod tasks;
#[cfg(test)]
pub(crate) mod test_config;
use connection::error::{Error, Result}; use connection::error::{Error, Result};
pub use crypto::CryptoMode; pub use crypto::CryptoMode;
pub(crate) use crypto::CryptoState; pub(crate) use crypto::CryptoState;
pub use decode_mode::DecodeMode; pub use decode_mode::DecodeMode;
pub use mix_mode::MixMode;
#[cfg(test)]
pub use test_config::*;
#[cfg(feature = "builtin-queue")] #[cfg(feature = "builtin-queue")]
use crate::tracks::TrackQueue; use crate::tracks::TrackQueue;
use crate::{ use crate::{
events::EventData, events::EventData,
input::Input, input::Input,
tracks::{self, Track, TrackHandle}, tracks::{Track, TrackHandle},
Config, Config,
ConnectionInfo, ConnectionInfo,
Event, Event,
@@ -41,6 +47,8 @@ use core::{
task::{Context, Poll}, task::{Context, Poll},
}; };
use flume::{r#async::RecvFut, SendError, Sender}; use flume::{r#async::RecvFut, SendError, Sender};
#[cfg(feature = "builtin-queue")]
use std::time::Duration;
use tasks::message::CoreMessage; use tasks::message::CoreMessage;
use tracing::instrument; use tracing::instrument;
@@ -54,8 +62,13 @@ pub struct Driver {
config: Config, config: Config,
self_mute: bool, self_mute: bool,
sender: Sender<CoreMessage>, sender: Sender<CoreMessage>,
// Making this an Option is an abhorrent hack to coerce the borrow checker
// into letting us have an &TrackQueue at the same time as an &mut Driver.
// This is probably preferable to cloning the driver: Arc<...> should be nonzero
// and if the compiler's smart we'll just codegen a pointer swap. It definitely makes
// use of NonZero.
#[cfg(feature = "builtin-queue")] #[cfg(feature = "builtin-queue")]
queue: TrackQueue, queue: Option<TrackQueue>,
} }
impl Driver { impl Driver {
@@ -63,6 +76,7 @@ impl Driver {
/// ///
/// This will create the core voice tasks in the background. /// This will create the core voice tasks in the background.
#[inline] #[inline]
#[must_use]
pub fn new(config: Config) -> Self { pub fn new(config: Config) -> Self {
let sender = Self::start_inner(config.clone()); let sender = Self::start_inner(config.clone());
@@ -71,7 +85,7 @@ impl Driver {
self_mute: false, self_mute: false,
sender, sender,
#[cfg(feature = "builtin-queue")] #[cfg(feature = "builtin-queue")]
queue: Default::default(), queue: Some(TrackQueue::default()),
} }
} }
@@ -136,63 +150,45 @@ impl Driver {
self.self_mute self.self_mute
} }
/// Plays audio from a source, returning a handle for further control. /// Plays audio from an input, returning a handle for further control.
/// #[instrument(skip(self, input))]
/// This can be a source created via [`ffmpeg`] or [`ytdl`]. pub fn play_input(&mut self, input: Input) -> TrackHandle {
/// self.play(input.into())
/// [`ffmpeg`]: crate::input::ffmpeg
/// [`ytdl`]: crate::input::ytdl
#[instrument(skip(self))]
pub fn play_source(&mut self, source: Input) -> TrackHandle {
let (player, handle) = super::create_player(source);
self.send(CoreMessage::AddTrack(player));
handle
} }
/// Plays audio from a source, returning a handle for further control. /// Plays audio from an input, returning a handle for further control.
/// ///
/// Unlike [`play_source`], this stops all other sources attached /// Unlike [`Self::play_input`], this stops all other inputs attached
/// to the channel. /// to the channel.
/// #[instrument(skip(self, input))]
/// [`play_source`]: Driver::play_source pub fn play_only_input(&mut self, input: Input) -> TrackHandle {
#[instrument(skip(self))] self.play_only(input.into())
pub fn play_only_source(&mut self, source: Input) -> TrackHandle {
let (player, handle) = super::create_player(source);
self.send(CoreMessage::SetTrack(Some(player)));
handle
} }
/// Plays audio from a [`Track`] object. /// Plays audio from a [`Track`] object.
/// ///
/// This will be one half of the return value of [`create_player`]. /// The main difference between this function and [`Self::play_input`] is
/// The main difference between this function and [`play_source`] is
/// that this allows for direct manipulation of the [`Track`] object /// that this allows for direct manipulation of the [`Track`] object
/// before it is passed over to the voice and mixing contexts. /// before it is passed over to the voice and mixing contexts.
/// #[instrument(skip(self, track))]
/// [`create_player`]: crate::tracks::create_player pub fn play(&mut self, track: Track) -> TrackHandle {
/// [`create_player`]: crate::tracks::Track let (handle, ctx) = track.into_context();
/// [`play_source`]: Driver::play_source self.send(CoreMessage::AddTrack(ctx));
#[instrument(skip(self))]
pub fn play(&mut self, track: Track) { handle
self.send(CoreMessage::AddTrack(track));
} }
/// Exclusively plays audio from a [`Track`] object. /// Exclusively plays audio from a [`Track`] object.
/// ///
/// This will be one half of the return value of [`create_player`]. /// As in [`Self::play_only_input`], this stops all other sources attached to the
/// As in [`play_only_source`], this stops all other sources attached to the /// channel. Like [`Self::play`], however, this allows for direct manipulation of the
/// channel. Like [`play`], however, this allows for direct manipulation of the
/// [`Track`] object before it is passed over to the voice and mixing contexts. /// [`Track`] object before it is passed over to the voice and mixing contexts.
/// #[instrument(skip(self, track))]
/// [`create_player`]: crate::tracks::create_player pub fn play_only(&mut self, track: Track) -> TrackHandle {
/// [`Track`]: crate::tracks::Track let (handle, ctx) = track.into_context();
/// [`play_only_source`]: Driver::play_only_source self.send(CoreMessage::SetTrack(Some(ctx)));
/// [`play`]: Driver::play
#[instrument(skip(self))] handle
pub fn play_only(&mut self, track: Track) {
self.send(CoreMessage::SetTrack(Some(track)));
} }
/// Sets the bitrate for encoding Opus packets sent along /// Sets the bitrate for encoding Opus packets sent along
@@ -204,20 +200,20 @@ impl Driver {
/// Alternatively, `Auto` and `Max` remain available. /// Alternatively, `Auto` and `Max` remain available.
#[instrument(skip(self))] #[instrument(skip(self))]
pub fn set_bitrate(&mut self, bitrate: Bitrate) { pub fn set_bitrate(&mut self, bitrate: Bitrate) {
self.send(CoreMessage::SetBitrate(bitrate)) self.send(CoreMessage::SetBitrate(bitrate));
} }
/// Stops playing audio from all sources, if any are set. /// Stops playing audio from all sources, if any are set.
#[instrument(skip(self))] #[instrument(skip(self))]
pub fn stop(&mut self) { pub fn stop(&mut self) {
self.send(CoreMessage::SetTrack(None)) self.send(CoreMessage::SetTrack(None));
} }
/// Sets the configuration for this driver (and parent `Call`, if applicable). /// Sets the configuration for this driver (and parent `Call`, if applicable).
#[instrument(skip(self))] #[instrument(skip(self))]
pub fn set_config(&mut self, config: Config) { pub fn set_config(&mut self, config: Config) {
self.config = config.clone(); self.config = config.clone();
self.send(CoreMessage::SetConfig(config)) self.send(CoreMessage::SetConfig(config));
} }
/// Returns a view of this driver's configuration. /// Returns a view of this driver's configuration.
@@ -237,7 +233,6 @@ impl Driver {
/// within the supplied function or closure. *Taking excess time could prevent /// within the supplied function or closure. *Taking excess time could prevent
/// timely sending of packets, causing audio glitches and delays*. /// timely sending of packets, causing audio glitches and delays*.
/// ///
/// [`Track`]: crate::tracks::Track
/// [`TrackEvent`]: crate::events::TrackEvent /// [`TrackEvent`]: crate::events::TrackEvent
/// [`EventContext`]: crate::events::EventContext /// [`EventContext`]: crate::events::EventContext
#[instrument(skip(self, action))] #[instrument(skip(self, action))]
@@ -267,41 +262,53 @@ impl Driver {
/// Returns a reference to this driver's built-in queue. /// Returns a reference to this driver's built-in queue.
/// ///
/// Requires the `"builtin-queue"` feature. /// Requires the `"builtin-queue"` feature.
/// Queue additions should be made via [`enqueue`] and /// Queue additions should be made via [`Driver::enqueue`] and
/// [`enqueue_source`]. /// [`Driver::enqueue_input`].
/// #[must_use]
/// [`enqueue`]: Driver::enqueue
/// [`enqueue_source`]: Driver::enqueue_source
pub fn queue(&self) -> &TrackQueue { pub fn queue(&self) -> &TrackQueue {
&self.queue self.queue
.as_ref()
.expect("Queue: The only case this can fail is if a previous queue operation panicked.")
} }
/// Adds an audio [`Input`] to this driver's built-in queue. /// Adds an audio [`Input`] to this driver's built-in queue.
/// ///
/// Requires the `"builtin-queue"` feature. /// Requires the `"builtin-queue"` feature.
/// pub async fn enqueue_input(&mut self, input: Input) -> TrackHandle {
/// [`Input`]: crate::input::Input self.enqueue(input.into()).await
pub fn enqueue_source(&mut self, source: Input) -> TrackHandle {
let (track, handle) = tracks::create_player(source);
self.enqueue(track);
handle
} }
/// Adds an existing [`Track`] to this driver's built-in queue. /// Adds an existing [`Track`] to this driver's built-in queue.
/// ///
/// Requires the `"builtin-queue"` feature. /// Requires the `"builtin-queue"` feature.
pub async fn enqueue(&mut self, mut track: Track) -> TrackHandle {
let preload_time = TrackQueue::get_preload_time(&mut track).await;
self.enqueue_with_preload(track, preload_time)
}
/// Add an existing [`Track`] to the queue, using a known time to preload the next track.
/// ///
/// [`Track`]: crate::tracks::Track /// See [`TrackQueue::add_with_preload`] for how `preload_time` is used.
pub fn enqueue(&mut self, mut track: Track) { ///
self.queue.add_raw(&mut track); /// Requires the `"builtin-queue"` feature.
self.play(track); pub fn enqueue_with_preload(
&mut self,
track: Track,
preload_time: Option<Duration>,
) -> TrackHandle {
let queue = self.queue.take().expect(
"Enqueue: The only case this can fail is if a previous queue operation panicked.",
);
let handle = queue.add_with_preload(track, self, preload_time);
self.queue = Some(queue);
handle
} }
} }
impl Default for Driver { impl Default for Driver {
fn default() -> Self { fn default() -> Self {
Self::new(Default::default()) Self::new(Config::default())
} }
} }
@@ -309,7 +316,7 @@ impl Drop for Driver {
/// Leaves the current connected voice channel, if connected to one, and /// Leaves the current connected voice channel, if connected to one, and
/// forgets all configurations relevant to this Handler. /// forgets all configurations relevant to this Handler.
fn drop(&mut self) { fn drop(&mut self) {
let _ = self.sender.send(CoreMessage::Poison); drop(self.sender.send(CoreMessage::Poison));
} }
} }
@@ -317,8 +324,6 @@ impl Drop for Driver {
/// ///
/// This future awaits the *result* of a connection; the driver /// This future awaits the *result* of a connection; the driver
/// is messaged at the time of the call. /// is messaged at the time of the call.
///
/// [`Driver::connect`]: Driver::connect
pub struct Connect { pub struct Connect {
inner: RecvFut<'static, Result<()>>, inner: RecvFut<'static, Result<()>>,
} }

4
src/driver/retry/mod.rs
View File

@@ -28,7 +28,7 @@ pub struct Retry {
impl Default for Retry { impl Default for Retry {
fn default() -> Self { fn default() -> Self {
Self { Self {
strategy: Strategy::Backoff(Default::default()), strategy: Strategy::Backoff(ExponentialBackoff::default()),
retry_limit: Some(5), retry_limit: Some(5),
} }
} }
@@ -40,7 +40,7 @@ impl Retry {
last_wait: Option<Duration>, last_wait: Option<Duration>,
attempts: usize, attempts: usize,
) -> Option<Duration> { ) -> Option<Duration> {
if self.retry_limit.map(|a| attempts < a).unwrap_or(true) { if self.retry_limit.map_or(true, |a| attempts < a) {
Some(self.strategy.retry_in(last_wait)) Some(self.strategy.retry_in(last_wait))
} else { } else {
None None

2
src/driver/retry/strategy.rs
View File

@@ -58,7 +58,7 @@ impl Default for ExponentialBackoff {
impl ExponentialBackoff { impl ExponentialBackoff {
pub(crate) fn retry_in(&self, last_wait: Option<Duration>) -> Duration { pub(crate) fn retry_in(&self, last_wait: Option<Duration>) -> Duration {
let attempt = last_wait.map(|t| 2 * t).unwrap_or(self.min); let attempt = last_wait.map_or(self.min, |t| 2 * t);
let perturb = (1.0 - (self.jitter * 2.0 * (random::<f32>() - 1.0))) let perturb = (1.0 - (self.jitter * 2.0 * (random::<f32>() - 1.0)))
.max(0.0) .max(0.0)
.min(2.0); .min(2.0);

7
src/driver/tasks/disposal.rs
View File

@@ -9,10 +9,5 @@ use tracing::instrument;
/// to prevent deadline misses. /// to prevent deadline misses.
#[instrument(skip(mix_rx))] #[instrument(skip(mix_rx))]
pub(crate) fn runner(mix_rx: Receiver<DisposalMessage>) { pub(crate) fn runner(mix_rx: Receiver<DisposalMessage>) {
loop { while mix_rx.recv().is_ok() {}
match mix_rx.recv() {
Err(_) | Ok(DisposalMessage::Poison) => break,
_ => {},
}
}
} }

4
src/driver/tasks/error.rs
View File

@@ -32,9 +32,7 @@ impl Error {
pub(crate) fn should_trigger_connect(&self) -> bool { pub(crate) fn should_trigger_connect(&self) -> bool {
matches!( matches!(
self, self,
Error::InterconnectFailure(Recipient::AuxNetwork) Error::InterconnectFailure(Recipient::AuxNetwork | Recipient::UdpRx | Recipient::UdpTx)
| Error::InterconnectFailure(Recipient::UdpRx)
| Error::InterconnectFailure(Recipient::UdpTx)
) )
} }

84
src/driver/tasks/events.rs
View File

@@ -1,7 +1,7 @@
use super::message::*; use super::message::*;
use crate::{ use crate::{
events::{EventStore, GlobalEvents, TrackEvent}, events::{EventStore, GlobalEvents, TrackEvent},
tracks::{TrackHandle, TrackState}, tracks::{ReadyState, TrackHandle, TrackState},
}; };
use flume::Receiver; use flume::Receiver;
use tracing::{debug, info, instrument, trace}; use tracing::{debug, info, instrument, trace};
@@ -14,14 +14,13 @@ pub(crate) async fn runner(_interconnect: Interconnect, evt_rx: Receiver<EventMe
let mut states: Vec<TrackState> = vec![]; let mut states: Vec<TrackState> = vec![];
let mut handles: Vec<TrackHandle> = vec![]; let mut handles: Vec<TrackHandle> = vec![];
loop { while let Ok(msg) = evt_rx.recv_async().await {
use EventMessage::*; match msg {
match evt_rx.recv_async().await { EventMessage::AddGlobalEvent(data) => {
Ok(AddGlobalEvent(data)) => {
info!("Global event added."); info!("Global event added.");
global.add_event(data); global.add_event(data);
}, },
Ok(AddTrackEvent(i, data)) => { EventMessage::AddTrackEvent(i, data) => {
info!("Adding event to track {}.", i); info!("Adding event to track {}.", i);
let event_store = events let event_store = events
@@ -33,7 +32,7 @@ pub(crate) async fn runner(_interconnect: Interconnect, evt_rx: Receiver<EventMe
event_store.add_event(data, state.position); event_store.add_event(data, state.position);
}, },
Ok(FireCoreEvent(ctx)) => { EventMessage::FireCoreEvent(ctx) => {
let ctx = ctx.to_user_context(); let ctx = ctx.to_user_context();
let evt = ctx let evt = ctx
.to_core_event() .to_core_event()
@@ -43,19 +42,17 @@ pub(crate) async fn runner(_interconnect: Interconnect, evt_rx: Receiver<EventMe
global.fire_core_event(evt, ctx).await; global.fire_core_event(evt, ctx).await;
}, },
Ok(RemoveGlobalEvents) => { EventMessage::RemoveGlobalEvents => {
global.remove_handlers(); global.remove_handlers();
}, },
Ok(AddTrack(store, state, handle)) => { EventMessage::AddTrack(store, state, handle) => {
events.push(store); events.push(store);
states.push(state); states.push(state);
handles.push(handle); handles.push(handle);
info!("Event state for track {} added", events.len()); info!("Event state for track {} added", events.len());
}, },
Ok(ChangeState(i, change)) => { EventMessage::ChangeState(i, change) => {
use TrackStateChange::*;
let max_states = states.len(); let max_states = states.len();
debug!( debug!(
"Changing state for track {} of {}: {:?}", "Changing state for track {} of {}: {:?}",
@@ -67,53 +64,74 @@ pub(crate) async fn runner(_interconnect: Interconnect, evt_rx: Receiver<EventMe
.expect("Event thread was given an illegal state index for ChangeState."); .expect("Event thread was given an illegal state index for ChangeState.");
match change { match change {
Mode(mode) => { TrackStateChange::Mode(mut mode) => {
let old = state.playing; std::mem::swap(&mut state.playing, &mut mode);
state.playing = mode; if state.playing != mode {
if old != mode { global.fire_track_event(state.playing.as_track_event(), i);
global.fire_track_event(mode.as_track_event(), i); if let Some(other_evts) = state.playing.also_fired_track_events() {
for evt in other_evts {
global.fire_track_event(evt, i);
}
}
} }
}, },
Volume(vol) => { TrackStateChange::Volume(vol) => {
state.volume = vol; state.volume = vol;
}, },
Position(pos) => { TrackStateChange::Position(pos) => {
// Currently, only Tick should fire time events. // Currently, only Tick should fire time events.
state.position = pos; state.position = pos;
}, },
Loops(loops, user_set) => { TrackStateChange::Loops(loops, user_set) => {
state.loops = loops; state.loops = loops;
if !user_set { if !user_set {
global.fire_track_event(TrackEvent::Loop, i); global.fire_track_event(TrackEvent::Loop, i);
} }
}, },
Total(new) => { TrackStateChange::Total(new) => {
// Massive, unprecedented state changes. // Massive, unprecedented state changes.
*state = new; *state = new;
}, },
TrackStateChange::Ready(ready_state) => {
state.ready = ready_state;
match ready_state {
ReadyState::Playable => {
global.fire_track_event(TrackEvent::Playable, i);
},
ReadyState::Preparing => {
global.fire_track_event(TrackEvent::Preparing, i);
},
ReadyState::Uninitialised => {},
}
},
} }
}, },
Ok(RemoveTrack(i)) => { EventMessage::RemoveAllTracks => {
info!("Event state for track {} of {} removed.", i, events.len());
events.swap_remove(i);
states.swap_remove(i);
handles.swap_remove(i);
},
Ok(RemoveAllTracks) => {
info!("Event state for all tracks removed."); info!("Event state for all tracks removed.");
events.clear(); events.clear();
states.clear(); states.clear();
handles.clear(); handles.clear();
}, },
Ok(Tick) => { EventMessage::Tick => {
// NOTE: this should fire saved up blocks of state change evts. // NOTE: this should fire saved up blocks of state change evts.
global.tick(&mut events, &mut states, &mut handles).await; global.tick(&mut events, &mut states, &mut handles).await;
let mut i = 0;
while i < states.len() {
if states[i].playing.is_done() {
info!("Event state for track {} of {} removed.", i, events.len());
events.swap_remove(i);
states.swap_remove(i);
handles.swap_remove(i);
} else {
i += 1;
}
}
}, },
Err(_) | Ok(Poison) => { EventMessage::Poison => break,
break;
},
} }
} }

16
src/driver/tasks/message/core.rs
View File

@@ -3,20 +3,18 @@
use crate::{ use crate::{
driver::{connection::error::Error, Bitrate, Config}, driver::{connection::error::Error, Bitrate, Config},
events::{context_data::DisconnectReason, EventData}, events::{context_data::DisconnectReason, EventData},
tracks::Track, tracks::{Track, TrackCommand, TrackHandle},
ConnectionInfo, ConnectionInfo,
}; };
use flume::Sender; use flume::{Receiver, Sender};
#[allow(clippy::large_enum_variant)]
#[derive(Debug)]
pub enum CoreMessage { pub enum CoreMessage {
ConnectWithResult(ConnectionInfo, Sender<Result<(), Error>>), ConnectWithResult(ConnectionInfo, Sender<Result<(), Error>>),
RetryConnect(usize), RetryConnect(usize),
SignalWsClosure(usize, ConnectionInfo, Option<DisconnectReason>), SignalWsClosure(usize, ConnectionInfo, Option<DisconnectReason>),
Disconnect, Disconnect,
SetTrack(Option<Track>), SetTrack(Option<TrackContext>),
AddTrack(Track), AddTrack(TrackContext),
SetBitrate(Bitrate), SetBitrate(Bitrate),
AddEvent(EventData), AddEvent(EventData),
RemoveGlobalEvents, RemoveGlobalEvents,
@@ -27,3 +25,9 @@ pub enum CoreMessage {
RebuildInterconnect, RebuildInterconnect,
Poison, Poison,
} }
pub struct TrackContext {
pub track: Track,
pub handle: TrackHandle,
pub receiver: Receiver<TrackCommand>,
}

7
src/driver/tasks/message/disposal.rs
View File

@@ -1,9 +1,8 @@
#![allow(missing_docs)] #![allow(missing_docs)]
use crate::tracks::Track; use crate::{driver::tasks::mixer::InternalTrack, tracks::TrackHandle};
pub enum DisposalMessage { pub enum DisposalMessage {
Track(Track), Track(Box<InternalTrack>),
Handle(TrackHandle),
Poison,
} }

4
src/driver/tasks/message/events.rs
View File

@@ -2,7 +2,7 @@
use crate::{ use crate::{
events::{CoreContext, EventData, EventStore}, events::{CoreContext, EventData, EventStore},
tracks::{LoopState, PlayMode, TrackHandle, TrackState}, tracks::{LoopState, PlayMode, ReadyState, TrackHandle, TrackState},
}; };
use std::time::Duration; use std::time::Duration;
@@ -16,7 +16,6 @@ pub enum EventMessage {
AddTrack(EventStore, TrackState, TrackHandle), AddTrack(EventStore, TrackState, TrackHandle),
ChangeState(usize, TrackStateChange), ChangeState(usize, TrackStateChange),
RemoveTrack(usize),
RemoveAllTracks, RemoveAllTracks,
Tick, Tick,
@@ -31,4 +30,5 @@ pub enum TrackStateChange {
// Bool indicates user-set. // Bool indicates user-set.
Loops(LoopState, bool), Loops(LoopState, bool),
Total(TrackState), Total(TrackState),
Ready(ReadyState),
} }

28
src/driver/tasks/message/mixer.rs
View File

@@ -1,12 +1,14 @@
#![allow(missing_docs)] #![allow(missing_docs)]
use super::{Interconnect, UdpRxMessage, UdpTxMessage, WsMessage}; use super::{Interconnect, TrackContext, UdpRxMessage, UdpTxMessage, WsMessage};
use crate::{ use crate::{
driver::{Bitrate, Config, CryptoState}, driver::{Bitrate, Config, CryptoState},
tracks::Track, input::{AudioStreamError, Compose, Parsed},
}; };
use flume::Sender; use flume::Sender;
use std::sync::Arc;
use symphonia_core::{errors::Error as SymphoniaError, formats::SeekedTo};
use xsalsa20poly1305::XSalsa20Poly1305 as Cipher; use xsalsa20poly1305::XSalsa20Poly1305 as Cipher;
pub struct MixerConnection { pub struct MixerConnection {
@@ -16,16 +18,9 @@ pub struct MixerConnection {
pub udp_tx: Sender<UdpTxMessage>, pub udp_tx: Sender<UdpTxMessage>,
} }
impl Drop for MixerConnection {
fn drop(&mut self) {
let _ = self.udp_rx.send(UdpRxMessage::Poison);
let _ = self.udp_tx.send(UdpTxMessage::Poison);
}
}
pub enum MixerMessage { pub enum MixerMessage {
AddTrack(Track), AddTrack(TrackContext),
SetTrack(Option<Track>), SetTrack(Option<TrackContext>),
SetBitrate(Bitrate), SetBitrate(Bitrate),
SetConfig(Config), SetConfig(Config),
@@ -40,3 +35,14 @@ pub enum MixerMessage {
Poison, Poison,
} }
pub enum MixerInputResultMessage {
CreateErr(Arc<AudioStreamError>),
ParseErr(Arc<SymphoniaError>),
Seek(
Parsed,
Option<Box<dyn Compose>>,
Result<SeekedTo, Arc<SymphoniaError>>,
),
Built(Parsed, Option<Box<dyn Compose>>),
}

11
src/driver/tasks/message/mod.rs
View File

@@ -23,11 +23,11 @@ pub struct Interconnect {
impl Interconnect { impl Interconnect {
pub fn poison(&self) { pub fn poison(&self) {
let _ = self.events.send(EventMessage::Poison); drop(self.events.send(EventMessage::Poison));
} }
pub fn poison_all(&self) { pub fn poison_all(&self) {
let _ = self.mixer.send(MixerMessage::Poison); drop(self.mixer.send(MixerMessage::Poison));
self.poison(); self.poison();
} }
@@ -46,8 +46,9 @@ impl Interconnect {
}); });
// Make mixer aware of new targets... // Make mixer aware of new targets...
let _ = self drop(
.mixer self.mixer
.send(MixerMessage::ReplaceInterconnect(self.clone())); .send(MixerMessage::ReplaceInterconnect(self.clone())),
);
} }
} }

2
src/driver/tasks/message/udp_rx.rs
View File

@@ -6,6 +6,4 @@ use crate::driver::Config;
pub enum UdpRxMessage { pub enum UdpRxMessage {
SetConfig(Config), SetConfig(Config),
ReplaceInterconnect(Interconnect), ReplaceInterconnect(Interconnect),
Poison,
} }

6
src/driver/tasks/message/udp_tx.rs
View File

@@ -1,6 +1,4 @@
#![allow(missing_docs)] #![allow(missing_docs)]
pub enum UdpTxMessage { // TODO: do something cheaper.
Packet(Vec<u8>), // TODO: do something cheaper. pub type UdpTxMessage = Vec<u8>;
Poison,
}

3
src/driver/tasks/message/ws.rs
View File

@@ -3,12 +3,9 @@
use super::Interconnect; use super::Interconnect;
use crate::ws::WsStream; use crate::ws::WsStream;
#[allow(dead_code)]
pub enum WsMessage { pub enum WsMessage {
Ws(Box<WsStream>), Ws(Box<WsStream>),
ReplaceInterconnect(Interconnect), ReplaceInterconnect(Interconnect),
SetKeepalive(f64), SetKeepalive(f64),
Speaking(bool), Speaking(bool),
Poison,
} }

628
src/driver/tasks/mixer.rs
View File

@@ -1,628 +0,0 @@
use super::{disposal, error::Result, message::*};
use crate::{
constants::*,
tracks::{PlayMode, Track},
Config,
};
use audiopus::{
coder::Encoder as OpusEncoder,
softclip::SoftClip,
Application as CodingMode,
Bitrate,
Channels,
};
use discortp::{
rtp::{MutableRtpPacket, RtpPacket},
MutablePacket,
};
use flume::{Receiver, Sender, TryRecvError};
use rand::random;
use std::{convert::TryInto, time::Instant};
use tokio::runtime::Handle;
use tracing::{debug, error, instrument};
use xsalsa20poly1305::TAG_SIZE;
pub struct Mixer {
pub async_handle: Handle,
pub bitrate: Bitrate,
pub config: Config,
pub conn_active: Option<MixerConnection>,
pub deadline: Instant,
pub disposer: Sender<DisposalMessage>,
pub encoder: OpusEncoder,
pub interconnect: Interconnect,
pub mix_rx: Receiver<MixerMessage>,
pub muted: bool,
pub packet: [u8; VOICE_PACKET_MAX],
pub prevent_events: bool,
pub silence_frames: u8,
pub skip_sleep: bool,
pub soft_clip: SoftClip,
pub tracks: Vec<Track>,
pub ws: Option<Sender<WsMessage>>,
}
fn new_encoder(bitrate: Bitrate) -> Result<OpusEncoder> {
let mut encoder = OpusEncoder::new(SAMPLE_RATE, Channels::Stereo, CodingMode::Audio)?;
encoder.set_bitrate(bitrate)?;
Ok(encoder)
}
impl Mixer {
pub fn new(
mix_rx: Receiver<MixerMessage>,
async_handle: Handle,
interconnect: Interconnect,
config: Config,
) -> Self {
let bitrate = DEFAULT_BITRATE;
let encoder = new_encoder(bitrate)
.expect("Failed to create encoder in mixing thread with known-good values.");
let soft_clip = SoftClip::new(Channels::Stereo);
let mut packet = [0u8; VOICE_PACKET_MAX];
let mut rtp = MutableRtpPacket::new(&mut packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_version(RTP_VERSION);
rtp.set_payload_type(RTP_PROFILE_TYPE);
rtp.set_sequence(random::<u16>().into());
rtp.set_timestamp(random::<u32>().into());
let tracks = Vec::with_capacity(1.max(config.preallocated_tracks));
// Create an object disposal thread here.
let (disposer, disposal_rx) = flume::unbounded();
std::thread::spawn(move || disposal::runner(disposal_rx));
Self {
async_handle,
bitrate,
config,
conn_active: None,
deadline: Instant::now(),
disposer,
encoder,
interconnect,
mix_rx,
muted: false,
packet,
prevent_events: false,
silence_frames: 0,
skip_sleep: false,
soft_clip,
tracks,
ws: None,
}
}
fn run(&mut self) {
let mut events_failure = false;
let mut conn_failure = false;
'runner: loop {
if self.conn_active.is_some() {
loop {
match self.mix_rx.try_recv() {
Ok(m) => {
let (events, conn, should_exit) = self.handle_message(m);
events_failure |= events;
conn_failure |= conn;
if should_exit {
break 'runner;
}
},
Err(TryRecvError::Disconnected) => {
break 'runner;
},
Err(TryRecvError::Empty) => {
break;
},
};
}
// The above action may have invalidated the connection; need to re-check!
if self.conn_active.is_some() {
if let Err(e) = self.cycle().and_then(|_| self.audio_commands_events()) {
events_failure |= e.should_trigger_interconnect_rebuild();
conn_failure |= e.should_trigger_connect();
debug!("Mixer thread cycle: {:?}", e);
}
}
} else {
match self.mix_rx.recv() {
Ok(m) => {
let (events, conn, should_exit) = self.handle_message(m);
events_failure |= events;
conn_failure |= conn;
if should_exit {
break 'runner;
}
},
Err(_) => {
break 'runner;
},
}
}
// event failure? rebuild interconnect.
// ws or udp failure? full connect
// (soft reconnect is covered by the ws task.)
//
// in both cases, send failure is fatal,
// but will only occur on disconnect.
// expecting this is fairly noisy, so exit silently.
if events_failure {
self.prevent_events = true;
let sent = self
.interconnect
.core
.send(CoreMessage::RebuildInterconnect);
events_failure = false;
if sent.is_err() {
break;
}
}
if conn_failure {
self.conn_active = None;
let sent = self.interconnect.core.send(CoreMessage::FullReconnect);
conn_failure = false;
if sent.is_err() {
break;
}
}
}
}
#[inline]
fn handle_message(&mut self, msg: MixerMessage) -> (bool, bool, bool) {
let mut events_failure = false;
let mut conn_failure = false;
let mut should_exit = false;
use MixerMessage::*;
let error = match msg {
AddTrack(mut t) => {
t.source.prep_with_handle(self.async_handle.clone());
self.add_track(t)
},
SetTrack(t) => {
self.tracks.clear();
let mut out = self.fire_event(EventMessage::RemoveAllTracks);
if let Some(mut t) = t {
t.source.prep_with_handle(self.async_handle.clone());
// Do this unconditionally: this affects local state infallibly,
// with the event installation being the remote part.
if let Err(e) = self.add_track(t) {
out = Err(e);
}
}
out
},
SetBitrate(b) => {
self.bitrate = b;
if let Err(e) = self.set_bitrate(b) {
error!("Failed to update bitrate {:?}", e);
}
Ok(())
},
SetMute(m) => {
self.muted = m;
Ok(())
},
SetConn(conn, ssrc) => {
self.conn_active = Some(conn);
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_ssrc(ssrc);
rtp.set_sequence(random::<u16>().into());
rtp.set_timestamp(random::<u32>().into());
self.deadline = Instant::now();
Ok(())
},
DropConn => {
self.conn_active = None;
Ok(())
},
ReplaceInterconnect(i) => {
self.prevent_events = false;
if let Some(ws) = &self.ws {
conn_failure |= ws.send(WsMessage::ReplaceInterconnect(i.clone())).is_err();
}
if let Some(conn) = &self.conn_active {
conn_failure |= conn
.udp_rx
.send(UdpRxMessage::ReplaceInterconnect(i.clone()))
.is_err();
}
self.interconnect = i;
self.rebuild_tracks()
},
SetConfig(new_config) => {
self.config = new_config.clone();
if self.tracks.capacity() < self.config.preallocated_tracks {
self.tracks
.reserve(self.config.preallocated_tracks - self.tracks.len());
}
if let Some(conn) = &self.conn_active {
conn_failure |= conn
.udp_rx
.send(UdpRxMessage::SetConfig(new_config))
.is_err();
}
Ok(())
},
RebuildEncoder => match new_encoder(self.bitrate) {
Ok(encoder) => {
self.encoder = encoder;
Ok(())
},
Err(e) => {
error!("Failed to rebuild encoder. Resetting bitrate. {:?}", e);
self.bitrate = DEFAULT_BITRATE;
self.encoder = new_encoder(self.bitrate)
.expect("Failed fallback rebuild of OpusEncoder with safe inputs.");
Ok(())
},
},
Ws(new_ws_handle) => {
self.ws = new_ws_handle;
Ok(())
},
Poison => {
should_exit = true;
Ok(())
},
};
if let Err(e) = error {
events_failure |= e.should_trigger_interconnect_rebuild();
conn_failure |= e.should_trigger_connect();
}
(events_failure, conn_failure, should_exit)
}
#[inline]
fn fire_event(&self, event: EventMessage) -> Result<()> {
// As this task is responsible for noticing the potential death of an event context,
// it's responsible for not forcibly recreating said context repeatedly.
if !self.prevent_events {
self.interconnect.events.send(event)?;
Ok(())
} else {
Ok(())
}
}
#[inline]
fn add_track(&mut self, mut track: Track) -> Result<()> {
let evts = track.events.take().unwrap_or_default();
let state = track.state();
let handle = track.handle.clone();
self.tracks.push(track);
self.interconnect
.events
.send(EventMessage::AddTrack(evts, state, handle))?;
Ok(())
}
// rebuilds the event thread's view of each track, in event of a full rebuild.
#[inline]
fn rebuild_tracks(&mut self) -> Result<()> {
for track in self.tracks.iter_mut() {
let evts = track.events.take().unwrap_or_default();
let state = track.state();
let handle = track.handle.clone();
self.interconnect
.events
.send(EventMessage::AddTrack(evts, state, handle))?;
}
Ok(())
}
#[inline]
fn audio_commands_events(&mut self) -> Result<()> {
// Apply user commands.
for (i, track) in self.tracks.iter_mut().enumerate() {
// This causes fallible event system changes,
// but if the event thread has died then we'll certainly
// detect that on the tick later.
// Changes to play state etc. MUST all be handled.
track.process_commands(i, &self.interconnect);
}
// TODO: do without vec?
let mut i = 0;
let mut to_remove = Vec::with_capacity(self.tracks.len());
while i < self.tracks.len() {
let track = self
.tracks
.get_mut(i)
.expect("Tried to remove an illegal track index.");
if track.playing.is_done() {
let p_state = track.playing();
let to_drop = self.tracks.swap_remove(i);
to_remove.push(i);
self.fire_event(EventMessage::ChangeState(
i,
TrackStateChange::Mode(p_state),
))?;
let _ = self.disposer.send(DisposalMessage::Track(to_drop));
} else {
i += 1;
}
}
// Tick
self.fire_event(EventMessage::Tick)?;
// Then do removals.
for i in &to_remove[..] {
self.fire_event(EventMessage::RemoveTrack(*i))?;
}
Ok(())
}
#[inline]
fn march_deadline(&mut self) {
if self.skip_sleep {
return;
}
std::thread::sleep(self.deadline.saturating_duration_since(Instant::now()));
self.deadline += TIMESTEP_LENGTH;
}
pub fn cycle(&mut self) -> Result<()> {
let mut mix_buffer = [0f32; STEREO_FRAME_SIZE];
// Walk over all the audio files, combining into one audio frame according
// to volume, play state, etc.
let mut mix_len = {
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
// self.mix_tracks(&mut payload[TAG_SIZE..], &mut mix_buffer)
mix_tracks(
&mut payload[TAG_SIZE..],
&mut mix_buffer,
&mut self.tracks,
&self.interconnect,
self.prevent_events,
)
};
self.soft_clip.apply((&mut mix_buffer[..]).try_into()?)?;
if self.muted {
mix_len = MixType::MixedPcm(0);
}
if mix_len == MixType::MixedPcm(0) {
if self.silence_frames > 0 {
self.silence_frames -= 1;
// Explicit "Silence" frame.
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
(&mut payload[TAG_SIZE..TAG_SIZE + SILENT_FRAME.len()])
.copy_from_slice(&SILENT_FRAME[..]);
mix_len = MixType::Passthrough(SILENT_FRAME.len());
} else {
// Per official guidelines, send 5x silence BEFORE we stop speaking.
if let Some(ws) = &self.ws {
// NOTE: this should prevent a catastrophic thread pileup.
// A full reconnect might cause an inner closed connection.
// It's safer to leave the central task to clean this up and
// pass the mixer a new channel.
let _ = ws.send(WsMessage::Speaking(false));
}
self.march_deadline();
return Ok(());
}
} else {
self.silence_frames = 5;
}
if let Some(ws) = &self.ws {
ws.send(WsMessage::Speaking(true))?;
}
self.march_deadline();
self.prep_and_send_packet(mix_buffer, mix_len)?;
Ok(())
}
fn set_bitrate(&mut self, bitrate: Bitrate) -> Result<()> {
self.encoder.set_bitrate(bitrate).map_err(Into::into)
}
#[inline]
fn prep_and_send_packet(&mut self, buffer: [f32; 1920], mix_len: MixType) -> Result<()> {
let conn = self
.conn_active
.as_mut()
.expect("Shouldn't be mixing packets without access to a cipher + UDP dest.");
let index = {
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
let crypto_mode = conn.crypto_state.kind();
let payload_len = match mix_len {
MixType::Passthrough(opus_len) => opus_len,
MixType::MixedPcm(_samples) => {
let total_payload_space = payload.len() - crypto_mode.payload_suffix_len();
self.encoder.encode_float(
&buffer[..STEREO_FRAME_SIZE],
&mut payload[TAG_SIZE..total_payload_space],
)?
},
};
let final_payload_size = conn
.crypto_state
.write_packet_nonce(&mut rtp, TAG_SIZE + payload_len);
conn.crypto_state.kind().encrypt_in_place(
&mut rtp,
&conn.cipher,
final_payload_size,
)?;
RtpPacket::minimum_packet_size() + final_payload_size
};
// TODO: This is dog slow, don't do this.
// Can we replace this with a shared ring buffer + semaphore?
// i.e., do something like double/triple buffering in graphics.
conn.udp_tx
.send(UdpTxMessage::Packet(self.packet[..index].to_vec()))?;
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_sequence(rtp.get_sequence() + 1);
rtp.set_timestamp(rtp.get_timestamp() + MONO_FRAME_SIZE as u32);
Ok(())
}
}
#[derive(Debug, Eq, PartialEq)]
enum MixType {
Passthrough(usize),
MixedPcm(usize),
}
#[inline]
fn mix_tracks<'a>(
opus_frame: &'a mut [u8],
mix_buffer: &mut [f32; STEREO_FRAME_SIZE],
tracks: &mut Vec<Track>,
interconnect: &Interconnect,
prevent_events: bool,
) -> MixType {
let mut len = 0;
// Opus frame passthrough.
// This requires that we have only one track, who has volume 1.0, and an
// Opus codec type.
let do_passthrough = tracks.len() == 1 && {
let track = &tracks[0];
(track.volume - 1.0).abs() < f32::EPSILON && track.source.supports_passthrough()
};
for (i, track) in tracks.iter_mut().enumerate() {
let vol = track.volume;
let stream = &mut track.source;
if track.playing != PlayMode::Play {
continue;
}
let (temp_len, opus_len) = if do_passthrough {
(0, track.source.read_opus_frame(opus_frame).ok())
} else {
(stream.mix(mix_buffer, vol), None)
};
len = len.max(temp_len);
if temp_len > 0 || opus_len.is_some() {
track.step_frame();
} else if track.do_loop() {
if let Ok(time) = track.seek_time(Default::default()) {
// have to reproduce self.fire_event here
// to circumvent the borrow checker's lack of knowledge.
//
// In event of error, one of the later event calls will
// trigger the event thread rebuild: it is more prudent that
// the mixer works as normal right now.
if !prevent_events {
let _ = interconnect.events.send(EventMessage::ChangeState(
i,
TrackStateChange::Position(time),
));
let _ = interconnect.events.send(EventMessage::ChangeState(
i,
TrackStateChange::Loops(track.loops, false),
));
}
}
} else {
track.end();
}
if let Some(opus_len) = opus_len {
return MixType::Passthrough(opus_len);
}
}
MixType::MixedPcm(len)
}
/// The mixing thread is a synchronous context due to its compute-bound nature.
///
/// We pass in an async handle for the benefit of some Input classes (e.g., restartables)
/// who need to run their restart code elsewhere and return blank data until such time.
#[instrument(skip(interconnect, mix_rx, async_handle))]
pub(crate) fn runner(
interconnect: Interconnect,
mix_rx: Receiver<MixerMessage>,
async_handle: Handle,
config: Config,
) {
let mut mixer = Mixer::new(mix_rx, async_handle, interconnect, config);
mixer.run();
let _ = mixer.disposer.send(DisposalMessage::Poison);
}

444
src/driver/tasks/mixer/mix_logic.rs Normal file
View File

@@ -0,0 +1,444 @@
use super::*;
/// Mix a track's audio stream into either the shared mixing buffer, or directly into the output
/// packet ("passthrough") when possible.
///
/// Passthrough is highest performance, but the source MUST be opus, have 20ms frames, and be the only
/// live track. In this case we copy the opus-encoded data with no changes. Otherwise, we fall back to
/// below.
///
/// There are a few functional requirements here for non-passthrough mixing that make it tricky:
/// * Input frame lengths are not congruent with what we need to send (i.e., 26.12ms in MP3 vs
/// needed 20ms).
/// * Input audio arrives at a different sample rate from required (i.e., 44.1 vs needed 48 kHz).
/// * Input data may not be `f32`s.
/// * Input data may not match stereo/mono of desired output.
///
/// All of the above challenges often happen at once. The rough pipeline in processing is:
///
/// until source end or 20 ms taken:
/// (use previous frame 'til empty / get new frame) -> [resample] -> [audio += vol * (sample as f32)]
///
/// Typically, we mix between a subset of the input packet and the output buf because the 20ms window
/// straddles packet boundaries. If there's enough space AND 48kHz AND receive f32s, then we use a fast
/// path.
///
/// In the mono -> stereo case, we duplicate across all target channels. In stereo -> mono, we average
/// the samples from each channel.
///
/// To avoid needing to hold onto resampled data longer than one mix cycle, we take enough input samples
/// to fill a chunk of the mixer (e.g., 10ms == 20ms / 2) so that they will all be used.
///
/// This is a fairly annoying piece of code to reason about, mainly because you need to hold so many
/// internal positions into: the mix buffer, resample buffers, and previous/current packets
/// for a stream.
#[inline]
pub fn mix_symph_indiv(
// shared buffer to mix into.
symph_mix: &mut AudioBuffer<f32>,
// buffer to hold built up packet
resample_scratch: &mut AudioBuffer<f32>,
// the input stream to use
input: &mut Parsed,
// resampler state and positions into partially read packets
local_state: &mut DecodeState,
// volume of this source
volume: f32,
// window into the output UDP buffer to copy opus frames into.
// This is set to `Some` IF passthrough is possible (i.e., one live source).
mut opus_slot: Option<&mut [u8]>,
) -> (MixType, MixStatus) {
let mut samples_written = 0;
let mut resample_in_progress = false;
let mut track_status = MixStatus::Live;
let codec_type = input.decoder.codec_params().codec;
resample_scratch.clear();
while samples_written != MONO_FRAME_SIZE {
// fetch a packet: either in progress, passthrough (early exit), or
let source_packet = if local_state.inner_pos != 0 {
Some(input.decoder.last_decoded())
} else if let Ok(pkt) = input.format.next_packet() {
if pkt.track_id() != input.track_id {
continue;
}
let buf = pkt.buf();
// Opus packet passthrough special case.
if codec_type == CODEC_TYPE_OPUS && local_state.passthrough != Passthrough::Block {
if let Some(slot) = opus_slot.as_mut() {
let sample_ct = buf
.try_into()
.and_then(|buf| audiopus::packet::nb_samples(buf, SAMPLE_RATE));
// We don't actually block passthrough until a few violations are
// seen. The main one is that most Opus tracks end on a sub-20ms
// frame, particularly on Youtube.
// However, a frame that's bigger than the target buffer is an instant block.
let buf_size_fatal = buf.len() <= slot.len();
if match sample_ct {
Ok(MONO_FRAME_SIZE) => true,
_ => !local_state.record_and_check_passthrough_strike_final(buf_size_fatal),
} {
slot.write_all(buf)
.expect("Bounds check performed, and failure will block passthrough.");
return (MixType::Passthrough(buf.len()), MixStatus::Live);
}
}
}
input
.decoder
.decode(&pkt)
.map_err(|e| {
track_status = e.into();
})
.ok()
} else {
track_status = MixStatus::Ended;
None
};
// Cleanup: failed to get the next packet, but still have to convert and mix scratch.
if source_packet.is_none() {
if resample_in_progress {
// fill up remainder of buf with zeroes, resample, mix
let (chan_c, resampler, rs_out_buf) = local_state.resampler.as_mut().unwrap();
let in_len = resample_scratch.frames();
let to_render = resampler.input_frames_next().saturating_sub(in_len);
if to_render != 0 {
resample_scratch.render_reserved(Some(to_render));
for plane in resample_scratch.planes_mut().planes() {
for val in &mut plane[in_len..] {
*val = 0.0f32;
}
}
}
// Luckily, we make use of the WHOLE input buffer here.
resampler
.process_into_buffer(
&resample_scratch.planes().planes()[..*chan_c],
rs_out_buf,
None,
)
.unwrap();
// Calculate true end position using sample rate math
let ratio = (rs_out_buf[0].len() as f32) / (resample_scratch.frames() as f32);
let out_samples = (ratio * (in_len as f32)).round() as usize;
mix_resampled(rs_out_buf, symph_mix, samples_written, volume);
samples_written += out_samples;
}
break;
}
let source_packet = source_packet.unwrap();
let in_rate = source_packet.spec().rate;
if in_rate == SAMPLE_RATE_RAW as u32 {
// No need to resample: mix as standard.
let samples_marched = mix_over_ref(
&source_packet,
symph_mix,
local_state.inner_pos,
samples_written,
volume,
);
samples_written += samples_marched;
local_state.inner_pos += samples_marched;
local_state.inner_pos %= source_packet.frames();
} else {
// NOTE: this should NEVER change in one stream.
let chan_c = source_packet.spec().channels.count();
let (_, resampler, rs_out_buf) = local_state.resampler.get_or_insert_with(|| {
// TODO: integ. error handling here.
let resampler = FftFixedOut::new(
in_rate as usize,
SAMPLE_RATE_RAW,
RESAMPLE_OUTPUT_FRAME_SIZE,
4,
chan_c,
)
.expect("Failed to create resampler.");
let out_buf = resampler.output_buffer_allocate();
(chan_c, resampler, out_buf)
});
let inner_pos = local_state.inner_pos;
let pkt_frames = source_packet.frames();
if pkt_frames == 0 {
continue;
}
let needed_in_frames = resampler.input_frames_next();
let available_frames = pkt_frames - inner_pos;
let force_copy = resample_in_progress || needed_in_frames > available_frames;
if (!force_copy) && matches!(source_packet, AudioBufferRef::F32(_)) {
// This is the only case where we can pull off a straight resample...
// I would really like if this could be a slice of slices,
// but the technology just isn't there yet. And I don't feel like
// writing unsafe transformations to do so.
// NOTE: if let needed as if-let && {bool} is nightly only.
if let AudioBufferRef::F32(s_pkt) = source_packet {
let refs: Vec<&[f32]> = s_pkt
.planes()
.planes()
.iter()
.map(|s| &s[inner_pos..][..needed_in_frames])
.collect();
local_state.inner_pos += needed_in_frames;
local_state.inner_pos %= pkt_frames;
resampler
.process_into_buffer(&refs, rs_out_buf, None)
.unwrap();
} else {
unreachable!()
}
} else {
// We either lack enough samples, or have the wrong data format, forcing
// a conversion/copy into the buffer.
let old_scratch_len = resample_scratch.frames();
let missing_frames = needed_in_frames - old_scratch_len;
let frames_to_take = available_frames.min(missing_frames);
resample_scratch.render_reserved(Some(frames_to_take));
copy_into_resampler(
&source_packet,
resample_scratch,
inner_pos,
old_scratch_len,
frames_to_take,
);
local_state.inner_pos += frames_to_take;
local_state.inner_pos %= pkt_frames;
if resample_scratch.frames() == needed_in_frames {
resampler
.process_into_buffer(
&resample_scratch.planes().planes()[..chan_c],
rs_out_buf,
None,
)
.unwrap();
resample_scratch.clear();
resample_in_progress = false;
} else {
// Not enough data to fill the resampler: fetch more.
resample_in_progress = true;
continue;
}
};
let samples_marched = mix_resampled(rs_out_buf, symph_mix, samples_written, volume);
samples_written += samples_marched;
}
}
(MixType::MixedPcm(samples_written), track_status)
}
#[inline]
fn mix_over_ref(
source: &AudioBufferRef,
target: &mut AudioBuffer<f32>,
source_pos: usize,
dest_pos: usize,
volume: f32,
) -> usize {
match source {
AudioBufferRef::U8(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::U16(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::U24(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::U32(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::S8(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::S16(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::S24(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::S32(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::F32(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
AudioBufferRef::F64(v) => mix_symph_buffer(v, target, source_pos, dest_pos, volume),
}
}
#[inline]
fn mix_symph_buffer<S>(
source: &AudioBuffer<S>,
target: &mut AudioBuffer<f32>,
source_pos: usize,
dest_pos: usize,
volume: f32,
) -> usize
where
S: Sample + IntoSample<f32>,
{
// mix in source_packet[inner_pos..] til end of EITHER buffer.
let src_usable = source.frames() - source_pos;
let tgt_usable = target.frames() - dest_pos;
let mix_ct = src_usable.min(tgt_usable);
let target_chans = target.spec().channels.count();
let target_mono = target_chans == 1;
let source_chans = source.spec().channels.count();
let source_mono = source_chans == 1;
let source_planes = source.planes();
let source_raw_planes = source_planes.planes();
if source_mono {
// mix this signal into *all* output channels at req'd volume.
let source_plane = source_raw_planes[0];
for d_plane in (&mut *target.planes_mut().planes()).iter_mut() {
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct]
.iter_mut()
.zip(source_plane[source_pos..source_pos + mix_ct].iter())
{
*d += volume * (*s).into_sample();
}
}
} else if target_mono {
// mix all signals into the one target channel: reduce aggregate volume
// by n_channels.
let vol_adj = 1.0 / (source_chans as f32);
let mut t_planes = target.planes_mut();
let d_plane = &mut *t_planes.planes()[0];
for s_plane in source_raw_planes[..].iter() {
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct]
.iter_mut()
.zip(s_plane[source_pos..source_pos + mix_ct].iter())
{
*d += volume * vol_adj * (*s).into_sample();
}
}
} else {
// stereo -> stereo: don't change volume, map input -> output channels w/ no duplication
for (d_plane, s_plane) in (&mut *target.planes_mut().planes())
.iter_mut()
.zip(source_raw_planes[..].iter())
{
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct]
.iter_mut()
.zip(s_plane[source_pos..source_pos + mix_ct].iter())
{
*d += volume * (*s).into_sample();
}
}
}
mix_ct
}
#[inline]
fn mix_resampled(
source: &[Vec<f32>],
target: &mut AudioBuffer<f32>,
dest_pos: usize,
volume: f32,
) -> usize {
let mix_ct = source[0].len();
let target_chans = target.spec().channels.count();
let target_mono = target_chans == 1;
let source_chans = source.len();
let source_mono = source_chans == 1;
// see `mix_symph_buffer` for explanations of stereo<->mono logic.
if source_mono {
let source_plane = &source[0];
for d_plane in (&mut *target.planes_mut().planes()).iter_mut() {
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct]
.iter_mut()
.zip(source_plane)
{
*d += volume * s;
}
}
} else if target_mono {
let vol_adj = 1.0 / (source_chans as f32);
let mut t_planes = target.planes_mut();
let d_plane = &mut *t_planes.planes()[0];
for s_plane in source[..].iter() {
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct].iter_mut().zip(s_plane) {
*d += volume * vol_adj * s;
}
}
} else {
for (d_plane, s_plane) in (&mut *target.planes_mut().planes())
.iter_mut()
.zip(source[..].iter())
{
for (d, s) in d_plane[dest_pos..dest_pos + mix_ct].iter_mut().zip(s_plane) {
*d += volume * (*s);
}
}
}
mix_ct
}
#[inline]
pub(crate) fn copy_into_resampler(
source: &AudioBufferRef,
target: &mut AudioBuffer<f32>,
source_pos: usize,
dest_pos: usize,
len: usize,
) -> usize {
match source {
AudioBufferRef::U8(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::U16(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::U24(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::U32(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::S8(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::S16(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::S24(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::S32(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::F32(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
AudioBufferRef::F64(v) => copy_symph_buffer(v, target, source_pos, dest_pos, len),
}
}
#[inline]
fn copy_symph_buffer<S>(
source: &AudioBuffer<S>,
target: &mut AudioBuffer<f32>,
source_pos: usize,
dest_pos: usize,
len: usize,
) -> usize
where
S: Sample + IntoSample<f32>,
{
for (d_plane, s_plane) in (&mut *target.planes_mut().planes())
.iter_mut()
.zip(source.planes().planes()[..].iter())
{
for (d, s) in d_plane[dest_pos..dest_pos + len]
.iter_mut()
.zip(s_plane[source_pos..source_pos + len].iter())
{
*d = (*s).into_sample();
}
}
len
}

924
src/driver/tasks/mixer/mod.rs Normal file
View File

@@ -0,0 +1,924 @@
pub mod mix_logic;
mod pool;
mod result;
pub mod state;
pub mod track;
mod util;
use pool::*;
use result::*;
use state::*;
pub use track::*;
use super::{disposal, error::Result, message::*};
use crate::{
constants::*,
driver::MixMode,
events::EventStore,
input::{Input, Parsed},
tracks::{Action, LoopState, PlayError, PlayMode, TrackCommand, TrackHandle, TrackState, View},
Config,
};
use audiopus::{
coder::Encoder as OpusEncoder,
softclip::SoftClip,
Application as CodingMode,
Bitrate,
};
use discortp::{
rtp::{MutableRtpPacket, RtpPacket},
MutablePacket,
};
use flume::{Receiver, Sender, TryRecvError};
use rand::random;
use rubato::{FftFixedOut, Resampler};
use std::{
io::Write,
sync::Arc,
time::{Duration, Instant},
};
use symphonia_core::{
audio::{AudioBuffer, AudioBufferRef, Layout, SampleBuffer, Signal, SignalSpec},
codecs::CODEC_TYPE_OPUS,
conv::IntoSample,
formats::SeekTo,
sample::Sample,
units::Time,
};
use tokio::runtime::Handle;
use tracing::{debug, error, instrument, warn};
use xsalsa20poly1305::TAG_SIZE;
#[cfg(test)]
use crate::driver::test_config::{OutputMessage, OutputMode, TickStyle};
#[cfg(test)]
use discortp::Packet as _;
pub struct Mixer {
pub bitrate: Bitrate,
pub config: Arc<Config>,
pub conn_active: Option<MixerConnection>,
pub content_prep_sequence: u64,
pub deadline: Instant,
pub disposer: Sender<DisposalMessage>,
pub encoder: OpusEncoder,
pub interconnect: Interconnect,
pub mix_rx: Receiver<MixerMessage>,
pub muted: bool,
pub packet: [u8; VOICE_PACKET_MAX],
pub prevent_events: bool,
pub silence_frames: u8,
pub skip_sleep: bool,
pub soft_clip: SoftClip,
thread_pool: BlockyTaskPool,
pub ws: Option<Sender<WsMessage>>,
pub tracks: Vec<InternalTrack>,
track_handles: Vec<TrackHandle>,
sample_buffer: SampleBuffer<f32>,
symph_mix: AudioBuffer<f32>,
resample_scratch: AudioBuffer<f32>,
#[cfg(test)]
remaining_loops: Option<u64>,
}
fn new_encoder(bitrate: Bitrate, mix_mode: MixMode) -> Result<OpusEncoder> {
let mut encoder = OpusEncoder::new(SAMPLE_RATE, mix_mode.to_opus(), CodingMode::Audio)?;
encoder.set_bitrate(bitrate)?;
Ok(encoder)
}
impl Mixer {
pub fn new(
mix_rx: Receiver<MixerMessage>,
async_handle: Handle,
interconnect: Interconnect,
config: Config,
) -> Self {
let bitrate = DEFAULT_BITRATE;
let encoder = new_encoder(bitrate, config.mix_mode)
.expect("Failed to create encoder in mixing thread with known-good values.");
let soft_clip = SoftClip::new(config.mix_mode.to_opus());
let mut packet = [0u8; VOICE_PACKET_MAX];
let mut rtp = MutableRtpPacket::new(&mut packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_version(RTP_VERSION);
rtp.set_payload_type(RTP_PROFILE_TYPE);
rtp.set_sequence(random::<u16>().into());
rtp.set_timestamp(random::<u32>().into());
let tracks = Vec::with_capacity(1.max(config.preallocated_tracks));
let track_handles = Vec::with_capacity(1.max(config.preallocated_tracks));
// Create an object disposal thread here.
let (disposer, disposal_rx) = flume::unbounded();
std::thread::spawn(move || disposal::runner(disposal_rx));
let thread_pool = BlockyTaskPool::new(async_handle);
let symph_layout = config.mix_mode.symph_layout();
let config = config.into();
let sample_buffer = SampleBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
symphonia_core::audio::SignalSpec::new_with_layout(
SAMPLE_RATE_RAW as u32,
symph_layout,
),
);
let symph_mix = AudioBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
symphonia_core::audio::SignalSpec::new_with_layout(
SAMPLE_RATE_RAW as u32,
symph_layout,
),
);
let resample_scratch = AudioBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, Layout::Stereo),
);
Self {
bitrate,
config,
conn_active: None,
content_prep_sequence: 0,
deadline: Instant::now(),
disposer,
encoder,
interconnect,
mix_rx,
muted: false,
packet,
prevent_events: false,
silence_frames: 0,
skip_sleep: false,
soft_clip,
thread_pool,
ws: None,
tracks,
track_handles,
sample_buffer,
symph_mix,
resample_scratch,
#[cfg(test)]
remaining_loops: None,
}
}
fn run(&mut self) {
let mut events_failure = false;
let mut conn_failure = false;
#[cfg(test)]
let ignore_check = self.config.override_connection.is_some();
#[cfg(not(test))]
let ignore_check = false;
'runner: loop {
if self.conn_active.is_some() || ignore_check {
loop {
match self.mix_rx.try_recv() {
Ok(m) => {
let (events, conn, should_exit) = self.handle_message(m);
events_failure |= events;
conn_failure |= conn;
if should_exit {
break 'runner;
}
},
Err(TryRecvError::Disconnected) => {
break 'runner;
},
Err(TryRecvError::Empty) => {
break;
},
};
}
// The above action may have invalidated the connection; need to re-check!
// Also, if we're in a test mode we should unconditionally run packet mixing code.
if self.conn_active.is_some() || ignore_check {
if let Err(e) = self.cycle().and_then(|_| self.audio_commands_events()) {
events_failure |= e.should_trigger_interconnect_rebuild();
conn_failure |= e.should_trigger_connect();
debug!("Mixer thread cycle: {:?}", e);
}
}
} else {
match self.mix_rx.recv() {
Ok(m) => {
let (events, conn, should_exit) = self.handle_message(m);
events_failure |= events;
conn_failure |= conn;
if should_exit {
break 'runner;
}
},
Err(_) => {
break 'runner;
},
}
}
// event failure? rebuild interconnect.
// ws or udp failure? full connect
// (soft reconnect is covered by the ws task.)
//
// in both cases, send failure is fatal,
// but will only occur on disconnect.
// expecting this is fairly noisy, so exit silently.
if events_failure {
self.prevent_events = true;
let sent = self
.interconnect
.core
.send(CoreMessage::RebuildInterconnect);
events_failure = false;
if sent.is_err() {
break;
}
}
if conn_failure {
self.conn_active = None;
let sent = self.interconnect.core.send(CoreMessage::FullReconnect);
conn_failure = false;
if sent.is_err() {
break;
}
}
}
}
#[inline]
fn handle_message(&mut self, msg: MixerMessage) -> (bool, bool, bool) {
let mut events_failure = false;
let mut conn_failure = false;
let mut should_exit = false;
let error = match msg {
MixerMessage::AddTrack(t) => self.add_track(t),
MixerMessage::SetTrack(t) => {
self.tracks.clear();
let mut out = self.fire_event(EventMessage::RemoveAllTracks);
if let Some(t) = t {
// Do this unconditionally: this affects local state infallibly,
// with the event installation being the remote part.
if let Err(e) = self.add_track(t) {
out = Err(e);
}
}
out
},
MixerMessage::SetBitrate(b) => {
self.bitrate = b;
if let Err(e) = self.set_bitrate(b) {
error!("Failed to update bitrate {:?}", e);
}
Ok(())
},
MixerMessage::SetMute(m) => {
self.muted = m;
Ok(())
},
MixerMessage::SetConn(conn, ssrc) => {
self.conn_active = Some(conn);
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_ssrc(ssrc);
rtp.set_sequence(random::<u16>().into());
rtp.set_timestamp(random::<u32>().into());
self.deadline = Instant::now();
Ok(())
},
MixerMessage::DropConn => {
self.conn_active = None;
Ok(())
},
MixerMessage::ReplaceInterconnect(i) => {
self.prevent_events = false;
if let Some(ws) = &self.ws {
conn_failure |= ws.send(WsMessage::ReplaceInterconnect(i.clone())).is_err();
}
if let Some(conn) = &self.conn_active {
conn_failure |= conn
.udp_rx
.send(UdpRxMessage::ReplaceInterconnect(i.clone()))
.is_err();
}
self.interconnect = i;
self.rebuild_tracks()
},
MixerMessage::SetConfig(new_config) => {
if new_config.mix_mode != self.config.mix_mode {
self.soft_clip = SoftClip::new(new_config.mix_mode.to_opus());
if let Ok(enc) = new_encoder(self.bitrate, new_config.mix_mode) {
self.encoder = enc;
} else {
self.bitrate = DEFAULT_BITRATE;
self.encoder = new_encoder(self.bitrate, new_config.mix_mode)
.expect("Failed fallback rebuild of OpusEncoder with safe inputs.");
}
let sl = new_config.mix_mode.symph_layout();
self.sample_buffer = SampleBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, sl),
);
self.symph_mix = AudioBuffer::<f32>::new(
MONO_FRAME_SIZE as u64,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, sl),
);
}
self.config = Arc::new(new_config.clone());
if self.tracks.capacity() < self.config.preallocated_tracks {
self.tracks
.reserve(self.config.preallocated_tracks - self.tracks.len());
}
if let Some(conn) = &self.conn_active {
conn_failure |= conn
.udp_rx
.send(UdpRxMessage::SetConfig(new_config))
.is_err();
}
Ok(())
},
MixerMessage::RebuildEncoder => match new_encoder(self.bitrate, self.config.mix_mode) {
Ok(encoder) => {
self.encoder = encoder;
Ok(())
},
Err(e) => {
error!("Failed to rebuild encoder. Resetting bitrate. {:?}", e);
self.bitrate = DEFAULT_BITRATE;
self.encoder = new_encoder(self.bitrate, self.config.mix_mode)
.expect("Failed fallback rebuild of OpusEncoder with safe inputs.");
Ok(())
},
},
MixerMessage::Ws(new_ws_handle) => {
self.ws = new_ws_handle;
Ok(())
},
MixerMessage::Poison => {
should_exit = true;
Ok(())
},
};
if let Err(e) = error {
events_failure |= e.should_trigger_interconnect_rebuild();
conn_failure |= e.should_trigger_connect();
}
(events_failure, conn_failure, should_exit)
}
#[inline]
fn fire_event(&self, event: EventMessage) -> Result<()> {
// As this task is responsible for noticing the potential death of an event context,
// it's responsible for not forcibly recreating said context repeatedly.
if !self.prevent_events {
self.interconnect.events.send(event)?;
}
Ok(())
}
#[inline]
pub fn add_track(&mut self, track: TrackContext) -> Result<()> {
let (track, evts, state, handle) = InternalTrack::decompose_track(track);
self.tracks.push(track);
self.track_handles.push(handle.clone());
self.interconnect
.events
.send(EventMessage::AddTrack(evts, state, handle))?;
Ok(())
}
// rebuilds the event thread's view of each track, in event of a full rebuild.
#[inline]
fn rebuild_tracks(&mut self) -> Result<()> {
for (track, handle) in self.tracks.iter().zip(self.track_handles.iter()) {
let evts = EventStore::default();
let state = track.state();
let handle = handle.clone();
self.interconnect
.events
.send(EventMessage::AddTrack(evts, state, handle))?;
}
Ok(())
}
#[inline]
fn audio_commands_events(&mut self) -> Result<()> {
// Apply user commands.
for (i, track) in self.tracks.iter_mut().enumerate() {
// This causes fallible event system changes,
// but if the event thread has died then we'll certainly
// detect that on the tick later.
// Changes to play state etc. MUST all be handled.
let action = track.process_commands(i, &self.interconnect);
if let Some(req) = action.seek_point {
track.seek(
i,
req,
&self.interconnect,
&self.thread_pool,
&self.config,
self.prevent_events,
);
}
if let Some(callback) = action.make_playable {
if let Err(e) = track.get_or_ready_input(
i,
&self.interconnect,
&self.thread_pool,
&self.config,
self.prevent_events,
) {
track.callbacks.make_playable = Some(callback);
if let Some(fail) = e.as_user() {
track.playing = PlayMode::Errored(fail);
}
if let Some(req) = e.into_seek_request() {
track.seek(
i,
req,
&self.interconnect,
&self.thread_pool,
&self.config,
self.prevent_events,
);
}
} else {
// Track is already ready: don't register callback and just act.
drop(callback.send(Ok(())));
}
}
}
let mut i = 0;
while i < self.tracks.len() {
let track = self
.tracks
.get_mut(i)
.expect("Tried to remove an illegal track index.");
if track.playing.is_done() {
let p_state = track.playing.clone();
let to_drop = self.tracks.swap_remove(i);
drop(
self.disposer
.send(DisposalMessage::Track(Box::new(to_drop))),
);
let to_drop = self.track_handles.swap_remove(i);
drop(self.disposer.send(DisposalMessage::Handle(to_drop)));
self.fire_event(EventMessage::ChangeState(
i,
TrackStateChange::Mode(p_state),
))?;
} else {
i += 1;
}
}
// Tick -- receive side also handles removals in same manner after it increments
// times etc.
self.fire_event(EventMessage::Tick)?;
Ok(())
}
#[cfg(test)]
fn _march_deadline(&mut self) {
match &self.config.tick_style {
TickStyle::Timed => {
std::thread::sleep(self.deadline.saturating_duration_since(Instant::now()));
self.deadline += TIMESTEP_LENGTH;
},
TickStyle::UntimedWithExecLimit(rx) => {
if self.remaining_loops.is_none() {
if let Ok(new_val) = rx.recv() {
self.remaining_loops = Some(new_val.wrapping_sub(1));
}
}
if let Some(cnt) = self.remaining_loops.as_mut() {
if *cnt == 0 {
self.remaining_loops = None;
} else {
*cnt = cnt.wrapping_sub(1);
}
}
},
}
}
#[cfg(not(test))]
#[inline(always)]
#[allow(clippy::inline_always)] // Justified, this is a very very hot path
fn _march_deadline(&mut self) {
std::thread::sleep(self.deadline.saturating_duration_since(Instant::now()));
self.deadline += TIMESTEP_LENGTH;
}
#[inline]
fn march_deadline(&mut self) {
if self.skip_sleep {
return;
}
self._march_deadline();
}
pub fn cycle(&mut self) -> Result<()> {
let mut mix_buffer = [0f32; STEREO_FRAME_SIZE];
// symph_mix is an `AudioBuffer` (planar format), we need to convert this
// later into an interleaved `SampleBuffer` for libopus.
self.symph_mix.clear();
self.symph_mix.render_reserved(Some(MONO_FRAME_SIZE));
self.resample_scratch.clear();
// Walk over all the audio files, combining into one audio frame according
// to volume, play state, etc.
let mut mix_len = {
let out = self.mix_tracks();
self.sample_buffer.copy_interleaved_typed(&self.symph_mix);
out
};
if self.muted {
mix_len = MixType::MixedPcm(0);
}
// Explicit "Silence" frame handling: if there is no mixed data, we must send
// ~5 frames of silence (unless another good audio frame appears) before we
// stop sending RTP frames.
if mix_len == MixType::MixedPcm(0) {
if self.silence_frames > 0 {
self.silence_frames -= 1;
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
(&mut payload[TAG_SIZE..TAG_SIZE + SILENT_FRAME.len()])
.copy_from_slice(&SILENT_FRAME[..]);
mix_len = MixType::Passthrough(SILENT_FRAME.len());
} else {
// Per official guidelines, send 5x silence BEFORE we stop speaking.
if let Some(ws) = &self.ws {
// NOTE: this explicit `drop` should prevent a catastrophic thread pileup.
// A full reconnect might cause an inner closed connection.
// It's safer to leave the central task to clean this up and
// pass the mixer a new channel.
drop(ws.send(WsMessage::Speaking(false)));
}
self.march_deadline();
#[cfg(test)]
match &self.config.override_connection {
Some(OutputMode::Raw(tx)) =>
drop(tx.send(crate::driver::test_config::TickMessage::NoEl)),
Some(OutputMode::Rtp(tx)) =>
drop(tx.send(crate::driver::test_config::TickMessage::NoEl)),
None => {},
}
return Ok(());
}
} else {
self.silence_frames = 5;
if let MixType::MixedPcm(n) = mix_len {
// FIXME: When impling #134, prevent this copy from happening if softclip disabled.
// Offer sample_buffer.samples() to prep_and_send_packet.
// to apply soft_clip, we need this to be in a normal f32 buffer.
// unfortunately, SampleBuffer does not expose a `.samples_mut()`.
// hence, an extra copy...
let samples_to_copy = self.config.mix_mode.channels() * n;
(&mut mix_buffer[..samples_to_copy])
.copy_from_slice(&self.sample_buffer.samples()[..samples_to_copy]);
self.soft_clip.apply(
(&mut mix_buffer[..])
.try_into()
.expect("Mix buffer is known to have a valid sample count (softclip)."),
)?;
}
}
if let Some(ws) = &self.ws {
ws.send(WsMessage::Speaking(true))?;
}
// Wait till the right time to send this packet:
// usually a 20ms tick, in test modes this is either a finite number of runs or user input.
self.march_deadline();
#[cfg(test)]
if let Some(OutputMode::Raw(tx)) = &self.config.override_connection {
let msg = match mix_len {
MixType::Passthrough(len) if len == SILENT_FRAME.len() => OutputMessage::Silent,
MixType::Passthrough(len) => {
let rtp = RtpPacket::new(&self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload();
let opus_frame = (&payload[TAG_SIZE..][..len]).to_vec();
OutputMessage::Passthrough(opus_frame)
},
MixType::MixedPcm(_) => OutputMessage::Mixed(
mix_buffer[..self.config.mix_mode.sample_count_in_frame()].to_vec(),
),
};
drop(tx.send(msg.into()));
} else {
self.prep_and_send_packet(&mix_buffer, mix_len)?;
}
#[cfg(not(test))]
self.prep_and_send_packet(&mix_buffer, mix_len)?;
// Zero out all planes of the mix buffer if any audio was written.
if matches!(mix_len, MixType::MixedPcm(a) if a > 0) {
for plane in self.symph_mix.planes_mut().planes() {
plane.fill(0.0);
}
}
Ok(())
}
fn set_bitrate(&mut self, bitrate: Bitrate) -> Result<()> {
self.encoder.set_bitrate(bitrate).map_err(Into::into)
}
#[inline]
fn prep_and_send_packet(&mut self, buffer: &[f32; 1920], mix_len: MixType) -> Result<()> {
let conn = self
.conn_active
.as_mut()
.expect("Shouldn't be mixing packets without access to a cipher + UDP dest.");
let index = {
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
let crypto_mode = conn.crypto_state.kind();
// If passthrough, Opus payload in place already.
// Else encode into buffer with space for AEAD encryption headers.
let payload_len = match mix_len {
MixType::Passthrough(opus_len) => opus_len,
MixType::MixedPcm(_samples) => {
let total_payload_space = payload.len() - crypto_mode.payload_suffix_len();
self.encoder.encode_float(
&buffer[..self.config.mix_mode.sample_count_in_frame()],
&mut payload[TAG_SIZE..total_payload_space],
)?
},
};
let final_payload_size = conn
.crypto_state
.write_packet_nonce(&mut rtp, TAG_SIZE + payload_len);
// Packet encryption ignored in test modes.
#[cfg(not(test))]
let encrypt = true;
#[cfg(test)]
let encrypt = self.config.override_connection.is_none();
if encrypt {
conn.crypto_state.kind().encrypt_in_place(
&mut rtp,
&conn.cipher,
final_payload_size,
)?;
}
RtpPacket::minimum_packet_size() + final_payload_size
};
#[cfg(test)]
if let Some(OutputMode::Rtp(tx)) = &self.config.override_connection {
// Test mode: send unencrypted (compressed) packets to local receiver.
drop(tx.send(self.packet[..index].to_vec().into()));
} else {
conn.udp_tx.send(self.packet[..index].to_vec())?;
}
#[cfg(not(test))]
{
// Normal operation: send encrypted payload to UDP Tx task.
// TODO: This is dog slow, don't do this.
// Can we replace this with a shared ring buffer + semaphore?
// or the BBQueue crate?
conn.udp_tx.send(self.packet[..index].to_vec())?;
}
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
rtp.set_sequence(rtp.get_sequence() + 1);
rtp.set_timestamp(rtp.get_timestamp() + MONO_FRAME_SIZE as u32);
Ok(())
}
#[inline]
fn mix_tracks(&mut self) -> MixType {
// Get a slice of bytes to write in data for Opus packet passthrough.
let mut rtp = MutableRtpPacket::new(&mut self.packet[..]).expect(
"FATAL: Too few bytes in self.packet for RTP header.\
(Blame: VOICE_PACKET_MAX?)",
);
let payload = rtp.payload_mut();
let opus_frame = &mut payload[TAG_SIZE..];
// Opus frame passthrough.
// This requires that we have only one PLAYING track, who has volume 1.0, and an
// Opus codec type (verified later in mix_symph_indiv).
//
// We *could* cache the number of live tracks separately, but that makes this
// quite fragile given all the ways a user can alter the PlayMode.
let mut num_live = 0;
let mut last_live_vol = 1.0;
for track in &self.tracks {
if track.playing.is_playing() {
num_live += 1;
last_live_vol = track.volume;
}
}
let do_passthrough = num_live == 1 && (last_live_vol - 1.0).abs() < f32::EPSILON;
let mut len = 0;
for (i, track) in self.tracks.iter_mut().enumerate() {
let vol = track.volume;
// This specifically tries to get tracks who are "preparing",
// so that event handlers and the like can all be fired without
// the track being in a `Play` state.
if !track.should_check_input() {
continue;
}
let should_play = track.playing.is_playing();
let input = track.get_or_ready_input(
i,
&self.interconnect,
&self.thread_pool,
&self.config,
self.prevent_events,
);
let (input, mix_state) = match input {
Ok(i) => i,
Err(InputReadyingError::Waiting) => continue,
Err(InputReadyingError::NeedsSeek(req)) => {
track.seek(
i,
req,
&self.interconnect,
&self.thread_pool,
&self.config,
self.prevent_events,
);
continue;
},
// TODO: allow for retry in given time.
Err(e) => {
if let Some(fail) = e.as_user() {
track.playing = PlayMode::Errored(fail);
}
continue;
},
};
// Now that we have dealt with potential errors in preparing tracks,
// only do any mixing if the track is to be played!
if !should_play {
continue;
}
let (mix_type, status) = mix_logic::mix_symph_indiv(
&mut self.symph_mix,
&mut self.resample_scratch,
input,
mix_state,
vol,
do_passthrough.then(|| &mut *opus_frame),
);
let return_here = if let MixType::MixedPcm(pcm_len) = mix_type {
len = len.max(pcm_len);
false
} else {
if mix_state.passthrough == Passthrough::Inactive {
input.decoder.reset();
}
mix_state.passthrough = Passthrough::Active;
true
};
// FIXME: allow Ended to trigger a seek/loop/revisit in the same mix cycle?
// Would this be possible with special-casing to mark some inputs as fast
// to recreate? Probably not doable in the general case.
match status {
MixStatus::Live => track.step_frame(),
MixStatus::Errored(e) =>
track.playing = PlayMode::Errored(PlayError::Decode(e.into())),
MixStatus::Ended if track.do_loop() => {
drop(self.track_handles[i].seek(Duration::default()));
if !self.prevent_events {
// position update is sent out later, when the seek concludes.
drop(self.interconnect.events.send(EventMessage::ChangeState(
i,
TrackStateChange::Loops(track.loops, false),
)));
}
},
MixStatus::Ended => {
track.end();
},
}
// This needs to happen here due to borrow checker shenanigans.
if return_here {
return mix_type;
}
}
MixType::MixedPcm(len)
}
}
/// The mixing thread is a synchronous context due to its compute-bound nature.
///
/// We pass in an async handle for the benefit of some Input classes (e.g., restartables)
/// who need to run their restart code elsewhere and return blank data until such time.
#[instrument(skip(interconnect, mix_rx, async_handle))]
pub(crate) fn runner(
interconnect: Interconnect,
mix_rx: Receiver<MixerMessage>,
async_handle: Handle,
config: Config,
) {
Mixer::new(mix_rx, async_handle, interconnect, config).run();
}

148
src/driver/tasks/mixer/pool.rs Normal file
View File

@@ -0,0 +1,148 @@
use super::util::copy_seek_to;
use crate::{
driver::tasks::message::MixerInputResultMessage,
input::{AudioStream, AudioStreamError, Compose, Input, LiveInput, Parsed},
Config,
};
use flume::Sender;
use parking_lot::RwLock;
use std::{result::Result as StdResult, sync::Arc, time::Duration};
use symphonia_core::{
formats::{SeekMode, SeekTo},
io::MediaSource,
};
use tokio::runtime::Handle;
#[derive(Clone)]
pub struct BlockyTaskPool {
pool: Arc<RwLock<rusty_pool::ThreadPool>>,
handle: Handle,
}
impl BlockyTaskPool {
pub fn new(handle: Handle) -> Self {
Self {
pool: Arc::new(RwLock::new(rusty_pool::ThreadPool::new(
1,
64,
Duration::from_secs(300),
))),
handle,
}
}
pub fn create(
&self,
callback: Sender<MixerInputResultMessage>,
input: Input,
seek_time: Option<SeekTo>,
config: Arc<Config>,
) {
// Moves an Input from Lazy -> Live.
// We either do this on this pool, or move it to the tokio executor as the source requires.
// This takes a seek_time to pass on and execute *after* parsing (i.e., back-seek on
// read-only stream).
match input {
Input::Lazy(mut lazy) => {
let far_pool = self.clone();
if lazy.should_create_async() {
self.handle.spawn(async move {
let out = lazy.create_async().await;
far_pool.send_to_parse(out, lazy, callback, seek_time, config);
});
} else {
let pool = self.pool.read();
pool.execute(move || {
let out = lazy.create();
far_pool.send_to_parse(out, lazy, callback, seek_time, config);
});
}
},
Input::Live(live, maybe_create) =>
self.parse(config, callback, live, maybe_create, seek_time),
}
}
pub fn send_to_parse(
&self,
create_res: StdResult<AudioStream<Box<dyn MediaSource>>, AudioStreamError>,
rec: Box<dyn Compose>,
callback: Sender<MixerInputResultMessage>,
seek_time: Option<SeekTo>,
config: Arc<Config>,
) {
match create_res {
Ok(o) => {
self.parse(config, callback, LiveInput::Raw(o), Some(rec), seek_time);
},
Err(e) => {
drop(callback.send(MixerInputResultMessage::CreateErr(e.into())));
},
}
}
pub fn parse(
&self,
config: Arc<Config>,
callback: Sender<MixerInputResultMessage>,
input: LiveInput,
rec: Option<Box<dyn Compose>>,
seek_time: Option<SeekTo>,
) {
let pool_clone = self.clone();
let pool = self.pool.read();
pool.execute(
move || match input.promote(config.codec_registry, config.format_registry) {
Ok(LiveInput::Parsed(parsed)) => match seek_time {
// If seek time is zero, then wipe it out.
// Some formats (MKV) make SeekTo(0) require a backseek to realign with the
// current page.
Some(seek_time) if !super::util::seek_to_is_zero(&seek_time) => {
pool_clone.seek(callback, parsed, rec, seek_time, false, config);
},
_ => {
drop(callback.send(MixerInputResultMessage::Built(parsed, rec)));
},
},
Ok(_) => unreachable!(),
Err(e) => {
drop(callback.send(MixerInputResultMessage::ParseErr(e.into())));
},
},
);
}
pub fn seek(
&self,
callback: Sender<MixerInputResultMessage>,
mut input: Parsed,
rec: Option<Box<dyn Compose>>,
seek_time: SeekTo,
// Not all of symphonia's formats bother to return SeekErrorKind::ForwardOnly.
// So, we need *this* flag.
backseek_needed: bool,
config: Arc<Config>,
) {
let pool_clone = self.clone();
let pool = self.pool.read();
pool.execute(move || match rec {
Some(rec) if (!input.supports_backseek) && backseek_needed => {
pool_clone.create(callback, Input::Lazy(rec), Some(seek_time), config);
},
_ => {
let seek_result = input
.format
.seek(SeekMode::Accurate, copy_seek_to(&seek_time));
input.decoder.reset();
drop(callback.send(MixerInputResultMessage::Seek(
input,
rec,
seek_result.map_err(Arc::new),
)));
},
});
}
}

55
src/driver/tasks/mixer/result.rs Normal file
View File

@@ -0,0 +1,55 @@
use crate::{
input::AudioStreamError,
tracks::{PlayError, SeekRequest},
};
use std::sync::Arc;
use symphonia_core::errors::Error as SymphoniaError;
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub enum MixType {
Passthrough(usize),
MixedPcm(usize),
}
pub enum MixStatus {
Live,
Ended,
Errored(SymphoniaError),
}
impl From<SymphoniaError> for MixStatus {
fn from(e: SymphoniaError) -> Self {
Self::Errored(e)
}
}
// The Symph errors are Arc'd here since if they come up, they will always
// be Arc'd anyway via into_user.
#[derive(Clone, Debug)]
pub enum InputReadyingError {
Parsing(Arc<SymphoniaError>),
Creation(Arc<AudioStreamError>),
Seeking(Arc<SymphoniaError>),
Dropped,
Waiting,
NeedsSeek(SeekRequest),
}
impl InputReadyingError {
pub fn as_user(&self) -> Option<PlayError> {
match self {
Self::Parsing(e) => Some(PlayError::Parse(e.clone())),
Self::Creation(e) => Some(PlayError::Create(e.clone())),
Self::Seeking(e) => Some(PlayError::Seek(e.clone())),
_ => None,
}
}
pub fn into_seek_request(self) -> Option<SeekRequest> {
if let Self::NeedsSeek(a) = self {
Some(a)
} else {
None
}
}
}

104
src/driver/tasks/mixer/state.rs Normal file
View File

@@ -0,0 +1,104 @@
use crate::{
constants::OPUS_PASSTHROUGH_STRIKE_LIMIT,
driver::tasks::message::*,
input::{Compose, Input, LiveInput, Metadata, Parsed},
tracks::{ReadyState, SeekRequest},
};
use flume::Receiver;
use rubato::FftFixedOut;
use std::time::Instant;
pub enum InputState {
NotReady(Input),
Preparing(PreparingInfo),
Ready(Parsed, Option<Box<dyn Compose>>),
}
impl InputState {
pub fn metadata(&mut self) -> Option<Metadata> {
if let Self::Ready(parsed, _) = self {
Some(parsed.into())
} else {
None
}
}
pub fn ready_state(&self) -> ReadyState {
match self {
Self::NotReady(_) => ReadyState::Uninitialised,
Self::Preparing(_) => ReadyState::Preparing,
Self::Ready(_, _) => ReadyState::Playable,
}
}
}
impl From<Input> for InputState {
fn from(val: Input) -> Self {
match val {
a @ Input::Lazy(_) => Self::NotReady(a),
Input::Live(live, rec) => match live {
LiveInput::Parsed(p) => Self::Ready(p, rec),
other => Self::NotReady(Input::Live(other, rec)),
},
}
}
}
pub struct PreparingInfo {
#[allow(dead_code)]
/// Time this request was fired.
pub time: Instant,
/// Used to handle seek requests fired while a track was being created (or a seek was in progress).
pub queued_seek: Option<SeekRequest>,
/// Callback from the thread pool to indicate the result of creating/parsing this track.
pub callback: Receiver<MixerInputResultMessage>,
}
pub struct DecodeState {
pub inner_pos: usize,
pub resampler: Option<(usize, FftFixedOut<f32>, Vec<Vec<f32>>)>,
pub passthrough: Passthrough,
pub passthrough_violations: u8,
}
impl DecodeState {
pub fn reset(&mut self) {
self.inner_pos = 0;
self.resampler = None;
}
pub fn record_and_check_passthrough_strike_final(&mut self, fatal: bool) -> bool {
self.passthrough_violations = self.passthrough_violations.saturating_add(1);
let blocked = fatal || self.passthrough_violations > OPUS_PASSTHROUGH_STRIKE_LIMIT;
if blocked {
self.passthrough = Passthrough::Block;
}
blocked
}
}
impl Default for DecodeState {
fn default() -> Self {
Self {
inner_pos: 0,
resampler: None,
passthrough: Passthrough::Inactive,
passthrough_violations: 0,
}
}
}
/// Simple state to manage decoder resets etc.
///
/// Inactive->Active transitions should trigger a reset.
///
/// Block should be used if a source contains known-bad packets:
/// it's unlikely that packet sizes will vary, but if they do then
/// we can't passthrough (and every attempt will trigger a codec reset,
/// which probably won't sound too smooth).
#[derive(Clone, Copy, Eq, PartialEq)]
pub enum Passthrough {
Active,
Inactive,
Block,
}

400
src/driver/tasks/mixer/track.rs Normal file
View File

@@ -0,0 +1,400 @@
use crate::tracks::{ReadyState, SeekRequest};
use std::result::Result as StdResult;
use symphonia_core::errors::Error as SymphError;
use super::*;
pub struct InternalTrack {
pub(crate) playing: PlayMode,
pub(crate) volume: f32,
pub(crate) input: InputState,
pub(crate) mix_state: DecodeState,
pub(crate) position: Duration,
pub(crate) play_time: Duration,
pub(crate) commands: Receiver<TrackCommand>,
pub(crate) loops: LoopState,
pub(crate) callbacks: Callbacks,
}
impl<'a> InternalTrack {
pub(crate) fn decompose_track(
val: TrackContext,
) -> (Self, EventStore, TrackState, TrackHandle) {
let TrackContext {
handle,
track,
receiver,
} = val;
let out = InternalTrack {
playing: track.playing,
volume: track.volume,
input: InputState::from(track.input),
mix_state: DecodeState::default(),
position: Duration::default(),
play_time: Duration::default(),
commands: receiver,
loops: track.loops,
callbacks: Callbacks::default(),
};
let state = out.state();
(out, track.events, state, handle)
}
pub(crate) fn state(&self) -> TrackState {
let ready = self.input.ready_state();
TrackState {
playing: self.playing.clone(),
volume: self.volume,
position: self.position,
play_time: self.play_time,
loops: self.loops,
ready,
}
}
pub(crate) fn view(&'a mut self) -> View<'a> {
let ready = self.input.ready_state();
View {
position: &self.position,
play_time: &self.play_time,
volume: &mut self.volume,
meta: self.input.metadata(),
ready,
playing: &mut self.playing,
loops: &mut self.loops,
}
}
pub(crate) fn process_commands(&mut self, index: usize, ic: &Interconnect) -> Action {
// Note: disconnection and an empty channel are both valid,
// and should allow the audio object to keep running as intended.
// We also need to export a target seek point to the mixer, if known.
let mut action = Action::default();
// Note that interconnect failures are not currently errors.
// In correct operation, the event thread should never panic,
// but it receiving status updates is secondary do actually
// doing the work.
while let Ok(cmd) = self.commands.try_recv() {
match cmd {
TrackCommand::Play => {
self.playing.change_to(PlayMode::Play);
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing.clone()),
)));
},
TrackCommand::Pause => {
self.playing.change_to(PlayMode::Pause);
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing.clone()),
)));
},
TrackCommand::Stop => {
self.playing.change_to(PlayMode::Stop);
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Mode(self.playing.clone()),
)));
},
TrackCommand::Volume(vol) => {
self.volume = vol;
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Volume(self.volume),
)));
},
TrackCommand::Seek(req) => action.seek_point = Some(req),
TrackCommand::AddEvent(evt) => {
drop(ic.events.send(EventMessage::AddTrackEvent(index, evt)));
},
TrackCommand::Do(func) => {
if let Some(indiv_action) = func(self.view()) {
action.combine(indiv_action);
}
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Total(self.state()),
)));
},
TrackCommand::Request(tx) => {
drop(tx.send(self.state()));
},
TrackCommand::Loop(loops) => {
self.loops = loops;
drop(ic.events.send(EventMessage::ChangeState(
index,
TrackStateChange::Loops(self.loops, true),
)));
},
TrackCommand::MakePlayable(callback) => action.make_playable = Some(callback),
}
}
action
}
pub(crate) fn do_loop(&mut self) -> bool {
match self.loops {
LoopState::Infinite => true,
LoopState::Finite(0) => false,
LoopState::Finite(ref mut n) => {
*n -= 1;
true
},
}
}
/// Steps playback location forward by one frame.
pub(crate) fn step_frame(&mut self) {
self.position += TIMESTEP_LENGTH;
self.play_time += TIMESTEP_LENGTH;
}
pub(crate) fn should_check_input(&self) -> bool {
self.playing.is_playing() || matches!(self.input, InputState::Preparing(_))
}
pub(crate) fn end(&mut self) -> &mut Self {
self.playing.change_to(PlayMode::End);
self
}
/// Readies the requested input state.
///
/// Returns the usable version of the audio if available, and whether the track should be deleted.
pub(crate) fn get_or_ready_input(
&'a mut self,
id: usize,
interconnect: &Interconnect,
pool: &BlockyTaskPool,
config: &Arc<Config>,
prevent_events: bool,
) -> StdResult<(&'a mut Parsed, &'a mut DecodeState), InputReadyingError> {
let input = &mut self.input;
let mix_state = &mut self.mix_state;
let (out, queued_seek) = match input {
InputState::NotReady(_) => {
let (tx, rx) = flume::bounded(1);
let mut state = InputState::Preparing(PreparingInfo {
time: Instant::now(),
queued_seek: None,
callback: rx,
});
std::mem::swap(&mut state, input);
match state {
InputState::NotReady(a @ Input::Lazy(_)) => {
pool.create(tx, a, None, config.clone());
},
InputState::NotReady(Input::Live(audio, rec)) => {
pool.parse(config.clone(), tx, audio, rec, None);
},
_ => unreachable!(),
}
if !prevent_events {
drop(interconnect.events.send(EventMessage::ChangeState(
id,
TrackStateChange::Ready(ReadyState::Preparing),
)));
}
(Err(InputReadyingError::Waiting), None)
},
InputState::Preparing(info) => {
let queued_seek = info.queued_seek.take();
let orig_out = match info.callback.try_recv() {
Ok(MixerInputResultMessage::Built(parsed, rec)) => {
*input = InputState::Ready(parsed, rec);
mix_state.reset();
// possible TODO: set position to the true track position here?
// ISSUE: need to get next_packet to see its `ts`, but inner_pos==0
// will trigger next packet to be taken at mix time.
if !prevent_events {
drop(interconnect.events.send(EventMessage::ChangeState(
id,
TrackStateChange::Ready(ReadyState::Playable),
)));
}
self.callbacks.playable();
if let InputState::Ready(ref mut parsed, _) = input {
Ok(parsed)
} else {
unreachable!()
}
},
Ok(MixerInputResultMessage::Seek(parsed, rec, seek_res)) => {
match seek_res {
Ok(pos) =>
if let Some(time_base) = parsed.decoder.codec_params().time_base {
// Update track's position to match the actual timestamp the
// seek landed at.
let new_time = time_base.calc_time(pos.actual_ts);
let time_in_float = new_time.seconds as f64 + new_time.frac;
self.position =
std::time::Duration::from_secs_f64(time_in_float);
self.callbacks.seeked(self.position);
self.callbacks.playable();
if !prevent_events {
drop(interconnect.events.send(EventMessage::ChangeState(
id,
TrackStateChange::Position(self.position),
)));
drop(interconnect.events.send(EventMessage::ChangeState(
id,
TrackStateChange::Ready(ReadyState::Playable),
)));
}
// Our decoder state etc. must be reset.
// (Symphonia decoder state reset in the thread pool during
// the operation.)
mix_state.reset();
*input = InputState::Ready(parsed, rec);
if let InputState::Ready(ref mut parsed, _) = input {
Ok(parsed)
} else {
unreachable!()
}
} else {
Err(InputReadyingError::Seeking(
SymphError::Unsupported("Track had no recorded time base.")
.into(),
))
},
Err(e) => Err(InputReadyingError::Seeking(e)),
}
},
Ok(MixerInputResultMessage::CreateErr(e)) =>
Err(InputReadyingError::Creation(e)),
Ok(MixerInputResultMessage::ParseErr(e)) => Err(InputReadyingError::Parsing(e)),
Err(TryRecvError::Disconnected) => Err(InputReadyingError::Dropped),
Err(TryRecvError::Empty) => Err(InputReadyingError::Waiting),
};
let orig_out = orig_out.map(|a| (a, mix_state));
if let Err(ref e) = orig_out {
if let Some(e) = e.as_user() {
self.callbacks.readying_error(e);
}
}
(orig_out, queued_seek)
},
InputState::Ready(ref mut parsed, _) => (Ok((parsed, mix_state)), None),
};
match (out, queued_seek) {
(Ok(_), Some(request)) => Err(InputReadyingError::NeedsSeek(request)),
(a, _) => a,
}
}
pub(crate) fn seek(
&mut self,
id: usize,
request: SeekRequest,
interconnect: &Interconnect,
pool: &BlockyTaskPool,
config: &Arc<Config>,
prevent_events: bool,
) {
if let InputState::Preparing(p) = &mut self.input {
p.queued_seek = Some(request);
return;
}
// might be a little topsy turvy: rethink me.
let SeekRequest { time, callback } = request;
self.callbacks.seek = Some(callback);
if !prevent_events {
drop(interconnect.events.send(EventMessage::ChangeState(
id,
TrackStateChange::Ready(ReadyState::Preparing),
)));
}
let backseek_needed = time < self.position;
let time = Time::from(time.as_secs_f64());
let mut ts = SeekTo::Time {
time,
track_id: None,
};
let (tx, rx) = flume::bounded(1);
let state = std::mem::replace(
&mut self.input,
InputState::Preparing(PreparingInfo {
time: Instant::now(),
callback: rx,
queued_seek: None,
}),
);
match state {
InputState::Ready(p, r) => {
if let SeekTo::Time { time: _, track_id } = &mut ts {
*track_id = Some(p.track_id);
}
pool.seek(tx, p, r, ts, backseek_needed, config.clone());
},
InputState::NotReady(lazy) => pool.create(tx, lazy, Some(ts), config.clone()),
InputState::Preparing(_) => unreachable!(), // Covered above.
}
}
}
#[derive(Debug, Default)]
pub struct Callbacks {
pub seek: Option<Sender<StdResult<Duration, PlayError>>>,
pub make_playable: Option<Sender<StdResult<(), PlayError>>>,
}
impl Callbacks {
fn readying_error(&mut self, err: PlayError) {
if let Some(callback) = self.seek.take() {
drop(callback.send(Err(err.clone())));
}
if let Some(callback) = self.make_playable.take() {
drop(callback.send(Err(err)));
}
}
fn playable(&mut self) {
if let Some(callback) = self.make_playable.take() {
drop(callback.send(Ok(())));
}
}
fn seeked(&mut self, time: Duration) {
if let Some(callback) = self.seek.take() {
drop(callback.send(Ok(time)));
}
}
}

20
src/driver/tasks/mixer/util.rs Normal file
View File

@@ -0,0 +1,20 @@
use symphonia_core::{formats::SeekTo, units::Time};
// SeekTo lacks Copy and Clone... somehow.
pub fn copy_seek_to(pos: &SeekTo) -> SeekTo {
match *pos {
SeekTo::Time { time, track_id } => SeekTo::Time { time, track_id },
SeekTo::TimeStamp { ts, track_id } => SeekTo::TimeStamp { ts, track_id },
}
}
pub fn seek_to_is_zero(pos: &SeekTo) -> bool {
match *pos {
SeekTo::Time { time, .. } =>
time == Time {
seconds: 0,
frac: 0.0,
},
SeekTo::TimeStamp { ts, .. } => ts == 0,
}
}

116
src/driver/tasks/mod.rs
View File

@@ -21,7 +21,7 @@ use crate::{
Config, Config,
ConnectionInfo, ConnectionInfo,
}; };
use flume::{Receiver, RecvError, Sender}; use flume::{Receiver, Sender};
use message::*; use message::*;
use tokio::{runtime::Handle, spawn, time::sleep as tsleep}; use tokio::{runtime::Handle, spawn, time::sleep as tsleep};
use tracing::{debug, instrument, trace}; use tracing::{debug, instrument, trace};
@@ -70,23 +70,21 @@ async fn runner(mut config: Config, rx: Receiver<CoreMessage>, tx: Sender<CoreMe
let mut retrying = None; let mut retrying = None;
let mut attempt_idx = 0; let mut attempt_idx = 0;
loop { while let Ok(msg) = rx.recv_async().await {
match rx.recv_async().await { match msg {
Ok(CoreMessage::ConnectWithResult(info, tx)) => { CoreMessage::ConnectWithResult(info, tx) => {
config = if let Some(new_config) = next_config.take() { config = if let Some(new_config) = next_config.take() {
let _ = interconnect drop(
.mixer interconnect
.send(MixerMessage::SetConfig(new_config.clone())); .mixer
.send(MixerMessage::SetConfig(new_config.clone())),
);
new_config new_config
} else { } else {
config config
}; };
if connection if connection.as_ref().map_or(true, |conn| conn.info != info) {
.as_ref()
.map(|conn| conn.info != info)
.unwrap_or(true)
{
// Only *actually* reconnect if the conn info changed, or we don't have an // Only *actually* reconnect if the conn info changed, or we don't have an
// active connection. // active connection.
// This allows the gateway component to keep sending join requests independent // This allows the gateway component to keep sending join requests independent
@@ -97,10 +95,10 @@ async fn runner(mut config: Config, rx: Receiver<CoreMessage>, tx: Sender<CoreMe
} else { } else {
// No reconnection was attempted as there's a valid, identical connection; // No reconnection was attempted as there's a valid, identical connection;
// tell the outside listener that the operation was a success. // tell the outside listener that the operation was a success.
let _ = tx.send(Ok(())); drop(tx.send(Ok(())));
} }
}, },
Ok(CoreMessage::RetryConnect(retry_idx)) => { CoreMessage::RetryConnect(retry_idx) => {
debug!("Retrying idx: {} (vs. {})", retry_idx, attempt_idx); debug!("Retrying idx: {} (vs. {})", retry_idx, attempt_idx);
if retry_idx == attempt_idx { if retry_idx == attempt_idx {
if let Some(progress) = retrying.take() { if let Some(progress) = retrying.take() {
@@ -110,68 +108,68 @@ async fn runner(mut config: Config, rx: Receiver<CoreMessage>, tx: Sender<CoreMe
} }
} }
}, },
Ok(CoreMessage::Disconnect) => { CoreMessage::Disconnect => {
let last_conn = connection.take(); let last_conn = connection.take();
let _ = interconnect.mixer.send(MixerMessage::DropConn); drop(interconnect.mixer.send(MixerMessage::DropConn));
let _ = interconnect.mixer.send(MixerMessage::RebuildEncoder); drop(interconnect.mixer.send(MixerMessage::RebuildEncoder));
if let Some(conn) = last_conn { if let Some(conn) = last_conn {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverDisconnect(InternalDisconnect { CoreContext::DriverDisconnect(InternalDisconnect {
kind: DisconnectKind::Runtime, kind: DisconnectKind::Runtime,
reason: None, reason: None,
info: conn.info.clone(), info: conn.info.clone(),
}), }),
)); )));
} }
}, },
Ok(CoreMessage::SignalWsClosure(ws_idx, ws_info, mut reason)) => { CoreMessage::SignalWsClosure(ws_idx, ws_info, mut reason) => {
// if idx is not a match, quash reason // if idx is not a match, quash reason
// (i.e., prevent users from mistakenly trying to reconnect for an *old* dead conn). // (i.e., prevent users from mistakenly trying to reconnect for an *old* dead conn).
// if it *is* a match, the conn needs to die! // if it *is* a match, the conn needs to die!
// (as the WS channel has truly given up the ghost). // (as the WS channel has truly given up the ghost).
if ws_idx != attempt_idx { if ws_idx == attempt_idx {
reason = None;
} else {
connection = None; connection = None;
let _ = interconnect.mixer.send(MixerMessage::DropConn); drop(interconnect.mixer.send(MixerMessage::DropConn));
let _ = interconnect.mixer.send(MixerMessage::RebuildEncoder); drop(interconnect.mixer.send(MixerMessage::RebuildEncoder));
} else {
reason = None;
} }
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverDisconnect(InternalDisconnect { CoreContext::DriverDisconnect(InternalDisconnect {
kind: DisconnectKind::Runtime, kind: DisconnectKind::Runtime,
reason, reason,
info: ws_info, info: ws_info,
}), }),
)); )));
}, },
Ok(CoreMessage::SetTrack(s)) => { CoreMessage::SetTrack(s) => {
let _ = interconnect.mixer.send(MixerMessage::SetTrack(s)); drop(interconnect.mixer.send(MixerMessage::SetTrack(s)));
}, },
Ok(CoreMessage::AddTrack(s)) => { CoreMessage::AddTrack(s) => {
let _ = interconnect.mixer.send(MixerMessage::AddTrack(s)); drop(interconnect.mixer.send(MixerMessage::AddTrack(s)));
}, },
Ok(CoreMessage::SetBitrate(b)) => { CoreMessage::SetBitrate(b) => {
let _ = interconnect.mixer.send(MixerMessage::SetBitrate(b)); drop(interconnect.mixer.send(MixerMessage::SetBitrate(b)));
}, },
Ok(CoreMessage::SetConfig(mut new_config)) => { CoreMessage::SetConfig(mut new_config) => {
next_config = Some(new_config.clone()); next_config = Some(new_config.clone());
new_config.make_safe(&config, connection.is_some()); new_config.make_safe(&config, connection.is_some());
let _ = interconnect.mixer.send(MixerMessage::SetConfig(new_config)); drop(interconnect.mixer.send(MixerMessage::SetConfig(new_config)));
}, },
Ok(CoreMessage::AddEvent(evt)) => { CoreMessage::AddEvent(evt) => {
let _ = interconnect.events.send(EventMessage::AddGlobalEvent(evt)); drop(interconnect.events.send(EventMessage::AddGlobalEvent(evt)));
}, },
Ok(CoreMessage::RemoveGlobalEvents) => { CoreMessage::RemoveGlobalEvents => {
let _ = interconnect.events.send(EventMessage::RemoveGlobalEvents); drop(interconnect.events.send(EventMessage::RemoveGlobalEvents));
}, },
Ok(CoreMessage::Mute(m)) => { CoreMessage::Mute(m) => {
let _ = interconnect.mixer.send(MixerMessage::SetMute(m)); drop(interconnect.mixer.send(MixerMessage::SetMute(m)));
}, },
Ok(CoreMessage::Reconnect) => { CoreMessage::Reconnect => {
if let Some(mut conn) = connection.take() { if let Some(mut conn) = connection.take() {
// try once: if interconnect, try again. // try once: if interconnect, try again.
// if still issue, full connect. // if still issue, full connect.
@@ -201,16 +199,16 @@ async fn runner(mut config: Config, rx: Receiver<CoreMessage>, tx: Sender<CoreMe
.attempt(&mut retrying, &interconnect, &config) .attempt(&mut retrying, &interconnect, &config)
.await; .await;
} else if let Some(ref connection) = &connection { } else if let Some(ref connection) = &connection {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverReconnect(InternalConnect { CoreContext::DriverReconnect(InternalConnect {
info: connection.info.clone(), info: connection.info.clone(),
ssrc: connection.ssrc, ssrc: connection.ssrc,
}), }),
)); )));
} }
} }
}, },
Ok(CoreMessage::FullReconnect) => CoreMessage::FullReconnect =>
if let Some(conn) = connection.take() { if let Some(conn) = connection.take() {
let info = conn.info.clone(); let info = conn.info.clone();
@@ -218,12 +216,10 @@ async fn runner(mut config: Config, rx: Receiver<CoreMessage>, tx: Sender<CoreMe
.attempt(&mut retrying, &interconnect, &config) .attempt(&mut retrying, &interconnect, &config)
.await; .await;
}, },
Ok(CoreMessage::RebuildInterconnect) => { CoreMessage::RebuildInterconnect => {
interconnect.restart_volatile_internals(); interconnect.restart_volatile_internals();
}, },
Err(RecvError::Disconnected) | Ok(CoreMessage::Poison) => { CoreMessage::Poison => break,
break;
},
} }
} }
@@ -275,22 +271,22 @@ impl ConnectionRetryData {
match self.flavour { match self.flavour {
ConnectionFlavour::Connect(tx) => { ConnectionFlavour::Connect(tx) => {
// Other side may not be listening: this is fine. // Other side may not be listening: this is fine.
let _ = tx.send(Ok(())); drop(tx.send(Ok(())));
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverConnect(InternalConnect { CoreContext::DriverConnect(InternalConnect {
info: connection.info.clone(), info: connection.info.clone(),
ssrc: connection.ssrc, ssrc: connection.ssrc,
}), }),
)); )));
}, },
ConnectionFlavour::Reconnect => { ConnectionFlavour::Reconnect => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverReconnect(InternalConnect { CoreContext::DriverReconnect(InternalConnect {
info: connection.info.clone(), info: connection.info.clone(),
ssrc: connection.ssrc, ssrc: connection.ssrc,
}), }),
)); )));
}, },
} }
@@ -304,7 +300,7 @@ impl ConnectionRetryData {
spawn(async move { spawn(async move {
tsleep(t).await; tsleep(t).await;
let _ = remote_ic.core.send(CoreMessage::RetryConnect(idx)); drop(remote_ic.core.send(CoreMessage::RetryConnect(idx)));
}); });
self.attempts += 1; self.attempts += 1;
@@ -325,24 +321,24 @@ impl ConnectionRetryData {
match self.flavour { match self.flavour {
ConnectionFlavour::Connect(tx) => { ConnectionFlavour::Connect(tx) => {
// See above. // See above.
let _ = tx.send(Err(why)); drop(tx.send(Err(why)));
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverDisconnect(InternalDisconnect { CoreContext::DriverDisconnect(InternalDisconnect {
kind: DisconnectKind::Connect, kind: DisconnectKind::Connect,
reason, reason,
info: self.info, info: self.info,
}), }),
)); )));
}, },
ConnectionFlavour::Reconnect => { ConnectionFlavour::Reconnect => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::DriverDisconnect(InternalDisconnect { CoreContext::DriverDisconnect(InternalDisconnect {
kind: DisconnectKind::Reconnect, kind: DisconnectKind::Reconnect,
reason, reason,
info: self.info, info: self.info,
}), }),
)); )));
}, },
} }
} }

87
src/driver/tasks/udp_rx.rs
View File

@@ -5,7 +5,7 @@ use super::{
}; };
use crate::{ use crate::{
constants::*, constants::*,
driver::DecodeMode, driver::{CryptoMode, DecodeMode},
events::{internal_data::*, CoreContext}, events::{internal_data::*, CoreContext},
}; };
use audiopus::{ use audiopus::{
@@ -53,27 +53,25 @@ enum PacketDecodeSize {
impl PacketDecodeSize { impl PacketDecodeSize {
fn bump_up(self) -> Self { fn bump_up(self) -> Self {
use PacketDecodeSize::*;
match self { match self {
TwentyMillis => ThirtyMillis, Self::TwentyMillis => Self::ThirtyMillis,
ThirtyMillis => FortyMillis, Self::ThirtyMillis => Self::FortyMillis,
FortyMillis => SixtyMillis, Self::FortyMillis => Self::SixtyMillis,
SixtyMillis | Max => Max, Self::SixtyMillis | Self::Max => Self::Max,
} }
} }
fn can_bump_up(self) -> bool { fn can_bump_up(self) -> bool {
self != PacketDecodeSize::Max self != Self::Max
} }
fn len(self) -> usize { fn len(self) -> usize {
use PacketDecodeSize::*;
match self { match self {
TwentyMillis => STEREO_FRAME_SIZE, Self::TwentyMillis => STEREO_FRAME_SIZE,
ThirtyMillis => (STEREO_FRAME_SIZE / 2) * 3, Self::ThirtyMillis => (STEREO_FRAME_SIZE / 2) * 3,
FortyMillis => 2 * STEREO_FRAME_SIZE, Self::FortyMillis => 2 * STEREO_FRAME_SIZE,
SixtyMillis => 3 * STEREO_FRAME_SIZE, Self::SixtyMillis => 3 * STEREO_FRAME_SIZE,
Max => 6 * STEREO_FRAME_SIZE, Self::Max => 6 * STEREO_FRAME_SIZE,
} }
} }
} }
@@ -86,7 +84,7 @@ enum SpeakingDelta {
} }
impl SsrcState { impl SsrcState {
fn new(pkt: RtpPacket<'_>) -> Self { fn new(pkt: &RtpPacket<'_>) -> Self {
Self { Self {
silent_frame_count: 5, // We do this to make the first speech packet fire an event. silent_frame_count: 5, // We do this to make the first speech packet fire an event.
decoder: OpusDecoder::new(SAMPLE_RATE, Channels::Stereo) decoder: OpusDecoder::new(SAMPLE_RATE, Channels::Stereo)
@@ -98,7 +96,7 @@ impl SsrcState {
fn process( fn process(
&mut self, &mut self,
pkt: RtpPacket<'_>, pkt: &RtpPacket<'_>,
data_offset: usize, data_offset: usize,
data_trailer: usize, data_trailer: usize,
decode_mode: DecodeMode, decode_mode: DecodeMode,
@@ -198,11 +196,10 @@ impl SsrcState {
// and then remember that. // and then remember that.
loop { loop {
let tried_audio_len = self.decoder.decode( let tried_audio_len = self.decoder.decode(
Some((&data[start..]).try_into()?), Some(data[start..].try_into()?),
(&mut out[..]).try_into()?, (&mut out[..]).try_into()?,
false, false,
); );
match tried_audio_len { match tried_audio_len {
Ok(audio_len) => { Ok(audio_len) => {
// Decoding to stereo: audio_len refers to sample count irrespective of channel count. // Decoding to stereo: audio_len refers to sample count irrespective of channel count.
@@ -243,7 +240,6 @@ struct UdpRx {
config: Config, config: Config,
packet_buffer: [u8; VOICE_PACKET_MAX], packet_buffer: [u8; VOICE_PACKET_MAX],
rx: Receiver<UdpRxMessage>, rx: Receiver<UdpRxMessage>,
udp_socket: Arc<UdpSocket>, udp_socket: Arc<UdpSocket>,
} }
@@ -256,15 +252,14 @@ impl UdpRx {
self.process_udp_message(interconnect, len); self.process_udp_message(interconnect, len);
} }
msg = self.rx.recv_async() => { msg = self.rx.recv_async() => {
use UdpRxMessage::*;
match msg { match msg {
Ok(ReplaceInterconnect(i)) => { Ok(UdpRxMessage::ReplaceInterconnect(i)) => {
*interconnect = i; *interconnect = i;
}, },
Ok(SetConfig(c)) => { Ok(UdpRxMessage::SetConfig(c)) => {
self.config = c; self.config = c;
}, },
Ok(Poison) | Err(_) => break, Err(flume::RecvError::Disconnected) => break,
} }
} }
} }
@@ -284,7 +279,7 @@ impl UdpRx {
match demux::demux_mut(packet) { match demux::demux_mut(packet) {
DemuxedMut::Rtp(mut rtp) => { DemuxedMut::Rtp(mut rtp) => {
if !rtp_valid(rtp.to_immutable()) { if !rtp_valid(&rtp.to_immutable()) {
error!("Illegal RTP message received."); error!("Illegal RTP message received.");
return; return;
} }
@@ -303,9 +298,10 @@ impl UdpRx {
None None
}; };
let rtp = rtp.to_immutable();
let (rtp_body_start, rtp_body_tail, decrypted) = packet_data.unwrap_or_else(|| { let (rtp_body_start, rtp_body_tail, decrypted) = packet_data.unwrap_or_else(|| {
( (
crypto_mode.payload_prefix_len(), CryptoMode::payload_prefix_len(),
crypto_mode.payload_suffix_len(), crypto_mode.payload_suffix_len(),
false, false,
) )
@@ -314,10 +310,10 @@ impl UdpRx {
let entry = self let entry = self
.decoder_map .decoder_map
.entry(rtp.get_ssrc()) .entry(rtp.get_ssrc())
.or_insert_with(|| SsrcState::new(rtp.to_immutable())); .or_insert_with(|| SsrcState::new(&rtp));
if let Ok((delta, audio)) = entry.process( if let Ok((delta, audio)) = entry.process(
rtp.to_immutable(), &rtp,
rtp_body_start, rtp_body_start,
rtp_body_tail, rtp_body_tail,
self.config.decode_mode, self.config.decode_mode,
@@ -325,32 +321,32 @@ impl UdpRx {
) { ) {
match delta { match delta {
SpeakingDelta::Start => { SpeakingDelta::Start => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::SpeakingUpdate(InternalSpeakingUpdate { CoreContext::SpeakingUpdate(InternalSpeakingUpdate {
ssrc: rtp.get_ssrc(), ssrc: rtp.get_ssrc(),
speaking: true, speaking: true,
}), }),
)); )));
}, },
SpeakingDelta::Stop => { SpeakingDelta::Stop => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::SpeakingUpdate(InternalSpeakingUpdate { CoreContext::SpeakingUpdate(InternalSpeakingUpdate {
ssrc: rtp.get_ssrc(), ssrc: rtp.get_ssrc(),
speaking: false, speaking: false,
}), }),
)); )));
}, },
_ => {}, SpeakingDelta::Same => {},
} }
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::VoicePacket(InternalVoicePacket { CoreContext::VoicePacket(InternalVoicePacket {
audio, audio,
packet: rtp.from_packet(), packet: rtp.from_packet(),
payload_offset: rtp_body_start, payload_offset: rtp_body_start,
payload_end_pad: rtp_body_tail, payload_end_pad: rtp_body_tail,
}), }),
)); )));
} else { } else {
warn!("RTP decoding/processing failed."); warn!("RTP decoding/processing failed.");
} }
@@ -370,26 +366,23 @@ impl UdpRx {
let (start, tail) = packet_data.unwrap_or_else(|| { let (start, tail) = packet_data.unwrap_or_else(|| {
( (
crypto_mode.payload_prefix_len(), CryptoMode::payload_prefix_len(),
crypto_mode.payload_suffix_len(), crypto_mode.payload_suffix_len(),
) )
}); });
let _ = drop(interconnect.events.send(EventMessage::FireCoreEvent(
interconnect CoreContext::RtcpPacket(InternalRtcpPacket {
.events packet: rtcp.from_packet(),
.send(EventMessage::FireCoreEvent(CoreContext::RtcpPacket( payload_offset: start,
InternalRtcpPacket { payload_end_pad: tail,
packet: rtcp.from_packet(), }),
payload_offset: start, )));
payload_end_pad: tail,
},
)));
}, },
DemuxedMut::FailedParse(t) => { DemuxedMut::FailedParse(t) => {
warn!("Failed to parse message of type {:?}.", t); warn!("Failed to parse message of type {:?}.", t);
}, },
_ => { DemuxedMut::TooSmall => {
warn!("Illegal UDP packet from voice server."); warn!("Illegal UDP packet from voice server.");
}, },
} }
@@ -408,7 +401,7 @@ pub(crate) async fn runner(
let mut state = UdpRx { let mut state = UdpRx {
cipher, cipher,
decoder_map: Default::default(), decoder_map: HashMap::new(),
config, config,
packet_buffer: [0u8; VOICE_PACKET_MAX], packet_buffer: [0u8; VOICE_PACKET_MAX],
rx, rx,
@@ -421,6 +414,6 @@ pub(crate) async fn runner(
} }
#[inline] #[inline]
fn rtp_valid(packet: RtpPacket<'_>) -> bool { fn rtp_valid(packet: &RtpPacket<'_>) -> bool {
packet.get_version() == RTP_VERSION && packet.get_payload_type() == RTP_PROFILE_TYPE packet.get_version() == RTP_VERSION && packet.get_payload_type() == RTP_PROFILE_TYPE
} }

10
src/driver/tasks/udp_tx.rs
View File

@@ -12,7 +12,6 @@ use tracing::{error, instrument, trace};
struct UdpTx { struct UdpTx {
ssrc: u32, ssrc: u32,
rx: Receiver<UdpTxMessage>, rx: Receiver<UdpTxMessage>,
udp_tx: Arc<UdpSocket>, udp_tx: Arc<UdpSocket>,
} }
@@ -26,7 +25,6 @@ impl UdpTx {
let mut ka_time = Instant::now() + UDP_KEEPALIVE_GAP; let mut ka_time = Instant::now() + UDP_KEEPALIVE_GAP;
loop { loop {
use UdpTxMessage::*;
match timeout_at(ka_time, self.rx.recv_async()).await { match timeout_at(ka_time, self.rx.recv_async()).await {
Err(_) => { Err(_) => {
trace!("Sending UDP Keepalive."); trace!("Sending UDP Keepalive.");
@@ -36,16 +34,12 @@ impl UdpTx {
} }
ka_time += UDP_KEEPALIVE_GAP; ka_time += UDP_KEEPALIVE_GAP;
}, },
Ok(Ok(Packet(p))) => Ok(Ok(p)) =>
if let Err(e) = self.udp_tx.send(&p[..]).await { if let Err(e) = self.udp_tx.send(&p[..]).await {
error!("Fatal UDP packet send error: {:?}.", e); error!("Fatal UDP packet send error: {:?}.", e);
break; break;
}, },
Ok(Err(e)) => { Ok(Err(flume::RecvError::Disconnected)) => {
error!("Fatal UDP packet receive error: {:?}.", e);
break;
},
Ok(Ok(Poison)) => {
break; break;
}, },
} }

16
src/driver/tasks/ws.rs
View File

@@ -140,7 +140,7 @@ impl AuxNetwork {
} }
} }
}, },
Err(_) | Ok(WsMessage::Poison) => { Err(flume::RecvError::Disconnected) => {
break; break;
}, },
} }
@@ -151,13 +151,13 @@ impl AuxNetwork {
self.dont_send = true; self.dont_send = true;
if should_reconnect { if should_reconnect {
let _ = interconnect.core.send(CoreMessage::Reconnect); drop(interconnect.core.send(CoreMessage::Reconnect));
} else { } else {
let _ = interconnect.core.send(CoreMessage::SignalWsClosure( drop(interconnect.core.send(CoreMessage::SignalWsClosure(
self.attempt_idx, self.attempt_idx,
self.info.clone(), self.info.clone(),
ws_reason, ws_reason,
)); )));
break; break;
} }
} }
@@ -186,17 +186,17 @@ impl AuxNetwork {
fn process_ws(&mut self, interconnect: &Interconnect, value: GatewayEvent) { fn process_ws(&mut self, interconnect: &Interconnect, value: GatewayEvent) {
match value { match value {
GatewayEvent::Speaking(ev) => { GatewayEvent::Speaking(ev) => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::SpeakingStateUpdate(ev), CoreContext::SpeakingStateUpdate(ev),
)); )));
}, },
GatewayEvent::ClientConnect(ev) => { GatewayEvent::ClientConnect(ev) => {
debug!("Received discontinued ClientConnect: {:?}", ev); debug!("Received discontinued ClientConnect: {:?}", ev);
}, },
GatewayEvent::ClientDisconnect(ev) => { GatewayEvent::ClientDisconnect(ev) => {
let _ = interconnect.events.send(EventMessage::FireCoreEvent( drop(interconnect.events.send(EventMessage::FireCoreEvent(
CoreContext::ClientDisconnect(ev), CoreContext::ClientDisconnect(ev),
)); )));
}, },
GatewayEvent::HeartbeatAck(ev) => { GatewayEvent::HeartbeatAck(ev) => {
if let Some(nonce) = self.last_heartbeat_nonce.take() { if let Some(nonce) = self.last_heartbeat_nonce.take() {

251
src/driver/test_config.rs Normal file
View File

@@ -0,0 +1,251 @@
#![allow(missing_docs)]
use flume::{Receiver, Sender};
use crate::{
tracks::{PlayMode, TrackHandle, TrackState},
Event,
EventContext,
EventHandler,
TrackEvent,
};
use std::time::Duration;
#[allow(dead_code)]
#[derive(Clone, Debug)]
pub enum TickStyle {
Timed,
UntimedWithExecLimit(Receiver<u64>),
}
#[derive(Clone, Debug, PartialEq)]
pub enum OutputMessage {
Passthrough(Vec<u8>),
Mixed(Vec<f32>),
Silent,
}
#[allow(dead_code)]
impl OutputMessage {
pub fn is_passthrough(&self) -> bool {
matches!(self, Self::Passthrough(_))
}
pub fn is_mixed(&self) -> bool {
matches!(self, Self::Mixed(_))
}
pub fn is_mixed_with_nonzero_signal(&self) -> bool {
if let Self::Mixed(data) = self {
data.iter().any(|v| *v != 0.0f32)
} else {
false
}
}
pub fn is_explicit_silence(&self) -> bool {
*self == Self::Silent
}
}
#[allow(dead_code)]
#[derive(Clone, Debug)]
pub enum OutputMode {
Raw(Sender<TickMessage<OutputMessage>>),
Rtp(Sender<TickMessage<Vec<u8>>>),
}
#[allow(dead_code)]
#[derive(Clone, Debug)]
pub enum TickMessage<T> {
El(T),
NoEl,
}
impl<T> From<T> for TickMessage<T> {
fn from(val: T) -> Self {
TickMessage::El(val)
}
}
impl From<TickMessage<OutputMessage>> for OutputPacket {
fn from(val: TickMessage<OutputMessage>) -> Self {
match val {
TickMessage::El(e) => OutputPacket::Raw(e),
TickMessage::NoEl => OutputPacket::Empty,
}
}
}
impl From<TickMessage<Vec<u8>>> for OutputPacket {
fn from(val: TickMessage<Vec<u8>>) -> Self {
match val {
TickMessage::El(e) => OutputPacket::Rtp(e),
TickMessage::NoEl => OutputPacket::Empty,
}
}
}
#[derive(Clone, Debug, PartialEq)]
pub enum OutputPacket {
Raw(OutputMessage),
Rtp(Vec<u8>),
Empty,
}
impl OutputPacket {
pub fn raw(&self) -> Option<&OutputMessage> {
if let Self::Raw(o) = self {
Some(o)
} else {
None
}
}
}
#[derive(Clone, Debug)]
pub enum OutputReceiver {
Raw(Receiver<TickMessage<OutputMessage>>),
Rtp(Receiver<TickMessage<Vec<u8>>>),
}
#[derive(Clone)]
pub struct DriverTestHandle {
pub rx: OutputReceiver,
pub tx: Sender<u64>,
}
impl DriverTestHandle {
pub fn recv(&self) -> OutputPacket {
match &self.rx {
OutputReceiver::Raw(rx) => rx.recv().unwrap().into(),
OutputReceiver::Rtp(rx) => rx.recv().unwrap().into(),
}
}
pub async fn recv_async(&self) -> OutputPacket {
match &self.rx {
OutputReceiver::Raw(rx) => rx.recv_async().await.unwrap().into(),
OutputReceiver::Rtp(rx) => rx.recv_async().await.unwrap().into(),
}
}
pub fn len(&self) -> usize {
match &self.rx {
OutputReceiver::Raw(rx) => rx.len(),
OutputReceiver::Rtp(rx) => rx.len(),
}
}
pub fn wait(&self, n_ticks: u64) {
for _i in 0..n_ticks {
drop(self.recv());
}
}
pub async fn wait_async(&self, n_ticks: u64) {
for _i in 0..n_ticks {
drop(self.recv_async().await);
}
}
pub async fn spawn_ticker(&self) {
let remote = self.clone();
tokio::spawn(async move {
loop {
remote.skip(1).await;
tokio::time::sleep(Duration::from_millis(1)).await;
}
});
}
pub fn wait_noisy(&self, n_ticks: u64) {
for _i in 0..n_ticks {
match self.recv() {
OutputPacket::Empty => eprintln!("pkt: Nothing"),
OutputPacket::Rtp(p) => eprintln!("pkt: RTP[{}B]", p.len()),
OutputPacket::Raw(OutputMessage::Silent) => eprintln!("pkt: Raw-Silent"),
OutputPacket::Raw(OutputMessage::Passthrough(p)) =>
eprintln!("pkt: Raw-Passthrough[{}B]", p.len()),
OutputPacket::Raw(OutputMessage::Mixed(p)) =>
eprintln!("pkt: Raw-Mixed[{}B]", p.len()),
}
}
}
pub async fn skip(&self, n_ticks: u64) {
self.tick(n_ticks);
self.wait_async(n_ticks).await;
}
pub fn tick(&self, n_ticks: u64) {
if n_ticks == 0 {
panic!("Number of ticks to advance driver/mixer must be >= 1.");
}
self.tx.send(n_ticks).unwrap();
}
pub async fn ready_track(
&self,
handle: &TrackHandle,
tick_wait: Option<Duration>,
) -> TrackState {
let (tx, rx) = flume::bounded(1);
let (err_tx, err_rx) = flume::bounded(1);
struct SongPlayable {
tx: Sender<TrackState>,
}
#[async_trait::async_trait]
impl EventHandler for SongPlayable {
async fn act(&self, ctx: &crate::EventContext<'_>) -> Option<Event> {
if let EventContext::Track(&[(state, _)]) = ctx {
drop(self.tx.send(state.clone()));
}
Some(Event::Cancel)
}
}
struct SongErred {
tx: Sender<PlayMode>,
}
#[async_trait::async_trait]
impl EventHandler for SongErred {
async fn act(&self, ctx: &crate::EventContext<'_>) -> Option<Event> {
if let EventContext::Track(&[(state, _)]) = ctx {
drop(self.tx.send(state.playing.clone()));
}
Some(Event::Cancel)
}
}
handle
.add_event(Event::Track(TrackEvent::Playable), SongPlayable { tx })
.expect("Adding track evt should not fail before any ticks.");
handle
.add_event(Event::Track(TrackEvent::Error), SongErred { tx: err_tx })
.expect("Adding track evt should not fail before any ticks.");
loop {
self.tick(1);
tokio::time::sleep(tick_wait.unwrap_or_else(|| Duration::from_millis(20))).await;
self.wait_async(1).await;
match err_rx.try_recv() {
Ok(e) => panic!("Error reported on track: {:?}", e),
Err(flume::TryRecvError::Empty | flume::TryRecvError::Disconnected) => {},
}
match rx.try_recv() {
Ok(val) => return val,
Err(flume::TryRecvError::Disconnected) => panic!(),
Err(flume::TryRecvError::Empty) => {},
}
}
}
}

40
src/error.rs
View File

@@ -4,12 +4,12 @@
use futures::channel::mpsc::TrySendError; use futures::channel::mpsc::TrySendError;
#[cfg(feature = "serenity")] #[cfg(feature = "serenity")]
use serenity::gateway::InterMessage; use serenity::gateway::InterMessage;
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
use std::{error::Error, fmt}; use std::{error::Error, fmt};
#[cfg(feature = "twilight")] #[cfg(feature = "twilight")]
use twilight_gateway::{cluster::ClusterCommandError, shard::CommandError}; use twilight_gateway::{cluster::ClusterCommandError, shard::CommandError};
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
#[derive(Debug)] #[derive(Debug)]
#[non_exhaustive] #[non_exhaustive]
/// Error returned when a manager or call handler is /// Error returned when a manager or call handler is
@@ -36,11 +36,7 @@ pub enum JoinError {
/// ///
/// [the `Call`'s configuration]: crate::Config /// [the `Call`'s configuration]: crate::Config
TimedOut, TimedOut,
/// The given guild ID was zero. #[cfg(feature = "driver")]
IllegalGuild,
/// The given channel ID was zero.
IllegalChannel,
#[cfg(feature = "driver-core")]
/// The driver failed to establish a voice connection. /// The driver failed to establish a voice connection.
/// ///
/// *Users should `leave` the server on the gateway before /// *Users should `leave` the server on the gateway before
@@ -57,7 +53,7 @@ pub enum JoinError {
TwilightShard(CommandError), TwilightShard(CommandError),
} }
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
impl JoinError { impl JoinError {
/// Indicates whether this failure may have left (or been /// Indicates whether this failure may have left (or been
/// caused by) Discord's gateway state being in an /// caused by) Discord's gateway state being in an
@@ -69,7 +65,7 @@ impl JoinError {
matches!(self, JoinError::TimedOut) matches!(self, JoinError::TimedOut)
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Indicates whether this failure can be reattempted via /// Indicates whether this failure can be reattempted via
/// [`Driver::connect`] with retreived connection info. /// [`Driver::connect`] with retreived connection info.
/// ///
@@ -82,7 +78,7 @@ impl JoinError {
} }
} }
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
impl fmt::Display for JoinError { impl fmt::Display for JoinError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "failed to join voice channel: ")?; write!(f, "failed to join voice channel: ")?;
@@ -91,9 +87,7 @@ impl fmt::Display for JoinError {
JoinError::NoSender => write!(f, "no gateway destination"), JoinError::NoSender => write!(f, "no gateway destination"),
JoinError::NoCall => write!(f, "tried to leave a non-existent call"), JoinError::NoCall => write!(f, "tried to leave a non-existent call"),
JoinError::TimedOut => write!(f, "gateway response from Discord timed out"), JoinError::TimedOut => write!(f, "gateway response from Discord timed out"),
JoinError::IllegalGuild => write!(f, "target guild ID was zero"), #[cfg(feature = "driver")]
JoinError::IllegalChannel => write!(f, "target channel ID was zero"),
#[cfg(feature = "driver-core")]
JoinError::Driver(_) => write!(f, "establishing connection failed"), JoinError::Driver(_) => write!(f, "establishing connection failed"),
#[cfg(feature = "serenity")] #[cfg(feature = "serenity")]
JoinError::Serenity(e) => e.fmt(f), JoinError::Serenity(e) => e.fmt(f),
@@ -105,7 +99,7 @@ impl fmt::Display for JoinError {
} }
} }
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
impl Error for JoinError { impl Error for JoinError {
fn source(&self) -> Option<&(dyn Error + 'static)> { fn source(&self) -> Option<&(dyn Error + 'static)> {
match self { match self {
@@ -113,9 +107,7 @@ impl Error for JoinError {
JoinError::NoSender => None, JoinError::NoSender => None,
JoinError::NoCall => None, JoinError::NoCall => None,
JoinError::TimedOut => None, JoinError::TimedOut => None,
JoinError::IllegalGuild => None, #[cfg(feature = "driver")]
JoinError::IllegalChannel => None,
#[cfg(feature = "driver-core")]
JoinError::Driver(e) => Some(e), JoinError::Driver(e) => Some(e),
#[cfg(feature = "serenity")] #[cfg(feature = "serenity")]
JoinError::Serenity(e) => e.source(), JoinError::Serenity(e) => e.source(),
@@ -127,40 +119,40 @@ impl Error for JoinError {
} }
} }
#[cfg(all(feature = "serenity", feature = "gateway-core"))] #[cfg(all(feature = "serenity", feature = "gateway"))]
impl From<TrySendError<InterMessage>> for JoinError { impl From<TrySendError<InterMessage>> for JoinError {
fn from(e: TrySendError<InterMessage>) -> Self { fn from(e: TrySendError<InterMessage>) -> Self {
JoinError::Serenity(e) JoinError::Serenity(e)
} }
} }
#[cfg(all(feature = "twilight", feature = "gateway-core"))] #[cfg(all(feature = "twilight", feature = "gateway"))]
impl From<CommandError> for JoinError { impl From<CommandError> for JoinError {
fn from(e: CommandError) -> Self { fn from(e: CommandError) -> Self {
JoinError::TwilightShard(e) JoinError::TwilightShard(e)
} }
} }
#[cfg(all(feature = "twilight", feature = "gateway-core"))] #[cfg(all(feature = "twilight", feature = "gateway"))]
impl From<ClusterCommandError> for JoinError { impl From<ClusterCommandError> for JoinError {
fn from(e: ClusterCommandError) -> Self { fn from(e: ClusterCommandError) -> Self {
JoinError::TwilightCluster(e) JoinError::TwilightCluster(e)
} }
} }
#[cfg(all(feature = "driver-core", feature = "gateway-core"))] #[cfg(all(feature = "driver", feature = "gateway"))]
impl From<ConnectionError> for JoinError { impl From<ConnectionError> for JoinError {
fn from(e: ConnectionError) -> Self { fn from(e: ConnectionError) -> Self {
JoinError::Driver(e) JoinError::Driver(e)
} }
} }
#[cfg(feature = "gateway-core")] #[cfg(feature = "gateway")]
/// Convenience type for Discord gateway error handling. /// Convenience type for Discord gateway error handling.
pub type JoinResult<T> = Result<T, JoinError>; pub type JoinResult<T> = Result<T, JoinError>;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
pub use crate::{ pub use crate::{
driver::connection::error::{Error as ConnectionError, Result as ConnectionResult}, driver::connection::error::{Error as ConnectionError, Result as ConnectionResult},
tracks::{TrackError, TrackResult}, tracks::{ControlError, PlayError, TrackResult},
}; };

26
src/events/context/data/disconnect.rs
View File

@@ -84,21 +84,19 @@ pub enum DisconnectReason {
impl From<&ConnectionError> for DisconnectReason { impl From<&ConnectionError> for DisconnectReason {
fn from(e: &ConnectionError) -> Self { fn from(e: &ConnectionError) -> Self {
use ConnectionError::*;
match e { match e {
AttemptDiscarded => Self::AttemptDiscarded, ConnectionError::AttemptDiscarded => Self::AttemptDiscarded,
CryptoModeInvalid ConnectionError::CryptoModeInvalid
| CryptoModeUnavailable | ConnectionError::CryptoModeUnavailable
| EndpointUrl | ConnectionError::EndpointUrl
| ExpectedHandshake | ConnectionError::ExpectedHandshake
| IllegalDiscoveryResponse | ConnectionError::IllegalDiscoveryResponse
| IllegalIp | ConnectionError::IllegalIp
| Json(_) => Self::ProtocolViolation, | ConnectionError::Json(_) => Self::ProtocolViolation,
Io(_) => Self::Io, ConnectionError::Io(_) => Self::Io,
Crypto(_) | InterconnectFailure(_) => Self::Internal, ConnectionError::Crypto(_) | ConnectionError::InterconnectFailure(_) => Self::Internal,
Ws(ws) => ws.into(), ConnectionError::Ws(ws) => ws.into(),
TimedOut => Self::TimedOut, ConnectionError::TimedOut => Self::TimedOut,
} }
} }
} }

39
src/events/context/mod.rs
View File

@@ -62,17 +62,17 @@ pub enum CoreContext {
impl<'a> CoreContext { impl<'a> CoreContext {
pub(crate) fn to_user_context(&'a self) -> EventContext<'a> { pub(crate) fn to_user_context(&'a self) -> EventContext<'a> {
use CoreContext::*;
match self { match self {
SpeakingStateUpdate(evt) => EventContext::SpeakingStateUpdate(*evt), Self::SpeakingStateUpdate(evt) => EventContext::SpeakingStateUpdate(*evt),
SpeakingUpdate(evt) => EventContext::SpeakingUpdate(SpeakingUpdateData::from(evt)), Self::SpeakingUpdate(evt) =>
VoicePacket(evt) => EventContext::VoicePacket(VoiceData::from(evt)), EventContext::SpeakingUpdate(SpeakingUpdateData::from(evt)),
RtcpPacket(evt) => EventContext::RtcpPacket(RtcpData::from(evt)), Self::VoicePacket(evt) => EventContext::VoicePacket(VoiceData::from(evt)),
ClientDisconnect(evt) => EventContext::ClientDisconnect(*evt), Self::RtcpPacket(evt) => EventContext::RtcpPacket(RtcpData::from(evt)),
DriverConnect(evt) => EventContext::DriverConnect(ConnectData::from(evt)), Self::ClientDisconnect(evt) => EventContext::ClientDisconnect(*evt),
DriverReconnect(evt) => EventContext::DriverReconnect(ConnectData::from(evt)), Self::DriverConnect(evt) => EventContext::DriverConnect(ConnectData::from(evt)),
DriverDisconnect(evt) => EventContext::DriverDisconnect(DisconnectData::from(evt)), Self::DriverReconnect(evt) => EventContext::DriverReconnect(ConnectData::from(evt)),
Self::DriverDisconnect(evt) =>
EventContext::DriverDisconnect(DisconnectData::from(evt)),
} }
} }
} }
@@ -80,18 +80,17 @@ impl<'a> CoreContext {
impl EventContext<'_> { impl EventContext<'_> {
/// Retreive the event class for an event (i.e., when matching) /// Retreive the event class for an event (i.e., when matching)
/// an event against the registered listeners. /// an event against the registered listeners.
#[must_use]
pub fn to_core_event(&self) -> Option<CoreEvent> { pub fn to_core_event(&self) -> Option<CoreEvent> {
use EventContext::*;
match self { match self {
SpeakingStateUpdate(_) => Some(CoreEvent::SpeakingStateUpdate), Self::SpeakingStateUpdate(_) => Some(CoreEvent::SpeakingStateUpdate),
SpeakingUpdate(_) => Some(CoreEvent::SpeakingUpdate), Self::SpeakingUpdate(_) => Some(CoreEvent::SpeakingUpdate),
VoicePacket(_) => Some(CoreEvent::VoicePacket), Self::VoicePacket(_) => Some(CoreEvent::VoicePacket),
RtcpPacket(_) => Some(CoreEvent::RtcpPacket), Self::RtcpPacket(_) => Some(CoreEvent::RtcpPacket),
ClientDisconnect(_) => Some(CoreEvent::ClientDisconnect), Self::ClientDisconnect(_) => Some(CoreEvent::ClientDisconnect),
DriverConnect(_) => Some(CoreEvent::DriverConnect), Self::DriverConnect(_) => Some(CoreEvent::DriverConnect),
DriverReconnect(_) => Some(CoreEvent::DriverReconnect), Self::DriverReconnect(_) => Some(CoreEvent::DriverReconnect),
DriverDisconnect(_) => Some(CoreEvent::DriverDisconnect), Self::DriverDisconnect(_) => Some(CoreEvent::DriverDisconnect),
_ => None, _ => None,
} }
} }

2
src/events/core.rs
View File

@@ -9,7 +9,7 @@
/// when a client leaves the session ([`ClientDisconnect`]), voice packets ([`VoicePacket`]), and /// when a client leaves the session ([`ClientDisconnect`]), voice packets ([`VoicePacket`]), and
/// telemetry data ([`RtcpPacket`]). The format of voice packets is described by [`VoiceData`]. /// telemetry data ([`RtcpPacket`]). The format of voice packets is described by [`VoiceData`].
/// ///
/// To detect when a user connects, you must correlate gateway (e.g., VoiceStateUpdate) events /// To detect when a user connects, you must correlate gateway (e.g., `VoiceStateUpdate`) events
/// from the main part of your bot. /// from the main part of your bot.
/// ///
/// To obtain a user's SSRC, you must use [`SpeakingStateUpdate`] events. /// To obtain a user's SSRC, you must use [`SpeakingStateUpdate`] events.

44
src/events/store.rs
View File

@@ -1,7 +1,7 @@
use super::*; use super::*;
use crate::{ use crate::{
constants::*, constants::*,
tracks::{PlayMode, TrackHandle, TrackState}, tracks::{ReadyState, TrackHandle, TrackState},
}; };
use std::{ use std::{
collections::{BinaryHeap, HashMap}, collections::{BinaryHeap, HashMap},
@@ -24,8 +24,9 @@ pub struct EventStore {
impl EventStore { impl EventStore {
/// Creates a new event store to be used globally. /// Creates a new event store to be used globally.
#[must_use]
pub fn new() -> Self { pub fn new() -> Self {
Default::default() Self::default()
} }
/// Creates a new event store to be used within a [`Track`]. /// Creates a new event store to be used within a [`Track`].
@@ -34,6 +35,7 @@ impl EventStore {
/// a track has been registered. /// a track has been registered.
/// ///
/// [`Track`]: crate::tracks::Track /// [`Track`]: crate::tracks::Track
#[must_use]
pub fn new_local() -> Self { pub fn new_local() -> Self {
EventStore { EventStore {
local_only: true, local_only: true,
@@ -53,21 +55,20 @@ impl EventStore {
return; return;
} }
use Event::*;
match evt.event { match evt.event {
Core(c) => { Event::Core(c) => {
self.untimed self.untimed
.entry(c.into()) .entry(c.into())
.or_insert_with(Vec::new) .or_insert_with(Vec::new)
.push(evt); .push(evt);
}, },
Track(t) => { Event::Track(t) => {
self.untimed self.untimed
.entry(t.into()) .entry(t.into())
.or_insert_with(Vec::new) .or_insert_with(Vec::new)
.push(evt); .push(evt);
}, },
Delayed(_) | Periodic(_, _) => { Event::Delayed(_) | Event::Periodic(_, _) => {
self.timed.push(evt); self.timed.push(evt);
}, },
_ => { _ => {
@@ -105,15 +106,12 @@ impl EventStore {
/// Processes all events due up to and including `now`. /// Processes all events due up to and including `now`.
pub(crate) fn timed_event_ready(&self, now: Duration) -> bool { pub(crate) fn timed_event_ready(&self, now: Duration) -> bool {
self.timed self.timed.peek().map_or(false, |evt| {
.peek() evt.fire_time
.map(|evt| { .as_ref()
evt.fire_time .expect("Timed event must have a fire_time.")
.as_ref() <= &now
.expect("Timed event must have a fire_time.") })
<= &now
})
.unwrap_or(false)
} }
/// Processes all events attached to the given track event. /// Processes all events attached to the given track event.
@@ -183,9 +181,9 @@ impl GlobalEvents {
pub(crate) async fn tick( pub(crate) async fn tick(
&mut self, &mut self,
events: &mut Vec<EventStore>, events: &mut [EventStore],
states: &mut Vec<TrackState>, states: &mut [TrackState],
handles: &mut Vec<TrackHandle>, handles: &mut [TrackHandle],
) { ) {
// Global timed events // Global timed events
self.time += TIMESTEP_LENGTH; self.time += TIMESTEP_LENGTH;
@@ -199,7 +197,7 @@ impl GlobalEvents {
// Local timed events // Local timed events
for (i, state) in states.iter_mut().enumerate() { for (i, state) in states.iter_mut().enumerate() {
if state.playing == PlayMode::Play { if state.playing.is_playing() && state.ready == ReadyState::Playable {
state.step_frame(); state.step_frame();
let event_store = events let event_store = events
@@ -215,7 +213,7 @@ impl GlobalEvents {
} }
} }
for (evt, indices) in self.awaiting_tick.iter() { for (evt, indices) in &self.awaiting_tick {
let untimed = (*evt).into(); let untimed = (*evt).into();
if !indices.is_empty() { if !indices.is_empty() {
@@ -223,7 +221,7 @@ impl GlobalEvents {
} }
// Local untimed track events. // Local untimed track events.
for &i in indices.iter() { for &i in indices {
let event_store = events let event_store = events
.get_mut(i) .get_mut(i)
.expect("Missing store index for Tick (local untimed)."); .expect("Missing store index for Tick (local untimed).");
@@ -261,12 +259,12 @@ impl GlobalEvents {
self.store self.store
.process_untimed(self.time, untimed, EventContext::Track(&global_ctx[..])) .process_untimed(self.time, untimed, EventContext::Track(&global_ctx[..]))
.await .await;
} }
} }
// Now drain vecs. // Now drain vecs.
for (_evt, indices) in self.awaiting_tick.iter_mut() { for indices in self.awaiting_tick.values_mut() {
indices.clear(); indices.clear();
} }
} }

12
src/events/track.rs
View File

@@ -1,3 +1,6 @@
// TODO: Could this be a bitset? Could accelerate lookups,
// allow easy joint subscription & remove Vecs for related evt handling?
/// Track events correspond to certain actions or changes /// Track events correspond to certain actions or changes
/// of state, such as a track finishing, looping, or being /// of state, such as a track finishing, looping, or being
/// manually stopped. Voice core events occur on receipt of /// manually stopped. Voice core events occur on receipt of
@@ -14,7 +17,8 @@ pub enum TrackEvent {
/// ///
/// This event will not fire when a track first starts, /// This event will not fire when a track first starts,
/// but will fire when a track changes from, e.g., paused to playing. /// but will fire when a track changes from, e.g., paused to playing.
/// This is most relevant for queue users. /// This is most relevant for queue users: queued tracks placed into a
/// non-empty queue are initlally paused, and are later moved to `Play`.
Play, Play,
/// The attached track has been paused. /// The attached track has been paused.
Pause, Pause,
@@ -22,4 +26,10 @@ pub enum TrackEvent {
End, End,
/// The attached track has looped. /// The attached track has looped.
Loop, Loop,
/// The attached track is being readied or recreated.
Preparing,
/// The attached track has become playable.
Playable,
/// The attached track has encountered a runtime or initialisation error.
Error,
} }

45
src/handler.rs
View File

@@ -1,4 +1,4 @@
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use crate::{driver::Driver, error::ConnectionResult}; use crate::{driver::Driver, error::ConnectionResult};
use crate::{ use crate::{
error::{JoinError, JoinResult}, error::{JoinError, JoinResult},
@@ -12,7 +12,7 @@ use flume::Sender;
use std::fmt::Debug; use std::fmt::Debug;
use tracing::instrument; use tracing::instrument;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use std::ops::{Deref, DerefMut}; use std::ops::{Deref, DerefMut};
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
@@ -24,7 +24,7 @@ enum Return {
// second indicates that the driver successfully connected. // second indicates that the driver successfully connected.
// The first is needed to cancel a timeout as the driver can/should // The first is needed to cancel a timeout as the driver can/should
// have separate connection timing/retry config. // have separate connection timing/retry config.
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
Conn(Sender<()>, Sender<ConnectionResult<()>>), Conn(Sender<()>, Sender<ConnectionResult<()>>),
} }
@@ -37,12 +37,12 @@ enum Return {
/// [`Driver`]: struct@Driver /// [`Driver`]: struct@Driver
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct Call { pub struct Call {
#[cfg(not(feature = "driver-core"))] #[cfg(not(feature = "driver"))]
config: Config, config: Config,
connection: Option<(ConnectionProgress, Return)>, connection: Option<(ConnectionProgress, Return)>,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// The internal controller of the voice connection monitor thread. /// The internal controller of the voice connection monitor thread.
driver: Driver, driver: Driver,
@@ -73,12 +73,7 @@ impl Call {
G: Into<GuildId> + Debug, G: Into<GuildId> + Debug,
U: Into<UserId> + Debug, U: Into<UserId> + Debug,
{ {
Self::new_raw_cfg( Self::new_raw_cfg(guild_id.into(), Some(ws), user_id.into(), Config::default())
guild_id.into(),
Some(ws),
user_id.into(),
Default::default(),
)
} }
/// Creates a new Call, configuring the driver as specified. /// Creates a new Call, configuring the driver as specified.
@@ -107,7 +102,7 @@ impl Call {
G: Into<GuildId> + Debug, G: Into<GuildId> + Debug,
U: Into<UserId> + Debug, U: Into<UserId> + Debug,
{ {
Self::new_raw_cfg(guild_id.into(), None, user_id.into(), Default::default()) Self::new_raw_cfg(guild_id.into(), None, user_id.into(), Config::default())
} }
/// Creates a new standalone Call from the given configuration file. /// Creates a new standalone Call from the given configuration file.
@@ -123,10 +118,10 @@ impl Call {
fn new_raw_cfg(guild_id: GuildId, ws: Option<Shard>, user_id: UserId, config: Config) -> Self { fn new_raw_cfg(guild_id: GuildId, ws: Option<Shard>, user_id: UserId, config: Config) -> Self {
Call { Call {
#[cfg(not(feature = "driver-core"))] #[cfg(not(feature = "driver"))]
config, config,
connection: None, connection: None,
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
driver: Driver::new(config), driver: Driver::new(config),
guild_id, guild_id,
self_deaf: false, self_deaf: false,
@@ -141,9 +136,9 @@ impl Call {
match &self.connection { match &self.connection {
Some((ConnectionProgress::Complete(c), Return::Info(tx))) => { Some((ConnectionProgress::Complete(c), Return::Info(tx))) => {
// It's okay if the receiver hung up. // It's okay if the receiver hung up.
let _ = tx.send(c.clone()); drop(tx.send(c.clone()));
}, },
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
Some((ConnectionProgress::Complete(c), Return::Conn(first_tx, driver_tx))) => { Some((ConnectionProgress::Complete(c), Return::Conn(first_tx, driver_tx))) => {
// It's okay if the receiver hung up. // It's okay if the receiver hung up.
let _ = first_tx.send(()); let _ = first_tx.send(());
@@ -195,7 +190,7 @@ impl Call {
self.leave().await?; self.leave().await?;
true true
} else if conn.0.channel_id() == channel_id { } else if conn.0.channel_id() == channel_id {
let _ = tx.send(completion_generator(self)); drop(tx.send(completion_generator(self)));
false false
} else { } else {
// not in progress, and/or a channel change. // not in progress, and/or a channel change.
@@ -206,7 +201,7 @@ impl Call {
}) })
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
/// Connect or switch to the given voice channel by its Id. /// Connect or switch to the given voice channel by its Id.
/// ///
/// This function acts as a future in two stages: /// This function acts as a future in two stages:
@@ -227,7 +222,7 @@ impl Call {
self._join(channel_id.into()).await self._join(channel_id.into()).await
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
async fn _join(&mut self, channel_id: ChannelId) -> JoinResult<Join> { async fn _join(&mut self, channel_id: ChannelId) -> JoinResult<Join> {
let (tx, rx) = flume::unbounded(); let (tx, rx) = flume::unbounded();
let (gw_tx, gw_rx) = flume::unbounded(); let (gw_tx, gw_rx) = flume::unbounded();
@@ -359,7 +354,7 @@ impl Call {
fn leave_local(&mut self) { fn leave_local(&mut self) {
self.connection = None; self.connection = None;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
self.driver.leave(); self.driver.leave();
} }
@@ -376,7 +371,7 @@ impl Call {
pub async fn mute(&mut self, mute: bool) -> JoinResult<()> { pub async fn mute(&mut self, mute: bool) -> JoinResult<()> {
self.self_mute = mute; self.self_mute = mute;
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
self.driver.mute(mute); self.driver.mute(mute);
self.update().await self.update().await
@@ -419,7 +414,7 @@ impl Call {
where where
C: Into<ChannelId> + Debug, C: Into<ChannelId> + Debug,
{ {
self._update_state(session_id, channel_id.map(|c| c.into())) self._update_state(session_id, channel_id.map(Into::into));
} }
fn _update_state(&mut self, session_id: String, channel_id: Option<ChannelId>) { fn _update_state(&mut self, session_id: String, channel_id: Option<ChannelId>) {
@@ -460,7 +455,7 @@ impl Call {
} }
} }
#[cfg(not(feature = "driver-core"))] #[cfg(not(feature = "driver"))]
impl Call { impl Call {
/// Access this call handler's configuration. /// Access this call handler's configuration.
pub fn config(&self) -> &Config { pub fn config(&self) -> &Config {
@@ -478,7 +473,7 @@ impl Call {
} }
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
impl Deref for Call { impl Deref for Call {
type Target = Driver; type Target = Driver;
@@ -487,7 +482,7 @@ impl Deref for Call {
} }
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
impl DerefMut for Call { impl DerefMut for Call {
fn deref_mut(&mut self) -> &mut Self::Target { fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.driver &mut self.driver

45
src/id.rs
View File

@@ -1,6 +1,6 @@
//! Newtypes around Discord IDs for library cross-compatibility. //! Newtypes around Discord IDs for library cross-compatibility.
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
use crate::model::id::{GuildId as DriverGuild, UserId as DriverUser}; use crate::model::id::{GuildId as DriverGuild, UserId as DriverUser};
#[cfg(feature = "serenity")] #[cfg(feature = "serenity")]
use serenity::model::id::{ use serenity::model::id::{
@@ -8,7 +8,10 @@ use serenity::model::id::{
GuildId as SerenityGuild, GuildId as SerenityGuild,
UserId as SerenityUser, UserId as SerenityUser,
}; };
use std::fmt::{Display, Formatter, Result as FmtResult}; use std::{
fmt::{Display, Formatter, Result as FmtResult},
num::NonZeroU64,
};
#[cfg(feature = "twilight")] #[cfg(feature = "twilight")]
use twilight_model::id::{ use twilight_model::id::{
marker::{ChannelMarker, GuildMarker, UserMarker}, marker::{ChannelMarker, GuildMarker, UserMarker},
@@ -16,16 +19,16 @@ use twilight_model::id::{
}; };
/// ID of a Discord voice/text channel. /// ID of a Discord voice/text channel.
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq)] #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
pub struct ChannelId(pub u64); pub struct ChannelId(pub NonZeroU64);
/// ID of a Discord guild (colloquially, "server"). /// ID of a Discord guild (colloquially, "server").
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq)] #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
pub struct GuildId(pub u64); pub struct GuildId(pub NonZeroU64);
/// ID of a Discord user. /// ID of a Discord user.
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq)] #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)]
pub struct UserId(pub u64); pub struct UserId(pub NonZeroU64);
impl Display for ChannelId { impl Display for ChannelId {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult { fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
@@ -33,8 +36,8 @@ impl Display for ChannelId {
} }
} }
impl From<u64> for ChannelId { impl From<NonZeroU64> for ChannelId {
fn from(id: u64) -> Self { fn from(id: NonZeroU64) -> Self {
Self(id) Self(id)
} }
} }
@@ -49,7 +52,7 @@ impl From<SerenityChannel> for ChannelId {
#[cfg(feature = "twilight")] #[cfg(feature = "twilight")]
impl From<TwilightId<ChannelMarker>> for ChannelId { impl From<TwilightId<ChannelMarker>> for ChannelId {
fn from(id: TwilightId<ChannelMarker>) -> Self { fn from(id: TwilightId<ChannelMarker>) -> Self {
Self(id.get().into()) Self(id.into_nonzero())
} }
} }
@@ -59,8 +62,8 @@ impl Display for GuildId {
} }
} }
impl From<u64> for GuildId { impl From<NonZeroU64> for GuildId {
fn from(id: u64) -> Self { fn from(id: NonZeroU64) -> Self {
Self(id) Self(id)
} }
} }
@@ -72,17 +75,17 @@ impl From<SerenityGuild> for GuildId {
} }
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
impl From<GuildId> for DriverGuild { impl From<GuildId> for DriverGuild {
fn from(id: GuildId) -> Self { fn from(id: GuildId) -> Self {
Self(id.0) Self(id.0.get())
} }
} }
#[cfg(feature = "twilight")] #[cfg(feature = "twilight")]
impl From<TwilightId<GuildMarker>> for GuildId { impl From<TwilightId<GuildMarker>> for GuildId {
fn from(id: TwilightId<GuildMarker>) -> Self { fn from(id: TwilightId<GuildMarker>) -> Self {
Self(id.get().into()) Self(id.into_nonzero())
} }
} }
@@ -92,8 +95,8 @@ impl Display for UserId {
} }
} }
impl From<u64> for UserId { impl From<NonZeroU64> for UserId {
fn from(id: u64) -> Self { fn from(id: NonZeroU64) -> Self {
Self(id) Self(id)
} }
} }
@@ -105,16 +108,16 @@ impl From<SerenityUser> for UserId {
} }
} }
#[cfg(feature = "driver-core")] #[cfg(feature = "driver")]
impl From<UserId> for DriverUser { impl From<UserId> for DriverUser {
fn from(id: UserId) -> Self { fn from(id: UserId) -> Self {
Self(id.0) Self(id.0.get())
} }
} }
#[cfg(feature = "twilight")] #[cfg(feature = "twilight")]
impl From<TwilightId<UserMarker>> for UserId { impl From<TwilightId<UserMarker>> for UserId {
fn from(id: TwilightId<UserMarker>) -> Self { fn from(id: TwilightId<UserMarker>) -> Self {
Self(id.get().into()) Self(id.into_nonzero())
} }
} }

33
src/info.rs
View File

@@ -13,15 +13,17 @@ impl ConnectionProgress {
channel_id, channel_id,
guild_id, guild_id,
user_id, user_id,
..Default::default() token: None,
endpoint: None,
session_id: None,
}) })
} }
pub(crate) fn get_connection_info(&self) -> Option<&ConnectionInfo> { pub(crate) fn get_connection_info(&self) -> Option<&ConnectionInfo> {
use ConnectionProgress::*; if let Self::Complete(c) = self {
match self { Some(c)
Complete(c) => Some(c), } else {
_ => None, None
} }
} }
@@ -53,10 +55,7 @@ impl ConnectionProgress {
} }
pub(crate) fn info(&self) -> Option<ConnectionInfo> { pub(crate) fn info(&self) -> Option<ConnectionInfo> {
match self { self.get_connection_info().cloned()
ConnectionProgress::Complete(conn_info) => Some(conn_info.clone()),
_ => None,
}
} }
pub(crate) fn apply_state_update(&mut self, session_id: String, channel_id: ChannelId) -> bool { pub(crate) fn apply_state_update(&mut self, session_id: String, channel_id: ChannelId) -> bool {
@@ -65,26 +64,24 @@ impl ConnectionProgress {
*self = ConnectionProgress::new(self.guild_id(), self.user_id(), channel_id); *self = ConnectionProgress::new(self.guild_id(), self.user_id(), channel_id);
} }
use ConnectionProgress::*;
match self { match self {
Complete(c) => { Self::Complete(c) => {
let should_reconn = c.session_id != session_id; let should_reconn = c.session_id != session_id;
c.session_id = session_id; c.session_id = session_id;
should_reconn should_reconn
}, },
Incomplete(i) => i Self::Incomplete(i) => i
.apply_state_update(session_id, channel_id) .apply_state_update(session_id, channel_id)
.map(|info| { .map(|info| {
*self = Complete(info); *self = Self::Complete(info);
}) })
.is_some(), .is_some(),
} }
} }
pub(crate) fn apply_server_update(&mut self, endpoint: String, token: String) -> bool { pub(crate) fn apply_server_update(&mut self, endpoint: String, token: String) -> bool {
use ConnectionProgress::*;
match self { match self {
Complete(c) => { Self::Complete(c) => {
let should_reconn = c.endpoint != endpoint || c.token != token; let should_reconn = c.endpoint != endpoint || c.token != token;
c.endpoint = endpoint; c.endpoint = endpoint;
@@ -92,10 +89,10 @@ impl ConnectionProgress {
should_reconn should_reconn
}, },
Incomplete(i) => i Self::Incomplete(i) => i
.apply_server_update(endpoint, token) .apply_server_update(endpoint, token)
.map(|info| { .map(|info| {
*self = Complete(info); *self = Self::Complete(info);
}) })
.is_some(), .is_some(),
} }
@@ -138,7 +135,7 @@ impl fmt::Debug for ConnectionInfo {
} }
} }
#[derive(Clone, Default)] #[derive(Clone)]
pub(crate) struct Partial { pub(crate) struct Partial {
pub channel_id: ChannelId, pub channel_id: ChannelId,
pub endpoint: Option<String>, pub endpoint: Option<String>,

332
src/input/adapters/async_adapter.rs Normal file
View File

@@ -0,0 +1,332 @@
use crate::input::AudioStreamError;
use async_trait::async_trait;
use flume::{Receiver, RecvError, Sender, TryRecvError};
use futures::{future::Either, stream::FuturesUnordered, FutureExt, StreamExt};
use ringbuf::*;
use std::{
io::{
Error as IoError,
ErrorKind as IoErrorKind,
Read,
Result as IoResult,
Seek,
SeekFrom,
Write,
},
sync::{
atomic::{AtomicBool, Ordering},
Arc,
},
};
use symphonia_core::io::MediaSource;
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncSeek, AsyncSeekExt},
sync::Notify,
};
struct AsyncAdapterSink {
bytes_in: Producer<u8>,
req_rx: Receiver<AdapterRequest>,
resp_tx: Sender<AdapterResponse>,
stream: Box<dyn AsyncMediaSource>,
notify_rx: Arc<Notify>,
}
impl AsyncAdapterSink {
async fn launch(mut self) {
let mut inner_buf = [0u8; 10 * 1024];
let mut read_region = 0..0;
let mut hit_end = false;
let mut blocked = false;
let mut pause_buf_moves = false;
let mut seek_res = None;
let mut seen_bytes = 0;
loop {
// if read_region is empty, refill from src.
// if that read is zero, tell other half.
// if WouldBlock, block on msg acquire,
// else non_block msg acquire.
if !pause_buf_moves {
if !hit_end && read_region.is_empty() {
if let Ok(n) = self.stream.read(&mut inner_buf).await {
read_region = 0..n;
if n == 0 {
drop(self.resp_tx.send_async(AdapterResponse::ReadZero).await);
hit_end = true;
}
seen_bytes += n as u64;
} else {
match self.stream.try_resume(seen_bytes).await {
Ok(s) => {
self.stream = s;
},
Err(_e) => break,
}
}
}
while !read_region.is_empty() && !blocked {
if let Ok(n_moved) = self
.bytes_in
.write(&inner_buf[read_region.start..read_region.end])
{
read_region.start += n_moved;
} else {
blocked = true;
}
}
}
let msg = if blocked || hit_end {
let mut fs = FuturesUnordered::new();
fs.push(Either::Left(self.req_rx.recv_async()));
fs.push(Either::Right(self.notify_rx.notified().map(|_| {
let o: Result<AdapterRequest, RecvError> = Ok(AdapterRequest::Wake);
o
})));
match fs.next().await {
Some(Ok(a)) => a,
_ => break,
}
} else {
match self.req_rx.try_recv() {
Ok(a) => a,
Err(TryRecvError::Empty) => continue,
_ => break,
}
};
match msg {
AdapterRequest::Wake => blocked = false,
AdapterRequest::ByteLen => {
drop(
self.resp_tx
.send_async(AdapterResponse::ByteLen(self.stream.byte_len().await))
.await,
);
},
AdapterRequest::Seek(pos) => {
pause_buf_moves = true;
drop(self.resp_tx.send_async(AdapterResponse::SeekClear).await);
seek_res = Some(self.stream.seek(pos).await);
},
AdapterRequest::SeekCleared => {
if let Some(res) = seek_res.take() {
drop(
self.resp_tx
.send_async(AdapterResponse::SeekResult(res))
.await,
);
}
pause_buf_moves = false;
},
}
}
}
}
/// An adapter for converting an async media source into a synchronous one
/// usable by symphonia.
///
/// This adapter takes a source implementing `AsyncRead`, and allows the receive side to
/// pass along seek requests needed. This allows for passing bytes from exclusively `AsyncRead`
/// streams (e.g., hyper HTTP sessions) to Songbird.
pub struct AsyncAdapterStream {
bytes_out: Consumer<u8>,
can_seek: bool,
// Note: this is Atomic just to work around the need for
// check_messages to take &self rather than &mut.
finalised: AtomicBool,
req_tx: Sender<AdapterRequest>,
resp_rx: Receiver<AdapterResponse>,
notify_tx: Arc<Notify>,
}
impl AsyncAdapterStream {
/// Wrap and pull from an async file stream, with an intermediate ring-buffer of size `buf_len`
/// between the async and sync halves.
#[must_use]
pub fn new(stream: Box<dyn AsyncMediaSource>, buf_len: usize) -> AsyncAdapterStream {
let (bytes_in, bytes_out) = RingBuffer::new(buf_len).split();
let (resp_tx, resp_rx) = flume::unbounded();
let (req_tx, req_rx) = flume::unbounded();
let can_seek = stream.is_seekable();
let notify_rx = Arc::new(Notify::new());
let notify_tx = notify_rx.clone();
let sink = AsyncAdapterSink {
bytes_in,
req_rx,
resp_tx,
stream,
notify_rx,
};
let stream = AsyncAdapterStream {
bytes_out,
can_seek,
finalised: false.into(),
req_tx,
resp_rx,
notify_tx,
};
tokio::spawn(async move {
sink.launch().await;
});
stream
}
fn handle_messages(&self, block: bool) -> Option<AdapterResponse> {
loop {
match self.resp_rx.try_recv() {
Ok(AdapterResponse::ReadZero) => {
self.finalised.store(true, Ordering::Relaxed);
},
Ok(a) => break Some(a),
Err(TryRecvError::Empty) if !block => break None,
Err(TryRecvError::Disconnected) => break None,
Err(TryRecvError::Empty) => {},
}
}
}
fn is_dropped_and_clear(&self) -> bool {
self.resp_rx.is_empty() && self.resp_rx.is_disconnected()
}
fn check_dropped(&self) -> IoResult<()> {
if self.is_dropped_and_clear() {
Err(IoError::new(
IoErrorKind::UnexpectedEof,
"Async half was dropped.",
))
} else {
Ok(())
}
}
}
impl Read for AsyncAdapterStream {
fn read(&mut self, buf: &mut [u8]) -> IoResult<usize> {
// TODO: make this run via condvar instead?
// This needs to remain blocking or spin loopy
// Mainly because this is at odds with "keep CPU low."
loop {
drop(self.handle_messages(false));
match self.bytes_out.read(buf) {
Ok(n) => {
self.notify_tx.notify_one();
return Ok(n);
},
Err(e) if e.kind() == IoErrorKind::WouldBlock => {
// receive side must ABSOLUTELY be unblocked here.
self.notify_tx.notify_one();
if self.finalised.load(Ordering::Relaxed) {
return Ok(0);
}
self.check_dropped()?;
std::thread::yield_now();
},
a => {
println!("Misc err {:?}", a);
return a;
},
}
}
}
}
impl Seek for AsyncAdapterStream {
fn seek(&mut self, pos: SeekFrom) -> IoResult<u64> {
if !self.can_seek {
return Err(IoError::new(
IoErrorKind::Unsupported,
"Async half does not support seek operations.",
));
}
self.check_dropped()?;
let _ = self.req_tx.send(AdapterRequest::Seek(pos));
// wait for async to tell us that it has stopped writing,
// then clear buf and allow async to write again.
self.finalised.store(false, Ordering::Relaxed);
match self.handle_messages(true) {
Some(AdapterResponse::SeekClear) => {},
None => self.check_dropped().map(|_| unreachable!())?,
_ => unreachable!(),
}
self.bytes_out.discard(self.bytes_out.capacity());
let _ = self.req_tx.send(AdapterRequest::SeekCleared);
match self.handle_messages(true) {
Some(AdapterResponse::SeekResult(a)) => a,
None => self.check_dropped().map(|_| unreachable!()),
_ => unreachable!(),
}
}
}
impl MediaSource for AsyncAdapterStream {
fn is_seekable(&self) -> bool {
self.can_seek
}
fn byte_len(&self) -> Option<u64> {
self.check_dropped().ok()?;
let _ = self.req_tx.send(AdapterRequest::ByteLen);
match self.handle_messages(true) {
Some(AdapterResponse::ByteLen(a)) => a,
None => self.check_dropped().ok().map(|_| unreachable!()),
_ => unreachable!(),
}
}
}
enum AdapterRequest {
Wake,
Seek(SeekFrom),
SeekCleared,
ByteLen,
}
enum AdapterResponse {
SeekResult(IoResult<u64>),
SeekClear,
ByteLen(Option<u64>),
ReadZero,
}
/// An async port of symphonia's [`MediaSource`].
///
/// Streams which are not seekable should implement `AsyncSeek` such that all operations
/// fail with `Unsupported`, and implement `fn is_seekable(&self) -> { false }`.
///
/// [`MediaSource`]: MediaSource
#[async_trait]
pub trait AsyncMediaSource: AsyncRead + AsyncSeek + Send + Sync + Unpin {
/// Returns if the source is seekable. This may be an expensive operation.
fn is_seekable(&self) -> bool;
/// Returns the length in bytes, if available. This may be an expensive operation.
async fn byte_len(&self) -> Option<u64>;
/// Tries to recreate this stream in event of an error, resuming from the given offset.
async fn try_resume(
&mut self,
_offset: u64,
) -> Result<Box<dyn AsyncMediaSource>, AudioStreamError> {
Err(AudioStreamError::Unsupported)
}
}

537
src/input/adapters/cached/compressed.rs Normal file
View File

@@ -0,0 +1,537 @@
use super::{compressed_cost_per_sec, default_config, CodecCacheError, ToAudioBytes};
use crate::{
constants::*,
input::{
codecs::{dca::*, CODEC_REGISTRY, PROBE},
AudioStream,
Input,
LiveInput,
},
};
use audiopus::{
coder::{Encoder as OpusEncoder, GenericCtl},
Application,
Bitrate,
Channels,
Error as OpusError,
ErrorCode as OpusErrorCode,
SampleRate,
};
use byteorder::{LittleEndian, ReadBytesExt, WriteBytesExt};
use std::{
convert::TryInto,
io::{
Cursor,
Error as IoError,
ErrorKind as IoErrorKind,
Read,
Result as IoResult,
Seek,
SeekFrom,
},
mem,
sync::atomic::{AtomicUsize, Ordering},
};
use streamcatcher::{
Config as ScConfig,
NeedsBytes,
Stateful,
Transform,
TransformPosition,
TxCatcher,
};
use symphonia_core::{
audio::Channels as SChannels,
codecs::CodecRegistry,
io::MediaSource,
meta::{MetadataRevision, StandardTagKey, Value},
probe::{Probe, ProbedMetadata},
};
use tracing::{debug, trace};
pub struct Config {
/// Registry of audio codecs supported by the driver.
///
/// Defaults to [`CODEC_REGISTRY`], which adds audiopus-based Opus codec support
/// to all of Symphonia's default codecs.
///
/// [`CODEC_REGISTRY`]: static@CODEC_REGISTRY
pub codec_registry: &'static CodecRegistry,
/// Registry of the muxers and container formats supported by the driver.
///
/// Defaults to [`PROBE`], which includes all of Symphonia's default format handlers
/// and DCA format support.
///
/// [`PROBE`]: static@PROBE
pub format_registry: &'static Probe,
/// Configuration for the inner streamcatcher instance.
///
/// Notably, this governs size hints and resize logic.
pub streamcatcher: ScConfig,
}
impl Default for Config {
fn default() -> Self {
Self {
codec_registry: &CODEC_REGISTRY,
format_registry: &PROBE,
streamcatcher: ScConfig::default(),
}
}
}
impl Config {
pub fn default_from_cost(cost_per_sec: usize) -> Self {
let streamcatcher = default_config(cost_per_sec);
Self {
streamcatcher,
..Default::default()
}
}
}
/// A wrapper around an existing [`Input`] which compresses
/// the input using the Opus codec before storing it in memory.
///
/// The main purpose of this wrapper is to enable seeking on
/// incompatible sources and to ease resource consumption for
/// commonly reused/shared tracks. If only one Opus-compressed track
/// is playing at a time, then this removes the runtime decode cost
/// from the driver.
///
/// This is intended for use with larger, repeatedly used audio
/// tracks shared between sources, and stores the sound data
/// retrieved as **compressed Opus audio**.
///
/// Internally, this stores the stream and its metadata as a DCA1 file,
/// which can be written out to disk for later use.
///
/// [`Input`]: crate::input::Input
#[derive(Clone)]
pub struct Compressed {
/// Inner shared bytestore.
pub raw: TxCatcher<ToAudioBytes, OpusCompressor>,
}
impl Compressed {
/// Wrap an existing [`Input`] with an in-memory store, compressed using Opus.
///
/// [`Input`]: Input
pub async fn new(source: Input, bitrate: Bitrate) -> Result<Self, CodecCacheError> {
Self::with_config(source, bitrate, None).await
}
/// Wrap an existing [`Input`] with an in-memory store, compressed using Opus, with
/// custom configuration for both Symphonia and the backing store.
///
/// [`Input`]: Input
pub async fn with_config(
source: Input,
bitrate: Bitrate,
config: Option<Config>,
) -> Result<Self, CodecCacheError> {
let input = match source {
Input::Lazy(mut r) => {
let created = if r.should_create_async() {
r.create_async().await.map_err(CodecCacheError::from)
} else {
tokio::task::spawn_blocking(move || r.create().map_err(CodecCacheError::from))
.await
.map_err(CodecCacheError::from)
.and_then(|v| v)
};
created.map(LiveInput::Raw)
},
Input::Live(LiveInput::Parsed(_), _) => Err(CodecCacheError::StreamNotAtStart),
Input::Live(a, _rec) => Ok(a),
}?;
let cost_per_sec = compressed_cost_per_sec(bitrate);
let config = config.unwrap_or_else(|| Config::default_from_cost(cost_per_sec));
let promoted = tokio::task::spawn_blocking(move || {
input.promote(config.codec_registry, config.format_registry)
})
.await??;
// If success, guaranteed to be Parsed
let mut parsed = if let LiveInput::Parsed(parsed) = promoted {
parsed
} else {
unreachable!()
};
// TODO: apply length hint.
// if config.length_hint.is_none() {
// if let Some(dur) = metadata.duration {
// apply_length_hint(&mut config, dur, cost_per_sec);
// }
// }
let track_info = parsed.decoder.codec_params();
let chan_count = track_info.channels.map_or(2, SChannels::count);
let (channels, stereo) = if chan_count >= 2 {
(Channels::Stereo, true)
} else {
(Channels::Mono, false)
};
let mut encoder = OpusEncoder::new(SampleRate::Hz48000, channels, Application::Audio)?;
encoder.set_bitrate(bitrate)?;
let codec_type = parsed.decoder.codec_params().codec;
let encoding = config
.codec_registry
.get_codec(codec_type)
.map(|v| v.short_name.to_string());
let format_meta_hold = parsed.format.metadata();
let format_meta = format_meta_hold.current();
let metadata = create_metadata(
&mut parsed.meta,
format_meta,
&encoder,
chan_count as u8,
encoding,
)?;
let mut metabytes = b"DCA1\0\0\0\0".to_vec();
let orig_len = metabytes.len();
serde_json::to_writer(&mut metabytes, &metadata)?;
let meta_len = (metabytes.len() - orig_len)
.try_into()
.map_err(|_| CodecCacheError::MetadataTooLarge)?;
(&mut metabytes[4..][..mem::size_of::<i32>()])
.write_i32::<LittleEndian>(meta_len)
.expect("Magic byte writing location guaranteed to be well-founded.");
let source = ToAudioBytes::new(parsed, Some(2));
let raw = config
.streamcatcher
.build_tx(source, OpusCompressor::new(encoder, stereo, metabytes))?;
Ok(Self { raw })
}
/// Acquire a new handle to this object, creating a new
/// view of the existing cached data from the beginning.
#[must_use]
pub fn new_handle(&self) -> Self {
Self {
raw: self.raw.new_handle(),
}
}
}
fn create_metadata(
probe_metadata: &mut ProbedMetadata,
track_metadata: Option<&MetadataRevision>,
opus: &OpusEncoder,
channels: u8,
encoding: Option<String>,
) -> Result<DcaMetadata, CodecCacheError> {
let dca = DcaInfo {
version: 1,
tool: Tool {
name: env!("CARGO_PKG_NAME").into(),
version: env!("CARGO_PKG_VERSION").into(),
url: Some(env!("CARGO_PKG_HOMEPAGE").into()),
author: Some(env!("CARGO_PKG_AUTHORS").into()),
},
};
let abr = match opus.bitrate()? {
Bitrate::BitsPerSecond(i) => Some(i as u64),
Bitrate::Auto => None,
Bitrate::Max => Some(510_000),
};
let mode = match opus.application()? {
Application::Voip => "voip",
Application::Audio => "music",
Application::LowDelay => "lowdelay",
}
.to_string();
let sample_rate = opus.sample_rate()? as u32;
let opus = Opus {
mode,
sample_rate,
frame_size: MONO_FRAME_BYTE_SIZE as u64,
abr,
vbr: opus.vbr()?,
channels: channels.min(2),
};
let mut origin = Origin {
source: Some("file".into()),
abr: None,
channels: Some(channels),
encoding,
url: None,
};
let mut info = Info {
title: None,
artist: None,
album: None,
genre: None,
cover: None,
comments: None,
};
if let Some(meta) = probe_metadata.get() {
apply_meta_to_dca(&mut info, &mut origin, meta.current());
}
apply_meta_to_dca(&mut info, &mut origin, track_metadata);
Ok(DcaMetadata {
dca,
opus,
info: Some(info),
origin: Some(origin),
extra: None,
})
}
fn apply_meta_to_dca(info: &mut Info, origin: &mut Origin, src_meta: Option<&MetadataRevision>) {
if let Some(meta) = src_meta {
for tag in meta.tags() {
match tag.std_key {
Some(StandardTagKey::Album) =>
if let Value::String(s) = &tag.value {
info.album = Some(s.clone());
},
Some(StandardTagKey::Artist) =>
if let Value::String(s) = &tag.value {
info.artist = Some(s.clone());
},
Some(StandardTagKey::Comment) =>
if let Value::String(s) = &tag.value {
info.comments = Some(s.clone());
},
Some(StandardTagKey::Genre) =>
if let Value::String(s) = &tag.value {
info.genre = Some(s.clone());
},
Some(StandardTagKey::TrackTitle) =>
if let Value::String(s) = &tag.value {
info.title = Some(s.clone());
},
Some(StandardTagKey::Url | StandardTagKey::UrlSource) => {
if let Value::String(s) = &tag.value {
origin.url = Some(s.clone());
}
},
_ => {},
}
}
for _visual in meta.visuals() {
// FIXME: will require MIME type inspection and Base64 conversion.
}
}
}
/// Transform applied inside [`Compressed`], converting a floating-point PCM
/// input stream into a DCA-framed Opus stream.
///
/// Created and managed by [`Compressed`].
///
/// [`Compressed`]: Compressed
#[derive(Debug)]
pub struct OpusCompressor {
prepend: Option<Cursor<Vec<u8>>>,
encoder: OpusEncoder,
last_frame: Vec<u8>,
stereo_input: bool,
frame_pos: usize,
audio_bytes: AtomicUsize,
}
impl OpusCompressor {
fn new(encoder: OpusEncoder, stereo_input: bool, prepend: Vec<u8>) -> Self {
Self {
prepend: Some(Cursor::new(prepend)),
encoder,
last_frame: Vec::with_capacity(4000),
stereo_input,
frame_pos: 0,
audio_bytes: AtomicUsize::default(),
}
}
}
impl<T> Transform<T> for OpusCompressor
where
T: Read,
{
fn transform_read(&mut self, src: &mut T, buf: &mut [u8]) -> IoResult<TransformPosition> {
if let Some(prepend) = self.prepend.as_mut() {
match prepend.read(buf)? {
0 => {},
n => return Ok(TransformPosition::Read(n)),
}
}
self.prepend = None;
let output_start = mem::size_of::<u16>();
let mut eof = false;
let mut raw_len = 0;
let mut out = None;
let mut sample_buf = [0f32; STEREO_FRAME_SIZE];
let (samples_in_frame, interleaved_count) = if self.stereo_input {
(STEREO_FRAME_SIZE, 2)
} else {
(MONO_FRAME_SIZE, 1)
};
// Purge old frame and read new, if needed.
if self.frame_pos == self.last_frame.len() + output_start || self.last_frame.is_empty() {
self.last_frame.resize(self.last_frame.capacity(), 0);
// We can't use `read_f32_into` because we can't guarantee the buffer will be filled.
// However, we can guarantee that reads will be channel aligned at least!
for el in sample_buf[..samples_in_frame].chunks_mut(interleaved_count) {
match src.read_f32_into::<LittleEndian>(el) {
Ok(_) => {
raw_len += interleaved_count;
},
Err(e) if e.kind() == IoErrorKind::UnexpectedEof => {
eof = true;
break;
},
Err(e) => {
out = Some(Err(e));
break;
},
}
}
if out.is_none() && raw_len > 0 {
loop {
// NOTE: we don't index by raw_len because the last frame can be too small
// to occupy a "whole packet". Zero-padding is the correct behaviour.
match self
.encoder
.encode_float(&sample_buf[..samples_in_frame], &mut self.last_frame[..])
{
Ok(pkt_len) => {
trace!("Next packet to write has {:?}", pkt_len);
self.frame_pos = 0;
self.last_frame.truncate(pkt_len);
break;
},
Err(OpusError::Opus(OpusErrorCode::BufferTooSmall)) => {
// If we need more capacity to encode this frame, then take it.
trace!("Resizing inner buffer (+256).");
self.last_frame.resize(self.last_frame.len() + 256, 0);
},
Err(e) => {
debug!("Read error {:?} {:?} {:?}.", e, out, raw_len);
out = Some(Err(IoError::new(IoErrorKind::Other, e)));
break;
},
}
}
}
}
if out.is_none() {
// Write from frame we have.
let start = if self.frame_pos < output_start {
(&mut buf[..output_start])
.write_i16::<LittleEndian>(self.last_frame.len() as i16)
.expect(
"Minimum bytes requirement for Opus (2) should mean that an i16 \
may always be written.",
);
self.frame_pos += output_start;
trace!("Wrote frame header: {}.", self.last_frame.len());
output_start
} else {
0
};
let out_pos = self.frame_pos - output_start;
let remaining = self.last_frame.len() - out_pos;
let write_len = remaining.min(buf.len() - start);
buf[start..start + write_len]
.copy_from_slice(&self.last_frame[out_pos..out_pos + write_len]);
self.frame_pos += write_len;
trace!("Appended {} to inner store", write_len);
out = Some(Ok(write_len + start));
}
// NOTE: use of raw_len here preserves true sample length even if
// stream is extended to 20ms boundary.
out.unwrap_or_else(|| Err(IoError::new(IoErrorKind::Other, "Unclear.")))
.map(|compressed_sz| {
self.audio_bytes
.fetch_add(raw_len * mem::size_of::<f32>(), Ordering::Release);
if eof {
TransformPosition::Finished
} else {
TransformPosition::Read(compressed_sz)
}
})
}
}
impl NeedsBytes for OpusCompressor {
fn min_bytes_required(&self) -> usize {
2
}
}
impl Stateful for OpusCompressor {
type State = usize;
fn state(&self) -> Self::State {
self.audio_bytes.load(Ordering::Acquire)
}
}
impl Read for Compressed {
fn read(&mut self, buf: &mut [u8]) -> IoResult<usize> {
self.raw.read(buf)
}
}
impl Seek for Compressed {
fn seek(&mut self, pos: SeekFrom) -> IoResult<u64> {
self.raw.seek(pos)
}
}
impl MediaSource for Compressed {
fn is_seekable(&self) -> bool {
true
}
fn byte_len(&self) -> Option<u64> {
if self.raw.is_finished() {
Some(self.raw.len() as u64)
} else {
None
}
}
}
impl From<Compressed> for Input {
fn from(val: Compressed) -> Input {
let input = Box::new(val);
Input::Live(LiveInput::Raw(AudioStream { input, hint: None }), None)
}
}

142
src/input/adapters/cached/decompressed.rs Normal file
View File

@@ -0,0 +1,142 @@
use super::{compressed::Config, CodecCacheError, ToAudioBytes};
use crate::{
constants::SAMPLE_RATE_RAW,
input::{AudioStream, Input, LiveInput, RawAdapter},
};
use std::io::{Read, Result as IoResult, Seek, SeekFrom};
use streamcatcher::Catcher;
use symphonia_core::{audio::Channels, io::MediaSource};
/// A wrapper around an existing [`Input`] which caches
/// the decoded and converted audio data locally in memory
/// as `f32`-format PCM data.
///
/// The main purpose of this wrapper is to enable seeking on
/// incompatible sources (i.e., ffmpeg output) and to ease resource
/// consumption for commonly reused/shared tracks. [`Compressed`]
/// offers similar functionality with different
/// tradeoffs.
///
/// This is intended for use with small, repeatedly used audio
/// tracks shared between sources, and stores the sound data
/// retrieved in **uncompressed floating point** form to minimise the
/// cost of audio processing when mixing several tracks together.
/// This must be used sparingly: these cost a significant
/// *3 Mbps (375 kiB/s)*, or 131 MiB of RAM for a 6 minute song.
///
/// [`Input`]: crate::input::Input
/// [`Compressed`]: super::Compressed
#[derive(Clone)]
pub struct Decompressed {
/// Inner shared bytestore.
pub raw: Catcher<RawAdapter<ToAudioBytes>>,
}
impl Decompressed {
/// Wrap an existing [`Input`] with an in-memory store, decompressed into `f32` PCM audio.
///
/// [`Input`]: Input
pub async fn new(source: Input) -> Result<Self, CodecCacheError> {
Self::with_config(source, None).await
}
/// Wrap an existing [`Input`] with an in-memory store, decompressed into `f32` PCM audio,
/// with custom configuration for both Symphonia and the backing store.
///
/// [`Input`]: Input
pub async fn with_config(
source: Input,
config: Option<Config>,
) -> Result<Self, CodecCacheError> {
let input = match source {
Input::Lazy(mut r) => {
let created = if r.should_create_async() {
r.create_async().await.map_err(CodecCacheError::from)
} else {
tokio::task::spawn_blocking(move || r.create().map_err(CodecCacheError::from))
.await
.map_err(CodecCacheError::from)
.and_then(|v| v)
};
created.map(LiveInput::Raw)
},
Input::Live(LiveInput::Parsed(_), _) => Err(CodecCacheError::StreamNotAtStart),
Input::Live(a, _rec) => Ok(a),
}?;
let cost_per_sec = super::raw_cost_per_sec(true);
let config = config.unwrap_or_else(|| Config::default_from_cost(cost_per_sec));
let promoted = tokio::task::spawn_blocking(move || {
input.promote(config.codec_registry, config.format_registry)
})
.await??;
// If success, guaranteed to be Parsed
let parsed = if let LiveInput::Parsed(parsed) = promoted {
parsed
} else {
unreachable!()
};
let track_info = parsed.decoder.codec_params();
let chan_count = track_info
.channels
.map(Channels::count)
.ok_or(CodecCacheError::UnknownChannelCount)?;
let sample_rate = SAMPLE_RATE_RAW as u32;
let source = RawAdapter::new(
ToAudioBytes::new(parsed, Some(chan_count)),
sample_rate,
chan_count as u32,
);
let raw = config.streamcatcher.build(source)?;
Ok(Self { raw })
}
/// Acquire a new handle to this object, creating a new
/// view of the existing cached data from the beginning.
#[must_use]
pub fn new_handle(&self) -> Self {
Self {
raw: self.raw.new_handle(),
}
}
}
impl Read for Decompressed {
fn read(&mut self, buf: &mut [u8]) -> IoResult<usize> {
self.raw.read(buf)
}
}
impl Seek for Decompressed {
fn seek(&mut self, pos: SeekFrom) -> IoResult<u64> {
self.raw.seek(pos)
}
}
impl MediaSource for Decompressed {
fn is_seekable(&self) -> bool {
true
}
fn byte_len(&self) -> Option<u64> {
if self.raw.is_finished() {
Some(self.raw.len() as u64)
} else {
None
}
}
}
impl From<Decompressed> for Input {
fn from(val: Decompressed) -> Input {
let input = Box::new(val);
Input::Live(LiveInput::Raw(AudioStream { input, hint: None }), None)
}
}

146
src/input/adapters/cached/error.rs Normal file
View File

@@ -0,0 +1,146 @@
use crate::input::AudioStreamError;
use audiopus::error::Error as OpusError;
use serde_json::Error as JsonError;
use std::{
error::Error as StdError,
fmt::{Display, Formatter, Result as FmtResult},
};
use streamcatcher::CatcherError;
use symphonia_core::errors::Error as SymphError;
use tokio::task::JoinError;
/// Errors encountered using a [`Memory`] cached source.
///
/// [`Memory`]: super::Memory
#[derive(Debug)]
pub enum Error {
/// The audio stream could not be created.
Create(AudioStreamError),
/// The audio stream failed to be created due to a panic in `spawn_blocking`.
CreatePanicked,
/// Streamcatcher's configuration was illegal, and the cache could not be created.
Streamcatcher(CatcherError),
/// The input stream had already been read (i.e., `Parsed`) and so the whole stream
/// could not be used.
StreamNotAtStart,
}
impl Display for Error {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
match self {
Self::Create(c) => f.write_fmt(format_args!("failed to create audio stream: {}", c)),
Self::CreatePanicked => f.write_str("sync thread panicked while creating stream"),
Self::Streamcatcher(s) =>
f.write_fmt(format_args!("illegal streamcatcher config: {}", s)),
Self::StreamNotAtStart =>
f.write_str("stream cannot have been pre-read/parsed, missing headers"),
}
}
}
impl StdError for Error {}
impl From<AudioStreamError> for Error {
fn from(val: AudioStreamError) -> Self {
Self::Create(val)
}
}
impl From<CatcherError> for Error {
fn from(val: CatcherError) -> Self {
Self::Streamcatcher(val)
}
}
impl From<JoinError> for Error {
fn from(_val: JoinError) -> Self {
Self::CreatePanicked
}
}
/// Errors encountered using a [`Compressed`] or [`Decompressed`] cached source.
///
/// [`Compressed`]: super::Compressed
/// [`Decompressed`]: super::Decompressed
#[derive(Debug)]
pub enum CodecCacheError {
/// The audio stream could not be created.
Create(AudioStreamError),
/// Symphonia failed to parse the container or decode the default stream.
Parse(SymphError),
/// The Opus encoder could not be created.
Opus(OpusError),
/// The file's metadata could not be converted to JSON.
MetadataEncoding(JsonError),
/// The input's metadata was too large after conversion to JSON to fit in a DCA file.
MetadataTooLarge,
/// The audio stream failed to be created due to a panic in `spawn_blocking`.
CreatePanicked,
/// The audio stream's channel count could not be determined.
UnknownChannelCount,
/// Streamcatcher's configuration was illegal, and the cache could not be created.
Streamcatcher(CatcherError),
/// The input stream had already been read (i.e., `Parsed`) and so the whole stream
/// could not be used.
StreamNotAtStart,
}
impl Display for CodecCacheError {
fn fmt(&self, f: &mut Formatter<'_>) -> FmtResult {
match self {
Self::Create(c) => f.write_fmt(format_args!("failed to create audio stream: {}", c)),
Self::Parse(p) => f.write_fmt(format_args!("failed to parse audio format: {}", p)),
Self::Opus(o) => f.write_fmt(format_args!("failed to create Opus encoder: {}", o)),
Self::MetadataEncoding(m) => f.write_fmt(format_args!(
"failed to convert track metadata to JSON: {}",
m
)),
Self::MetadataTooLarge => f.write_str("track metadata was too large, >= 32kiB"),
Self::CreatePanicked => f.write_str("sync thread panicked while creating stream"),
Self::UnknownChannelCount =>
f.write_str("audio stream's channel count could not be determined"),
Self::Streamcatcher(s) =>
f.write_fmt(format_args!("illegal streamcatcher config: {}", s)),
Self::StreamNotAtStart =>
f.write_str("stream cannot have been pre-read/parsed, missing headers"),
}
}
}
impl StdError for CodecCacheError {}
impl From<AudioStreamError> for CodecCacheError {
fn from(val: AudioStreamError) -> Self {
Self::Create(val)
}
}
impl From<CatcherError> for CodecCacheError {
fn from(val: CatcherError) -> Self {
Self::Streamcatcher(val)
}
}
impl From<JoinError> for CodecCacheError {
fn from(_val: JoinError) -> Self {
Self::CreatePanicked
}
}
impl From<JsonError> for CodecCacheError {
fn from(val: JsonError) -> Self {
Self::MetadataEncoding(val)
}
}
impl From<OpusError> for CodecCacheError {
fn from(val: OpusError) -> Self {
Self::Opus(val)
}
}
impl From<SymphError> for CodecCacheError {
fn from(val: SymphError) -> Self {
Self::Parse(val)
}
}

0
src/input/cached/hint.rs → src/input/adapters/cached/hint.rs
View File

111
src/input/adapters/cached/memory.rs Normal file
View File

@@ -0,0 +1,111 @@
use super::{default_config, raw_cost_per_sec, Error};
use crate::input::{AudioStream, Input, LiveInput};
use std::io::{Read, Result as IoResult, Seek};
use streamcatcher::{Catcher, Config};
use symphonia_core::io::MediaSource;
/// A wrapper around an existing [`Input`] which caches its data
/// in memory.
///
/// The main purpose of this wrapper is to enable fast seeking on
/// incompatible sources (i.e., HTTP streams) and to ease resource
/// consumption for commonly reused/shared tracks.
///
/// This consumes exactly as many bytes of memory as the input stream contains.
///
/// [`Input`]: Input
#[derive(Clone)]
pub struct Memory {
/// Inner shared bytestore.
pub raw: Catcher<Box<dyn MediaSource>>,
}
impl Memory {
/// Wrap an existing [`Input`] with an in-memory store with the same codec and framing.
///
/// [`Input`]: Input
pub async fn new(source: Input) -> Result<Self, Error> {
Self::with_config(source, None).await
}
/// Wrap an existing [`Input`] with an in-memory store with the same codec and framing.
///
/// `length_hint` may be used to control the size of the initial chunk, preventing
/// needless allocations and copies.
///
/// [`Input`]: Input
pub async fn with_config(source: Input, config: Option<Config>) -> Result<Self, Error> {
let input = match source {
Input::Lazy(mut r) => {
let created = if r.should_create_async() {
r.create_async().await
} else {
tokio::task::spawn_blocking(move || r.create()).await?
};
created.map(|v| v.input).map_err(Error::from)
},
Input::Live(LiveInput::Raw(a), _rec) => Ok(a.input),
Input::Live(LiveInput::Wrapped(a), _rec) =>
Ok(Box::new(a.input) as Box<dyn MediaSource>),
Input::Live(LiveInput::Parsed(_), _) => Err(Error::StreamNotAtStart),
}?;
let cost_per_sec = raw_cost_per_sec(true);
let config = config.unwrap_or_else(|| default_config(cost_per_sec));
// TODO: apply length hint.
// if config.length_hint.is_none() {
// if let Some(dur) = metadata.duration {
// apply_length_hint(&mut config, dur, cost_per_sec);
// }
// }
let raw = config.build(input)?;
Ok(Self { raw })
}
/// Acquire a new handle to this object, creating a new
/// view of the existing cached data from the beginning.
#[must_use]
pub fn new_handle(&self) -> Self {
Self {
raw: self.raw.new_handle(),
}
}
}
impl Read for Memory {
fn read(&mut self, buf: &mut [u8]) -> IoResult<usize> {
self.raw.read(buf)
}
}
impl Seek for Memory {
fn seek(&mut self, pos: std::io::SeekFrom) -> IoResult<u64> {
self.raw.seek(pos)
}
}
impl MediaSource for Memory {
fn is_seekable(&self) -> bool {
true
}
fn byte_len(&self) -> Option<u64> {
if self.raw.is_finished() {
Some(self.raw.len() as u64)
} else {
None
}
}
}
impl From<Memory> for Input {
fn from(val: Memory) -> Input {
let input = Box::new(val);
Input::Live(LiveInput::Raw(AudioStream { input, hint: None }), None)
}
}

11
src/input/cached/mod.rs → src/input/adapters/cached/mod.rs
View File

@@ -2,12 +2,14 @@
//! direct Opus frame passthrough. //! direct Opus frame passthrough.
mod compressed; mod compressed;
mod decompressed;
mod error;
mod hint; mod hint;
mod memory; mod memory;
#[cfg(test)] mod util;
mod tests;
pub use self::{compressed::*, hint::*, memory::*}; pub(crate) use self::util::*;
pub use self::{compressed::*, decompressed::*, error::*, hint::*, memory::*};
use crate::constants::*; use crate::constants::*;
use crate::input::utils; use crate::input::utils;
@@ -16,6 +18,7 @@ use std::{mem, time::Duration};
use streamcatcher::{Config, GrowthStrategy}; use streamcatcher::{Config, GrowthStrategy};
/// Estimates the cost, in B/s, of audio data compressed at the given bitrate. /// Estimates the cost, in B/s, of audio data compressed at the given bitrate.
#[must_use]
pub fn compressed_cost_per_sec(bitrate: Bitrate) -> usize { pub fn compressed_cost_per_sec(bitrate: Bitrate) -> usize {
let framing_cost_per_sec = AUDIO_FRAME_RATE * mem::size_of::<u16>(); let framing_cost_per_sec = AUDIO_FRAME_RATE * mem::size_of::<u16>();
@@ -29,6 +32,7 @@ pub fn compressed_cost_per_sec(bitrate: Bitrate) -> usize {
} }
/// Calculates the cost, in B/s, of raw floating-point audio data. /// Calculates the cost, in B/s, of raw floating-point audio data.
#[must_use]
pub fn raw_cost_per_sec(stereo: bool) -> usize { pub fn raw_cost_per_sec(stereo: bool) -> usize {
utils::timestamp_to_byte_count(Duration::from_secs(1), stereo) utils::timestamp_to_byte_count(Duration::from_secs(1), stereo)
} }
@@ -39,6 +43,7 @@ pub fn raw_cost_per_sec(stereo: bool) -> usize {
/// a constant chunk size of 5s worth of audio at the given bitrate estimate. /// a constant chunk size of 5s worth of audio at the given bitrate estimate.
/// ///
/// [`streamcatcher`]: https://docs.rs/streamcatcher/0.1.0/streamcatcher/struct.Config.html /// [`streamcatcher`]: https://docs.rs/streamcatcher/0.1.0/streamcatcher/struct.Config.html
#[must_use]
pub fn default_config(cost_per_sec: usize) -> Config { pub fn default_config(cost_per_sec: usize) -> Config {
Config::new().chunk_size(GrowthStrategy::Constant(5 * cost_per_sec)) Config::new().chunk_size(GrowthStrategy::Constant(5 * cost_per_sec))
} }

458
src/input/adapters/cached/util.rs Normal file
View File

@@ -0,0 +1,458 @@
use crate::{constants::*, driver::tasks::mixer::mix_logic, input::Parsed};
use byteorder::{LittleEndian, WriteBytesExt};
use rubato::{FftFixedOut, Resampler};
use std::{
io::{ErrorKind as IoErrorKind, Read, Result as IoResult, Seek, Write},
mem,
ops::Range,
};
use symphonia_core::{
audio::{AudioBuffer, AudioBufferRef, Layout, Signal, SignalSpec},
conv::IntoSample,
io::MediaSource,
sample::Sample,
};
const SAMPLE_LEN: usize = mem::size_of::<f32>();
/// Adapter for Symphonia sources into an interleaved f32 bytestream.
///
/// This will output `f32`s in LE byte order, matching the channel count
/// of the input.
pub struct ToAudioBytes {
chan_count: usize,
chan_limit: usize,
parsed: Parsed,
/// Position with parsed's last decoded frame.
inner_pos: Range<usize>,
resample: Option<ResampleState>,
done: bool,
interrupted_samples: Vec<f32>,
interrupted_byte_pos: Range<usize>,
}
struct ResampleState {
/// Used to hold outputs from resampling, *ready to be used*.
resampled_data: Vec<Vec<f32>>,
/// The actual resampler.
resampler: FftFixedOut<f32>,
/// Used to hold inputs to resampler across packet boundaries.
scratch: AudioBuffer<f32>,
/// The range of floats in `resampled_data` which have not yet
/// been read.
resample_pos: Range<usize>,
}
impl ToAudioBytes {
pub fn new(parsed: Parsed, chan_limit: Option<usize>) -> Self {
let track_info = parsed.decoder.codec_params();
let sample_rate = track_info.sample_rate.unwrap_or(SAMPLE_RATE_RAW as u32);
let maybe_layout = track_info.channel_layout;
let maybe_chans = track_info.channels;
let chan_count = if let Some(chans) = maybe_chans {
chans.count()
} else if let Some(layout) = maybe_layout {
match layout {
Layout::Mono => 1,
Layout::Stereo => 2,
Layout::TwoPointOne => 3,
Layout::FivePointOne => 6,
}
} else {
2
};
let chan_limit = chan_limit.unwrap_or(chan_count);
let resample = (sample_rate != SAMPLE_RATE_RAW as u32).then(|| {
let spec = if let Some(chans) = maybe_chans {
SignalSpec::new(SAMPLE_RATE_RAW as u32, chans)
} else if let Some(layout) = maybe_layout {
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, layout)
} else {
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, Layout::Stereo)
};
let scratch = AudioBuffer::<f32>::new(MONO_FRAME_SIZE as u64, spec);
// TODO: integ. error handling here.
let resampler = FftFixedOut::new(
sample_rate as usize,
SAMPLE_RATE_RAW,
RESAMPLE_OUTPUT_FRAME_SIZE,
4,
chan_count,
)
.expect("Failed to create resampler.");
let resampled_data = resampler.output_buffer_allocate();
ResampleState {
resampled_data,
resampler,
scratch,
resample_pos: 0..0,
}
});
Self {
chan_count,
chan_limit,
parsed,
inner_pos: 0..0,
resample,
done: false,
interrupted_samples: Vec::with_capacity(chan_count),
interrupted_byte_pos: 0..0,
}
}
pub fn num_channels(&self) -> usize {
self.chan_count.min(self.chan_limit)
}
fn is_done(&self) -> bool {
self.done
&& self.inner_pos.is_empty()
&& self.resample.as_ref().map_or(true, |v| {
v.scratch.frames() == 0 && v.resample_pos.is_empty()
})
&& self.interrupted_byte_pos.is_empty()
}
}
impl Read for ToAudioBytes {
fn read(&mut self, mut buf: &mut [u8]) -> IoResult<usize> {
// NOTE: this is disturbingly similar to the mixer code, but different enough that we can't
// just reuse it freely.
let orig_sz = buf.len();
let num_chans = self.num_channels();
while !buf.is_empty() && !self.is_done() {
// Work to clear interrupted channel floats.
while !buf.is_empty() && !self.interrupted_byte_pos.is_empty() {
let index_of_first_f32 = self.interrupted_byte_pos.start / SAMPLE_LEN;
let f32_inner_pos = self.interrupted_byte_pos.start % SAMPLE_LEN;
let f32_bytes_remaining = SAMPLE_LEN - f32_inner_pos;
let to_write = f32_bytes_remaining.min(buf.len());
let bytes = self.interrupted_samples[index_of_first_f32].to_le_bytes();
let written = buf.write(&bytes[f32_inner_pos..][..to_write])?;
self.interrupted_byte_pos.start += written;
}
// Clear out already produced resampled floats.
if let Some(resample) = self.resample.as_mut() {
if !buf.is_empty() && !resample.resample_pos.is_empty() {
let bytes_advanced = write_resample_buffer(
&resample.resampled_data,
buf,
&mut resample.resample_pos,
&mut self.interrupted_samples,
&mut self.interrupted_byte_pos,
num_chans,
);
buf = &mut buf[bytes_advanced..];
}
if !resample.resample_pos.is_empty() {
continue;
}
}
// Now work with new packets.
let source_packet = if !self.inner_pos.is_empty() {
Some(self.parsed.decoder.last_decoded())
} else if let Ok(pkt) = self.parsed.format.next_packet() {
if pkt.track_id() != self.parsed.track_id {
continue;
}
self.parsed
.decoder
.decode(&pkt)
.map(|pkt| {
self.inner_pos = 0..pkt.frames();
pkt
})
.ok()
} else {
// EOF.
None
};
if source_packet.is_none() {
self.done = true;
if let Some(resample) = self.resample.as_mut() {
if resample.scratch.frames() != 0 {
let data = &mut resample.resampled_data;
let resampler = &mut resample.resampler;
let in_len = resample.scratch.frames();
let to_render = resampler.input_frames_next().saturating_sub(in_len);
if to_render != 0 {
resample.scratch.render_reserved(Some(to_render));
for plane in resample.scratch.planes_mut().planes() {
for val in &mut plane[in_len..] {
*val = 0.0f32;
}
}
}
// Luckily, we make use of the WHOLE input buffer here.
resampler
.process_into_buffer(resample.scratch.planes().planes(), data, None)
.unwrap();
// Calculate true end position using sample rate math
let ratio = (data[0].len() as f32) / (resample.scratch.frames() as f32);
let out_samples = (ratio * (in_len as f32)).round() as usize;
resample.scratch.clear();
resample.resample_pos = 0..out_samples;
}
}
// Now go back and make use of the buffer.
// We have to do this here because we can't make any guarantees about
// the read site having enough space to hold all samples etc.
continue;
}
let source_packet = source_packet.unwrap();
if let Some(resample) = self.resample.as_mut() {
// Do a resample using the newest packet.
let pkt_frames = source_packet.frames();
if pkt_frames == 0 {
continue;
}
let needed_in_frames = resample.resampler.input_frames_next();
let available_frames = self.inner_pos.len();
let force_copy =
resample.scratch.frames() != 0 || needed_in_frames > available_frames;
if (!force_copy) && matches!(source_packet, AudioBufferRef::F32(_)) {
// This is the only case where we can pull off a straight resample...
// I.e., skip scratch.
// NOTE: if let needed as if-let && {bool} is nightly only.
if let AudioBufferRef::F32(s_pkt) = source_packet {
let refs: Vec<&[f32]> = s_pkt
.planes()
.planes()
.iter()
.map(|s| &s[self.inner_pos.start..][..needed_in_frames])
.collect();
self.inner_pos.start += needed_in_frames;
resample
.resampler
.process_into_buffer(&refs, &mut resample.resampled_data, None)
.unwrap();
} else {
unreachable!()
}
} else {
// We either lack enough samples, or have the wrong data format, forcing
// a conversion/copy into scratch.
let old_scratch_len = resample.scratch.frames();
let missing_frames = needed_in_frames - old_scratch_len;
let frames_to_take = available_frames.min(missing_frames);
resample.scratch.render_reserved(Some(frames_to_take));
mix_logic::copy_into_resampler(
&source_packet,
&mut resample.scratch,
self.inner_pos.start,
old_scratch_len,
frames_to_take,
);
self.inner_pos.start += frames_to_take;
if resample.scratch.frames() == needed_in_frames {
resample
.resampler
.process_into_buffer(
resample.scratch.planes().planes(),
&mut resample.resampled_data,
None,
)
.unwrap();
resample.scratch.clear();
} else {
continue;
}
}
resample.resample_pos = 0..resample.resampled_data[0].len();
} else {
// Newest packet may be used straight away: just convert format
// to ensure it's f32.
let bytes_advanced = write_out(
&source_packet,
buf,
&mut self.inner_pos,
&mut self.interrupted_samples,
&mut self.interrupted_byte_pos,
num_chans,
);
buf = &mut buf[bytes_advanced..];
}
}
Ok(orig_sz - buf.len())
}
}
impl Seek for ToAudioBytes {
fn seek(&mut self, _pos: std::io::SeekFrom) -> IoResult<u64> {
Err(IoErrorKind::Unsupported.into())
}
}
impl MediaSource for ToAudioBytes {
fn is_seekable(&self) -> bool {
false
}
fn byte_len(&self) -> Option<u64> {
None
}
}
#[inline]
fn write_out(
source: &AudioBufferRef,
target: &mut [u8],
source_pos: &mut Range<usize>,
spillover: &mut Vec<f32>,
spill_range: &mut Range<usize>,
num_chans: usize,
) -> usize {
match source {
AudioBufferRef::U8(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::U16(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::U24(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::U32(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::S8(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::S16(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::S24(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::S32(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::F32(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
AudioBufferRef::F64(v) =>
write_symph_buffer(v, target, source_pos, spillover, spill_range, num_chans),
}
}
#[inline]
fn write_symph_buffer<S>(
source: &AudioBuffer<S>,
buf: &mut [u8],
source_pos: &mut Range<usize>,
spillover: &mut Vec<f32>,
spill_range: &mut Range<usize>,
num_chans: usize,
) -> usize
where
S: Sample + IntoSample<f32>,
{
let float_space = buf.len() / SAMPLE_LEN;
let interleaved_space = float_space / num_chans;
let non_contiguous_end = (float_space % num_chans) != 0;
let remaining = source_pos.len();
let to_write = remaining.min(interleaved_space);
let need_spill = non_contiguous_end && to_write < remaining;
let samples_used = to_write + if need_spill { 1 } else { 0 };
let last_sample = source_pos.start + to_write;
if need_spill {
spillover.clear();
*spill_range = 0..num_chans * SAMPLE_LEN;
}
for (i, plane) in source.planes().planes()[..num_chans].iter().enumerate() {
for (j, sample) in plane[source_pos.start..][..to_write].iter().enumerate() {
// write this into the correct slot of buf.
let addr = ((j * num_chans) + i) * SAMPLE_LEN;
(&mut buf[addr..][..SAMPLE_LEN])
.write_f32::<LittleEndian>((*sample).into_sample())
.expect("Address known to exist by length checks.");
}
if need_spill {
spillover.push(plane[last_sample].into_sample());
}
}
source_pos.start += samples_used;
to_write * num_chans * SAMPLE_LEN
}
#[inline]
fn write_resample_buffer(
source: &[Vec<f32>],
buf: &mut [u8],
source_pos: &mut Range<usize>,
spillover: &mut Vec<f32>,
spill_range: &mut Range<usize>,
num_chans: usize,
) -> usize {
let float_space = buf.len() / SAMPLE_LEN;
let interleaved_space = float_space / num_chans;
let non_contiguous_end = (float_space % num_chans) != 0;
let remaining = source_pos.len();
let to_write = remaining.min(interleaved_space);
let need_spill = non_contiguous_end && to_write < remaining;
let samples_used = to_write + if need_spill { 1 } else { 0 };
let last_sample = source_pos.start + to_write;
if need_spill {
spillover.clear();
*spill_range = 0..num_chans * SAMPLE_LEN;
}
for (i, plane) in source[..num_chans].iter().enumerate() {
for (j, sample) in plane[source_pos.start..][..to_write].iter().enumerate() {
// write this into the correct slot of buf.
let addr = ((j * num_chans) + i) * SAMPLE_LEN;
(&mut buf[addr..][..SAMPLE_LEN])
.write_f32::<LittleEndian>(*sample)
.expect("Address well-formed according to bounds checks.");
}
if need_spill {
spillover.push(plane[last_sample]);
}
}
source_pos.start += samples_used;
to_write * num_chans * SAMPLE_LEN
}

64
src/input/child.rs → src/input/adapters/child.rs
View File

@@ -1,9 +1,10 @@
use super::*; use crate::input::{AudioStream, Input, LiveInput};
use std::{ use std::{
io::{BufReader, Read}, io::{Read, Result as IoResult},
mem, mem,
process::Child, process::Child,
}; };
use symphonia_core::io::{MediaSource, ReadOnlySource};
use tokio::runtime::Handle; use tokio::runtime::Handle;
use tracing::debug; use tracing::debug;
@@ -15,34 +16,7 @@ use tracing::debug;
/// make sure to use `From<Vec<Child>>`. Here, the *last* process in the `Vec` will be /// make sure to use `From<Vec<Child>>`. Here, the *last* process in the `Vec` will be
/// used as the audio byte source. /// used as the audio byte source.
#[derive(Debug)] #[derive(Debug)]
pub struct ChildContainer(Vec<Child>); pub struct ChildContainer(pub Vec<Child>);
impl ChildContainer {
/// Create a new [`ChildContainer`] from a child process
pub fn new(children: Vec<Child>) -> Self {
Self(children)
}
}
/// Create a [`Reader`] from a child process
pub fn children_to_reader<T>(children: Vec<Child>) -> Reader {
Reader::Pipe(BufReader::with_capacity(
STEREO_FRAME_SIZE * mem::size_of::<T>() * CHILD_BUFFER_LEN,
ChildContainer(children),
))
}
impl From<Child> for Reader {
fn from(container: Child) -> Self {
children_to_reader::<f32>(vec![container])
}
}
impl From<Vec<Child>> for Reader {
fn from(container: Vec<Child>) -> Self {
children_to_reader::<f32>(container)
}
}
impl Read for ChildContainer { impl Read for ChildContainer {
fn read(&mut self, buffer: &mut [u8]) -> IoResult<usize> { fn read(&mut self, buffer: &mut [u8]) -> IoResult<usize> {
@@ -53,6 +27,36 @@ impl Read for ChildContainer {
} }
} }
impl ChildContainer {
/// Create a new [`ChildContainer`] from a child process
#[must_use]
pub fn new(children: Vec<Child>) -> Self {
Self(children)
}
}
impl From<Child> for ChildContainer {
fn from(container: Child) -> Self {
Self(vec![container])
}
}
impl From<Vec<Child>> for ChildContainer {
fn from(container: Vec<Child>) -> Self {
Self(container)
}
}
impl From<ChildContainer> for Input {
fn from(val: ChildContainer) -> Self {
let audio_stream = AudioStream {
input: Box::new(ReadOnlySource::new(val)) as Box<dyn MediaSource>,
hint: None,
};
Input::Live(LiveInput::Raw(audio_stream), None)
}
}
impl Drop for ChildContainer { impl Drop for ChildContainer {
fn drop(&mut self) { fn drop(&mut self) {
let children = mem::take(&mut self.0); let children = mem::take(&mut self.0);

6
src/input/adapters/mod.rs Normal file
View File

@@ -0,0 +1,6 @@
mod async_adapter;
pub mod cached;
mod child;
mod raw_adapter;
pub use self::{async_adapter::*, child::*, raw_adapter::*};

114
src/input/adapters/raw_adapter.rs Normal file
View File

@@ -0,0 +1,114 @@
use crate::input::{AudioStream, Input, LiveInput};
use byteorder::{LittleEndian, WriteBytesExt};
use std::io::{ErrorKind as IoErrorKind, Read, Result as IoResult, Seek, SeekFrom, Write};
use symphonia::core::io::MediaSource;
// format header is a magic string, followed by two LE u32s (sample rate, channel count)
const FMT_HEADER: &[u8; 16] = b"SbirdRaw\0\0\0\0\0\0\0\0";
/// Adapter around a raw, interleaved, `f32` PCM byte stream.
///
/// This may be used to port legacy songbird audio sources to be compatible with
/// the symphonia backend, particularly those with unknown length (making WAV
/// unsuitable).
///
/// The format is described in [`RawReader`].
///
/// [`RawReader`]: crate::input::codecs::RawReader
pub struct RawAdapter<A> {
prepend: [u8; 16],
inner: A,
pos: u64,
}
impl<A: MediaSource> RawAdapter<A> {
/// Wrap an input PCM byte source to be readable by symphonia.
pub fn new(audio_source: A, sample_rate: u32, channel_count: u32) -> Self {
let mut prepend: [u8; 16] = *FMT_HEADER;
let mut write_space = &mut prepend[8..];
write_space
.write_u32::<LittleEndian>(sample_rate)
.expect("Prepend buffer is sized to include enough space for sample rate.");
write_space
.write_u32::<LittleEndian>(channel_count)
.expect("Prepend buffer is sized to include enough space for number of channels.");
Self {
prepend,
inner: audio_source,
pos: 0,
}
}
}
impl<A: MediaSource> Read for RawAdapter<A> {
fn read(&mut self, mut buf: &mut [u8]) -> IoResult<usize> {
let out = if self.pos < self.prepend.len() as u64 {
let upos = self.pos as usize;
let remaining = self.prepend.len() - upos;
let to_write = buf.len().min(remaining);
buf.write(&self.prepend[upos..][..to_write])
} else {
self.inner.read(buf)
};
if let Ok(n) = out {
self.pos += n as u64;
}
out
}
}
impl<A: MediaSource> Seek for RawAdapter<A> {
fn seek(&mut self, pos: SeekFrom) -> IoResult<u64> {
if self.is_seekable() {
let target_pos = match pos {
SeekFrom::Start(p) => p,
SeekFrom::End(_) => return Err(IoErrorKind::Unsupported.into()),
SeekFrom::Current(p) if p.unsigned_abs() > self.pos =>
return Err(IoErrorKind::InvalidInput.into()),
SeekFrom::Current(p) => (self.pos as i64 + p) as u64,
};
let out = if target_pos as usize <= self.prepend.len() {
self.inner.rewind().map(|_| 0)
} else {
self.inner.seek(SeekFrom::Start(target_pos))
};
match out {
Ok(0) => self.pos = target_pos,
Ok(a) => self.pos = a + self.prepend.len() as u64,
_ => {},
}
out.map(|_| self.pos)
} else {
Err(IoErrorKind::Unsupported.into())
}
}
}
impl<A: MediaSource> MediaSource for RawAdapter<A> {
fn is_seekable(&self) -> bool {
self.inner.is_seekable()
}
fn byte_len(&self) -> Option<u64> {
self.inner.byte_len().map(|m| m + self.prepend.len() as u64)
}
}
impl<A: MediaSource + Send + Sync + 'static> From<RawAdapter<A>> for Input {
fn from(val: RawAdapter<A>) -> Self {
let live = LiveInput::Raw(AudioStream {
input: Box::new(val),
hint: None,
});
Input::Live(live, None)
}
}

12
src/input/audiostream.rs Normal file
View File

@@ -0,0 +1,12 @@
use symphonia_core::probe::Hint;
/// An unread byte stream for an audio file.
pub struct AudioStream<T: Send> {
/// The wrapped file stream.
///
/// An input stream *must not* have been read into past the start of the
/// audio container's header.
pub input: T,
/// Extension and MIME type information which may help guide format selection.
pub hint: Option<Hint>,
}

303
src/input/cached/compressed.rs
View File

@@ -1,303 +0,0 @@
use super::{apply_length_hint, compressed_cost_per_sec, default_config};
use crate::{
constants::*,
input::{
error::{Error, Result},
CodecType,
Container,
Input,
Metadata,
Reader,
},
};
use audiopus::{
coder::Encoder as OpusEncoder,
Application,
Bitrate,
Channels,
Error as OpusError,
ErrorCode as OpusErrorCode,
SampleRate,
};
use byteorder::{LittleEndian, ReadBytesExt, WriteBytesExt};
use std::{
convert::TryInto,
io::{Error as IoError, ErrorKind as IoErrorKind, Read, Result as IoResult},
mem,
sync::atomic::{AtomicUsize, Ordering},
};
use streamcatcher::{Config, NeedsBytes, Stateful, Transform, TransformPosition, TxCatcher};
use tracing::{debug, trace};
/// A wrapper around an existing [`Input`] which compresses
/// the input using the Opus codec before storing it in memory.
///
/// The main purpose of this wrapper is to enable seeking on
/// incompatible sources (i.e., ffmpeg output) and to ease resource
/// consumption for commonly reused/shared tracks. [`Restartable`]
/// and [`Memory`] offer the same functionality with different
/// tradeoffs.
///
/// This is intended for use with larger, repeatedly used audio
/// tracks shared between sources, and stores the sound data
/// retrieved as **compressed Opus audio**. There is an associated memory cost,
/// but this is far smaller than using a [`Memory`].
///
/// [`Input`]: Input
/// [`Memory`]: super::Memory
/// [`Restartable`]: crate::input::restartable::Restartable
#[derive(Clone, Debug)]
pub struct Compressed {
/// Inner shared bytestore.
pub raw: TxCatcher<Box<Input>, OpusCompressor>,
/// Metadata moved out of the captured source.
pub metadata: Metadata,
/// Stereo-ness of the captured source.
pub stereo: bool,
}
impl Compressed {
/// Wrap an existing [`Input`] with an in-memory store, compressed using Opus.
///
/// [`Input`]: Input
/// [`Metadata.duration`]: ../struct.Metadata.html#structfield.duration
pub fn new(source: Input, bitrate: Bitrate) -> Result<Self> {
Self::with_config(source, bitrate, None)
}
/// Wrap an existing [`Input`] with an in-memory store, compressed using Opus.
///
/// `config.length_hint` may be used to control the size of the initial chunk, preventing
/// needless allocations and copies. If this is not present, the value specified in
/// `source`'s [`Metadata::duration`] will be used.
///
/// [`Input`]: Input
/// [`Metadata::duration`]: crate::input::Metadata::duration
pub fn with_config(source: Input, bitrate: Bitrate, config: Option<Config>) -> Result<Self> {
let channels = if source.stereo {
Channels::Stereo
} else {
Channels::Mono
};
let mut encoder = OpusEncoder::new(SampleRate::Hz48000, channels, Application::Audio)?;
encoder.set_bitrate(bitrate)?;
Self::with_encoder(source, encoder, config)
}
/// Wrap an existing [`Input`] with an in-memory store, compressed using a user-defined
/// Opus encoder.
///
/// `length_hint` functions as in [`new`]. This function's behaviour is undefined if your encoder
/// has a different sample rate than 48kHz, and if the decoder has a different channel count from the source.
///
/// [`Input`]: Input
/// [`new`]: Compressed::new
pub fn with_encoder(
mut source: Input,
encoder: OpusEncoder,
config: Option<Config>,
) -> Result<Self> {
let bitrate = encoder.bitrate()?;
let cost_per_sec = compressed_cost_per_sec(bitrate);
let stereo = source.stereo;
let metadata = source.metadata.take();
let mut config = config.unwrap_or_else(|| default_config(cost_per_sec));
// apply length hint.
if config.length_hint.is_none() {
if let Some(dur) = metadata.duration {
apply_length_hint(&mut config, dur, cost_per_sec);
}
}
let raw = config
.build_tx(Box::new(source), OpusCompressor::new(encoder, stereo))
.map_err(Error::Streamcatcher)?;
Ok(Self {
raw,
metadata,
stereo,
})
}
/// Acquire a new handle to this object, creating a new
/// view of the existing cached data from the beginning.
pub fn new_handle(&self) -> Self {
Self {
raw: self.raw.new_handle(),
metadata: self.metadata.clone(),
stereo: self.stereo,
}
}
}
impl From<Compressed> for Input {
fn from(src: Compressed) -> Self {
Input::new(
true,
Reader::Compressed(src.raw),
CodecType::Opus
.try_into()
.expect("Default decoder values are known to be valid."),
Container::Dca { first_frame: 0 },
Some(src.metadata),
)
}
}
/// Transform applied inside [`Compressed`], converting a floating-point PCM
/// input stream into a DCA-framed Opus stream.
///
/// Created and managed by [`Compressed`].
///
/// [`Compressed`]: Compressed
#[derive(Debug)]
pub struct OpusCompressor {
encoder: OpusEncoder,
last_frame: Vec<u8>,
stereo_input: bool,
frame_pos: usize,
audio_bytes: AtomicUsize,
}
impl OpusCompressor {
fn new(encoder: OpusEncoder, stereo_input: bool) -> Self {
Self {
encoder,
last_frame: Vec::with_capacity(4000),
stereo_input,
frame_pos: 0,
audio_bytes: Default::default(),
}
}
}
impl<T> Transform<T> for OpusCompressor
where
T: Read,
{
fn transform_read(&mut self, src: &mut T, buf: &mut [u8]) -> IoResult<TransformPosition> {
let output_start = mem::size_of::<u16>();
let mut eof = false;
let mut raw_len = 0;
let mut out = None;
let mut sample_buf = [0f32; STEREO_FRAME_SIZE];
let samples_in_frame = if self.stereo_input {
STEREO_FRAME_SIZE
} else {
MONO_FRAME_SIZE
};
// Purge old frame and read new, if needed.
if self.frame_pos == self.last_frame.len() + output_start || self.last_frame.is_empty() {
self.last_frame.resize(self.last_frame.capacity(), 0);
// We can't use `read_f32_into` because we can't guarantee the buffer will be filled.
for el in sample_buf[..samples_in_frame].iter_mut() {
match src.read_f32::<LittleEndian>() {
Ok(sample) => {
*el = sample;
raw_len += 1;
},
Err(e) if e.kind() == IoErrorKind::UnexpectedEof => {
eof = true;
break;
},
Err(e) => {
out = Some(Err(e));
break;
},
}
}
if out.is_none() && raw_len > 0 {
loop {
// NOTE: we don't index by raw_len because the last frame can be too small
// to occupy a "whole packet". Zero-padding is the correct behaviour.
match self
.encoder
.encode_float(&sample_buf[..samples_in_frame], &mut self.last_frame[..])
{
Ok(pkt_len) => {
trace!("Next packet to write has {:?}", pkt_len);
self.frame_pos = 0;
self.last_frame.truncate(pkt_len);
break;
},
Err(OpusError::Opus(OpusErrorCode::BufferTooSmall)) => {
// If we need more capacity to encode this frame, then take it.
trace!("Resizing inner buffer (+256).");
self.last_frame.resize(self.last_frame.len() + 256, 0);
},
Err(e) => {
debug!("Read error {:?} {:?} {:?}.", e, out, raw_len);
out = Some(Err(IoError::new(IoErrorKind::Other, e)));
break;
},
}
}
}
}
if out.is_none() {
// Write from frame we have.
let start = if self.frame_pos < output_start {
(&mut buf[..output_start])
.write_i16::<LittleEndian>(self.last_frame.len() as i16)
.expect(
"Minimum bytes requirement for Opus (2) should mean that an i16 \
may always be written.",
);
self.frame_pos += output_start;
trace!("Wrote frame header: {}.", self.last_frame.len());
output_start
} else {
0
};
let out_pos = self.frame_pos - output_start;
let remaining = self.last_frame.len() - out_pos;
let write_len = remaining.min(buf.len() - start);
buf[start..start + write_len]
.copy_from_slice(&self.last_frame[out_pos..out_pos + write_len]);
self.frame_pos += write_len;
trace!("Appended {} to inner store", write_len);
out = Some(Ok(write_len + start));
}
// NOTE: use of raw_len here preserves true sample length even if
// stream is extended to 20ms boundary.
out.unwrap_or_else(|| Err(IoError::new(IoErrorKind::Other, "Unclear.")))
.map(|compressed_sz| {
self.audio_bytes
.fetch_add(raw_len * mem::size_of::<f32>(), Ordering::Release);
if eof {
TransformPosition::Finished
} else {
TransformPosition::Read(compressed_sz)
}
})
}
}
impl NeedsBytes for OpusCompressor {
fn min_bytes_required(&self) -> usize {
2
}
}
impl Stateful for OpusCompressor {
type State = usize;
fn state(&self) -> Self::State {
self.audio_bytes.load(Ordering::Acquire)
}
}

116
src/input/cached/memory.rs
View File

@@ -1,116 +0,0 @@
use super::{apply_length_hint, default_config, raw_cost_per_sec};
use crate::input::{
error::{Error, Result},
CodecType,
Container,
Input,
Metadata,
Reader,
};
use std::convert::{TryFrom, TryInto};
use streamcatcher::{Catcher, Config};
/// A wrapper around an existing [`Input`] which caches
/// the decoded and converted audio data locally in memory.
///
/// The main purpose of this wrapper is to enable seeking on
/// incompatible sources (i.e., ffmpeg output) and to ease resource
/// consumption for commonly reused/shared tracks. [`Restartable`]
/// and [`Compressed`] offer the same functionality with different
/// tradeoffs.
///
/// This is intended for use with small, repeatedly used audio
/// tracks shared between sources, and stores the sound data
/// retrieved in **uncompressed floating point** form to minimise the
/// cost of audio processing. This is a significant *3 Mbps (375 kiB/s)*,
/// or 131 MiB of RAM for a 6 minute song.
///
/// [`Input`]: Input
/// [`Compressed`]: super::Compressed
/// [`Restartable`]: crate::input::restartable::Restartable
#[derive(Clone, Debug)]
pub struct Memory {
/// Inner shared bytestore.
pub raw: Catcher<Box<Reader>>,
/// Metadata moved out of the captured source.
pub metadata: Metadata,
/// Codec used to read the inner bytestore.
pub kind: CodecType,
/// Stereo-ness of the captured source.
pub stereo: bool,
/// Framing mechanism for the inner bytestore.
pub container: Container,
}
impl Memory {
/// Wrap an existing [`Input`] with an in-memory store with the same codec and framing.
///
/// [`Input`]: Input
pub fn new(source: Input) -> Result<Self> {
Self::with_config(source, None)
}
/// Wrap an existing [`Input`] with an in-memory store with the same codec and framing.
///
/// `length_hint` may be used to control the size of the initial chunk, preventing
/// needless allocations and copies. If this is not present, the value specified in
/// `source`'s [`Metadata::duration`] will be used, assuming that the source is uncompressed.
///
/// [`Input`]: Input
/// [`Metadata::duration`]: crate::input::Metadata::duration
pub fn with_config(mut source: Input, config: Option<Config>) -> Result<Self> {
let stereo = source.stereo;
let kind = (&source.kind).into();
let container = source.container;
let metadata = source.metadata.take();
let cost_per_sec = raw_cost_per_sec(stereo);
let mut config = config.unwrap_or_else(|| default_config(cost_per_sec));
// apply length hint.
if config.length_hint.is_none() {
if let Some(dur) = metadata.duration {
apply_length_hint(&mut config, dur, cost_per_sec);
}
}
let raw = config
.build(Box::new(source.reader))
.map_err(Error::Streamcatcher)?;
Ok(Self {
raw,
metadata,
kind,
stereo,
container,
})
}
/// Acquire a new handle to this object, creating a new
/// view of the existing cached data from the beginning.
pub fn new_handle(&self) -> Self {
Self {
raw: self.raw.new_handle(),
metadata: self.metadata.clone(),
kind: self.kind,
stereo: self.stereo,
container: self.container,
}
}
}
impl TryFrom<Memory> for Input {
type Error = Error;
fn try_from(src: Memory) -> Result<Self> {
Ok(Input::new(
src.stereo,
Reader::Memory(src.raw),
src.kind.try_into()?,
src.container,
Some(src.metadata),
))
}
}

90
src/input/cached/tests.rs
View File

@@ -1,90 +0,0 @@
use super::*;
use crate::{
constants::*,
input::{error::Error, Codec, Container, Input},
test_utils::*,
};
use audiopus::{coder::Decoder, Bitrate, Channels, SampleRate};
use byteorder::{LittleEndian, ReadBytesExt};
use std::{
convert::TryInto,
io::{Cursor, Read},
};
#[tokio::test]
async fn streamcatcher_preserves_file() {
let input = make_sine(50 * MONO_FRAME_SIZE, true);
let input_len = input.len();
let mut raw = default_config(raw_cost_per_sec(true))
.build(Cursor::new(input.clone()))
.map_err(Error::Streamcatcher)
.unwrap();
let mut out_buf = vec![];
let read = raw.read_to_end(&mut out_buf).unwrap();
assert_eq!(input_len, read);
assert_eq!(input, out_buf);
}
#[test]
fn compressed_scans_frames_decodes_mono() {
let data = one_s_compressed_sine(false);
run_through_dca(data.raw);
}
#[test]
fn compressed_scans_frames_decodes_stereo() {
let data = one_s_compressed_sine(true);
run_through_dca(data.raw);
}
#[test]
fn compressed_triggers_valid_passthrough() {
let mut input = Input::from(one_s_compressed_sine(true));
assert!(input.supports_passthrough());
let mut opus_buf = [0u8; 10_000];
let mut signal_buf = [0i16; 1920];
let opus_len = input.read_opus_frame(&mut opus_buf[..]).unwrap();
let mut decoder = Decoder::new(SampleRate::Hz48000, Channels::Stereo).unwrap();
decoder
.decode(
Some((&opus_buf[..opus_len]).try_into().unwrap()),
(&mut signal_buf[..]).try_into().unwrap(),
false,
)
.unwrap();
}
fn one_s_compressed_sine(stereo: bool) -> Compressed {
let data = make_sine(50 * MONO_FRAME_SIZE, stereo);
let input = Input::new(stereo, data.into(), Codec::FloatPcm, Container::Raw, None);
Compressed::new(input, Bitrate::BitsPerSecond(128_000)).unwrap()
}
fn run_through_dca(mut src: impl Read) {
let mut decoder = Decoder::new(SampleRate::Hz48000, Channels::Stereo).unwrap();
let mut pkt_space = [0u8; 10_000];
let mut signals = [0i16; 1920];
while let Ok(frame_len) = src.read_i16::<LittleEndian>() {
let pkt_len = src.read(&mut pkt_space[..frame_len as usize]).unwrap();
decoder
.decode(
Some((&pkt_space[..pkt_len]).try_into().unwrap()),
(&mut signals[..]).try_into().unwrap(),
false,
)
.unwrap();
}
}

99
src/input/codec/mod.rs
View File

@@ -1,99 +0,0 @@
//! Decoding schemes for input audio bytestreams.
mod opus;
pub use self::opus::OpusDecoderState;
use super::*;
use std::{fmt::Debug, mem};
/// State used to decode input bytes of an [`Input`].
///
/// [`Input`]: Input
#[non_exhaustive]
#[derive(Clone, Debug)]
pub enum Codec {
/// The inner bytestream is encoded using the Opus codec, to be decoded
/// using the given state.
///
/// Must be combined with a non-[`Raw`] container.
///
/// [`Raw`]: Container::Raw
Opus(OpusDecoderState),
/// The inner bytestream is encoded using raw `i16` samples.
///
/// Must be combined with a [`Raw`] container.
///
/// [`Raw`]: Container::Raw
Pcm,
/// The inner bytestream is encoded using raw `f32` samples.
///
/// Must be combined with a [`Raw`] container.
///
/// [`Raw`]: Container::Raw
FloatPcm,
}
impl From<&Codec> for CodecType {
fn from(f: &Codec) -> Self {
use Codec::*;
match f {
Opus(_) => Self::Opus,
Pcm => Self::Pcm,
FloatPcm => Self::FloatPcm,
}
}
}
/// Type of data being passed into an [`Input`].
///
/// [`Input`]: Input
#[non_exhaustive]
#[derive(Copy, Clone, Debug)]
pub enum CodecType {
/// The inner bytestream is encoded using the Opus codec.
///
/// Must be combined with a non-[`Raw`] container.
///
/// [`Raw`]: Container::Raw
Opus,
/// The inner bytestream is encoded using raw `i16` samples.
///
/// Must be combined with a [`Raw`] container.
///
/// [`Raw`]: Container::Raw
Pcm,
/// The inner bytestream is encoded using raw `f32` samples.
///
/// Must be combined with a [`Raw`] container.
///
/// [`Raw`]: Container::Raw
FloatPcm,
}
impl CodecType {
/// Returns the length of a single output sample, in bytes.
pub fn sample_len(&self) -> usize {
use CodecType::*;
match self {
Opus | FloatPcm => mem::size_of::<f32>(),
Pcm => mem::size_of::<i16>(),
}
}
}
impl TryFrom<CodecType> for Codec {
type Error = Error;
fn try_from(f: CodecType) -> Result<Self> {
use CodecType::*;
match f {
Opus => Ok(Codec::Opus(OpusDecoderState::new()?)),
Pcm => Ok(Codec::Pcm),
FloatPcm => Ok(Codec::FloatPcm),
}
}
}

43
src/input/codec/opus.rs
View File

@@ -1,43 +0,0 @@
use crate::constants::*;
use audiopus::{coder::Decoder as OpusDecoder, Channels, Error as OpusError};
use parking_lot::Mutex;
use std::sync::Arc;
#[derive(Clone, Debug)]
/// Inner state used to decode Opus input sources.
pub struct OpusDecoderState {
/// Inner decoder used to convert opus frames into a stream of samples.
pub decoder: Arc<Mutex<OpusDecoder>>,
/// Controls whether this source allows direct Opus frame passthrough.
/// Defaults to `true`.
///
/// Enabling this flag is a promise from the programmer to the audio core
/// that the source has been encoded at 48kHz, using 20ms long frames.
/// If you cannot guarantee this, disable this flag (or else risk nasal demons)
/// and bizarre audio behaviour.
pub allow_passthrough: bool,
pub(crate) current_frame: Vec<f32>,
pub(crate) frame_pos: usize,
pub(crate) should_reset: bool,
}
impl OpusDecoderState {
/// Creates a new decoder, having stereo output at 48kHz.
pub fn new() -> Result<Self, OpusError> {
Ok(Self::from_decoder(OpusDecoder::new(
SAMPLE_RATE,
Channels::Stereo,
)?))
}
/// Creates a new decoder pre-configured by the user.
pub fn from_decoder(decoder: OpusDecoder) -> Self {
Self {
decoder: Arc::new(Mutex::new(decoder)),
allow_passthrough: true,
current_frame: Vec::with_capacity(STEREO_FRAME_SIZE),
frame_pos: 0,
should_reset: false,
}
}
}

53
src/input/codecs/dca/metadata.rs Normal file
View File

@@ -0,0 +1,53 @@
use serde::{Deserialize, Serialize};
#[derive(Debug, Deserialize, Serialize)]
pub struct DcaMetadata {
pub dca: DcaInfo,
pub opus: Opus,
pub info: Option<Info>,
pub origin: Option<Origin>,
pub extra: Option<serde_json::Value>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct DcaInfo {
pub version: u64,
pub tool: Tool,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Tool {
pub name: String,
pub version: String,
pub url: Option<String>,
pub author: Option<String>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Opus {
pub mode: String,
pub sample_rate: u32,
pub frame_size: u64,
pub abr: Option<u64>,
pub vbr: bool,
pub channels: u8,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Info {
pub title: Option<String>,
pub artist: Option<String>,
pub album: Option<String>,
pub genre: Option<String>,
pub cover: Option<String>,
pub comments: Option<String>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct Origin {
pub source: Option<String>,
pub abr: Option<u64>,
pub channels: Option<u8>,
pub encoding: Option<String>,
pub url: Option<String>,
}

343
src/input/codecs/dca/mod.rs Normal file
View File

@@ -0,0 +1,343 @@
mod metadata;
pub use self::metadata::*;
use crate::constants::{SAMPLE_RATE, SAMPLE_RATE_RAW};
use std::io::{Seek, SeekFrom};
use symphonia::core::{
codecs::{CodecParameters, CODEC_TYPE_OPUS},
errors::{self as symph_err, Error as SymphError, Result as SymphResult, SeekErrorKind},
formats::prelude::*,
io::{MediaSource, MediaSourceStream, ReadBytes, SeekBuffered},
meta::{Metadata as SymphMetadata, MetadataBuilder, MetadataLog, StandardTagKey, Tag, Value},
probe::{Descriptor, Instantiate, QueryDescriptor},
sample::SampleFormat,
units::TimeStamp,
};
impl QueryDescriptor for DcaReader {
fn query() -> &'static [Descriptor] {
&[symphonia_core::support_format!(
"dca",
"DCA[0/1] Opus Wrapper",
&["dca"],
&[],
&[b"DCA1"]
)]
}
fn score(_context: &[u8]) -> u8 {
255
}
}
struct SeekAccel {
frame_offsets: Vec<(TimeStamp, u64)>,
seek_index_fill_rate: u16,
next_ts: TimeStamp,
}
impl SeekAccel {
fn new(options: FormatOptions, first_frame_byte_pos: u64) -> Self {
let per_s = options.seek_index_fill_rate;
let next_ts = (per_s as u64) * (SAMPLE_RATE_RAW as u64);
Self {
frame_offsets: vec![(0, first_frame_byte_pos)],
seek_index_fill_rate: per_s,
next_ts,
}
}
fn update(&mut self, ts: TimeStamp, pos: u64) {
if ts >= self.next_ts {
self.next_ts += (self.seek_index_fill_rate as u64) * (SAMPLE_RATE_RAW as u64);
self.frame_offsets.push((ts, pos));
}
}
fn get_seek_pos(&self, ts: TimeStamp) -> (TimeStamp, u64) {
let index = self.frame_offsets.partition_point(|&(o_ts, _)| o_ts <= ts) - 1;
self.frame_offsets[index]
}
}
/// [DCA\[0/1\]](https://github.com/bwmarrin/dca) Format reader for Symphonia.
pub struct DcaReader {
source: MediaSourceStream,
track: Option<Track>,
metas: MetadataLog,
seek_accel: SeekAccel,
curr_ts: TimeStamp,
max_ts: Option<TimeStamp>,
held_packet: Option<Packet>,
}
impl FormatReader for DcaReader {
fn try_new(mut source: MediaSourceStream, options: &FormatOptions) -> SymphResult<Self> {
// Read in the magic number to verify it's a DCA file.
let magic = source.read_quad_bytes()?;
// FIXME: make use of the new options.enable_gapless to apply the opus coder delay.
let read_meta = match &magic {
b"DCA1" => true,
_ if &magic[..3] == b"DCA" => {
return symph_err::unsupported_error("unsupported DCA version");
},
_ => {
source.seek_buffered_rel(-4);
false
},
};
let mut codec_params = CodecParameters::new();
codec_params
.for_codec(CODEC_TYPE_OPUS)
.with_max_frames_per_packet(1)
.with_sample_rate(SAMPLE_RATE_RAW as u32)
.with_time_base(TimeBase::new(1, SAMPLE_RATE_RAW as u32))
.with_sample_format(SampleFormat::F32);
let mut metas = MetadataLog::default();
if read_meta {
let size = source.read_u32()?;
// Sanity check
if (size as i32) < 2 {
return symph_err::decode_error("missing DCA1 metadata block");
}
let raw_json = source.read_boxed_slice_exact(size as usize)?;
let metadata: DcaMetadata = serde_json::from_slice::<DcaMetadata>(&raw_json)
.map_err(|_| SymphError::DecodeError("malformed DCA1 metadata block"))?;
let mut revision = MetadataBuilder::new();
if let Some(info) = metadata.info {
if let Some(t) = info.title {
revision.add_tag(Tag::new(
Some(StandardTagKey::TrackTitle),
"title",
Value::String(t),
));
}
if let Some(t) = info.album {
revision.add_tag(Tag::new(
Some(StandardTagKey::Album),
"album",
Value::String(t),
));
}
if let Some(t) = info.artist {
revision.add_tag(Tag::new(
Some(StandardTagKey::Artist),
"artist",
Value::String(t),
));
}
if let Some(t) = info.genre {
revision.add_tag(Tag::new(
Some(StandardTagKey::Genre),
"genre",
Value::String(t),
));
}
if let Some(t) = info.comments {
revision.add_tag(Tag::new(
Some(StandardTagKey::Comment),
"comments",
Value::String(t),
));
}
if let Some(_t) = info.cover {
// TODO: Add visual, figure out MIME types.
}
}
if let Some(origin) = metadata.origin {
if let Some(t) = origin.url {
revision.add_tag(Tag::new(Some(StandardTagKey::Url), "url", Value::String(t)));
}
}
metas.push(revision.metadata());
}
let bytes_read = source.pos();
Ok(Self {
source,
track: Some(Track {
id: 0,
language: None,
codec_params,
}),
metas,
seek_accel: SeekAccel::new(*options, bytes_read),
curr_ts: 0,
max_ts: None,
held_packet: None,
})
}
fn cues(&self) -> &[Cue] {
// No cues in DCA...
&[]
}
fn metadata(&mut self) -> SymphMetadata<'_> {
self.metas.metadata()
}
fn seek(&mut self, _mode: SeekMode, to: SeekTo) -> SymphResult<SeekedTo> {
let can_backseek = self.source.is_seekable();
let track = if self.track.is_none() {
return symph_err::seek_error(SeekErrorKind::Unseekable);
} else {
self.track.as_ref().unwrap()
};
let rate = track.codec_params.sample_rate;
let ts = match to {
SeekTo::Time { time, .. } =>
if let Some(rate) = rate {
TimeBase::new(1, rate).calc_timestamp(time)
} else {
return symph_err::seek_error(SeekErrorKind::Unseekable);
},
SeekTo::TimeStamp { ts, .. } => ts,
};
if let Some(max_ts) = self.max_ts {
if ts > max_ts {
return symph_err::seek_error(SeekErrorKind::OutOfRange);
}
}
let backseek_needed = self.curr_ts > ts;
if backseek_needed && !can_backseek {
return symph_err::seek_error(SeekErrorKind::ForwardOnly);
}
let (accel_seek_ts, accel_seek_pos) = self.seek_accel.get_seek_pos(ts);
if backseek_needed || accel_seek_pos > self.source.pos() {
self.source.seek(SeekFrom::Start(accel_seek_pos))?;
self.curr_ts = accel_seek_ts;
}
while let Ok(pkt) = self.next_packet() {
let pts = pkt.ts;
let dur = pkt.dur;
let track_id = pkt.track_id();
if (pts..pts + dur).contains(&ts) {
self.held_packet = Some(pkt);
return Ok(SeekedTo {
track_id,
required_ts: ts,
actual_ts: pts,
});
}
}
symph_err::seek_error(SeekErrorKind::OutOfRange)
}
fn tracks(&self) -> &[Track] {
// DCA tracks can hold only one track by design.
// Of course, a zero-length file is technically allowed,
// in which case no track.
if let Some(track) = self.track.as_ref() {
std::slice::from_ref(track)
} else {
&[]
}
}
fn default_track(&self) -> Option<&Track> {
self.track.as_ref()
}
fn next_packet(&mut self) -> SymphResult<Packet> {
if let Some(pkt) = self.held_packet.take() {
return Ok(pkt);
}
let frame_pos = self.source.pos();
let p_len = match self.source.read_u16() {
Ok(len) => len as i16,
Err(eof) => {
self.max_ts = Some(self.curr_ts);
return Err(eof.into());
},
};
if p_len < 0 {
return symph_err::decode_error("DCA frame header had a negative length.");
}
let buf = self.source.read_boxed_slice_exact(p_len as usize)?;
let checked_buf = buf[..].try_into().or_else(|_| {
symph_err::decode_error("Packet was not a valid Opus Packet: too large for audiopus.")
})?;
let sample_ct = audiopus::packet::nb_samples(checked_buf, SAMPLE_RATE).or_else(|_| {
symph_err::decode_error(
"Packet was not a valid Opus packet: couldn't read sample count.",
)
})? as u64;
let out = Packet::new_from_boxed_slice(0, self.curr_ts, sample_ct, buf);
self.seek_accel.update(self.curr_ts, frame_pos);
self.curr_ts += sample_ct;
Ok(out)
}
fn into_inner(self: Box<Self>) -> MediaSourceStream {
self.source
}
}
#[cfg(test)]
mod tests {
use crate::input::input_tests::*;
use crate::{constants::test_data::FILE_DCA_TARGET, input::File};
// NOTE: this covers youtube audio in a non-copyright-violating way, since
// those depend on an HttpRequest internally anyhow.
#[tokio::test]
#[ntest::timeout(10_000)]
async fn dca_track_plays() {
track_plays_passthrough(|| File::new(FILE_DCA_TARGET)).await;
}
#[tokio::test]
#[ntest::timeout(10_000)]
async fn dca_forward_seek_correct() {
forward_seek_correct(|| File::new(FILE_DCA_TARGET)).await;
}
#[tokio::test]
#[ntest::timeout(10_000)]
async fn dca_backward_seek_correct() {
backward_seek_correct(|| File::new(FILE_DCA_TARGET)).await;
}
#[tokio::test]
#[ntest::timeout(10_000)]
async fn opus_passthrough_when_other_tracks_paused() {
track_plays_passthrough_when_is_only_active(|| File::new(FILE_DCA_TARGET)).await;
}
}

34
src/input/codecs/mod.rs Normal file
View File

@@ -0,0 +1,34 @@
//! Codec registries extending Symphonia's probe and registry formats with Opus and DCA support.
pub(crate) mod dca;
mod opus;
mod raw;
pub use self::{dca::DcaReader, opus::OpusDecoder, raw::*};
use lazy_static::lazy_static;
use symphonia::{
core::{codecs::CodecRegistry, probe::Probe},
default::*,
};
lazy_static! {
/// Default Symphonia CodecRegistry, including the (audiopus-backed)
/// Opus codec.
pub static ref CODEC_REGISTRY: CodecRegistry = {
let mut registry = CodecRegistry::new();
register_enabled_codecs(&mut registry);
registry.register_all::<OpusDecoder>();
registry
};
}
lazy_static! {
/// Default Symphonia Probe, including DCA format support.
pub static ref PROBE: Probe = {
let mut probe = Probe::default();
probe.register_all::<DcaReader>();
probe.register_all::<RawReader>();
register_enabled_formats(&mut probe);
probe
};
}

167
src/input/codecs/opus.rs Normal file
View File

@@ -0,0 +1,167 @@
use crate::constants::*;
use audiopus::{
coder::{Decoder as AudiopusDecoder, GenericCtl},
Channels,
Error as OpusError,
ErrorCode,
};
use symphonia_core::{
audio::{AsAudioBufferRef, AudioBuffer, AudioBufferRef, Layout, Signal, SignalSpec},
codecs::{
CodecDescriptor,
CodecParameters,
Decoder,
DecoderOptions,
FinalizeResult,
CODEC_TYPE_OPUS,
},
errors::{decode_error, Result as SymphResult},
formats::Packet,
};
/// Opus decoder for symphonia, based on libopus v1.3 (via [`audiopus`]).
pub struct OpusDecoder {
inner: AudiopusDecoder,
params: CodecParameters,
buf: AudioBuffer<f32>,
rawbuf: Vec<f32>,
}
/// # SAFETY
/// The underlying Opus decoder (currently) requires only a `&self` parameter
/// to decode given packets, which is likely a mistaken decision.
///
/// This struct makes stronger assumptions and only touches FFI decoder state with a
/// `&mut self`, preventing data races via `&OpusDecoder` as required by `impl Sync`.
/// No access to other internal state relies on unsafety or crosses FFI.
unsafe impl Sync for OpusDecoder {}
impl OpusDecoder {
fn decode_inner(&mut self, packet: &Packet) -> SymphResult<()> {
let s_ct = loop {
let pkt = if packet.buf().is_empty() {
None
} else if let Ok(checked_pkt) = packet.buf().try_into() {
Some(checked_pkt)
} else {
return decode_error("Opus packet was too large (greater than i32::MAX bytes).");
};
let out_space = (&mut self.rawbuf[..]).try_into().expect("The following logic expands this buffer safely below i32::MAX, and we throw our own error.");
match self.inner.decode_float(pkt, out_space, false) {
Ok(v) => break v,
Err(OpusError::Opus(ErrorCode::BufferTooSmall)) => {
// double the buffer size
// correct behav would be to mirror the decoder logic in the udp_rx set.
let new_size = (self.rawbuf.len() * 2).min(std::i32::MAX as usize);
if new_size == self.rawbuf.len() {
return decode_error("Opus frame too big: cannot expand opus frame decode buffer any further.");
}
self.rawbuf.resize(new_size, 0.0);
self.buf = AudioBuffer::new(
self.rawbuf.len() as u64 / 2,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, Layout::Stereo),
);
},
Err(e) => {
tracing::error!("Opus decode error: {:?}", e);
return decode_error("Opus decode error: see 'tracing' logs.");
},
}
};
self.buf.clear();
self.buf.render_reserved(Some(s_ct));
// Forcibly assuming stereo, for now.
for ch in 0..2 {
let iter = self.rawbuf.chunks_exact(2).map(|chunk| chunk[ch]);
for (tgt, src) in self.buf.chan_mut(ch).iter_mut().zip(iter) {
*tgt = src;
}
}
Ok(())
}
}
impl Decoder for OpusDecoder {
fn try_new(params: &CodecParameters, _options: &DecoderOptions) -> SymphResult<Self> {
let inner = AudiopusDecoder::new(SAMPLE_RATE, Channels::Stereo).unwrap();
let mut params = params.clone();
params.with_sample_rate(SAMPLE_RATE_RAW as u32);
Ok(Self {
inner,
params,
buf: AudioBuffer::new(
MONO_FRAME_SIZE as u64,
SignalSpec::new_with_layout(SAMPLE_RATE_RAW as u32, Layout::Stereo),
),
rawbuf: vec![0.0f32; STEREO_FRAME_SIZE],
})
}
fn supported_codecs() -> &'static [CodecDescriptor] {
&[symphonia_core::support_codec!(
CODEC_TYPE_OPUS,
"opus",
"libopus (1.3+, audiopus)"
)]
}
fn codec_params(&self) -> &CodecParameters {
&self.params
}
fn decode(&mut self, packet: &Packet) -> SymphResult<AudioBufferRef<'_>> {
if let Err(e) = self.decode_inner(packet) {
self.buf.clear();
Err(e)
} else {
Ok(self.buf.as_audio_buffer_ref())
}
}
fn reset(&mut self) {
let _ = self.inner.reset_state();
}
fn finalize(&mut self) -> FinalizeResult {
FinalizeResult::default()
}
fn last_decoded(&self) -> AudioBufferRef {
self.buf.as_audio_buffer_ref()
}
}
#[cfg(test)]
mod tests {
use crate::{
constants::test_data::FILE_WEBM_TARGET,
input::{input_tests::*, File},
};
// NOTE: this covers youtube audio in a non-copyright-violating way, since
// those depend on an HttpRequest internally anyhow.
#[tokio::test]
#[ntest::timeout(10_000)]
async fn webm_track_plays() {
track_plays_passthrough(|| File::new(FILE_WEBM_TARGET)).await;
}
#[tokio::test]
#[ntest::timeout(10_000)]
async fn webm_forward_seek_correct() {
forward_seek_correct(|| File::new(FILE_WEBM_TARGET)).await;
}
#[tokio::test]
#[ntest::timeout(10_000)]
async fn webm_backward_seek_correct() {
backward_seek_correct(|| File::new(FILE_WEBM_TARGET)).await;
}
}

182
src/input/codecs/raw.rs Normal file
View File

@@ -0,0 +1,182 @@
use std::io::{Seek, SeekFrom};
use symphonia::core::{
audio::Channels,
codecs::{CodecParameters, CODEC_TYPE_PCM_F32LE},
errors::{self as symph_err, Result as SymphResult, SeekErrorKind},
formats::prelude::*,
io::{MediaSource, MediaSourceStream, ReadBytes, SeekBuffered},
meta::{Metadata as SymphMetadata, MetadataLog},
probe::{Descriptor, Instantiate, QueryDescriptor},
units::TimeStamp,
};
impl QueryDescriptor for RawReader {
fn query() -> &'static [Descriptor] {
&[symphonia_core::support_format!(
"raw",
"Raw arbitrary-length f32 audio container.",
&["rawf32"],
&[],
&[b"SbirdRaw"]
)]
}
fn score(_context: &[u8]) -> u8 {
255
}
}
/// Symphonia support for a simple container for raw f32-PCM data of unknown duration.
///
/// Contained files have a simple header:
/// * the 8-byte signature `b"SbirdRaw"`,
/// * the sample rate, as a little-endian `u32`,
/// * the channel count, as a little-endian `u32`.
///
/// The remainder of the file is interleaved little-endian `f32` samples.
pub struct RawReader {
source: MediaSourceStream,
track: Track,
meta: MetadataLog,
curr_ts: TimeStamp,
max_ts: Option<TimeStamp>,
}
impl FormatReader for RawReader {
fn try_new(mut source: MediaSourceStream, _options: &FormatOptions) -> SymphResult<Self> {
let mut magic = [0u8; 8];
ReadBytes::read_buf_exact(&mut source, &mut magic[..])?;
if &magic != b"SbirdRaw" {
source.seek_buffered_rel(-(magic.len() as isize));
return symph_err::decode_error("rawf32: illegal magic byte sequence.");
}
let sample_rate = source.read_u32()?;
let n_chans = source.read_u32()?;
let chans = match n_chans {
1 => Channels::FRONT_LEFT,
2 => Channels::FRONT_LEFT | Channels::FRONT_RIGHT,
_ =>
return symph_err::decode_error(
"rawf32: channel layout is not stereo or mono for fmt_pcm",
),
};
let mut codec_params = CodecParameters::new();
codec_params
.for_codec(CODEC_TYPE_PCM_F32LE)
.with_bits_per_coded_sample((std::mem::size_of::<f32>() as u32) * 8)
.with_bits_per_sample((std::mem::size_of::<f32>() as u32) * 8)
.with_sample_rate(sample_rate)
.with_time_base(TimeBase::new(1, sample_rate))
.with_sample_format(symphonia_core::sample::SampleFormat::F32)
.with_max_frames_per_packet(sample_rate as u64 / 50)
.with_channels(chans);
Ok(Self {
source,
track: Track {
id: 0,
language: None,
codec_params,
},
meta: MetadataLog::default(),
curr_ts: 0,
max_ts: None,
})
}
fn cues(&self) -> &[Cue] {
&[]
}
fn metadata(&mut self) -> SymphMetadata<'_> {
self.meta.metadata()
}
fn seek(&mut self, _mode: SeekMode, to: SeekTo) -> SymphResult<SeekedTo> {
let can_backseek = self.source.is_seekable();
let track = &self.track;
let rate = track.codec_params.sample_rate;
let ts = match to {
SeekTo::Time { time, .. } =>
if let Some(rate) = rate {
TimeBase::new(1, rate).calc_timestamp(time)
} else {
return symph_err::seek_error(SeekErrorKind::Unseekable);
},
SeekTo::TimeStamp { ts, .. } => ts,
};
if let Some(max_ts) = self.max_ts {
if ts > max_ts {
return symph_err::seek_error(SeekErrorKind::OutOfRange);
}
}
let backseek_needed = self.curr_ts > ts;
if backseek_needed && !can_backseek {
return symph_err::seek_error(SeekErrorKind::ForwardOnly);
}
let chan_count = track
.codec_params
.channels
.expect("Channel count is built into format.")
.count() as u64;
let seek_pos = 16 + (std::mem::size_of::<f32>() as u64) * (ts * chan_count);
self.source.seek(SeekFrom::Start(seek_pos))?;
self.curr_ts = ts;
Ok(SeekedTo {
track_id: track.id,
required_ts: ts,
actual_ts: ts,
})
}
fn tracks(&self) -> &[Track] {
std::slice::from_ref(&self.track)
}
fn default_track(&self) -> Option<&Track> {
Some(&self.track)
}
fn next_packet(&mut self) -> SymphResult<Packet> {
let track = &self.track;
let rate = track
.codec_params
.sample_rate
.expect("Sample rate is built into format.") as usize;
let chan_count = track
.codec_params
.channels
.expect("Channel count is built into format.")
.count();
let sample_unit = std::mem::size_of::<f32>() * chan_count;
// Aim for 20ms (50Hz).
let buf = self.source.read_boxed_slice((rate / 50) * sample_unit)?;
let sample_ct = (buf.len() / sample_unit) as u64;
let out = Packet::new_from_boxed_slice(0, self.curr_ts, sample_ct, buf);
self.curr_ts += sample_ct;
Ok(out)
}
fn into_inner(self: Box<Self>) -> MediaSourceStream {
self.source
}
}

40
src/input/compose.rs Normal file
View File

@@ -0,0 +1,40 @@
use super::{AudioStream, AudioStreamError, AuxMetadata};
use symphonia_core::io::MediaSource;
/// Data and behaviour required to instantiate a lazy audio source.
#[async_trait::async_trait]
pub trait Compose: Send {
/// Create a source synchronously.
///
/// If [`should_create_async`] returns `false`, this method will chosen at runtime.
///
/// [`should_create_async`]: Self::should_create_async
fn create(&mut self) -> Result<AudioStream<Box<dyn MediaSource>>, AudioStreamError>;
/// Create a source asynchronously.
///
/// If [`should_create_async`] returns `true`, this method will chosen at runtime.
///
/// [`should_create_async`]: Self::should_create_async
async fn create_async(&mut self)
-> Result<AudioStream<Box<dyn MediaSource>>, AudioStreamError>;
/// Determines whether this source will be instantiated using [`create`] or [`create_async`].
///
/// Songbird will create the audio stream using either a dynamically sized thread pool,
/// or a task on the async runtime it was spawned in respectively. Users do not need to
/// support both these methods.
///
/// [`create_async`]: Self::create_async
/// [`create`]: Self::create
fn should_create_async(&self) -> bool;
/// Requests auxiliary metadata which can be accessed without parsing the file.
///
/// This method will never be called by songbird but allows, for instance, access to metadata
/// which might only be visible to a web crawler e.g., uploader or source URL.
async fn aux_metadata(&mut self) -> Result<AuxMetadata, AudioStreamError> {
Err(AudioStreamError::Unsupported)
}
}

8
src/input/container/frame.rs
View File

@@ -1,8 +0,0 @@
/// Information used in audio frame detection.
#[derive(Clone, Copy, Debug)]
pub struct Frame {
/// Length of this frame's header, in bytes.
pub header_len: usize,
/// Payload length, in bytes.
pub frame_len: usize,
}

Some files were not shown because too many files have changed in this diff Show More