This PR Introduces a new `VoiceTick` event which collects and reorders all RTP packets to smooth over network instability, as well as to synchronise user audio streams. Raw packet events have been moved to `RtpPacket`, while `SpeakingUpdate`s have been removed as they can be easily computed using the `silent`/`speaking` audio maps included in each event.
Closes#146.
This PR makes use of `SampleBuffer::samples_mut` to remove a 7680B stack allocation in general, and memcopy when softclip is used. This appears to offer ~1.5% performance boost according to `cargo make bench`.
Adds a new field to Config, disposer, an Option<Sender<DisposalMessage>> responsible for dropping the DisposalMessage on a separate thread.
If this is not set, and the Config is passed into manager::Songbird, a thread is spawned for this purpose (which previously was spawned per driver).
If this is not set, and the Config is passed directly into Driver or Call, a thread is spawned locally, which is the current behavior as there is no where to store the Sender.
This disposer is then used in Driver as previously, to run possibly blocking destructors (which should only block the disposal thread). I cannot see this disposal thread getting overloaded, but if it is the DisposalMessages will simply be queued in the flume channel until it can be dropped.
Co-authored-by: Kyle Simpson <kyleandrew.simpson@gmail.com>
`SsrcState` objects are created on a per-user basis when "receive" is enabled, but were previously never destroyed. This PR adds some shared dashmaps for the WS task to communicate SSRC-to-ID mappings to the UDP Rx task, as well as any disconnections. Additionally, decoder state is pruned a default 1 minute after a user last speaks.
This was tested using `cargo make ready` and via `examples/serenity/voice_receive/`.
Closes#133
Adds the "receive" feature, which is disabled by default. When this is disabled, the UDP receive task is not compiled and not run, and as an optimisation the UDP receive buffer size is set to 0. All related events are also removed.
This also removes the UDP Tx task, and moves packet and keepalive sends back into the mixer thread. This allows us to entirely remove channels and various allocations between the mixer and an async task created only for sending data (i.e., fewer memcopies).
If "receive" is enabled, UDP sends are now non-blocking due to technical constraints -- failure to send is non-fatal, but *will* drop affected packets. Given that blocking on a UDP send indicates that the OS cannot clear send buffers fast enough, this should alleviate OS load.
Closes#131.
This places songbird, serenity, and twilight onto the same WS library, hopefully reducing the compile overhead for everyone.
Tested using `cargo make ready` and by running `examples/voice`.
Closes#129.
This adds the `use_softclip` field to `Config`, which can currently provide a ~10us reduction in mixing cost from both a removed memcpy and the softclip itself.
This PR was tested using cargo make ready.
Closes#134.
This extensive PR rewrites the internal mixing logic of the driver to use symphonia for parsing and decoding audio data, and rubato to resample audio. Existing logic to decode DCA and Opus formats/data have been reworked as plugins for symphonia. The main benefit is that we no longer need to keep yt-dlp and ffmpeg processes alive, saving a lot of memory and CPU: all decoding can be done in Rust! In exchange, we now need to do a lot of the HTTP handling and resumption ourselves, but this is still a huge net positive.
`Input`s have been completely reworked such that all default (non-cached) sources are lazy by default, and are no longer covered by a special-case `Restartable`. These now span a gamut from a `Compose` (lazy), to a live source, to a fully `Parsed` source. As mixing is still sync, this includes adapters for `AsyncRead`/`AsyncSeek`, and HTTP streams.
`Track`s have been reworked so that they only contain initialisation state for each track. `TrackHandles` are only created once a `Track`/`Input` has been handed over to the driver, replacing `create_player` and related functions. `TrackHandle::action` now acts on a `View` of (im)mutable state, and can request seeks/readying via `Action`.
Per-track event handling has also been improved -- we can now determine and propagate the reason behind individual track errors due to the new backend. Some `TrackHandle` commands (seek etc.) benefit from this, and now use internal callbacks to signal completion.
Due to associated PRs on felixmcfelix/songbird from avid testers, this includes general clippy tweaks, API additions, and other repo-wide cleanup. Thanks go out to the below co-authors.
Co-authored-by: Gnome! <45660393+GnomedDev@users.noreply.github.com>
Co-authored-by: Alakh <36898190+alakhpc@users.noreply.github.com>
Decrypt logic had two locations where the nonce would be separated from the payload without verifying the buffer size first, causing a panic for small packets.
Nonce and header removal now return an error if there are insufficient bytes.
Tested using `cargo make ready`, with some new tests to check that small packets simply return an `Err(...)`, and that encryption/decryption still function.
Sending poison messages should suffice to kill the voice session: attempting to `.leave()`. Fixes#88.
This was tested using `cargo make ready` and the modified `serenity/voice/` example.
This PR adds several enhancements to Driver connection logic:
* Driver (re)connection attempts now have a default timeout of around 10s.
* The driver will now attempt to retry full connection attempts using a user-provided strategy: currently, this defaults to 5 attempts under an exponential backoff strategy.
* The driver will now fire `DriverDisconnect` events at the end of any session -- this unifies (re)connection failure events with session expiry as seen in #76, which should provide users with enough detail to know *which* voice channel to reconnect to. Users still need to be careful to read the session/channel IDs to ensure that they aren't overwriting another join.
This has been tested using `cargo make ready`, and by setting low timeouts to force failures in the voice receive example (with some additional error handlers).
Closes#68.
This PR makes many of the types under `EventContext` separate `#[non_exhaustive]` structs. This makes it more feasible to add further information to connection and packet events as required in future. On this note, driver (re)connection events now include the SSRC supplied by Discord and the domain name which was connected to.
In addition, this fixes global timed events to return a list of all live tracks, and extensively details/documents events at a high level.
This was tested using `cargo make ready`.
This is a simple organisational change which moves `crate::Bitrate` to `crate::driver::Bitrate` to slightly clean up the crate root.
This has been tested using `cargo make ready`.
This change fixes tasks hanging due to rare cases of messages being lost between full Discord reconnections by placing a configurable timeout on the `ConnectionInfo` responses. This is a companion fix to [serenity#1255](https://github.com/serenity-rs/serenity/pull/1255). To make this doable, `Config`s are now used by all versions of `Songbird`/`Call`, and relevant functions are added to simplify setup with configuration. These are now non-exhaustive, correcting an earlier oversight. For future extensibility, this PR moves the return type of `join`/`join_gateway` into a custom future (no longer leaking flume's `RecvFut` type).
Additionally, this fixes the Makefile's feature sets for driver/gateway-only compilation.
This is a breaking change in:
* the return types of `join`/`join_gateway`
* moving `crate::driver::Config` -> `crate::Config`,
* `Config` and `JoinError` becoming `#[non_breaking]`.
This was tested via `cargo make ready`, and by testing `examples/serenity/voice_receive` with various timeout settings.
Leaving (rather than removing) a call would cause the driver to crash as it would try to use a non-existent connection immediately after it had been invalidated.
This has been tested using a modified `examples/serenity/voice_storage`, felyne, and via `cargo make ready`.
This change reduces many log levels to debug, particularly where errors are likely to be triggered by undocumented Discord messages or by threads exiting in an unpredictable way. This also reduces the task entry/exit messages to `trace`.
This PR has been tested via `cargo make ready`, and by manually inspecting logs at `debug` and `info` levels running `examples/serenity/voice`.
This change prevents mixer threads from waking every 20ms without an active voice connection. This was leading to unacceptably high CPU usage in cases where users needed to preserve this state between many active connections. Additionally, this modifies the documentation of `Songbird::leave` to emphasise why users would prefer to `remove` their calls.
This was tested by examining the CPU usage in task manager before and after the change was made, using a control of 10k manually created `Driver` instances. After creation is finished, the Drivers no longer saturate a 6-core laptop Intel i7 (while they very much did so before).
Closes#42.
Knowing your own SSRC is useful for handling RTCP packets, which may detail information about *ourselves* rather than another host. In theory, at least: this confirms that Discord just sends ReceiverReports containing your own packet stats.
This would have been better to fit into Driver(Re)Connect, but that would be a breaking change: when this change is made, `SsrcKnown` shall be deprecated.
Adds support to the library for tokio 0.2 backward-compatibility. This should hopefully benefit, and prevent lavalink-rs from being blocked on this feature.
These can be reached using, e.g., `gateway-tokio-02`, `driver-tokio-02`, `serenity-rustls-tokio-02`, and `serenity-native-tokio-02` features.
Naturally, this requires some jiggering about with features and the underlying CI, which has been taken care of. Twilight can't be handled in this way, as their last tokio 0.2 version uses the deprecated Discord Gateway v6.
Linux/Unix requires that processes be waited, which is unfortunate as Windows lets us abandon them to the murderous whims of the OS. This PR adds Unix-specific behaviour to send a SIGINT before waiting on the process, and adds an additional thread per call for asset disposal on all platforms.
Closes#38.
---
* Close processes by SIGINT and wait on Unix
This seems to remedy the Linux-specific zombie processes. Addition of
nix as a dependency *should* be fine on Windows, since I believe it
compiles to an empty crate.
* Dispose of Tracks on auxiliary thread
This adds a mechanism for the mixer threads to perform potentially expensive deallocation/cleanup outside of the main loop, preventing deadline misses etc. This should make misbehaving `wait`s a bit more friendly.
Moves to the faster dashmap in the Songbird management struct, as the final v4 brought back the `entry` API that I was needing to use it safely.
Also handles some new clippy lints.
These should allow bots to hook up events to a variety of important connection events as required. This was primarily motivated by the user who raised dcb6ad9.
Although I really would have liked to squeeze in (finite) reconnection attempts with exponential backoff, so that automated repeat attempts could be neatly handled, `Config` was accidentally *not* made non-exhaustive. Adding this and its needed configuration would then be a breaking change. This should warn users about an accidentally dead connection, until the next version can be put forth.
Closes#26.
This will also prevent a full reconnect failure from endlessly spamming attempts and error logs. I'll follow this up by looking into decent reconnection strategies, although sadly these won't be configurable until the next semver break due to an oversight on my part.
Migrates to the new version of tokio, requiring channel and sleep changes in a few locations. Additionally points to the in-tree v0.3 version of twilight.
* Driver Benchmarks
Benchmarks driver use cases for single packet send,
multiple packet send, float vs opus, and the cost of
head-of-queue track removal.
Mix costs for large packet counts are also included.
This is a prelude to the optimisations discussed in
#21.
* Typo in benchmark
* Place Opus packet directly into packet buffer
Cleans up some other logic surrounding this, too. Gets a 16.9% perf improvement on opus packet passthrough (sub 5us here).
* Better track removal
In theory this should be faster, but it aint. Keeping in case
reducing struct sizes down the line magically makes this
faster.
* Reduce size of Input, TrackHandle
Metadata is now boxed away. Similarly, TrackHandles are neatly Arc'd to reduce their size to pointer length (and mitigate the impact of copies if we add in more fields).
Main goal: a lot of nested future/result folding.
This mainly modifies error handling for Tracks and TrackHandles to be
more consistent, and hides the underlying channel result passing in
get_info. Errors returned should be far clearer, and are domain
specific rather than falling back to a very opaque use of the underlying
channel error. It should be clearer to users why their handle commands
failed, or why they can't make a ytdl track loop or similar.
Also fixed/cleaned up Songbird::join(_gateway) to return in a single
await, sparing the user from the underlying channel details and repeated
Errs. I was trying for some time to extend the same graces to `Call`,
but could not figure out a sane way to get a 'static version of the
first future in the chain (i.e., the gateway send) so that the whole
thing could happen after dropping the lock around the Call. I really
wanted to fix this to happen as a single folded await too, but I think
this might need some crazy hack or redesign.
Discord's web client on Firefox seems to send very large packets,
ranging from 20ms to 60ms at different times and systems. This adapts
the UDP Rx task to remember the largest packet needed for any SSRC
in a call and preallocate that much, which also allows it to decode
such packets.
Far cleaner and more reliable than the old doc-link pattern. Also allowed me to spot some event types and sources which should have been made non_exhaustive.
* Adds a uuid field to tracks and handles to make it easier to identify and match event sources after the fact.
* Adds optional feature "builtin-queue" to expose a queue on every driver, as a convenience for users who can guarantee they'll need a queue for every driver/call.
* Adds methods to queues to allow access to the currently running track handle, remove a specified queue entry, as well as to mutate the underlying queue from a closure.
Voice `CloseCode`s now map to a type rather than a collection of constants. Correct close code handling in this way terminates the websocket task when there is no likelihood of resuming, which was causing leftover tasks to spin at the `tokio::select` in some circumstances (i.e., ::leave, which keeps the `Driver` alive).
This implements a proof-of-concept for an improved audio frontend. The largest change is the introduction of events and event handling: both by time elapsed and by track events, such as ending or looping. Following on from this, the library now includes a basic, event-driven track queue system (which people seem to ask for unusually often). A new sample, `examples/13_voice_events`, demonstrates both the `TrackQueue` system and some basic events via the `~queue` and `~play_fade` commands.
Locks are removed from around the control of `Audio` objects, which should allow the backend to be moved to a more granular futures-based backend solution in a cleaner way.
