feat(postit): add native WebRTC calls with TURN IP masking

Replace the direct chat call panel with native Android and Linux WebRTC engines, enforce relay-only privacy, and cover consent, lifecycle, rendering and coturn interoperability. Document deployment and upgrade Avalonia to 12.1.6.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Paul Schneider 2026-11-11 09:24:59 +00:00
commit d812266ee0
Signed by: notazof
GPG key ID: AD4B9B545FBD7D12
28 changed files with 2568 additions and 61 deletions

View file

@ -5,6 +5,10 @@
### Added
* [PostIt/Calls] Replace the chat's direct VLC/SRTP call panel with native WebRTC on Android and Linux: official libwebrtc/Camera2 and GStreamer webrtcbin, existing consent/SignalR signaling, ICE/STUN/coturn, and fail-closed relay-only IP masking. Cancel pending startup and release media on hangup, navigation, interruption and backgrounding. Add privacy/lifecycle, real Linux media/rendering and debug Android interoperability regressions, coturn deployment instructions and a protected-server-secret configuration example.
* [PostIt] Upgrade Avalonia to 12.1.6 and align the headless and native Skia test rendering packages.
* [PostIt/PayPal] Add a native subscription page reachable from Home, profile and About: load server-priced Discovery/Premium offers, prepare or resume checkout with the current bearer session, open only PayPal for approval, reconcile confirmed payments and balances, and cancel with confirmation. Add quota-exempt bearer-only endpoints on Yavsc.Org, sharing checkout/cancellation logic with the existing web flow, and regression coverage. Browser return never confirms payment.
* [Resource usage] Let users with the `Administrator` role continue making requests after reaching their consumption quota, while still recording their usage.

View file

@ -28,6 +28,61 @@ dotnet publish -c Release
and then, install the app, from the `src/PostIt/PostIt.Android/bin/Release/net10.0/PostIt.apk`
## PostIt native WebRTC calls (Android and Linux)
Calls are opt-in on the HTTPS chat host. Configure `AudioVideoCalls` there:
```json
{
"AudioVideoCalls": {
"Enabled": true,
"EnableLanCalls": false,
"StunUrls": ["stun:turn.example.org:3478"],
"TurnUrls": [
"turn:turn.example.org:3478?transport=udp",
"turn:turn.example.org:3478?transport=tcp",
"turns:turn.example.org:5349?transport=tcp"
]
}
}
```
Set `AudioVideoCalls__TurnSharedSecret` through the host's protected environment
or secret store, **not** in a committed settings file or in PostIt. Generate
a strong random secret (for example `openssl rand -hex 32`), install coturn and
adapt [coturn.conf.example](contrib/coturn.conf.example) outside the repository.
Set its `static-auth-secret` to the same value. The server issues separate,
short-lived REST/HMAC credentials only after invitation acceptance; clients
never receive the shared secret. Keep clocks synchronized.
Expose TURN on UDP/TCP 3478, TLS/TCP 5349, and the configured **UDP relay range**
(49160–49200 in the example). Use a publicly trusted certificate matching the
TURN hostname and ensure coturn can read its key. TURN is not an HTTP reverse
proxy route. If behind NAT, configure coturn's `external-ip=PUBLIC/PRIVATE`,
`listening-ip` and `relay-ip` for the actual deployment and forward the relay
range without changing port numbers. Restrict relay access to private services,
set bandwidth/capacity limits appropriate to the host, and avoid logging SDP,
credentials or candidates. Do not copy the loopback permissions used by tests.
Android 8+ bundles native libwebrtc from the official Maven AAR. Linux needs
Python 3 GI and GStreamer with `webrtcbin`, libnice, VP8, Opus, JPEG, V4L2 and
PulseAudio/PipeWire support. On Debian/Ubuntu:
```bash
sudo apt install python3-gi python3-gst-1.0 gir1.2-gst-plugins-bad-1.0 \
gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad gstreamer1.0-nice gstreamer1.0-pulseaudio
```
Join the same chat room, refresh/select camera and microphone, invite a peer,
then accept on the other client. Capture starts only after acceptance.
The profile's IP-masking setting forces **both participants** through TURN;
missing/unreachable TURN fails the call, never falls back to direct media.
Without masking, explicit IP-disclosure consent is required.
TURN still sees each client's address, and masking does not hide addresses
from the signaling host or local network. See the
[architecture and native regression commands](doc/architecture/audio-video-calls.md).
## PostIt.Desktop on Windows (per-user installer)

View file

@ -0,0 +1,32 @@
# Copy outside the repository; replace the realm, secret and certificate paths.
# Use the same randomly generated secret in AudioVideoCalls:TurnSharedSecret.
realm=turn.example.org
fingerprint
use-auth-secret
static-auth-secret=REPLACE_WITH_A_RANDOM_SERVER_ONLY_SECRET
listening-port=3478
tls-listening-port=5349
cert=/etc/letsencrypt/live/turn.example.org/fullchain.pem
pkey=/etc/letsencrypt/live/turn.example.org/privkey.pem
min-port=49160
max-port=49200
no-cli
no-multicast-peers
no-tcp-relay
user-quota=4
total-quota=512
stale-nonce=600
# Do not allow the relay to reach private infrastructure or loopback services.
denied-peer-ip=0.0.0.0-0.255.255.255
denied-peer-ip=10.0.0.0-10.255.255.255
denied-peer-ip=100.64.0.0-100.127.255.255
denied-peer-ip=127.0.0.0-127.255.255.255
denied-peer-ip=169.254.0.0-169.254.255.255
denied-peer-ip=172.16.0.0-172.31.255.255
denied-peer-ip=192.168.0.0-192.168.255.255
denied-peer-ip=224.0.0.0-255.255.255.255
denied-peer-ip=::
denied-peer-ip=::1
denied-peer-ip=fc00::-fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
denied-peer-ip=fe80::-febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff

View file

@ -2,12 +2,14 @@
## Statut et périmètre
Ce document décrit l'architecture cible et l'implémentation commencée.
Ce document décrit l'architecture cible, l'implementation native actuelle et
l'historique du transport direct.
La préférence de confidentialité est persistée et éditable dans le profil.
La signalisation est intégrée au chat, désactivée par défaut. Un premier
transport natif **Android et Desktop Linux en IPv4/IPv6 directe (LAN/VPN ou IP publique)** est raccordé :
captures Podcast existantes et SRTP direct, sans libwebrtc ni nouvelle
couche JNI WebRTC. Il ne réalise pas encore la cible Internet, TURN ou groupe.
La signalisation est intégrée au chat, désactivée par défaut. La page du chat
utilise maintenant **WebRTC natif sur Android et Desktop Linux**, avec
ICE/STUN/TURN et masquage IP impose par la politique serveur. L'ancien
transport VLC/SRTP n'est plus raccorde a cette page. Les groupes et les
clients navigateur ne sont pas implementes.
Il complète la [topologie de PostIt](postit.md) et les
[mécanismes de notification existants](notifications.md).
La cible couvre :
@ -18,7 +20,105 @@ La cible couvre :
- les appels de groupe ;
- les clients Browser, Android et Desktop.
## Première implémentation Android et Desktop Linux : SRTP direct IPv4/IPv6
## Implementation actuelle : WebRTC natif Android et Linux
- [WebRtcCallViewModel](../../src/PostIt/PostIt/ViewModels/Chat/WebRtcCallViewModel.cs)
conserve invitation/acceptation/fin via SignalR, sans demarrer la capture
avant consentement. Les SDP et candidats ICE utilisent les contrats existants.
- [AndroidWebRtcMedia](../../src/PostIt/PostIt.Android/Services/AndroidWebRtcMedia.cs)
heberge deux surfaces natives. Le pont Java
[WebRtcPeer](../../src/PostIt/PostIt.Android/Services/WebRtcPeer.java) utilise
l'AAR officiel `io.github.webrtc-sdk:android:150.7871.04`, Camera2,
l'audio natif et DTLS-SRTP. Les commandes et callbacks SDP/ICE restent en
Java sur un executeur unique ; pas de WebView ni de binding C# complet du SDK.
- [LinuxWebRtcMedia](../../src/PostIt/PostIt/Services/LinuxWebRtcMedia.cs)
pilote un processus Python GI prive utilisant GStreamer `webrtcbin`,
VP8/Opus, la camera V4L2 et le microphone PulseAudio/PipeWire.
Les trames decodees sont affichees dans Avalonia, l'audio sur la sortie native.
Le protocole JSON utilise uniquement stdin/stdout, pas de serveur local.
- Le serveur impose `RelayOnly` aux deux clients si l'un des comptes masque
son IP. Le client configure la politique native avant capture/negociation,
valide les SDP/candidats sortants et confirme le type du candidat local
de la paire ICE selectionnee. Aucun repli direct ni transport SRTP historique
n'est tente en cas d'echec TURN.
- Avec libnice, les metadonnees ICE `raddr`/`rport` des candidats relais sont
neutralisees a la frontiere de signalisation (RFC 8828), sans modifier
l'adresse du relais ni reecrire le SDP. UPnP est desactive. Les politiques
`relay` de webrtcbin et `force-relay` de libnice sont toutes deux activees.
Un SDP qui expose une adresse non relayee est refuse, pas corrige.
- Raccrocher, sortir de la page/salle, se deconnecter ou passer l'application
en arriere-plan annule aussi un demarrage natif en cours et libere les medias.
Un echec natif, une perte ICE ou 30 secondes sans progression audio/video
terminent l'appel explicitement. Il n'y a pas d'ICE restart/renegociation
automatique ni de migration transparente d'un appel existant.
Le panneau ne demande plus d'adresse ni de port RTP manuel. Sans masquage,
un consentement explicite au partage IP reste requis. Activer le masquage
dans le profil pendant un appel fait terminer cet appel par le serveur :
il faut rappeler pour appliquer la nouvelle politique.
Configuration coturn, certificats, ports et dependances Linux :
[INSTALL](../../INSTALL.md#postit-native-webrtc-calls-android-and-linux).
Les moteurs d'appel sont independants des captures/encodeurs Podcast ;
les Podcasts conservent leur chemin existant.
### Regressions natives
Tests Linux reels, sans compte ni camera, avec sources video/audio synthetiques,
decodage bidirectionnel, selection effective d'un relais coturn, controle des
adresses signalees et panne TURN sans repli :
```bash
python3 src/PostIt/PostIt.Tests/Native/webrtc_media_test.py -v
```
Installer coturn pour ces tests (ou definir `POSTIT_TURN_SERVER` vers son
executable). Ils demarrent un processus local prive, avec secret REST aleatoire
temporaire, et l'arretent en fin de test ; ils ne changent aucun service systeme.
Tests du consentement, de la politique, des interruptions et du rendu de vraies
trames dans Avalonia/Skia :
```bash
XDG_CONFIG_HOME="$(mktemp -d)" POSTIT_TEST_NATIVE_WEBRTC=1 \
dotnet test --project src/PostIt/PostIt.Tests \
--filter-class 'PostIt.Tests.WebRtcCallViewModelTests' \
--filter-class 'PostIt.Tests.LanCallViewModelTests'
```
Le test de rendu natif est ignore sans `POSTIT_TEST_NATIVE_WEBRTC=1`.
Les tests headless ordinaires conservent leur moteur de dessin simule.
Utiliser un dossier de configuration de test sans comptes reels et le supprimer
apres execution.
Le test Android physique est reserve a l'APK **Debug** installe : camera avant,
microphone, deux surfaces natives, pair Linux synthetique et coturn local,
aucun compte ni enregistrement. Il utilise `adb reverse` pour des ports TCP
temporaires et retire uniquement ses propres redirections :
```bash
POSTIT_TEST_ANDROID_WEBRTC=1 POSTIT_ANDROID_SERIAL=SERIAL \
python3 src/PostIt/PostIt.Tests/Native/webrtc_media_test.py \
NativeWebRtcTests.test_android_linux_relay_bidirectional_media -v
```
Ce controle local ne remplace pas un essai sur deux reseaux Internet distincts,
avec DNS/certificats publics et pare-feu de production.
Validation effectuee : 37 tests PostIt (dont rendu Linux avec Skia), 76 tests
serveur de signalisation et 5 regressions natives reussis. Sur le telephone
Android arm64, les deux roles appelant/appele ont recu plus de 100 trames
video et 500 paquets audio pendant au moins 10 secondes, avec deux rendus
natifs confirmes et un candidat local `relay` des deux cotes. La panne TURN
ne produit aucun candidat direct sur Android ou Linux. Les essais utilisent
la camera avant et le microphone Android reels, des sources Linux synthetiques
et un coturn TCP local via USB ; la camera et la sortie audio Linux materielles,
TURN/TLS et les reseaux Internet distincts restent a verifier.
## Historique : premiere implementation SRTP direct IPv4/IPv6
Les paragraphes suivants decrivent le transport precedent, conserve dans le
code et ses tests mais non propose par le panneau d'appel actuel.
Les adresses IPv4 publiques directement assignees a une interface locale active
sont egalement disponibles, avec la meme validation unicast cote client,
@ -508,45 +608,37 @@ les redémarrages ICE et renégociations restent à implémenter.
Configuration serveur, section `AudioVideoCalls` :
- `Enabled` : `false` par défaut, à conserver tant que les moteurs ne sont
pas validés ;
- `Enabled` : `false` par défaut, a activer apres configuration et validation
du deploiement ICE/TURN ;
- `StunUrls` : tableau d'URI `stun:` ou `stuns:` ;
- `TurnUrls` : tableau d'URI `turn:` ou `turns:`, avec transport optionnel ;
- `TurnSharedSecret` : secret REST coturn, **uniquement côté serveur**,
à fournir par le gestionnaire de secrets ou la variable d'environnement
`AudioVideoCalls__TurnSharedSecret`, jamais dans le dépôt.
Linux GStreamer prend en charge `stun:`, mais pas `stuns:` : utiliser TURN/TLS
pour ce dernier besoin.
Les credentials TURN temporaires sont calculés côté serveur avec le
format REST coturn (expiration, nonce aléatoire et HMAC-SHA1). Aucun secret
de longue durée n'est envoyé aux clients. Aucun service TURN n'a été
déployé ou configuré par cette tranche.
de longue durée n'est envoyé aux clients. Aucun service TURN de production
n'est deploye automatiquement ; seuls les tests lancent un coturn isole.
Le [transport PostIt](../../src/PostIt/PostIt/Services/CallSignalingClient.cs)
utilise la connexion authentifiée du `ChatViewModel`, sans seconde connexion
ni interrogation HTTP. Il expose les commandes et événements typés et
signale les interruptions ; il ne capture aucun média. L'interface ne
propose pas encore de bouton d'appel ; une éventuelle invitation est
signalée comme non utilisable faute de moteur natif, sans acceptation
automatique.
signale les interruptions ; il ne capture aucun média.
Le panneau WebRTC de la page du chat et les deux adaptateurs natifs
consomment maintenant cette signalisation, toujours sans acceptation automatique.
### Travail restant pour la cible Internet multi-plateforme
- Android : l'approche Java/JNI libwebrtc a été écartée à la demande du
projet ; aucun adaptateur libwebrtc expérimental n'est conservé.
L'adaptateur SRTP LAN réutilise désormais la capture Podcast ; la solution
compatible avec la cible Internet reste à choisir.
- Desktop Linux : le transport SRTP LAN est raccordé, mais ne remplace pas
les validations matérielles ni une solution de connectivité Internet.
- Côté partagé : relier consentement, états réels du moteur, mise en
sourdine, arrêt et libération des ressources à la signalisation.
- Valider un appel Android ↔ Desktop Linux avec caméra avant et microphone
réels, rendu et audio distants, VP8/Opus, puis transport TURN sélectionné
dans les statistiques, sans repli direct lorsque le masquage est demandé.
Les tests de cette tranche portent sur les contrats, autorisations, courses
d'acceptation, expirations, limites, interruption et préférence persistée.
La compilation des applications Android et Desktop ne prouve pas le
fonctionnement de WebRTC natif : le crash antérieur n'est pas résolu.
- Valider les cameras, microphones et sorties audio Linux materiels.
- Valider les appels sur deux reseaux Internet distincts, UDP/TCP/TLS,
DNS, certificats et politiques de pare-feu de production.
- Ajouter si necessaire les commandes de sourdine, renegociation/ICE restart
et renouvellement des credentials pour les appels longs.
- Les groupes/SFU, Browser et les autres plateformes restent hors perimetre.
## Retour sur l'essai Android du 25 octobre 2026
@ -584,4 +676,9 @@ impossibilité de WebRTC sur Android ou une défaillance du ChatHub.
Les essais Browser/GStreamer réussis utilisaient des sources synthétiques
et un décodage sans affichage côté GStreamer : ils ne valident pas
PostIt.Desktop Linux, ses périphériques réels ou son rendu Avalonia.
La validation multi-plateforme demeure incomplète.
Ces limites de l'ancien laboratoire ne constituent plus le bilan du moteur
actuel : les nouvelles regressions Android/GStreamer ci-dessus verifient
des medias bidirectionnels et des rendus reels via TURN. La cause precise
du crash de cet ancien prototype reste inconnue ; aucun crash natif n'a
ete reproduit pendant les nouveaux essais. Cela ne remplace pas les
validations Internet et materielles encore indiquees.

View file

@ -4,18 +4,19 @@
<!-- https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management -->
<PropertyGroup>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<AvaloniaVersionBase>12.1.4</AvaloniaVersionBase>
<AvaloniaVersionBase>12.1.6</AvaloniaVersionBase>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="Avalonia" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.AvaloniaEdit" Version="12.0.0" />
<PackageVersion Include="Avalonia.Themes.Fluent" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Desktop" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Browser" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Android" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Headless" Version="12.1.4" />
<PackageVersion Include="Avalonia.Headless" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Headless.Xunit" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Skia" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.Fonts.Inter" Version="$(AvaloniaVersionBase)" />
<PackageVersion Include="Avalonia.AvaloniaEdit" Version="12.0.0" />
<PackageVersion Include="DirectShowLib" Version="1.0.0" />
<PackageVersion Include="LibVLCSharp.Avalonia" Version="3.10.2" />
<PackageVersion Include="LibVLCSharp" Version="3.10.2" />

View file

@ -27,6 +27,10 @@ internal static class PlatformBootstrap
Platform.CreateVideoSurface = (player, replay) => new AndroidVideoSurface(player, replay);
if (System.OperatingSystem.IsAndroidVersionAtLeast(26))
{
Platform.CreateWebRtcMedia = () => System.OperatingSystem.IsAndroidVersionAtLeast(26)
? new AndroidWebRtcMedia()
: throw new System.PlatformNotSupportedException("Android 8 requis pour WebRTC.");
Platform.GetWebRtcDevices = AndroidLanCallMedia.GetDevices;
Platform.GetLanCallDevices = AndroidLanCallMedia.GetDevices;
Platform.CreateLanCallMedia = (address, port) =>
System.OperatingSystem.IsAndroidVersionAtLeast(26)

View file

@ -30,6 +30,10 @@
<PackageReference Include="Xamarin.AndroidX.Core.SplashScreen" />
<PackageReference Include="Xamarin.AndroidX.Browser" />
</ItemGroup>
<ItemGroup>
<AndroidMavenLibrary Include="io.github.webrtc-sdk:android" Version="150.7871.04" Bind="false" VerifyDependencies="false" />
<AndroidJavaSource Update="Services/WebRtcPeer.java" Bind="false" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\PostIt\PostIt.csproj" />
</ItemGroup>
@ -40,6 +44,7 @@
<Compile Include="..\PostIt.Tests\AndroidCameraFrameRotationTestActivity.cs" Link="Tests\AndroidCameraFrameRotationTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidPodcastBlurTestActivity.cs" Link="Tests\AndroidPodcastBlurTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidLanCallTestActivity.cs" Link="Tests\AndroidLanCallTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidWebRtcTestActivity.cs" Link="Tests\AndroidWebRtcTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidScreenAwakeTestActivity.cs" Link="Tests\AndroidScreenAwakeTestActivity.cs" />
</ItemGroup>
<!-- This is an Avalonia app, not a MAUI app: exclude the .NET MAUI VS Code

View file

@ -0,0 +1,164 @@
using System;
using System.Diagnostics;
using System.Runtime.Versioning;
using System.Threading;
using System.Threading.Tasks;
using Android.Content;
using Android.Runtime;
using Avalonia.Android;
using Avalonia.Controls;
using Avalonia.Platform;
using PostIt.Services;
namespace PostIt.Android.Services;
[SupportedOSPlatform("android26.0")]
internal sealed class AndroidWebRtcMedia : WebRtcNativeMedia
{
private readonly CancellationTokenSource _lifetime = new();
private readonly Peer _peer;
private readonly Surface _local;
private readonly Surface _remote;
private readonly Task _poll;
private readonly TaskCompletionSource _closed = new(TaskCreationOptions.RunContinuationsAsynchronously);
private bool _disposed;
public override Control LocalVideo => _local;
public override Control RemoteVideo => _remote;
public AndroidWebRtcMedia()
{
_peer = new Peer(MainActivity.Current ?? throw new InvalidOperationException("Activité Android indisponible."));
_local = new Surface(_peer.GetView("localView"));
_remote = new Surface(_peer.GetView("remoteView"));
_poll = Task.Run(PollAsync);
}
protected override async Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
{
var activity = MainActivity.Current ?? throw new InvalidOperationException("Activité Android indisponible.");
await PodcastPermissions.EnsureAsync(activity, microphone: true, ct, notifications: false);
if (!MainActivity.IsForeground)
throw new InvalidOperationException("Revenez dans PostIt pour demarrer l'appel.");
await Task.WhenAll(_local.Ready, _remote.Ready).WaitAsync(TimeSpan.FromSeconds(10), ct);
await SendCommandAsync(StartCommand(configuration), ct);
}
protected override Task SendCommandAsync(string json, CancellationToken ct)
{
ct.ThrowIfCancellationRequested();
_peer.Command(json);
return Task.CompletedTask;
}
private async Task PollAsync()
{
var stats = Stopwatch.StartNew();
var failed = false;
try
{
while (!_lifetime.IsCancellationRequested)
{
for (var i = 0; i < 64 && _peer.Poll() is { } message; i++)
{
if (message == "{\"type\":\"closed\"}") _closed.TrySetResult();
if (!failed)
{
try { ProcessMessage(message); }
catch (Exception error)
{
failed = true;
Fail(error);
if (_disposed) _closed.TrySetException(error);
}
}
}
if (!_disposed && !failed && Configuration is not null && stats.Elapsed >= TimeSpan.FromSeconds(1))
{
_peer.Command("{\"command\":\"stats\"}");
stats.Restart();
}
await Task.Delay(20, _lifetime.Token);
}
}
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
catch (Exception error) { Fail(error); _closed.TrySetException(error); }
}
public override async ValueTask DisposeAsync()
{
if (_disposed) return;
_disposed = true;
try
{
_peer.Command("{\"command\":\"close\"}");
await _closed.Task.WaitAsync(TimeSpan.FromSeconds(10));
}
finally
{
_lifetime.Cancel();
await _poll;
_peer.Dispose();
_lifetime.Dispose();
Closed();
}
}
private sealed class Surface(global::Android.Views.View view) : NativeControlHost
{
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
public Task Ready => _ready.Task;
protected override IPlatformHandle CreateNativeControlCore(IPlatformHandle parent)
{
_ready.TrySetResult();
return new AndroidViewControlHandle(view);
}
}
// Only this small Java bridge crosses JNI. SDP and ICE callbacks stay in Java,
// where the native SDK owns their lifetime and thread affinity.
private sealed class Peer : IDisposable
{
private readonly Java.Lang.Class _type;
private readonly Java.Lang.Object _instance;
private readonly Java.Lang.Reflect.Method _command;
private readonly Java.Lang.Reflect.Method _poll;
public Peer(Context context)
{
_type = Java.Lang.Class.ForName("fr.pschneider.postit.WebRtcPeer")!;
using var contextType = Java.Lang.Class.FromType(typeof(Context));
using var constructor = _type.GetConstructor(contextType);
_instance = constructor!.NewInstance(context)!;
using var stringType = Java.Lang.Class.FromType(typeof(Java.Lang.String));
_command = _type.GetMethod("command", stringType)!;
_poll = _type.GetMethod("poll")!;
}
public global::Android.Views.View GetView(string method)
{
using var getter = _type.GetMethod(method);
return getter!.Invoke(_instance)!.JavaCast<global::Android.Views.View>()!;
}
public void Command(string message)
{
using var value = new Java.Lang.String(message);
_command.Invoke(_instance, value);
}
public string? Poll()
{
using var result = _poll.Invoke(_instance);
return result?.ToString();
}
public void Dispose()
{
_command.Dispose();
_poll.Dispose();
_instance.Dispose();
_type.Dispose();
}
}
}

View file

@ -32,6 +32,7 @@ internal static class PodcastPermissions
if (!await pending.Task.WaitAsync(ct))
throw new UnauthorizedAccessException(notifications
? "Autorisez la camera, les notifications et, si selectionne, le microphone pour diffuser."
: microphone ? "Autorisez la camera et le microphone pour l'appel."
: "Autorisez la camera pour la previsualisation.");
}

View file

@ -0,0 +1,323 @@
package fr.pschneider.postit;
import android.content.Context;
import org.json.JSONArray;
import org.json.JSONObject;
import org.webrtc.*;
import org.webrtc.audio.JavaAudioDeviceModule;
import java.util.*;
import java.util.concurrent.*;
/** Keeps all SDK operations and asynchronous SDP callbacks on one executor. */
public final class WebRtcPeer {
private final Context context;
private final EglBase egl = EglBase.create();
private final SurfaceViewRenderer local;
private final SurfaceViewRenderer remote;
private final ExecutorService executor = Executors.newSingleThreadExecutor();
private final ArrayBlockingQueue<String> events = new ArrayBlockingQueue<>(64);
private final List<IceCandidate> pendingLocal = new ArrayList<>();
private final List<IceCandidate> pendingRemote = new ArrayList<>();
private PeerConnectionFactory factory;
private PeerConnection peer;
private JavaAudioDeviceModule audioModule;
private VideoCapturer capturer;
private SurfaceTextureHelper texture;
private VideoSource videoSource;
private AudioSource audioSource;
private VideoTrack videoTrack;
private AudioTrack audioTrack;
private VideoTrack remoteTrack;
private boolean localDescriptionSent;
private boolean remoteDescriptionSet;
private volatile boolean closed;
private volatile boolean localRendered;
private volatile boolean remoteRendered;
public WebRtcPeer(Context context) {
this.context = context.getApplicationContext();
local = new SurfaceViewRenderer(context);
remote = new SurfaceViewRenderer(context);
local.init(egl.getEglBaseContext(), new RendererCommon.RendererEvents() {
public void onFirstFrameRendered() { localRendered = true; }
public void onFrameResolutionChanged(int width, int height, int rotation) {}
});
local.setMirror(true);
remote.init(egl.getEglBaseContext(), new RendererCommon.RendererEvents() {
public void onFirstFrameRendered() { remoteRendered = true; }
public void onFrameResolutionChanged(int width, int height, int rotation) {}
});
local.setEnableHardwareScaler(true);
remote.setEnableHardwareScaler(true);
}
public android.view.View localView() { return local; }
public android.view.View remoteView() { return remote; }
public String poll() { return events.poll(); }
private void enqueue(Runnable action) {
try {
executor.execute(() -> {
try { action.run(); }
catch (Exception failure) { error(failure.getMessage()); }
});
} catch (RejectedExecutionException failure) {
if (!closed) throw failure;
}
}
public void command(String json) {
enqueue(() -> {
try {
JSONObject message = new JSONObject(json);
switch (message.getString("command")) {
case "start": start(message.getJSONObject("configuration")); break;
case "signal": signal(message.getJSONObject("signal")); break;
case "stats": if (peer != null && !closed) peer.getStats(this::stats); break;
case "close": close(); break;
default: throw new IllegalArgumentException("Unknown WebRTC command");
}
} catch (Exception error) { error(error.getMessage()); }
});
}
private void emit(JSONObject value) {
if (!events.offer(value.toString())) {
events.clear();
events.offer("{\"type\":\"error\",\"message\":\"WebRTC event queue overflow\"}");
}
}
private void error(String message) {
try { emit(new JSONObject().put("type", "error").put("message", message)); }
catch (org.json.JSONException impossible) { throw new IllegalStateException(impossible); }
}
private void start(JSONObject configuration) throws Exception {
if (factory != null || closed) throw new IllegalStateException("WebRTC already started or closed");
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context)
.setEnableInternalTracer(false).createInitializationOptions());
audioModule = JavaAudioDeviceModule.builder(context).createAudioDeviceModule();
factory = PeerConnectionFactory.builder().setAudioDeviceModule(audioModule)
.setVideoEncoderFactory(new DefaultVideoEncoderFactory(egl.getEglBaseContext(), true, true))
.setVideoDecoderFactory(new DefaultVideoDecoderFactory(egl.getEglBaseContext())).createPeerConnectionFactory();
List<PeerConnection.IceServer> servers = new ArrayList<>();
JSONArray configuredServers = configuration.getJSONArray("iceServers");
boolean relay = configuration.getBoolean("relayOnly");
for (int i = 0; i < configuredServers.length(); i++) {
JSONObject item = configuredServers.getJSONObject(i);
JSONArray urls = item.getJSONArray("urls");
for (int j = 0; j < urls.length(); j++) {
String url = urls.getString(j);
if (relay && !url.startsWith("turn:") && !url.startsWith("turns:")) continue;
PeerConnection.IceServer.Builder server = PeerConnection.IceServer.builder(url);
if (!item.isNull("username")) server.setUsername(item.getString("username"));
if (!item.isNull("credential")) server.setPassword(item.getString("credential"));
servers.add(server.createIceServer());
}
}
PeerConnection.RTCConfiguration rtc = new PeerConnection.RTCConfiguration(servers);
rtc.sdpSemantics = PeerConnection.SdpSemantics.UNIFIED_PLAN;
rtc.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE;
rtc.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE;
rtc.iceTransportsType = relay ? PeerConnection.IceTransportsType.RELAY : PeerConnection.IceTransportsType.ALL;
peer = factory.createPeerConnection(rtc, new Observer());
if (peer == null) throw new IllegalStateException("Cannot create native WebRTC peer");
Camera2Enumerator cameras = new Camera2Enumerator(context);
String camera = configuration.getString("camera");
capturer = cameras.createCapturer(camera, new CameraVideoCapturer.CameraEventsHandler() {
public void onCameraError(String message) { error("Camera: " + message); }
public void onCameraDisconnected() { if (!closed) error("Camera disconnected"); }
public void onCameraFreezed(String message) { error("Camera frozen: " + message); }
public void onCameraOpening(String name) {}
public void onFirstFrameAvailable() {}
public void onCameraClosed() { if (!closed) error("Camera closed unexpectedly"); }
});
if (capturer == null) throw new IllegalArgumentException("Camera unavailable");
texture = SurfaceTextureHelper.create("PostIt-WebRTC-camera", egl.getEglBaseContext());
videoSource = factory.createVideoSource(false);
capturer.initialize(texture, context, videoSource.getCapturerObserver());
videoTrack = factory.createVideoTrack("postit-video", videoSource);
videoTrack.addSink(local);
audioSource = factory.createAudioSource(new MediaConstraints());
audioTrack = factory.createAudioTrack("postit-audio", audioSource);
peer.addTrack(audioTrack, Collections.singletonList("postit"));
peer.addTrack(videoTrack, Collections.singletonList("postit"));
capturer.startCapture(640, 480, 24);
if (configuration.getBoolean("caller")) peer.createOffer(new CreateDescription("offer"), new MediaConstraints());
}
private abstract class SdpCallback implements SdpObserver {
public void onCreateSuccess(SessionDescription description) {}
public void onSetSuccess() {}
public void onCreateFailure(String message) { error("SDP creation failed: " + message); }
public void onSetFailure(String message) { error("SDP application failed: " + message); }
}
private final class CreateDescription extends SdpCallback {
private final String kind;
CreateDescription(String kind) { this.kind = kind; }
public void onCreateSuccess(SessionDescription description) {
enqueue(() -> {
if (closed) return;
peer.setLocalDescription(new SdpCallback() {
public void onSetSuccess() {
enqueue(() -> {
if (closed) return;
try {
emit(new JSONObject().put("type", "signal").put("signal",
new JSONObject().put("kind", kind).put("sdp", description.description)));
localDescriptionSent = true;
for (IceCandidate candidate : pendingLocal) sendCandidate(candidate);
pendingLocal.clear();
} catch (Exception failure) { error(failure.getMessage()); }
});
}
}, description);
});
}
}
private void signal(JSONObject signal) throws Exception {
if (closed || peer == null) throw new IllegalStateException("WebRTC peer not running");
String kind = signal.getString("kind");
if (kind.equals("candidate")) {
IceCandidate candidate = new IceCandidate(signal.getString("sdpMid"),
signal.getInt("sdpMLineIndex"), signal.getString("candidate"));
if (remoteDescriptionSet) addCandidate(candidate);
else {
if (pendingRemote.size() >= 256) throw new IllegalStateException("Too many pending ICE candidates");
pendingRemote.add(candidate);
}
} else if (kind.equals("offer") || kind.equals("answer")) {
peer.setRemoteDescription(new SdpCallback() {
public void onSetSuccess() {
enqueue(() -> {
if (closed) return;
remoteDescriptionSet = true;
for (IceCandidate candidate : pendingRemote) addCandidate(candidate);
pendingRemote.clear();
if (kind.equals("offer")) peer.createAnswer(new CreateDescription("answer"), new MediaConstraints());
});
}
}, new SessionDescription(kind.equals("offer") ? SessionDescription.Type.OFFER : SessionDescription.Type.ANSWER,
signal.getString("sdp")));
} else throw new IllegalArgumentException("Invalid WebRTC signal");
}
private void addCandidate(IceCandidate candidate) {
peer.addIceCandidate(candidate, new AddIceObserver() {
public void onAddSuccess() {}
public void onAddFailure(String message) { error("ICE candidate rejected: " + message); }
});
}
private void sendCandidate(IceCandidate candidate) {
try {
emit(new JSONObject().put("type", "signal").put("signal", new JSONObject()
.put("kind", "candidate").put("candidate", candidate.sdp)
.put("sdpMid", candidate.sdpMid).put("sdpMLineIndex", candidate.sdpMLineIndex)));
} catch (Exception failure) { error(failure.getMessage()); }
}
private void stats(RTCStatsReport report) {
enqueue(() -> {
if (closed) return;
try {
Map<String, RTCStats> all = report.getStatsMap();
String selectedId = null, localType = null;
long audioPackets = 0, videoFrames = 0;
for (RTCStats item : all.values()) {
Map<String, Object> values = item.getMembers();
if (item.getType().equals("transport") && values.get("selectedCandidatePairId") instanceof String)
selectedId = (String)values.get("selectedCandidatePairId");
if (item.getType().equals("inbound-rtp")) {
String media = String.valueOf(values.getOrDefault("kind", values.get("mediaType")));
if (media.equals("audio")) audioPackets += number(values.get("packetsReceived"));
if (media.equals("video")) videoFrames += number(values.get("framesDecoded"));
}
}
RTCStats pair = all.get(selectedId);
if (pair != null) {
RTCStats candidate = all.get(String.valueOf(pair.getMembers().get("localCandidateId")));
if (candidate != null) localType = String.valueOf(candidate.getMembers().get("candidateType"));
}
String state = peer.connectionState().toString().toLowerCase(Locale.ROOT);
if (state.equals("connected") && localType == null) state = "checking";
emit(new JSONObject().put("type", "status").put("status", new JSONObject()
.put("connectionState", state).put("localCandidateType", localType)
.put("audioPackets", audioPackets).put("videoFrames", videoFrames)
.put("localVideoRendered", localRendered).put("remoteVideoRendered", remoteRendered)));
} catch (Exception failure) { error(failure.getMessage()); }
});
}
private static long number(Object value) { return value instanceof Number ? ((Number)value).longValue() : 0; }
private final class Observer implements PeerConnection.Observer {
public void onSignalingChange(PeerConnection.SignalingState state) {}
public void onIceConnectionChange(PeerConnection.IceConnectionState state) {}
public void onIceConnectionReceivingChange(boolean receiving) {}
public void onIceGatheringChange(PeerConnection.IceGatheringState state) {}
public void onIceCandidatesRemoved(IceCandidate[] candidates) {}
public void onAddStream(MediaStream stream) {}
public void onRemoveStream(MediaStream stream) {}
public void onDataChannel(DataChannel channel) {}
public void onRenegotiationNeeded() {}
public void onIceCandidate(IceCandidate candidate) {
enqueue(() -> {
if (closed) return;
if (localDescriptionSent) sendCandidate(candidate);
else if (pendingLocal.size() < 256) pendingLocal.add(candidate);
else error("Too many local ICE candidates");
});
}
public void onTrack(RtpTransceiver transceiver) {
MediaStreamTrack track = transceiver.getReceiver().track();
if (track instanceof VideoTrack) {
enqueue(() -> {
if (closed) return;
remoteTrack = (VideoTrack)track;
remoteTrack.addSink(remote);
});
}
}
}
private interface Cleanup { void run() throws Exception; }
private void release(Cleanup cleanup, List<String> failures) {
try { cleanup.run(); }
catch (Exception failure) { failures.add(failure.toString()); }
}
private void close() {
if (closed) return;
closed = true;
List<String> failures = new ArrayList<>();
if (capturer != null) release(() -> capturer.stopCapture(), failures);
if (remoteTrack != null) release(() -> remoteTrack.removeSink(remote), failures);
if (videoTrack != null) release(() -> videoTrack.removeSink(local), failures);
if (peer != null) {
release(() -> peer.close(), failures);
release(() -> peer.dispose(), failures);
}
if (videoTrack != null) release(() -> videoTrack.dispose(), failures);
if (audioTrack != null) release(() -> audioTrack.dispose(), failures);
if (capturer != null) release(() -> capturer.dispose(), failures);
if (texture != null) release(() -> texture.dispose(), failures);
if (videoSource != null) release(() -> videoSource.dispose(), failures);
if (audioSource != null) release(() -> audioSource.dispose(), failures);
if (factory != null) release(() -> factory.dispose(), failures);
if (audioModule != null) release(() -> audioModule.release(), failures);
new android.os.Handler(android.os.Looper.getMainLooper()).post(() -> {
release(() -> local.release(), failures);
release(() -> remote.release(), failures);
release(() -> egl.release(), failures);
if (!failures.isEmpty()) error("WebRTC cleanup failed: " + String.join("; ", failures));
try { emit(new JSONObject().put("type", "closed")); }
catch (Exception failure) { error(failure.getMessage()); }
});
executor.shutdown();
}
}

View file

@ -7,3 +7,5 @@
-keepattributes *Annotation*
-dontwarn org.videolan.libvlc.**
-dontwarn org.videolan.medialibrary.**
-keep class org.webrtc.** { *; }
-keep class fr.pschneider.postit.WebRtcPeer { *; }

View file

@ -0,0 +1,140 @@
#if ANDROID
using System;
using System.IO;
using System.Linq;
using System.Net.Sockets;
using System.Runtime.Versioning;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Android.App;
using Android.Content;
using Android.OS;
using Avalonia.Controls;
using Avalonia.Threading;
using PostIt.Android;
using PostIt.Android.Services;
using PostIt.Services;
using Yavsc.Abstract.Chat.Calls;
using Path = System.IO.Path;
namespace PostIt.Tests;
[Activity(Name = "fr.pschneider.postit.WebRtcTestActivity", Exported = true)]
[SupportedOSPlatform("android26.0")]
public sealed class AndroidWebRtcTestActivity : Activity
{
protected override async void OnCreate(Bundle? state)
{
base.OnCreate(state);
var result = Path.Combine(FilesDir!.AbsolutePath, "webrtc-test-result.txt");
File.WriteAllText(result, "RUNNING");
try
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(75));
await RunAsync(Intent?.GetIntExtra("port", 0) ?? 0, timeout.Token);
File.WriteAllText(result, "PASS\nnative WebRTC media and cleanup complete");
}
catch (Exception error) { File.WriteAllText(result, "FAIL\n" + error); }
finally { Finish(); }
}
private async Task RunAsync(int port, CancellationToken ct)
{
if (port is < 1024 or > 65535) throw new ArgumentOutOfRangeException(nameof(port));
StartActivity(new Intent(this, typeof(MainActivity)).AddFlags(ActivityFlags.NewTask));
while (!MainActivity.IsForeground || (Avalonia.Application.Current as App)?.View is null)
await Task.Delay(100, ct);
using var socket = new TcpClient();
await socket.ConnectAsync("127.0.0.1", port, ct);
using var stream = socket.GetStream();
using var reader = new StreamReader(stream, Encoding.UTF8, false, 4096, leaveOpen: true);
using var writer = new StreamWriter(stream, new UTF8Encoding(false), 4096, leaveOpen: true) { AutoFlush = true };
using var output = new SemaphoreSlim(1);
var callbackFailure = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
async Task EmitAsync(object value)
{
await output.WaitAsync(ct);
try { await writer.WriteLineAsync(JsonSerializer.Serialize(value, new JsonSerializerOptions(JsonSerializerDefaults.Web))); }
finally { output.Release(); }
}
void Emit(object value) => _ = EmitObservedAsync(value);
async Task EmitObservedAsync(object value)
{
try { await EmitAsync(value); }
catch (Exception error) { callbackFailure.TrySetException(error); }
}
var navigation = ((App)Avalonia.Application.Current!).View!.FindControl<NavigationPage>("NavRoot")
?? throw new InvalidOperationException("Main navigation unavailable.");
AndroidWebRtcMedia? media = null;
var attached = false;
try
{
await Dispatcher.UIThread.InvokeAsync(async () =>
{
media = new AndroidWebRtcMedia();
var grid = new Grid { RowDefinitions = new RowDefinitions("*,*") };
grid.Children.Add(media.LocalVideo);
Grid.SetRow(media.RemoteVideo, 1);
grid.Children.Add(media.RemoteVideo);
await navigation.PushAsync(new ContentPage { Content = grid });
attached = true;
});
var peer = media ?? throw new InvalidOperationException("Native UI unavailable.");
peer.Signal += signal => Emit(new { type = "signal", signal });
peer.StatusChanged += status =>
{
Emit(new { type = "status", status });
if (status.LocalVideoRendered) Emit(new { type = "rendered", target = "local", size = 1_000 });
if (status.RemoteVideoRendered) Emit(new { type = "rendered", target = "remote", size = 1_000 });
};
while (true)
{
var read = reader.ReadLineAsync(ct).AsTask();
var completed = await Task.WhenAny(read, peer.Completion, callbackFailure.Task);
if (completed != read)
{
await completed;
throw new IOException("Native WebRTC stopped unexpectedly.");
}
var line = await read;
if (line is null) throw new IOException("Test signaling disconnected.");
using var document = JsonDocument.Parse(line);
var root = document.RootElement;
switch (root.GetProperty("command").GetString())
{
case "start":
var configuration = root.GetProperty("configuration").Deserialize<WebRtcMediaConfiguration>(
new JsonSerializerOptions(JsonSerializerDefaults.Web))!;
var camera = AndroidLanCallMedia.GetDevices().Cameras.FirstOrDefault(c => c.Name == "Camera avant")
?? throw new InvalidOperationException("Front camera required.");
await Dispatcher.UIThread.InvokeAsync(() => peer.StartAsync(
configuration with { Camera = camera.Value, Microphone = "microphone" }, ct));
break;
case "signal":
var signal = root.GetProperty("signal").Deserialize<CallSignal>(
new JsonSerializerOptions(JsonSerializerDefaults.Web))!;
await peer.HandleSignalAsync(signal, ct);
break;
case "close":
await Dispatcher.UIThread.InvokeAsync(async () => await peer.DisposeAsync());
await EmitAsync(new { type = "closed" });
return;
default: throw new InvalidOperationException("Unknown test command.");
}
}
}
catch (Exception error)
{
await EmitAsync(new { type = "error", message = error.ToString() });
throw;
}
finally
{
if (media is not null) await Dispatcher.UIThread.InvokeAsync(async () => await media.DisposeAsync());
if (attached) await Dispatcher.UIThread.InvokeAsync(async () => await navigation.PopAsync());
}
}
}
#endif

View file

@ -0,0 +1,267 @@
"""Real GStreamer/coturn regression. No camera, account or public service is used."""
import base64
import hashlib
import hmac
import io
import json
import os
from pathlib import Path
import queue
import secrets
import shutil
import socket
import subprocess
import sys
import tempfile
import threading
import time
import unittest
ENGINE = Path(__file__).resolve().parents[2] / "PostIt/Services/WebRtc/gstreamer_peer.py"
class AndroidPeer:
"""Debug Activity over an adb-reversed, loopback-only test command socket."""
def __init__(self):
self.adb = [os.environ.get("POSTIT_ADB", "adb")]
if os.environ.get("POSTIT_ANDROID_SERIAL"):
self.adb += ["-s", os.environ["POSTIT_ANDROID_SERIAL"]]
self.exited = threading.Event()
self.reversed = []
with socket.socket() as listener:
listener.bind(("127.0.0.1", 0))
listener.listen(1)
listener.settimeout(35)
port = listener.getsockname()[1]
self.reverse(port)
try:
subprocess.run(self.adb + ["shell", "am", "start", "-f", "0x18000000", "-n",
"fr.pschneider.postit/.WebRtcTestActivity", "--ei", "port", str(port)],
check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=10)
self.socket, _ = listener.accept()
except BaseException:
self.remove_reverses()
raise
self.input = self.socket.makefile("w", encoding="utf-8")
self.output = self.socket.makefile("r", encoding="utf-8")
self.stdin = self.input
self.stdout = self.read()
self.stderr = io.StringIO()
def reverse(self, port):
subprocess.run(self.adb + ["reverse", f"tcp:{port}", f"tcp:{port}"], check=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=10)
self.reversed.append(port)
def remove_reverses(self):
for port in self.reversed:
subprocess.run(self.adb + ["reverse", "--remove", f"tcp:{port}"], check=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=10)
self.reversed.clear()
def read(self):
try:
yield from self.output
finally:
self.output.close()
self.exited.set()
def poll(self):
return 0 if self.exited.is_set() else None
def wait(self, timeout=None):
if not self.exited.wait(timeout):
raise subprocess.TimeoutExpired("Android test peer", timeout)
self.socket.close()
self.remove_reverses()
return 0
def kill(self):
self.socket.shutdown(socket.SHUT_RDWR)
self.socket.close()
class NativeWebRtcTests(unittest.TestCase):
def run_pair(self, relay, unavailable=False, android=False, android_caller=False):
processes = []
events = queue.Queue()
errors = []
server = None
server_log = None
with tempfile.TemporaryDirectory(prefix="postit-turn-") as directory:
try:
servers = []
if relay:
binary = os.environ.get("POSTIT_TURN_SERVER", shutil.which("turnserver"))
self.assertTrue(binary, "Install coturn or set POSTIT_TURN_SERVER")
with socket.socket() as reservation:
reservation.bind(("127.0.0.1", 0))
port = reservation.getsockname()[1]
secret = secrets.token_hex(32)
username = str(int(time.time()) + 600) + ":native-test"
password = base64.b64encode(hmac.new(secret.encode(), username.encode(), hashlib.sha1).digest()).decode()
server_log = open(Path(directory) / "coturn.log", "w+")
server = subprocess.Popen([
binary, "-n", "--listening-ip=127.0.0.1", "--relay-ip=127.0.0.1",
"--listening-port=" + str(port), "--min-port=52000", "--max-port=52100",
"--realm=postit-native-test", "--use-auth-secret", "--static-auth-secret=" + secret,
"--allow-loopback-peers", "--no-multicast-peers", "--no-tls", "--no-dtls",
"--no-cli", "--relay-threads=1", "--pidfile=" + directory + "/turn.pid",
"--log-file=stdout"
], stdout=server_log, stderr=subprocess.STDOUT)
deadline = time.monotonic() + 10
while True:
try:
with socket.create_connection(("127.0.0.1", port), timeout=0.2):
break
except OSError:
if time.monotonic() > deadline or server.poll() is not None:
server_log.seek(0)
self.fail("coturn failed to start: " + server_log.read()[-2000:])
time.sleep(0.1)
if unavailable:
server.terminate()
server.wait(timeout=5)
servers = [{"urls": [f"turn:127.0.0.1:{port}?transport=tcp"],
"username": username, "credential": password}]
def read(index, process):
for line in process.stdout:
try:
message = json.loads(line)
if message.get("type") == "frame":
events.put((index, {"type": "rendered", "target": message["target"],
"size": len(base64.b64decode(message["data"]))}))
else:
events.put((index, message))
except Exception as error:
errors.append(str(error))
def stderr(process):
for line in process.stderr:
errors.append(line.rstrip())
for index in range(2):
if android and index == 1:
process = AndroidPeer()
if relay:
process.reverse(port)
else:
process = subprocess.Popen([sys.executable, "-X", "faulthandler", "-u", str(ENGINE)],
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, text=True, bufsize=1)
processes.append(process)
threading.Thread(target=read, args=(index, process), daemon=True).start()
threading.Thread(target=stderr, args=(process,), daemon=True).start()
self.send(process, {"command": "start", "configuration": {
"iceServers": servers, "relayOnly": relay,
"caller": index == (1 if android_caller else 0),
"camera": "test", "microphone": "test"}})
stats = {}
rendered = set()
candidates = 0
descriptions = set()
started = time.monotonic()
deadline = started + (12 if unavailable else 35)
while time.monotonic() < deadline:
self.assertTrue(all(p.poll() is None for p in processes),
f"Native peer exited: {[p.poll() for p in processes]}, {errors[-20:]}")
try:
index, message = events.get(timeout=0.5)
except queue.Empty:
self.assertTrue(all(p.poll() is None for p in processes), str(errors[-10:]))
continue
kind = message["type"]
if kind == "error":
self.fail(message["message"] + "\n" + str(errors[-10:]))
if kind == "signal":
signal = message["signal"]
if relay:
candidate = signal.get("candidate")
if candidate:
tokens = candidate.split()
self.assertEqual("relay", tokens[7])
if "raddr" in tokens:
self.assertIn(tokens[tokens.index("raddr") + 1], ("0.0.0.0", "::"))
for line in (signal.get("sdp") or "").splitlines():
if line.startswith("a=candidate:"):
self.assertEqual("relay", line.split()[7])
if line.startswith(("o=", "c=")):
self.assertIn(line.split()[-1], ("0.0.0.0", "::", "127.0.0.1"))
if signal["kind"] == "candidate":
candidates += 1
else:
descriptions.add((index, signal["kind"]))
self.send(processes[1 - index], {"command": "signal", "signal": signal})
if kind == "rendered" and message["size"] > 100:
rendered.add((index, message["target"]))
if kind == "status":
stats[index] = message["status"]
if unavailable:
self.assertNotEqual("connected", message["status"]["connectionState"])
self.assertEqual(0, message["status"]["audioPackets"])
self.assertEqual(0, message["status"]["videoFrames"])
elif time.monotonic() - started >= 10 and len(stats) == 2 and len(rendered) == 4 and all(
s["connectionState"] == "connected" and s["audioPackets"] > 100
and s["videoFrames"] >= 100 for s in stats.values()
):
if relay:
self.assertTrue(all(s["localCandidateType"] == "relay" for s in stats.values()))
self.assertGreater(candidates, 0)
print(f"\nMedia verified: relay={relay}, android={android}, "
f"androidCaller={android_caller}, stats={stats}, rendered={len(rendered)}",
flush=True)
return
if unavailable:
caller = 1 if android_caller else 0
self.assertEqual({(caller, "offer"), (1 - caller, "answer")}, descriptions)
self.assertEqual({0, 1}, set(stats))
self.assertTrue(all((index, "local") in rendered for index in range(2)))
self.assertEqual(0, candidates, "TURN failure must never disclose direct candidates")
self.assertFalse(any(target == "remote" for _, target in rendered))
return
self.fail(f"No bidirectional decoded/rendered media: {stats}, {rendered}, {errors[-12:]}")
finally:
for process in processes:
if process.poll() is None:
self.send(process, {"command": "close"})
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
for stream in (process.stdin, process.stdout, process.stderr):
stream.close()
if server is not None and server.poll() is None:
server.terminate()
server.wait(timeout=5)
if server_log is not None:
server_log.close()
@staticmethod
def send(process, message):
process.stdin.write(json.dumps(message) + "\n")
process.stdin.flush()
def test_direct_bidirectional_media(self):
self.run_pair(False)
def test_relay_bidirectional_media_and_no_address_disclosure(self):
self.run_pair(True)
def test_turn_unavailable_never_falls_back_to_direct(self):
self.run_pair(True, unavailable=True)
@unittest.skipUnless(os.environ.get("POSTIT_TEST_ANDROID_WEBRTC") == "1", "Requires installed Debug APK and adb device")
def test_android_linux_relay_bidirectional_media(self):
for caller in (False, True):
with self.subTest(android_caller=caller):
self.run_pair(True, android=True, android_caller=caller)
@unittest.skipUnless(os.environ.get("POSTIT_TEST_ANDROID_WEBRTC") == "1", "Requires installed Debug APK and adb device")
def test_android_turn_unavailable_never_falls_back_to_direct(self):
self.run_pair(True, unavailable=True, android=True)
if __name__ == "__main__":
unittest.main()

View file

@ -18,6 +18,7 @@
<PackageReference Include="coverlet.collector" />
<PackageReference Include="Avalonia.Headless" />
<PackageReference Include="Avalonia.Headless.XUnit" />
<PackageReference Include="Avalonia.Skia" />
</ItemGroup>
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />

View file

@ -7,10 +7,14 @@ namespace PostIt.Tests;
public class TestAppBuilder
{
public static AppBuilder BuildAvaloniaApp() =>
AppBuilder.Configure<PostIt.App>()
.UseHeadless(new AvaloniaHeadlessPlatformOptions
{
UseHeadlessDrawing = true
});
public static AppBuilder BuildAvaloniaApp()
{
var nativeMedia = System.Environment.GetEnvironmentVariable("POSTIT_TEST_NATIVE_WEBRTC") == "1";
var builder = AppBuilder.Configure<PostIt.App>();
if (nativeMedia) builder.UseSkia();
return builder.UseHeadless(new AvaloniaHeadlessPlatformOptions
{
UseHeadlessDrawing = !nativeMedia
});
}
}

View file

@ -0,0 +1,294 @@
using Avalonia.Controls;
using Avalonia.Headless.XUnit;
using Avalonia.Media.Imaging;
using Avalonia.Threading;
using PostIt.Services;
using PostIt.ViewModels.Chat;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Tests;
public sealed class WebRtcCallViewModelTests
{
private static readonly CallIceServer[] Turn = [new(["turn:localhost:3478"], "temporary", "password")];
private const string Sdp = "v=0\r\no=- 1 1 IN IP4 127.0.0.1\r\nc=IN IP4 0.0.0.0\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\n";
[AvaloniaTheory]
[InlineData(true)]
[InlineData(false)]
public async Task Outgoing_waits_for_acceptance_and_passes_server_policy_to_native_engine(bool relay)
{
var signaling = new Signaling(relay);
var media = new Media();
await using var vm = Configure(signaling, media);
vm.AllowAddressDisclosure = !relay;
await vm.InviteCommand.ExecuteAsync(null);
Assert.True(vm.Busy);
Assert.Null(media.Configuration);
signaling.Ready();
await WaitAsync(() => media.Configuration is not null);
Assert.Equal(relay, media.Configuration!.RelayOnly);
Assert.True(media.Configuration.Caller);
Assert.Same(Turn, media.Configuration.IceServers);
media.Emit(new("offer", Sdp: Sdp));
await WaitAsync(() => signaling.Sent.Count == 1);
signaling.Receive(new("answer", Sdp: Sdp));
await WaitAsync(() => media.Received.Count == 1);
await vm.HangupAsync();
Assert.True(media.Disposed);
Assert.Null(vm.LocalVideo);
Assert.False(vm.Busy);
Assert.Contains(signaling.Call.Id, signaling.EndedCalls);
}
[AvaloniaFact]
public async Task Incoming_relay_needs_acceptance_but_never_requires_IP_disclosure()
{
var signaling = new Signaling(true);
var media = new Media();
await using var vm = Configure(signaling, media);
signaling.Invite();
await WaitAsync(() => vm.Incoming);
Assert.False(vm.AllowAddressDisclosure);
Assert.Null(media.Configuration);
await vm.AcceptCommand.ExecuteAsync(null);
await WaitAsync(() => media.Configuration is not null);
Assert.False(media.Configuration!.Caller);
media.Report(new("connected", "relay", 20, 20));
await WaitAsync(() => vm.Status.Contains("IP masquee"));
}
[AvaloniaFact]
public async Task Direct_call_without_consent_ends_before_any_capture()
{
var signaling = new Signaling(false);
var media = new Media();
await using var vm = Configure(signaling, media);
await vm.InviteCommand.ExecuteAsync(null);
Assert.Null(media.Configuration);
Assert.False(vm.Busy);
Assert.Contains("consentez", vm.Status);
Assert.Single(signaling.EndedCalls);
}
[AvaloniaTheory]
[InlineData(false)]
[InlineData(true)]
public async Task Missing_TURN_or_policy_downgrade_never_creates_a_native_engine(bool downgrade)
{
var signaling = new Signaling(true);
var created = false;
await using var vm = new WebRtcCallViewModel(signaling, () =>
{
created = true;
throw new InvalidOperationException("Should not create media");
})
{ Camera = new("Camera", "camera"), Microphone = new("Mic", "default"), Target = "bob", Room = "General" };
await vm.InviteCommand.ExecuteAsync(null);
signaling.Ready(downgrade ? signaling.Call with { RelayOnly = false } : null,
downgrade ? Turn : []);
await WaitAsync(() => !vm.Busy);
Assert.False(created);
Assert.Single(signaling.EndedCalls);
}
[AvaloniaTheory]
[InlineData("hangup")]
[InlineData("interrupted")]
[InlineData("ended")]
[InlineData("background")]
[InlineData("dispose")]
public async Task All_termination_paths_cancel_in_progress_native_startup(string reason)
{
var signaling = new Signaling(true);
var media = new Media { DelayStartup = true };
await using var vm = Configure(signaling, media);
await vm.InviteCommand.ExecuteAsync(null);
signaling.Ready();
await WaitAsync(() => media.Configuration is not null);
switch (reason)
{
case "hangup": await vm.HangupAsync(); break;
case "interrupted": signaling.Disconnect(); break;
case "ended": signaling.End(); break;
case "background": Platform.SuspendPlayback(); break;
case "dispose": await vm.DisposeAsync(); break;
}
await WaitAsync(() => media.Disposed);
Assert.False(vm.Busy);
Assert.Null(vm.RemoteVideo);
Assert.True(media.StartupCancelled);
}
[AvaloniaTheory]
[InlineData("host")]
[InlineData("srflx")]
[InlineData(null)]
public async Task Nonrelay_selected_path_is_rejected_and_media_is_stopped(string? candidateType)
{
var signaling = new Signaling(true);
var media = new Media();
await using var vm = Configure(signaling, media);
await vm.InviteCommand.ExecuteAsync(null);
signaling.Ready();
await WaitAsync(() => media.Configuration is not null);
media.Report(new("connected", candidateType, 20, 20));
await WaitAsync(() => media.Disposed);
Assert.Contains("chemin ICE", vm.Status);
}
[AvaloniaFact]
public async Task Leaking_candidate_is_not_sent_and_fails_closed()
{
var signaling = new Signaling(true);
var media = new Media();
await using var vm = Configure(signaling, media);
await vm.InviteCommand.ExecuteAsync(null);
signaling.Ready();
await WaitAsync(() => media.Configuration is not null);
media.Emit(new("candidate", Candidate: "candidate:1 1 UDP 1 192.0.2.1 1234 typ host",
SdpMid: "0", SdpMLineIndex: 0));
await WaitAsync(() => media.Disposed);
Assert.Empty(signaling.Sent);
Assert.Contains("candidat", vm.Status);
}
[AvaloniaFact]
public async Task Native_failure_is_visible_and_ends_the_server_call()
{
var signaling = new Signaling(true);
var media = new Media();
await using var vm = Configure(signaling, media);
await vm.InviteCommand.ExecuteAsync(null);
signaling.Ready();
await WaitAsync(() => media.Configuration is not null);
media.Fail();
await WaitAsync(() => media.Disposed);
Assert.Contains("native failure", vm.Status);
Assert.Single(signaling.EndedCalls);
}
[Theory]
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ host")]
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ srflx")]
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ relay raddr 198.51.100.1 rport 1234")]
public void Privacy_rejects_nonrelay_or_related_address_leaks(string candidate) =>
Assert.Throws<InvalidOperationException>(() => WebRtcPrivacy.ValidateSignal(
new("candidate", Candidate: candidate, SdpMid: "0", SdpMLineIndex: 0), true));
[Fact]
public void Privacy_accepts_redacted_relay_and_rejects_SDP_address_leaks()
{
WebRtcPrivacy.ValidateSignal(new("candidate",
Candidate: "candidate:1 1 UDP 1 192.0.2.1 1234 typ relay raddr 0.0.0.0 rport 0",
SdpMid: "0", SdpMLineIndex: 0), true);
WebRtcPrivacy.ValidateSignal(new("offer", Sdp: Sdp), true);
Assert.Throws<InvalidOperationException>(() =>
WebRtcPrivacy.ValidateSignal(new("offer", Sdp: Sdp.Replace("0.0.0.0", "198.51.100.1")), true));
}
public static bool NativeMediaAvailable => OperatingSystem.IsLinux() &&
Environment.GetEnvironmentVariable("POSTIT_TEST_NATIVE_WEBRTC") == "1";
[AvaloniaFact(Skip = "Requires Linux GStreamer/libnice and POSTIT_TEST_NATIVE_WEBRTC=1",
SkipUnless = nameof(NativeMediaAvailable))]
public async Task Linux_native_adapters_render_remote_bitmaps_and_receive_audio_in_both_directions()
{
await using var caller = new LinuxWebRtcMedia();
await using var callee = new LinuxWebRtcMedia();
var errors = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
timeout.CancelAfter(TimeSpan.FromSeconds(30));
void Forward(IWebRtcMedia destination, CallSignal signal) => Dispatcher.UIThread.Post(async () =>
{
try { await destination.HandleSignalAsync(signal, timeout.Token); }
catch (Exception error) { errors.TrySetException(error); }
});
caller.Signal += signal => Forward(callee, signal);
callee.Signal += signal => Forward(caller, signal);
WebRtcMediaStatus? callerStatus = null, calleeStatus = null;
caller.StatusChanged += status => callerStatus = status;
callee.StatusChanged += status => calleeStatus = status;
await callee.StartAsync(new([], false, false, "test", "test"), timeout.Token);
await caller.StartAsync(new([], false, true, "test", "test"), timeout.Token);
while (callerStatus is not { HasRemoteMedia: true } || calleeStatus is not { HasRemoteMedia: true } ||
((Image)caller.RemoteVideo).Source is not Bitmap || ((Image)callee.RemoteVideo).Source is not Bitmap)
{
if (errors.Task.IsCompleted) await errors.Task;
if (caller.Completion.IsCompleted) await caller.Completion;
if (callee.Completion.IsCompleted) await callee.Completion;
await Task.Delay(20, timeout.Token);
}
Assert.Equal(640, ((Bitmap)((Image)caller.RemoteVideo).Source!).PixelSize.Width);
Assert.Equal(360, ((Bitmap)((Image)callee.RemoteVideo).Source!).PixelSize.Height);
}
private static WebRtcCallViewModel Configure(Signaling signaling, Media media) => new(signaling, () => media)
{ Camera = new("Camera", "camera"), Microphone = new("Mic", "default"), Target = "bob", Room = "General" };
private static async Task WaitAsync(Func<bool> condition)
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
timeout.CancelAfter(TimeSpan.FromSeconds(5));
while (!condition()) await Task.Delay(10, timeout.Token);
}
private sealed class Media : IWebRtcMedia
{
private readonly TaskCompletionSource _completion = new(TaskCreationOptions.RunContinuationsAsynchronously);
public Control LocalVideo { get; } = new Border();
public Control RemoteVideo { get; } = new Border();
public event Action<CallSignal>? Signal;
public event Action<WebRtcMediaStatus>? StatusChanged;
public Task Completion => _completion.Task;
public WebRtcMediaConfiguration? Configuration { get; private set; }
public bool Disposed { get; private set; }
public bool StartupCancelled { get; private set; }
public bool DelayStartup { get; init; }
public List<CallSignal> Received { get; } = [];
public async Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
{
Configuration = configuration;
if (DelayStartup)
{
try { await Task.Delay(Timeout.Infinite, ct); }
catch (OperationCanceledException) when (ct.IsCancellationRequested)
{ StartupCancelled = true; throw; }
}
}
public Task HandleSignalAsync(CallSignal signal, CancellationToken ct)
{ Received.Add(signal); return Task.CompletedTask; }
public void Emit(CallSignal signal) => Signal?.Invoke(signal);
public void Report(WebRtcMediaStatus status) => StatusChanged?.Invoke(status);
public void Fail() => _completion.TrySetException(new IOException("native failure"));
public ValueTask DisposeAsync()
{ Disposed = true; _completion.TrySetResult(); return ValueTask.CompletedTask; }
}
private sealed class Signaling(bool relay) : IWebRtcCallSignaling
{
public event Action<CallSession>? Invitation;
public event Action<CallAccepted>? Accepted;
public event Action<CallEnded>? Ended;
public event Action<Guid, CallSignal>? Signal;
public event Action? Interrupted;
public CallSession Call { get; } = new(Guid.NewGuid(), "General", "alice", "bob", relay,
DateTimeOffset.UtcNow.AddMinutes(1));
public List<CallSignal> Sent { get; } = [];
public List<Guid> EndedCalls { get; } = [];
public Task<CallSession> InviteAsync(string room, string user, CancellationToken ct = default) => Task.FromResult(Call);
public Task<CallSession> InviteLanAsync(string room, string user, CancellationToken ct = default) =>
throw new InvalidOperationException("Legacy LAN transport must not be used");
public Task AcceptAsync(Guid callId, CancellationToken ct = default) { Ready(); return Task.CompletedTask; }
public Task EndAsync(Guid callId, CancellationToken ct = default)
{ EndedCalls.Add(callId); return Task.CompletedTask; }
public Task SendAsync(Guid callId, CallSignal signal, CancellationToken ct = default)
{ Sent.Add(signal); return Task.CompletedTask; }
public void Invite() => Invitation?.Invoke(Call);
public void Ready(CallSession? session = null, CallIceServer[]? servers = null) =>
Accepted?.Invoke(new(session ?? Call, servers ?? Turn));
public void Receive(CallSignal signal) => Signal?.Invoke(Call.Id, signal);
public void End() => Ended?.Invoke(new(Call.Id, "ended"));
public void Disconnect() => Interrupted?.Invoke();
}
}

View file

@ -19,6 +19,7 @@
</Content>
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Services/WebRtc/gstreamer_peer.py" LogicalName="PostIt.WebRtc.gstreamer_peer.py" />
<EmbeddedResource Include="postit-settings.json">
<LogicalName>PostIt.postit-settings.json</LogicalName>
</EmbeddedResource>

View file

@ -6,7 +6,7 @@ using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public sealed class CallSignalingClient : ILanCallSignaling, IDisposable
public sealed class CallSignalingClient : IWebRtcCallSignaling, IDisposable
{
private readonly HubConnection _connection;
private readonly IDisposable[] _subscriptions;

View file

@ -0,0 +1,32 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public sealed record WebRtcMediaConfiguration(CallIceServer[] IceServers, bool RelayOnly,
bool Caller, string Camera, string Microphone);
public sealed record WebRtcMediaStatus(string ConnectionState, string? LocalCandidateType,
long AudioPackets, long VideoFrames, bool LocalVideoRendered = false, bool RemoteVideoRendered = false)
{
public bool HasRemoteMedia => ConnectionState == "connected" && AudioPackets > 0 && VideoFrames > 0;
}
public interface IWebRtcMedia : IAsyncDisposable
{
Control LocalVideo { get; }
Control RemoteVideo { get; }
event Action<CallSignal>? Signal;
event Action<WebRtcMediaStatus>? StatusChanged;
Task Completion { get; }
Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct);
Task HandleSignalAsync(CallSignal signal, CancellationToken ct);
}
public interface IWebRtcCallSignaling : ILanCallSignaling
{
Task<CallSession> InviteAsync(string room, string user, CancellationToken ct = default);
}

View file

@ -0,0 +1,208 @@
using System;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Avalonia.Media;
using Avalonia.Media.Imaging;
using Avalonia.Threading;
namespace PostIt.Services;
public sealed class LinuxWebRtcMedia : WebRtcNativeMedia
{
private readonly Image _local = new() { Stretch = Stretch.Uniform };
private readonly Image _remote = new() { Stretch = Stretch.Uniform };
private readonly SemaphoreSlim _input = new(1);
private readonly CancellationTokenSource _lifetime = new();
private readonly StringBuilder _errors = new();
private readonly object _frameLock = new();
private byte[]? _localFrame;
private byte[]? _remoteFrame;
private bool _frameScheduled;
private Bitmap? _localBitmap;
private Bitmap? _remoteBitmap;
private Process? _process;
private Task? _reader;
private Task? _stderr;
private string? _script;
private bool _disposed;
public override Control LocalVideo => _local;
public override Control RemoteVideo => _remote;
public static PodcastCaptureDevices GetDevices() => new(
Directory.GetFiles("/dev", "video*").OrderBy(p => p).Select(p => new CaptureDevice(p, p)).ToArray(),
[new("Microphone système (PulseAudio/PipeWire)", "default")]);
protected override async Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
{
if (!OperatingSystem.IsLinux()) throw new PlatformNotSupportedException("GStreamer requiert Linux.");
_script = Path.Combine(Path.GetTempPath(), $"postit-webrtc-{Guid.NewGuid():N}.py");
await using (var resource = typeof(LinuxWebRtcMedia).Assembly.GetManifestResourceStream("PostIt.WebRtc.gstreamer_peer.py")
?? throw new InvalidOperationException("Adaptateur GStreamer introuvable."))
await using (var file = new FileStream(_script, FileMode.CreateNew, FileAccess.Write, FileShare.None))
await resource.CopyToAsync(file, ct);
_process = new Process
{
StartInfo = new ProcessStartInfo("python3")
{
ArgumentList = { "-u", _script },
UseShellExecute = false,
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true
}
};
if (!_process.Start()) throw new InvalidOperationException("Démarrage du moteur GStreamer impossible.");
_stderr = ReadErrorsAsync(_process);
_reader = ReadAsync(_process);
await SendCommandAsync(StartCommand(configuration), ct);
}
protected override async Task SendCommandAsync(string json, CancellationToken ct)
{
await _input.WaitAsync(ct);
try
{
if (_process is null || _process.HasExited)
throw new InvalidOperationException("Le moteur GStreamer est arrêté. Vérifiez Python GI, GStreamer WebRTC et libnice.");
await _process.StandardInput.WriteLineAsync(json.AsMemory(), ct);
await _process.StandardInput.FlushAsync(ct);
}
finally { _input.Release(); }
}
private async Task ReadErrorsAsync(Process process)
{
while (await process.StandardError.ReadLineAsync(_lifetime.Token) is { } line)
{
lock (_errors)
{
_errors.AppendLine(line);
if (_errors.Length > 4096) _errors.Remove(0, _errors.Length - 4096);
}
}
}
private async Task ReadAsync(Process process)
{
try
{
while (await process.StandardOutput.ReadLineAsync(_lifetime.Token) is { } line)
{
if (line.Length > 1500000) throw new InvalidOperationException("Trame native trop volumineuse.");
ProcessMessage(line);
}
await process.WaitForExitAsync(_lifetime.Token);
if (!_disposed)
{
if (_stderr is not null) await _stderr;
string details;
lock (_errors) details = _errors.ToString();
throw new InvalidOperationException($"Le moteur GStreamer s'est arrêté ({process.ExitCode}). {details}");
}
}
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
catch (Exception error) { Fail(error); }
}
protected override void OnFrame(JsonElement frame)
{
var image = frame.GetProperty("data").GetBytesFromBase64();
var target = frame.GetProperty("target").GetString();
if (target is not ("local" or "remote"))
throw new InvalidOperationException("Destination de trame WebRTC inconnue.");
lock (_frameLock)
{
if (_disposed) return;
if (target == "local") _localFrame = image;
else _remoteFrame = image;
if (_frameScheduled) return;
_frameScheduled = true;
}
Dispatcher.UIThread.Post(DisplayFrames);
}
private void DisplayFrames()
{
byte[]? local;
byte[]? remote;
lock (_frameLock)
{
_frameScheduled = false;
local = _localFrame;
remote = _remoteFrame;
_localFrame = _remoteFrame = null;
}
if (_disposed) return;
try
{
if (local is not null) Display(_local, local, ref _localBitmap);
if (remote is not null) Display(_remote, remote, ref _remoteBitmap);
}
catch (Exception error) { Fail(error); }
}
private static void Display(Image view, byte[] bytes, ref Bitmap? previous)
{
using var stream = new MemoryStream(bytes, writable: false);
var image = new Bitmap(stream);
view.Source = image;
previous?.Dispose();
previous = image;
}
public override async ValueTask DisposeAsync()
{
if (_disposed) return;
_disposed = true;
Exception? stopError = null;
try
{
if (_process is { HasExited: false } process)
{
try { await SendCommandAsync("{\"command\":\"close\"}", CancellationToken.None); }
catch (Exception error) when (error is IOException or InvalidOperationException)
{
stopError = error;
}
try { await process.WaitForExitAsync().WaitAsync(TimeSpan.FromSeconds(5)); }
catch (TimeoutException)
{
process.Kill(entireProcessTree: true);
await process.WaitForExitAsync();
throw new TimeoutException("Le moteur GStreamer n'a pas terminé son arrêt ; processus interrompu.");
}
}
}
finally
{
_lifetime.Cancel();
if (_reader is not null) await _reader;
if (_stderr is not null)
{
try { await _stderr; }
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
}
await Dispatcher.UIThread.InvokeAsync(() =>
{
_local.Source = _remote.Source = null;
_localBitmap?.Dispose();
_remoteBitmap?.Dispose();
});
_process?.Dispose();
if (_script is not null) File.Delete(_script);
_input.Dispose();
_lifetime.Dispose();
Closed();
}
if (stopError is not null)
throw new InvalidOperationException("La commande d'arret GStreamer a echoue.", stopError);
}
}

View file

@ -30,6 +30,8 @@ public static class Platform
public static Func<IServiceProvider>? CreateServiceProvider { get; set; }
public static Func<PodcastCaptureDevices>? GetLanCallDevices { get; set; }
public static Func<string, int, ILanCallMedia>? CreateLanCallMedia { get; set; }
public static Func<IWebRtcMedia>? CreateWebRtcMedia { get; set; }
public static Func<PodcastCaptureDevices>? GetWebRtcDevices { get; set; }
public static Func<PodcastCaptureDevices>? GetPodcastCaptureDevices { get; set; }
public static Func<string?, CancellationToken, Task>? RequestPodcastPermissionsAsync { get; set; }
public static Func<string, string?, string, CancellationToken, Task<IPodcastCapture>>? CreatePodcastCaptureAsync { get; set; }

View file

@ -0,0 +1,358 @@
import base64
import json
import queue
import sys
import threading
from urllib.parse import quote, urlsplit
import gi
gi.require_version("Gst", "1.0")
gi.require_version("GstApp", "1.0")
gi.require_version("GstSdp", "1.0")
gi.require_version("GstWebRTC", "1.0")
from gi.repository import GLib, Gst, GstSdp, GstWebRTC
Gst.init(None)
class Peer:
def __init__(self):
self.loop = GLib.MainLoop()
self.pipeline = None
self.rtc = None
self.outgoing = queue.Queue(64)
self.frames = {}
self.frame_lock = threading.Lock()
self.local_sent = False
self.remote_set = False
self.pending_local = []
self.pending_remote = []
self.mids = {}
self.closed = False
self.audio_packets = 0
self.video_frames = 0
self.writer = threading.Thread(target=self.write, daemon=True)
self.writer.start()
def emit(self, message):
try:
self.outgoing.put_nowait(message)
except queue.Full:
raise RuntimeError("WebRTC event queue overflow")
def write(self):
while True:
message = self.outgoing.get()
print(json.dumps(message, separators=(",", ":")), flush=True)
if message["type"] == "closed":
return
def fail(self, message):
self.emit({"type": "error", "message": str(message)})
if self.pipeline:
self.pipeline.set_state(Gst.State.NULL)
def command(self, message):
try:
command = message["command"]
if command == "start":
self.start(message["configuration"])
elif command == "signal":
self.signal(message["signal"])
elif command == "close":
self.close()
else:
raise ValueError("Unknown WebRTC command")
except Exception as error:
self.fail(error)
return GLib.SOURCE_REMOVE
def start(self, configuration):
if self.pipeline is not None:
raise RuntimeError("WebRTC already started")
for plugin in ("webrtcbin", "nicesrc", "vp8enc", "opusenc", "jpegenc"):
if not Gst.ElementFactory.find(plugin):
raise RuntimeError("Missing GStreamer plugin: " + plugin)
synthetic = configuration["camera"] == "test"
camera = "videotestsrc is-live=true" if synthetic else "v4l2src name=camera"
microphone = "audiotestsrc is-live=true" if synthetic else "pulsesrc name=microphone"
self.pipeline = Gst.parse_launch(
"webrtcbin name=rtc bundle-policy=max-bundle latency=100 "
+ camera + " ! videoconvert ! videoscale ! videorate ! "
"video/x-raw,width=640,height=360,framerate=15/1 ! tee name=local "
"local. ! queue leaky=downstream max-size-buffers=2 ! "
"vp8enc deadline=1 target-bitrate=900000 keyframe-max-dist=30 ! "
"rtpvp8pay pt=96 ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! rtc. "
"local. ! queue leaky=downstream max-size-buffers=1 ! videorate ! "
"video/x-raw,framerate=10/1 ! jpegenc ! appsink name=local_frames "
"emit-signals=true sync=false max-buffers=1 drop=true "
+ microphone + " ! audioconvert ! audioresample ! audio/x-raw,rate=48000,channels=1 ! "
"opusenc ! rtpopuspay pt=111 ! application/x-rtp,media=audio,encoding-name=OPUS,payload=111 ! rtc."
)
if not synthetic:
self.pipeline.get_by_name("camera").set_property("device", configuration["camera"])
if configuration["microphone"] != "default":
self.pipeline.get_by_name("microphone").set_property("device", configuration["microphone"])
self.synthetic = synthetic
self.relay_only = configuration["relayOnly"]
self.rtc = self.pipeline.get_by_name("rtc")
self.ice = self.rtc.get_property("ice-agent")
self.nice = self.ice.get_property("agent")
self.nice.set_property("upnp", False)
if configuration["relayOnly"]:
self.rtc.set_property("ice-transport-policy", GstWebRTC.WebRTCICETransportPolicy.RELAY)
self.nice.set_property("force-relay", True)
for server in configuration["iceServers"]:
for url in server["urls"]:
if url.startswith(("turn:", "turns:")):
parsed = urlsplit(url.replace(":", "://", 1))
credentials = quote(server["username"], safe="") + ":" + quote(server["credential"], safe="")
turn = parsed.scheme + "://" + credentials + "@" + parsed.netloc
if parsed.query:
turn += "?" + parsed.query
if not self.rtc.emit("add-turn-server", turn):
raise RuntimeError("Invalid TURN server configuration")
elif not configuration["relayOnly"] and url.startswith("stun:"):
self.rtc.set_property("stun-server", url.replace(":", "://", 1))
elif not configuration["relayOnly"] and url.startswith("stuns:"):
raise RuntimeError("GStreamer does not support STUN/TLS; configure STUN or TURN/TLS")
self.pipeline.get_by_name("local_frames").connect("new-sample", self.frame, "local")
self.rtc.connect("on-ice-candidate", self.candidate)
self.rtc.connect("pad-added", self.incoming)
bus = self.pipeline.get_bus()
bus.add_signal_watch()
bus.connect("message::error", self.bus_error)
if configuration["caller"]:
self.rtc.connect("on-negotiation-needed", self.offer)
if self.pipeline.set_state(Gst.State.PLAYING) == Gst.StateChangeReturn.FAILURE:
raise RuntimeError("Cannot start GStreamer camera/microphone pipeline")
GLib.timeout_add(100, self.flush_frames)
GLib.timeout_add(1000, self.stats)
def bus_error(self, bus, message):
error, _ = message.parse_error()
self.fail(error.message)
def offer(self, rtc):
GLib.idle_add(self.create_description, "offer")
def promise(self, callback):
def changed(promise, *_):
try:
reply = promise.get_reply()
if reply and reply.has_field("error"):
raise RuntimeError(reply.get_value("error").message)
# The reply is owned by the promise until this callback returns.
GLib.idle_add(callback, reply.copy() if reply is not None else None)
except Exception as error:
GLib.idle_add(self.fail, str(error))
return Gst.Promise.new_with_change_func(changed, None, None)
def create_description(self, kind):
if self.closed:
return GLib.SOURCE_REMOVE
def created(reply):
try:
description = reply.get_value(kind)
sdp_message = description.sdp
sdp = sdp_message.as_text()
mids = {}
index = -1
for line in sdp.splitlines():
if line.startswith("m="):
index += 1
elif line.startswith("a=mid:"):
mids[index] = line[6:]
def applied(_):
try:
self.mids = mids
self.emit({"type": "signal", "signal": {"kind": kind, "sdp": sdp}})
self.local_sent = True
for index, candidate in self.pending_local:
self.send_candidate(index, candidate)
self.pending_local.clear()
except Exception as error:
self.fail(error)
return GLib.SOURCE_REMOVE
self.rtc.emit("set-local-description", description, self.promise(applied))
except Exception as error:
self.fail(error)
return GLib.SOURCE_REMOVE
self.rtc.emit("create-" + kind, None, self.promise(created))
return GLib.SOURCE_REMOVE
def candidate(self, rtc, index, candidate):
GLib.idle_add(self.local_candidate, index, candidate)
def local_candidate(self, index, candidate):
if self.closed or not candidate:
return GLib.SOURCE_REMOVE
if self.local_sent:
self.send_candidate(index, candidate)
elif len(self.pending_local) < 256:
self.pending_local.append((index, candidate))
else:
self.fail("Too many local ICE candidates")
return GLib.SOURCE_REMOVE
def send_candidate(self, index, candidate):
if self.relay_only:
parts = candidate.split()
if len(parts) < 8 or parts[6:8] != ["typ", "relay"]:
self.fail("Relay-only ICE produced a non-relay candidate")
return
# RFC 8828: libnice exposes the base address in relay metadata.
# Hide only that metadata; never rewrite SDP or the relay endpoint.
for position in range(8, len(parts) - 1, 2):
if parts[position] == "raddr":
parts[position + 1] = "::" if ":" in parts[4] else "0.0.0.0"
elif parts[position] == "rport":
parts[position + 1] = "0"
candidate = " ".join(parts)
self.emit({"type": "signal", "signal": {
"kind": "candidate", "candidate": candidate,
"sdpMid": self.mids[index], "sdpMLineIndex": index}})
def signal(self, signal):
if signal["kind"] == "candidate":
if self.remote_set:
self.rtc.emit("add-ice-candidate", signal["sdpMLineIndex"], signal["candidate"])
elif len(self.pending_remote) < 256:
self.pending_remote.append(signal)
else:
raise RuntimeError("Too many pending remote ICE candidates")
else:
result, sdp = GstSdp.SDPMessage.new_from_text(signal["sdp"])
if result != GstSdp.SDPResult.OK:
raise ValueError("Invalid remote SDP")
kind = GstWebRTC.WebRTCSDPType.OFFER if signal["kind"] == "offer" else GstWebRTC.WebRTCSDPType.ANSWER
description = GstWebRTC.WebRTCSessionDescription.new(kind, sdp)
def applied(_):
self.remote_set = True
for candidate in self.pending_remote:
self.rtc.emit("add-ice-candidate", candidate["sdpMLineIndex"], candidate["candidate"])
self.pending_remote.clear()
if signal["kind"] == "offer":
self.create_description("answer")
return GLib.SOURCE_REMOVE
self.rtc.emit("set-remote-description", description, self.promise(applied))
def incoming(self, rtc, pad):
if pad.get_direction() != Gst.PadDirection.SRC:
return
decoder = Gst.ElementFactory.make("decodebin")
decoder.connect("pad-added", self.decoded)
self.pipeline.add(decoder)
decoder.sync_state_with_parent()
if pad.link(decoder.get_static_pad("sink")) != Gst.PadLinkReturn.OK:
GLib.idle_add(self.fail, "Cannot attach WebRTC decoder")
def decoded(self, decoder, pad):
media = pad.get_current_caps().get_structure(0).get_name()
if media.startswith("video/"):
sink = Gst.parse_bin_from_description(
"queue leaky=downstream max-size-buffers=2 ! videoconvert ! videoscale ! videorate ! "
"video/x-raw,width=640,height=360,framerate=10/1 ! jpegenc ! "
"appsink name=remote_frames emit-signals=true sync=false max-buffers=1 drop=true", True)
sink.get_by_name("remote_frames").connect("new-sample", self.frame, "remote")
elif media.startswith("audio/"):
sink = Gst.parse_bin_from_description(
"queue ! audioconvert ! audioresample ! "
+ ("fakesink" if self.synthetic else "autoaudiosink") + " name=remote_audio sync=false", True)
sink.get_by_name("remote_audio").get_static_pad("sink").add_probe(
Gst.PadProbeType.BUFFER, self.audio_buffer)
else:
GLib.idle_add(self.fail, "Unsupported remote media")
return
self.pipeline.add(sink)
sink.sync_state_with_parent()
if pad.link(sink.get_static_pad("sink")) != Gst.PadLinkReturn.OK:
GLib.idle_add(self.fail, "Cannot attach decoded media")
def audio_buffer(self, pad, probe):
self.audio_packets += 1
return Gst.PadProbeReturn.OK
def frame(self, sink, target):
sample = sink.emit("pull-sample")
if sample is None:
return Gst.FlowReturn.EOS
buffer = sample.get_buffer()
image = buffer.extract_dup(0, buffer.get_size())
if len(image) > 1024 * 1024:
GLib.idle_add(self.fail, "Decoded image exceeds frame limit")
return Gst.FlowReturn.ERROR
with self.frame_lock:
self.frames[target] = base64.b64encode(image).decode("ascii")
if target == "remote":
self.video_frames += 1
return Gst.FlowReturn.OK
def flush_frames(self):
if self.closed:
return GLib.SOURCE_REMOVE
with self.frame_lock:
frames, self.frames = self.frames, {}
for target, data in frames.items():
# Preview/render frames are disposable, unlike signaling.
if self.outgoing.qsize() < 4:
self.emit({"type": "frame", "target": target, "data": data})
return GLib.SOURCE_CONTINUE
def stats(self):
if self.closed:
return GLib.SOURCE_REMOVE
def received(reply):
if self.closed:
return GLib.SOURCE_REMOVE
try:
all_stats = {reply.nth_field_name(i): reply.get_value(reply.nth_field_name(i))
for i in range(reply.n_fields())}
selected_id = None
for item in all_stats.values():
if isinstance(item, Gst.Structure) and item.has_field("selected-candidate-pair-id"):
selected_id = item.get_value("selected-candidate-pair-id")
pair = all_stats.get(selected_id)
local = all_stats.get(pair.get_value("local-candidate-id")) if pair else None
local_type = local.get_value("candidate-type") if local else None
state = self.rtc.get_property("connection-state").value_nick
if state == "connected" and local_type is None:
state = "checking"
self.emit({"type": "status", "status": {
"connectionState": state, "localCandidateType": local_type,
"audioPackets": self.audio_packets, "videoFrames": self.video_frames}})
except Exception as error:
self.fail(error)
return GLib.SOURCE_REMOVE
self.rtc.emit("get-stats", None, self.promise(received))
return GLib.SOURCE_CONTINUE
def close(self):
self.closed = True
if self.pipeline:
self.pipeline.set_state(Gst.State.NULL)
self.emit({"type": "closed"})
self.loop.quit()
def main():
peer = Peer()
def read():
try:
for line in sys.stdin:
if len(line) > 65536:
raise ValueError("WebRTC command exceeds size limit")
GLib.idle_add(peer.command, json.loads(line))
except Exception as error:
GLib.idle_add(peer.fail, str(error))
GLib.idle_add(peer.close)
threading.Thread(target=read, daemon=True).start()
peer.loop.run()
peer.writer.join(timeout=3)
if __name__ == "__main__":
main()

View file

@ -0,0 +1,78 @@
using System;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public abstract class WebRtcNativeMedia : IWebRtcMedia
{
protected static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
private readonly TaskCompletionSource _completion = new(TaskCreationOptions.RunContinuationsAsynchronously);
protected WebRtcMediaConfiguration? Configuration { get; private set; }
public abstract Control LocalVideo { get; }
public abstract Control RemoteVideo { get; }
public event Action<CallSignal>? Signal;
public event Action<WebRtcMediaStatus>? StatusChanged;
public Task Completion => _completion.Task;
public async Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
{
if (Configuration is not null) throw new InvalidOperationException("Le moteur WebRTC est déjà démarré.");
WebRtcPrivacy.ValidateConfiguration(configuration);
Configuration = configuration;
await StartCoreAsync(configuration, ct);
}
protected abstract Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct);
protected abstract Task SendCommandAsync(string json, CancellationToken ct);
public abstract ValueTask DisposeAsync();
public Task HandleSignalAsync(CallSignal signal, CancellationToken ct)
{
// Privacy constrains our own addresses, not a peer's public TURN address.
WebRtcPrivacy.ValidateSignal(signal, false);
return SendCommandAsync(JsonSerializer.Serialize(new { command = "signal", signal }, JsonOptions), ct);
}
protected void ProcessMessage(string json)
{
using var document = JsonDocument.Parse(json);
var root = document.RootElement;
switch (root.GetProperty("type").GetString())
{
case "signal":
var signal = root.GetProperty("signal").Deserialize<CallSignal>(JsonOptions)
?? throw new InvalidOperationException("Signal natif vide.");
WebRtcPrivacy.ValidateSignal(signal, Configuration!.RelayOnly);
Signal?.Invoke(signal);
break;
case "status":
var status = root.GetProperty("status").Deserialize<WebRtcMediaStatus>(JsonOptions)
?? throw new InvalidOperationException("Statistiques WebRTC vides.");
WebRtcPrivacy.ValidateStatus(status, Configuration!.RelayOnly);
StatusChanged?.Invoke(status);
break;
case "error":
throw new InvalidOperationException(root.GetProperty("message").GetString());
case "closed":
_completion.TrySetResult();
break;
case "frame":
OnFrame(root);
break;
default:
throw new InvalidOperationException("Événement natif WebRTC inconnu.");
}
}
protected virtual void OnFrame(JsonElement frame) =>
throw new InvalidOperationException("Ce moteur ne transmet pas de trames décodées.");
protected void Fail(Exception error) => _completion.TrySetException(error);
protected void Closed() => _completion.TrySetResult();
protected static string StartCommand(WebRtcMediaConfiguration configuration) =>
JsonSerializer.Serialize(new { command = "start", configuration }, JsonOptions);
}

View file

@ -0,0 +1,73 @@
using System;
using System.Linq;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public static class WebRtcPrivacy
{
public static void ValidateConfiguration(WebRtcMediaConfiguration configuration)
{
if (configuration.RelayOnly && !configuration.IceServers.Any(server =>
server.Urls.Any(IsTurn) && !string.IsNullOrWhiteSpace(server.Username)
&& !string.IsNullOrWhiteSpace(server.Credential)))
throw new InvalidOperationException("Masquage IP : aucun relais TURN authentifié disponible. Aucun repli direct.");
}
public static bool IsTurn(string url) =>
url.StartsWith("turn:", StringComparison.Ordinal) || url.StartsWith("turns:", StringComparison.Ordinal);
public static void ValidateSignal(CallSignal signal, bool relayOnly)
{
if (signal.Endpoint is not null || signal.Kind is not ("offer" or "answer" or "candidate"))
throw new InvalidOperationException("Signal WebRTC invalide.");
if (signal.Kind == "candidate")
{
if (signal.Sdp is not null || signal.Candidate is not { Length: > 0 and <= 2048 } ||
!signal.Candidate.StartsWith("candidate:", StringComparison.Ordinal) ||
string.IsNullOrWhiteSpace(signal.SdpMid) || signal.SdpMid.Length > 256 ||
signal.SdpMLineIndex is not (>= 0 and <= 16))
throw new InvalidOperationException("Candidat ICE invalide.");
}
else if (signal.Sdp is not { Length: > 0 and <= 24576 } ||
!signal.Sdp.StartsWith("v=0\r\n", StringComparison.Ordinal) ||
signal.Candidate is not null)
throw new InvalidOperationException("Description SDP invalide.");
if (!relayOnly) return;
if (signal.Candidate is { } candidate) ValidateCandidate(candidate);
if (signal.Sdp is not { } sdp) return;
var lines = sdp.Split("\r\n", StringSplitOptions.RemoveEmptyEntries);
var candidates = lines.Where(l => l.StartsWith("a=candidate:", StringComparison.Ordinal)).ToArray();
foreach (var line in candidates) ValidateCandidate(line[2..]);
var relayAddresses = candidates.Select(l => l.Split(' ', StringSplitOptions.RemoveEmptyEntries)[4]).ToHashSet();
foreach (var line in lines)
{
if (line.StartsWith("o=", StringComparison.Ordinal) ||
line.StartsWith("c=", StringComparison.Ordinal) ||
(line.StartsWith("a=rtcp:", StringComparison.Ordinal) && line.Contains(" IN ", StringComparison.Ordinal)))
{
var address = line.Split(' ', StringSplitOptions.RemoveEmptyEntries).Last();
if (address is not ("0.0.0.0" or "::") &&
!(line.StartsWith("o=", StringComparison.Ordinal) && address == "127.0.0.1") &&
!relayAddresses.Contains(address))
throw new InvalidOperationException("Masquage IP : le SDP expose une adresse non relayée.");
}
}
}
private static void ValidateCandidate(string candidate)
{
var parts = candidate.Split(' ', StringSplitOptions.RemoveEmptyEntries);
if (parts.Length < 8 || parts[6] != "typ" || parts[7] != "relay")
throw new InvalidOperationException("Masquage IP : un candidat non relayé a été produit.");
for (var i = 8; i < parts.Length; i++)
if (parts[i] == "raddr" && (i + 1 >= parts.Length || parts[i + 1] is not ("0.0.0.0" or "::")))
throw new InvalidOperationException("Masquage IP : un candidat révèle son adresse associée.");
}
public static void ValidateStatus(WebRtcMediaStatus status, bool relayOnly)
{
if (relayOnly && status.ConnectionState == "connected" && status.LocalCandidateType != "relay")
throw new InvalidOperationException("Masquage IP : le chemin ICE sélectionné n'est pas confirmé comme relayé.");
}
}

View file

@ -56,11 +56,11 @@ namespace PostIt.ViewModels.Chat
public CallSignalingClient? Calls { get; private set; }
public bool SupportsLanCalls => Platform.CreateLanCallMedia != null ||
public bool SupportsVideoCalls => Platform.CreateWebRtcMedia != null ||
(OperatingSystem.IsLinux() && !OperatingSystem.IsAndroid());
[ObservableProperty]
public partial LanCallViewModel? LanCall { get; set; }
public partial WebRtcCallViewModel? VideoCall { get; set; }
[ObservableProperty]
public partial string CurrentRoom { get; set; } = "General";
@ -106,7 +106,7 @@ namespace PostIt.ViewModels.Chat
partial void OnSelectedRoomChanged(string? value)
{
if (LanCall != null) LanCall.Room = value ?? "";
if (VideoCall != null) VideoCall.Room = value ?? "";
FocusedMessages = value != null && _messagesByRoom.TryGetValue(value, out var msgs)
? msgs : _systemMessages;
FocusedUsers = value != null && _usersByRoom.TryGetValue(value, out var users)
@ -160,7 +160,7 @@ namespace PostIt.ViewModels.Chat
if (_transport == null || room == null) return;
try
{
if (LanCall != null) await LanCall.HangupAsync();
if (VideoCall != null) await VideoCall.HangupAsync();
await _transport.LeaveAsync(room);
JoinedRooms.Remove(room);
_messagesByRoom.Remove(room);
@ -210,11 +210,11 @@ namespace PostIt.ViewModels.Chat
private async Task DisconnectCoreAsync()
{
Exception? callError = null;
if (LanCall != null)
if (VideoCall != null)
{
try { await LanCall.DisposeAsync(); }
try { await VideoCall.DisposeAsync(); }
catch (Exception error) { callError = error; }
finally { LanCall = null; }
finally { VideoCall = null; }
}
Calls?.Dispose();
Calls = null;
@ -269,10 +269,10 @@ namespace PostIt.ViewModels.Chat
return;
if (_connection != null)
{
if (LanCall != null)
if (VideoCall != null)
{
await LanCall.DisposeAsync();
LanCall = null;
await VideoCall.DisposeAsync();
VideoCall = null;
}
Calls?.Dispose();
Calls = null;
@ -296,8 +296,8 @@ namespace PostIt.ViewModels.Chat
// Le hub est [Authorize] : on attache le Bearer OIDC.
Calls = new CallSignalingClient(_connection);
if (SupportsLanCalls)
LanCall = new LanCallViewModel(Calls, settings: _settings) { Room = SelectedRoom ?? "" };
if (SupportsVideoCalls)
VideoCall = new WebRtcCallViewModel(Calls, settings: _settings) { Room = SelectedRoom ?? "" };
else
Calls.Invitation += call => PostForCurrentSession(() =>
Log($"Invitation audio-vidéo de {call.Caller} dans {call.RoomName} ; transport non disponible sur cette plateforme."));

View file

@ -0,0 +1,364 @@
using System;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Avalonia.Threading;
using CommunityToolkit.Mvvm.ComponentModel;
using CommunityToolkit.Mvvm.Input;
using PostIt.Services;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.ViewModels.Chat;
public partial class WebRtcCallViewModel : ObservableObject, IAsyncDisposable
{
private readonly IWebRtcCallSignaling _signaling;
private readonly Func<IWebRtcMedia> _createMedia;
private readonly Settings? _settings;
private readonly SemaphoreSlim _events = new(1);
private readonly CancellationTokenSource _lifetime = new();
private CancellationTokenSource? _callLifetime;
private CallSession? _call;
private IWebRtcMedia? _media;
private WebRtcMediaStatus? _mediaStatus;
private Action<CallSignal>? _sendSignal;
private Action<WebRtcMediaStatus>? _receiveStatus;
private bool _caller;
private bool _accepted;
private bool _disposed;
private string? _camera;
private string? _microphone;
public ObservableCollection<CaptureDevice> Cameras { get; } = [];
public ObservableCollection<CaptureDevice> Microphones { get; } = [];
[ObservableProperty] public partial CaptureDevice? Camera { get; set; }
[ObservableProperty] public partial CaptureDevice? Microphone { get; set; }
[ObservableProperty] public partial bool AllowAddressDisclosure { get; set; }
[ObservableProperty] public partial string Target { get; set; } = "";
[ObservableProperty] public partial string Room { get; set; } = "";
[ObservableProperty] public partial string Status { get; set; } = "Appel WebRTC : aucun appel.";
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
public partial bool Busy { get; set; }
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
public partial bool Incoming { get; set; }
[ObservableProperty] public partial bool Preparing { get; set; }
[ObservableProperty] public partial Control? LocalVideo { get; set; }
[ObservableProperty] public partial Control? RemoteVideo { get; set; }
public bool CanConfigure => !Busy || Incoming;
public WebRtcCallViewModel(IWebRtcCallSignaling signaling, Func<IWebRtcMedia>? createMedia = null,
Settings? settings = null)
{
_signaling = signaling;
_settings = settings;
_createMedia = createMedia ?? Platform.CreateWebRtcMedia ?? (() => new LinuxWebRtcMedia());
signaling.Invitation += OnInvitation;
signaling.Accepted += OnAccepted;
signaling.Signal += OnSignal;
signaling.Ended += OnEnded;
signaling.Interrupted += OnInterrupted;
Platform.PlaybackSuspending += OnSuspending;
if (settings is not null)
{
foreach (var device in settings.PodcastCameras) Cameras.Add(device);
foreach (var device in settings.PodcastMicrophones.Where(d => !string.IsNullOrEmpty(d.Value)))
Microphones.Add(device);
RestoreSelection();
}
}
private void RestoreSelection()
{
Camera = Cameras.FirstOrDefault(d => d.Value == _settings?.PodcastCameraId) ?? Cameras.FirstOrDefault();
Microphone = Microphones.FirstOrDefault(d => d.Value == _settings?.PodcastMicrophoneId)
?? Microphones.FirstOrDefault();
}
[RelayCommand]
private void RefreshDevices()
{
try
{
if (!CanConfigure) throw new InvalidOperationException("Terminez l'appel avant de changer les peripheriques.");
var devices = (Platform.GetWebRtcDevices ?? LinuxWebRtcMedia.GetDevices)();
Cameras.Clear();
Microphones.Clear();
foreach (var device in devices.Cameras) Cameras.Add(device);
foreach (var device in devices.Microphones.Where(d => !string.IsNullOrEmpty(d.Value))) Microphones.Add(device);
RestoreSelection();
Status = Cameras.Count == 0 ? "Aucune camera detectee."
: "Peripheriques actualises. Le profil de masquage IP sera applique par le serveur.";
}
catch (Exception error) { Status = "Peripheriques indisponibles : " + error.Message; }
}
private void SelectDevices()
{
if (Camera is null || Microphone is null)
throw new InvalidOperationException("Une camera et un microphone sont requis.");
_camera = Camera.Value;
_microphone = Microphone.Value;
if (_settings is not null)
{
_settings.PodcastCameraId = _camera;
_settings.PodcastMicrophoneId = _microphone;
try { _settings.Save(); }
catch (Exception error) when (error is System.IO.IOException or UnauthorizedAccessException)
{
throw new InvalidOperationException("Sauvegarde des peripheriques impossible.", error);
}
}
}
private void CheckPolicy(CallSession session)
{
if (session.Transport != CallTransports.WebRtc)
throw new InvalidOperationException("Le transport demande n'est pas WebRTC.");
if (!session.RelayOnly && !AllowAddressDisclosure)
throw new InvalidOperationException(
"Activez le masquage IP dans votre profil ou consentez au partage IP pour cet appel.");
}
[RelayCommand]
private Task InviteAsync() => RunAsync(async () =>
{
if (Busy) throw new InvalidOperationException("Un appel est deja en cours.");
SelectDevices();
if (string.IsNullOrWhiteSpace(Target) || string.IsNullOrWhiteSpace(Room))
throw new InvalidOperationException("Selectionnez une salle jointe et un autre utilisateur.");
_callLifetime = CancellationTokenSource.CreateLinkedTokenSource(_lifetime.Token);
_caller = true;
Busy = true;
_call = await _signaling.InviteAsync(Room, Target.Trim(), _callLifetime.Token);
CheckPolicy(_call);
Status = $"Invitation envoyee a {_call.Callee} ; attente du consentement.";
});
[RelayCommand]
private Task AcceptAsync() => RunAsync(async () =>
{
if (_call is null || !Incoming) throw new InvalidOperationException("Aucune invitation a accepter.");
CheckPolicy(_call);
SelectDevices();
Incoming = false;
_accepted = true;
await _signaling.AcceptAsync(_call.Id, _callLifetime!.Token);
Status = "Invitation acceptee ; preparation des medias.";
});
[RelayCommand]
public Task HangupAsync()
{
_callLifetime?.Cancel();
return RunAsync(() => StopAsync("Appel termine.", true));
}
private void OnInvitation(CallSession call) => Queue(async () =>
{
if (call.Transport != CallTransports.WebRtc || _call is not null)
{
await _signaling.EndAsync(call.Id, _lifetime.Token);
Status = "Invitation refusee : transport incompatible ou appel deja en cours.";
return;
}
_call = call;
_caller = false;
_callLifetime = CancellationTokenSource.CreateLinkedTokenSource(_lifetime.Token);
Busy = Incoming = true;
Status = $"{call.Caller} propose un appel dans {call.RoomName}. "
+ (call.RelayOnly ? "Masquage IP : relais TURN obligatoire. " : "Mode direct : consentement au partage IP requis. ")
+ "Acceptez ou refusez.";
});
private void OnAccepted(CallAccepted accepted) => Queue(async () =>
{
if (_call?.Id != accepted.Session.Id) return;
if (_media is not null || (!_caller && !_accepted) ||
(_call.RelayOnly && !accepted.Session.RelayOnly))
throw new InvalidOperationException("Acceptation ou politique WebRTC inattendue.");
CheckPolicy(accepted.Session);
_call = accepted.Session;
Incoming = false;
var configuration = new WebRtcMediaConfiguration(accepted.IceServers, _call.RelayOnly,
_caller, _camera!, _microphone!);
WebRtcPrivacy.ValidateConfiguration(configuration);
_media = _createMedia();
var id = _call.Id;
var media = _media;
_sendSignal = signal => Queue(async () =>
{
if (_call?.Id != id || !ReferenceEquals(_media, media)) return;
WebRtcPrivacy.ValidateSignal(signal, _call.RelayOnly);
await _signaling.SendAsync(id, signal, _callLifetime!.Token);
});
_receiveStatus = status => Queue(() =>
{
if (_call?.Id == id && ReferenceEquals(_media, media))
{
WebRtcPrivacy.ValidateStatus(status, _call.RelayOnly);
_mediaStatus = status;
}
return Task.CompletedTask;
});
media.Signal += _sendSignal;
media.StatusChanged += _receiveStatus;
LocalVideo = media.LocalVideo;
RemoteVideo = media.RemoteVideo;
Status = _call.RelayOnly ? "Connexion WebRTC via TURN uniquement ; aucun repli direct."
: "Connexion WebRTC ICE/STUN/TURN.";
await media.StartAsync(configuration, _callLifetime!.Token);
_ = MonitorAsync(id, media, _callLifetime.Token);
});
private void OnSignal(Guid id, CallSignal signal) => Queue(async () =>
{
if (_call?.Id != id) return;
if (_media is null) throw new InvalidOperationException("Signal recu avant le consentement.");
WebRtcPrivacy.ValidateSignal(signal, false);
await _media.HandleSignalAsync(signal, _callLifetime!.Token);
});
private void OnEnded(CallEnded ended) => QueueStop(ended.CallId,
() => StopAsync("Appel termine : " + ended.Reason, false));
private void OnInterrupted() => QueueStop(null,
() => StopAsync("Connexion SignalR interrompue ; medias arretes.", false));
private void OnSuspending() => QueueStop(null,
() => StopAsync("Application en arriere-plan ; appel termine.", true));
private void QueueStop(Guid? id, Func<Task> action) => Dispatcher.UIThread.Post(async () =>
{
if (id is not null && _call?.Id != id) return;
_callLifetime?.Cancel();
await RunAsync(() => id is null || _call?.Id == id ? action() : Task.CompletedTask);
});
private async Task MonitorAsync(Guid id, IWebRtcMedia media, CancellationToken ct)
{
var progress = Stopwatch.StartNew();
var received = false;
long audio = 0, video = 0;
try
{
while (true)
{
ct.ThrowIfCancellationRequested();
if (media.Completion.IsCompleted)
{
await media.Completion;
throw new InvalidOperationException("Le moteur WebRTC s'est arrete.");
}
await Dispatcher.UIThread.InvokeAsync(() =>
{
if (_call?.Id != id || _mediaStatus is not { } status) return;
if (status.ConnectionState is "failed" or "closed" or "disconnected")
throw new InvalidOperationException("Connexion ICE interrompue ; aucun repli automatique.");
if (!status.HasRemoteMedia) return;
if (status.AudioPackets > audio && status.VideoFrames > video)
{
audio = status.AudioPackets;
video = status.VideoFrames;
progress.Restart();
}
received = true;
Status = _call.RelayOnly ? "Audio-video WebRTC etabli via TURN (IP masquee)."
: "Audio-video WebRTC etabli.";
});
if (progress.Elapsed > TimeSpan.FromSeconds(30))
throw new TimeoutException(received ? "Reception audio-video interrompue."
: "Aucun media distant. Verifiez ICE, TURN et les pare-feu ; aucun repli hors politique.");
await Task.Delay(500, ct);
}
}
catch (OperationCanceledException) when (ct.IsCancellationRequested) { }
catch (Exception error)
{
Queue(() => _call?.Id == id && ReferenceEquals(_media, media)
? FailAsync(error) : Task.CompletedTask);
}
}
private void Queue(Func<Task> action) => Dispatcher.UIThread.Post(async () => await RunAsync(action));
private async Task RunAsync(Func<Task> action)
{
await _events.WaitAsync();
try
{
if (_disposed) return;
Preparing = true;
await action();
}
catch (OperationCanceledException) when (
_lifetime.IsCancellationRequested || _callLifetime?.IsCancellationRequested == true) { }
catch (Exception error) { await FailAsync(error); }
finally { Preparing = false; _events.Release(); }
}
private async Task FailAsync(Exception error)
{
try { await StopAsync("Echec de l'appel : " + error.Message, true); }
catch (Exception cleanup)
{
Status = $"Echec de l'appel : {error.Message}. Nettoyage/signalisation : {cleanup.Message}";
}
}
private async Task StopAsync(string status, bool notify)
{
var call = _call;
_call = null;
Incoming = Busy = _accepted = false;
_callLifetime?.Cancel();
var media = _media;
_media = null;
_mediaStatus = null;
try
{
if (media is not null)
{
media.Signal -= _sendSignal;
media.StatusChanged -= _receiveStatus;
await media.DisposeAsync();
}
}
finally
{
LocalVideo = RemoteVideo = null;
_sendSignal = null;
_receiveStatus = null;
_callLifetime?.Dispose();
_callLifetime = null;
Status = status;
if (notify && call is not null)
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(5));
await _signaling.EndAsync(call.Id, timeout.Token);
}
}
}
public async ValueTask DisposeAsync()
{
if (_disposed) return;
_lifetime.Cancel();
_callLifetime?.Cancel();
await _events.WaitAsync();
try
{
if (_disposed) return;
_disposed = true;
_signaling.Invitation -= OnInvitation;
_signaling.Accepted -= OnAccepted;
_signaling.Signal -= OnSignal;
_signaling.Ended -= OnEnded;
_signaling.Interrupted -= OnInterrupted;
Platform.PlaybackSuspending -= OnSuspending;
try { await StopAsync("Appel arrete.", true); }
finally { _lifetime.Dispose(); }
}
finally { _events.Release(); }
}
}

View file

@ -104,17 +104,14 @@
Content="Envoyer" />
</Grid>
<Expander Grid.Row="3" Header="Appel direct IPv4/IPv6 (LAN/VPN ou IP publique)" IsVisible="{Binding SupportsLanCalls}" IsExpanded="True"
<Expander Grid.Row="3" Header="Appel video WebRTC" IsVisible="{Binding SupportsVideoCalls}" IsExpanded="True"
IsEnabled="{Binding IsConnected}" Margin="0,0,0,8">
<ScrollViewer MaxHeight="520">
<StackPanel DataContext="{Binding LanCall}" x:DataType="vm:LanCallViewModel" Spacing="8">
<TextBlock Text="Android et Desktop Linux : IPv4 ou IPv6 globale/privee directement assignee a la machine (hors IPv6 link-local). Les pairs doivent pouvoir joindre leurs adresses. SRTP chiffre les medias ; le pair connait votre IP. Ouvrir le port UDP choisi et le suivant. Sans TURN ni traversee NAT. Casque recommande (pas d'annulation d'echo)."
<StackPanel DataContext="{Binding VideoCall}" x:DataType="vm:WebRtcCallViewModel" Spacing="8">
<TextBlock Text="Android et Linux : medias chiffres par WebRTC (DTLS-SRTP), ICE/STUN/TURN. Activez le masquage IP dans votre profil pour imposer un relais TURN aux deux participants, sans repli direct. La camera et le microphone ne demarrent qu'apres acceptation."
TextWrapping="Wrap" />
<WrapPanel IsEnabled="{Binding CanConfigure}" IsVisible="{Binding !RemoteVideo}">
<Button Content="Actualiser les peripheriques" Command="{Binding RefreshDevicesCommand}" Margin="0,0,8,4" />
<ComboBox Width="160" ItemsSource="{Binding Addresses}" SelectedItem="{Binding Address}"
PlaceholderText="Adresse IPv4/IPv6 de cette machine" Margin="0,0,8,4" />
<TextBox Width="80" Text="{Binding Port}" PlaceholderText="Port pair" Margin="0,0,8,4" />
<ComboBox Width="190" ItemsSource="{Binding Cameras}" SelectedItem="{Binding Camera}"
PlaceholderText="Camera" Margin="0,0,8,4">
<ComboBox.ItemTemplate>
@ -133,7 +130,7 @@
</ComboBox>
<TextBox Width="170" Text="{Binding Target}" PlaceholderText="Pseudo du pair" Margin="0,0,8,4" />
</WrapPanel>
<CheckBox Content="Je consens a partager mon adresse IP avec le pair."
<CheckBox Content="Si le masquage IP est desactive, je consens a partager mon adresse IP avec le pair."
IsChecked="{Binding AllowAddressDisclosure}" IsEnabled="{Binding CanConfigure}"
IsVisible="{Binding !RemoteVideo}" />
<WrapPanel>

View file

@ -13,7 +13,7 @@ public partial class ChatPage : ContentPage
protected override async void OnDetachedFromVisualTree(VisualTreeAttachmentEventArgs e)
{
base.OnDetachedFromVisualTree(e);
if (DataContext is ChatViewModel { LanCall: { } call })
if (DataContext is ChatViewModel { VideoCall: { } call })
await call.HangupAsync();
}
}