feat(postit): add native WebRTC calls with TURN IP masking
Replace the direct chat call panel with native Android and Linux WebRTC engines, enforce relay-only privacy, and cover consent, lifecycle, rendering and coturn interoperability. Document deployment and upgrade Avalonia to 12.1.6. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
384e330a78
commit
d812266ee0
28 changed files with 2568 additions and 61 deletions
|
|
@ -5,6 +5,10 @@
|
|||
|
||||
### Added
|
||||
|
||||
* [PostIt/Calls] Replace the chat's direct VLC/SRTP call panel with native WebRTC on Android and Linux: official libwebrtc/Camera2 and GStreamer webrtcbin, existing consent/SignalR signaling, ICE/STUN/coturn, and fail-closed relay-only IP masking. Cancel pending startup and release media on hangup, navigation, interruption and backgrounding. Add privacy/lifecycle, real Linux media/rendering and debug Android interoperability regressions, coturn deployment instructions and a protected-server-secret configuration example.
|
||||
|
||||
* [PostIt] Upgrade Avalonia to 12.1.6 and align the headless and native Skia test rendering packages.
|
||||
|
||||
* [PostIt/PayPal] Add a native subscription page reachable from Home, profile and About: load server-priced Discovery/Premium offers, prepare or resume checkout with the current bearer session, open only PayPal for approval, reconcile confirmed payments and balances, and cancel with confirmation. Add quota-exempt bearer-only endpoints on Yavsc.Org, sharing checkout/cancellation logic with the existing web flow, and regression coverage. Browser return never confirms payment.
|
||||
|
||||
* [Resource usage] Let users with the `Administrator` role continue making requests after reaching their consumption quota, while still recording their usage.
|
||||
|
|
|
|||
55
INSTALL.md
55
INSTALL.md
|
|
@ -28,6 +28,61 @@ dotnet publish -c Release
|
|||
|
||||
and then, install the app, from the `src/PostIt/PostIt.Android/bin/Release/net10.0/PostIt.apk`
|
||||
|
||||
## PostIt native WebRTC calls (Android and Linux)
|
||||
|
||||
Calls are opt-in on the HTTPS chat host. Configure `AudioVideoCalls` there:
|
||||
|
||||
```json
|
||||
{
|
||||
"AudioVideoCalls": {
|
||||
"Enabled": true,
|
||||
"EnableLanCalls": false,
|
||||
"StunUrls": ["stun:turn.example.org:3478"],
|
||||
"TurnUrls": [
|
||||
"turn:turn.example.org:3478?transport=udp",
|
||||
"turn:turn.example.org:3478?transport=tcp",
|
||||
"turns:turn.example.org:5349?transport=tcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Set `AudioVideoCalls__TurnSharedSecret` through the host's protected environment
|
||||
or secret store, **not** in a committed settings file or in PostIt. Generate
|
||||
a strong random secret (for example `openssl rand -hex 32`), install coturn and
|
||||
adapt [coturn.conf.example](contrib/coturn.conf.example) outside the repository.
|
||||
Set its `static-auth-secret` to the same value. The server issues separate,
|
||||
short-lived REST/HMAC credentials only after invitation acceptance; clients
|
||||
never receive the shared secret. Keep clocks synchronized.
|
||||
|
||||
Expose TURN on UDP/TCP 3478, TLS/TCP 5349, and the configured **UDP relay range**
|
||||
(49160–49200 in the example). Use a publicly trusted certificate matching the
|
||||
TURN hostname and ensure coturn can read its key. TURN is not an HTTP reverse
|
||||
proxy route. If behind NAT, configure coturn's `external-ip=PUBLIC/PRIVATE`,
|
||||
`listening-ip` and `relay-ip` for the actual deployment and forward the relay
|
||||
range without changing port numbers. Restrict relay access to private services,
|
||||
set bandwidth/capacity limits appropriate to the host, and avoid logging SDP,
|
||||
credentials or candidates. Do not copy the loopback permissions used by tests.
|
||||
|
||||
Android 8+ bundles native libwebrtc from the official Maven AAR. Linux needs
|
||||
Python 3 GI and GStreamer with `webrtcbin`, libnice, VP8, Opus, JPEG, V4L2 and
|
||||
PulseAudio/PipeWire support. On Debian/Ubuntu:
|
||||
|
||||
```bash
|
||||
sudo apt install python3-gi python3-gst-1.0 gir1.2-gst-plugins-bad-1.0 \
|
||||
gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
|
||||
gstreamer1.0-plugins-bad gstreamer1.0-nice gstreamer1.0-pulseaudio
|
||||
```
|
||||
|
||||
Join the same chat room, refresh/select camera and microphone, invite a peer,
|
||||
then accept on the other client. Capture starts only after acceptance.
|
||||
The profile's IP-masking setting forces **both participants** through TURN;
|
||||
missing/unreachable TURN fails the call, never falls back to direct media.
|
||||
Without masking, explicit IP-disclosure consent is required.
|
||||
TURN still sees each client's address, and masking does not hide addresses
|
||||
from the signaling host or local network. See the
|
||||
[architecture and native regression commands](doc/architecture/audio-video-calls.md).
|
||||
|
||||
|
||||
## PostIt.Desktop on Windows (per-user installer)
|
||||
|
||||
|
|
|
|||
32
contrib/coturn.conf.example
Normal file
32
contrib/coturn.conf.example
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# Copy outside the repository; replace the realm, secret and certificate paths.
|
||||
# Use the same randomly generated secret in AudioVideoCalls:TurnSharedSecret.
|
||||
realm=turn.example.org
|
||||
fingerprint
|
||||
use-auth-secret
|
||||
static-auth-secret=REPLACE_WITH_A_RANDOM_SERVER_ONLY_SECRET
|
||||
listening-port=3478
|
||||
tls-listening-port=5349
|
||||
cert=/etc/letsencrypt/live/turn.example.org/fullchain.pem
|
||||
pkey=/etc/letsencrypt/live/turn.example.org/privkey.pem
|
||||
min-port=49160
|
||||
max-port=49200
|
||||
no-cli
|
||||
no-multicast-peers
|
||||
no-tcp-relay
|
||||
user-quota=4
|
||||
total-quota=512
|
||||
stale-nonce=600
|
||||
|
||||
# Do not allow the relay to reach private infrastructure or loopback services.
|
||||
denied-peer-ip=0.0.0.0-0.255.255.255
|
||||
denied-peer-ip=10.0.0.0-10.255.255.255
|
||||
denied-peer-ip=100.64.0.0-100.127.255.255
|
||||
denied-peer-ip=127.0.0.0-127.255.255.255
|
||||
denied-peer-ip=169.254.0.0-169.254.255.255
|
||||
denied-peer-ip=172.16.0.0-172.31.255.255
|
||||
denied-peer-ip=192.168.0.0-192.168.255.255
|
||||
denied-peer-ip=224.0.0.0-255.255.255.255
|
||||
denied-peer-ip=::
|
||||
denied-peer-ip=::1
|
||||
denied-peer-ip=fc00::-fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
|
||||
denied-peer-ip=fe80::-febf:ffff:ffff:ffff:ffff:ffff:ffff:ffff
|
||||
|
|
@ -2,12 +2,14 @@
|
|||
|
||||
## Statut et périmètre
|
||||
|
||||
Ce document décrit l'architecture cible et l'implémentation commencée.
|
||||
Ce document décrit l'architecture cible, l'implementation native actuelle et
|
||||
l'historique du transport direct.
|
||||
La préférence de confidentialité est persistée et éditable dans le profil.
|
||||
La signalisation est intégrée au chat, désactivée par défaut. Un premier
|
||||
transport natif **Android et Desktop Linux en IPv4/IPv6 directe (LAN/VPN ou IP publique)** est raccordé :
|
||||
captures Podcast existantes et SRTP direct, sans libwebrtc ni nouvelle
|
||||
couche JNI WebRTC. Il ne réalise pas encore la cible Internet, TURN ou groupe.
|
||||
La signalisation est intégrée au chat, désactivée par défaut. La page du chat
|
||||
utilise maintenant **WebRTC natif sur Android et Desktop Linux**, avec
|
||||
ICE/STUN/TURN et masquage IP impose par la politique serveur. L'ancien
|
||||
transport VLC/SRTP n'est plus raccorde a cette page. Les groupes et les
|
||||
clients navigateur ne sont pas implementes.
|
||||
Il complète la [topologie de PostIt](postit.md) et les
|
||||
[mécanismes de notification existants](notifications.md).
|
||||
La cible couvre :
|
||||
|
|
@ -18,7 +20,105 @@ La cible couvre :
|
|||
- les appels de groupe ;
|
||||
- les clients Browser, Android et Desktop.
|
||||
|
||||
## Première implémentation Android et Desktop Linux : SRTP direct IPv4/IPv6
|
||||
## Implementation actuelle : WebRTC natif Android et Linux
|
||||
|
||||
- [WebRtcCallViewModel](../../src/PostIt/PostIt/ViewModels/Chat/WebRtcCallViewModel.cs)
|
||||
conserve invitation/acceptation/fin via SignalR, sans demarrer la capture
|
||||
avant consentement. Les SDP et candidats ICE utilisent les contrats existants.
|
||||
- [AndroidWebRtcMedia](../../src/PostIt/PostIt.Android/Services/AndroidWebRtcMedia.cs)
|
||||
heberge deux surfaces natives. Le pont Java
|
||||
[WebRtcPeer](../../src/PostIt/PostIt.Android/Services/WebRtcPeer.java) utilise
|
||||
l'AAR officiel `io.github.webrtc-sdk:android:150.7871.04`, Camera2,
|
||||
l'audio natif et DTLS-SRTP. Les commandes et callbacks SDP/ICE restent en
|
||||
Java sur un executeur unique ; pas de WebView ni de binding C# complet du SDK.
|
||||
- [LinuxWebRtcMedia](../../src/PostIt/PostIt/Services/LinuxWebRtcMedia.cs)
|
||||
pilote un processus Python GI prive utilisant GStreamer `webrtcbin`,
|
||||
VP8/Opus, la camera V4L2 et le microphone PulseAudio/PipeWire.
|
||||
Les trames decodees sont affichees dans Avalonia, l'audio sur la sortie native.
|
||||
Le protocole JSON utilise uniquement stdin/stdout, pas de serveur local.
|
||||
- Le serveur impose `RelayOnly` aux deux clients si l'un des comptes masque
|
||||
son IP. Le client configure la politique native avant capture/negociation,
|
||||
valide les SDP/candidats sortants et confirme le type du candidat local
|
||||
de la paire ICE selectionnee. Aucun repli direct ni transport SRTP historique
|
||||
n'est tente en cas d'echec TURN.
|
||||
- Avec libnice, les metadonnees ICE `raddr`/`rport` des candidats relais sont
|
||||
neutralisees a la frontiere de signalisation (RFC 8828), sans modifier
|
||||
l'adresse du relais ni reecrire le SDP. UPnP est desactive. Les politiques
|
||||
`relay` de webrtcbin et `force-relay` de libnice sont toutes deux activees.
|
||||
Un SDP qui expose une adresse non relayee est refuse, pas corrige.
|
||||
- Raccrocher, sortir de la page/salle, se deconnecter ou passer l'application
|
||||
en arriere-plan annule aussi un demarrage natif en cours et libere les medias.
|
||||
Un echec natif, une perte ICE ou 30 secondes sans progression audio/video
|
||||
terminent l'appel explicitement. Il n'y a pas d'ICE restart/renegociation
|
||||
automatique ni de migration transparente d'un appel existant.
|
||||
|
||||
Le panneau ne demande plus d'adresse ni de port RTP manuel. Sans masquage,
|
||||
un consentement explicite au partage IP reste requis. Activer le masquage
|
||||
dans le profil pendant un appel fait terminer cet appel par le serveur :
|
||||
il faut rappeler pour appliquer la nouvelle politique.
|
||||
|
||||
Configuration coturn, certificats, ports et dependances Linux :
|
||||
[INSTALL](../../INSTALL.md#postit-native-webrtc-calls-android-and-linux).
|
||||
Les moteurs d'appel sont independants des captures/encodeurs Podcast ;
|
||||
les Podcasts conservent leur chemin existant.
|
||||
|
||||
### Regressions natives
|
||||
|
||||
Tests Linux reels, sans compte ni camera, avec sources video/audio synthetiques,
|
||||
decodage bidirectionnel, selection effective d'un relais coturn, controle des
|
||||
adresses signalees et panne TURN sans repli :
|
||||
|
||||
```bash
|
||||
python3 src/PostIt/PostIt.Tests/Native/webrtc_media_test.py -v
|
||||
```
|
||||
|
||||
Installer coturn pour ces tests (ou definir `POSTIT_TURN_SERVER` vers son
|
||||
executable). Ils demarrent un processus local prive, avec secret REST aleatoire
|
||||
temporaire, et l'arretent en fin de test ; ils ne changent aucun service systeme.
|
||||
|
||||
Tests du consentement, de la politique, des interruptions et du rendu de vraies
|
||||
trames dans Avalonia/Skia :
|
||||
|
||||
```bash
|
||||
XDG_CONFIG_HOME="$(mktemp -d)" POSTIT_TEST_NATIVE_WEBRTC=1 \
|
||||
dotnet test --project src/PostIt/PostIt.Tests \
|
||||
--filter-class 'PostIt.Tests.WebRtcCallViewModelTests' \
|
||||
--filter-class 'PostIt.Tests.LanCallViewModelTests'
|
||||
```
|
||||
|
||||
Le test de rendu natif est ignore sans `POSTIT_TEST_NATIVE_WEBRTC=1`.
|
||||
Les tests headless ordinaires conservent leur moteur de dessin simule.
|
||||
Utiliser un dossier de configuration de test sans comptes reels et le supprimer
|
||||
apres execution.
|
||||
|
||||
Le test Android physique est reserve a l'APK **Debug** installe : camera avant,
|
||||
microphone, deux surfaces natives, pair Linux synthetique et coturn local,
|
||||
aucun compte ni enregistrement. Il utilise `adb reverse` pour des ports TCP
|
||||
temporaires et retire uniquement ses propres redirections :
|
||||
|
||||
```bash
|
||||
POSTIT_TEST_ANDROID_WEBRTC=1 POSTIT_ANDROID_SERIAL=SERIAL \
|
||||
python3 src/PostIt/PostIt.Tests/Native/webrtc_media_test.py \
|
||||
NativeWebRtcTests.test_android_linux_relay_bidirectional_media -v
|
||||
```
|
||||
|
||||
Ce controle local ne remplace pas un essai sur deux reseaux Internet distincts,
|
||||
avec DNS/certificats publics et pare-feu de production.
|
||||
|
||||
Validation effectuee : 37 tests PostIt (dont rendu Linux avec Skia), 76 tests
|
||||
serveur de signalisation et 5 regressions natives reussis. Sur le telephone
|
||||
Android arm64, les deux roles appelant/appele ont recu plus de 100 trames
|
||||
video et 500 paquets audio pendant au moins 10 secondes, avec deux rendus
|
||||
natifs confirmes et un candidat local `relay` des deux cotes. La panne TURN
|
||||
ne produit aucun candidat direct sur Android ou Linux. Les essais utilisent
|
||||
la camera avant et le microphone Android reels, des sources Linux synthetiques
|
||||
et un coturn TCP local via USB ; la camera et la sortie audio Linux materielles,
|
||||
TURN/TLS et les reseaux Internet distincts restent a verifier.
|
||||
|
||||
## Historique : premiere implementation SRTP direct IPv4/IPv6
|
||||
|
||||
Les paragraphes suivants decrivent le transport precedent, conserve dans le
|
||||
code et ses tests mais non propose par le panneau d'appel actuel.
|
||||
|
||||
Les adresses IPv4 publiques directement assignees a une interface locale active
|
||||
sont egalement disponibles, avec la meme validation unicast cote client,
|
||||
|
|
@ -508,45 +608,37 @@ les redémarrages ICE et renégociations restent à implémenter.
|
|||
|
||||
Configuration serveur, section `AudioVideoCalls` :
|
||||
|
||||
- `Enabled` : `false` par défaut, à conserver tant que les moteurs ne sont
|
||||
pas validés ;
|
||||
- `Enabled` : `false` par défaut, a activer apres configuration et validation
|
||||
du deploiement ICE/TURN ;
|
||||
- `StunUrls` : tableau d'URI `stun:` ou `stuns:` ;
|
||||
- `TurnUrls` : tableau d'URI `turn:` ou `turns:`, avec transport optionnel ;
|
||||
- `TurnSharedSecret` : secret REST coturn, **uniquement côté serveur**,
|
||||
à fournir par le gestionnaire de secrets ou la variable d'environnement
|
||||
`AudioVideoCalls__TurnSharedSecret`, jamais dans le dépôt.
|
||||
|
||||
Linux GStreamer prend en charge `stun:`, mais pas `stuns:` : utiliser TURN/TLS
|
||||
pour ce dernier besoin.
|
||||
|
||||
Les credentials TURN temporaires sont calculés côté serveur avec le
|
||||
format REST coturn (expiration, nonce aléatoire et HMAC-SHA1). Aucun secret
|
||||
de longue durée n'est envoyé aux clients. Aucun service TURN n'a été
|
||||
déployé ou configuré par cette tranche.
|
||||
de longue durée n'est envoyé aux clients. Aucun service TURN de production
|
||||
n'est deploye automatiquement ; seuls les tests lancent un coturn isole.
|
||||
|
||||
Le [transport PostIt](../../src/PostIt/PostIt/Services/CallSignalingClient.cs)
|
||||
utilise la connexion authentifiée du `ChatViewModel`, sans seconde connexion
|
||||
ni interrogation HTTP. Il expose les commandes et événements typés et
|
||||
signale les interruptions ; il ne capture aucun média. L'interface ne
|
||||
propose pas encore de bouton d'appel ; une éventuelle invitation est
|
||||
signalée comme non utilisable faute de moteur natif, sans acceptation
|
||||
automatique.
|
||||
signale les interruptions ; il ne capture aucun média.
|
||||
Le panneau WebRTC de la page du chat et les deux adaptateurs natifs
|
||||
consomment maintenant cette signalisation, toujours sans acceptation automatique.
|
||||
|
||||
### Travail restant pour la cible Internet multi-plateforme
|
||||
|
||||
- Android : l'approche Java/JNI libwebrtc a été écartée à la demande du
|
||||
projet ; aucun adaptateur libwebrtc expérimental n'est conservé.
|
||||
L'adaptateur SRTP LAN réutilise désormais la capture Podcast ; la solution
|
||||
compatible avec la cible Internet reste à choisir.
|
||||
- Desktop Linux : le transport SRTP LAN est raccordé, mais ne remplace pas
|
||||
les validations matérielles ni une solution de connectivité Internet.
|
||||
- Côté partagé : relier consentement, états réels du moteur, mise en
|
||||
sourdine, arrêt et libération des ressources à la signalisation.
|
||||
- Valider un appel Android ↔ Desktop Linux avec caméra avant et microphone
|
||||
réels, rendu et audio distants, VP8/Opus, puis transport TURN sélectionné
|
||||
dans les statistiques, sans repli direct lorsque le masquage est demandé.
|
||||
|
||||
Les tests de cette tranche portent sur les contrats, autorisations, courses
|
||||
d'acceptation, expirations, limites, interruption et préférence persistée.
|
||||
La compilation des applications Android et Desktop ne prouve pas le
|
||||
fonctionnement de WebRTC natif : le crash antérieur n'est pas résolu.
|
||||
- Valider les cameras, microphones et sorties audio Linux materiels.
|
||||
- Valider les appels sur deux reseaux Internet distincts, UDP/TCP/TLS,
|
||||
DNS, certificats et politiques de pare-feu de production.
|
||||
- Ajouter si necessaire les commandes de sourdine, renegociation/ICE restart
|
||||
et renouvellement des credentials pour les appels longs.
|
||||
- Les groupes/SFU, Browser et les autres plateformes restent hors perimetre.
|
||||
|
||||
## Retour sur l'essai Android du 25 octobre 2026
|
||||
|
||||
|
|
@ -584,4 +676,9 @@ impossibilité de WebRTC sur Android ou une défaillance du ChatHub.
|
|||
Les essais Browser/GStreamer réussis utilisaient des sources synthétiques
|
||||
et un décodage sans affichage côté GStreamer : ils ne valident pas
|
||||
PostIt.Desktop Linux, ses périphériques réels ou son rendu Avalonia.
|
||||
La validation multi-plateforme demeure incomplète.
|
||||
Ces limites de l'ancien laboratoire ne constituent plus le bilan du moteur
|
||||
actuel : les nouvelles regressions Android/GStreamer ci-dessus verifient
|
||||
des medias bidirectionnels et des rendus reels via TURN. La cause precise
|
||||
du crash de cet ancien prototype reste inconnue ; aucun crash natif n'a
|
||||
ete reproduit pendant les nouveaux essais. Cela ne remplace pas les
|
||||
validations Internet et materielles encore indiquees.
|
||||
|
|
@ -4,18 +4,19 @@
|
|||
<!-- https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management -->
|
||||
<PropertyGroup>
|
||||
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
|
||||
<AvaloniaVersionBase>12.1.4</AvaloniaVersionBase>
|
||||
<AvaloniaVersionBase>12.1.6</AvaloniaVersionBase>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageVersion Include="Avalonia" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.AvaloniaEdit" Version="12.0.0" />
|
||||
<PackageVersion Include="Avalonia.Themes.Fluent" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Desktop" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Browser" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Android" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Headless" Version="12.1.4" />
|
||||
<PackageVersion Include="Avalonia.Headless" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Headless.Xunit" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Skia" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.Fonts.Inter" Version="$(AvaloniaVersionBase)" />
|
||||
<PackageVersion Include="Avalonia.AvaloniaEdit" Version="12.0.0" />
|
||||
<PackageVersion Include="DirectShowLib" Version="1.0.0" />
|
||||
<PackageVersion Include="LibVLCSharp.Avalonia" Version="3.10.2" />
|
||||
<PackageVersion Include="LibVLCSharp" Version="3.10.2" />
|
||||
|
|
|
|||
|
|
@ -27,6 +27,10 @@ internal static class PlatformBootstrap
|
|||
Platform.CreateVideoSurface = (player, replay) => new AndroidVideoSurface(player, replay);
|
||||
if (System.OperatingSystem.IsAndroidVersionAtLeast(26))
|
||||
{
|
||||
Platform.CreateWebRtcMedia = () => System.OperatingSystem.IsAndroidVersionAtLeast(26)
|
||||
? new AndroidWebRtcMedia()
|
||||
: throw new System.PlatformNotSupportedException("Android 8 requis pour WebRTC.");
|
||||
Platform.GetWebRtcDevices = AndroidLanCallMedia.GetDevices;
|
||||
Platform.GetLanCallDevices = AndroidLanCallMedia.GetDevices;
|
||||
Platform.CreateLanCallMedia = (address, port) =>
|
||||
System.OperatingSystem.IsAndroidVersionAtLeast(26)
|
||||
|
|
|
|||
|
|
@ -30,6 +30,10 @@
|
|||
<PackageReference Include="Xamarin.AndroidX.Core.SplashScreen" />
|
||||
<PackageReference Include="Xamarin.AndroidX.Browser" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<AndroidMavenLibrary Include="io.github.webrtc-sdk:android" Version="150.7871.04" Bind="false" VerifyDependencies="false" />
|
||||
<AndroidJavaSource Update="Services/WebRtcPeer.java" Bind="false" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\PostIt\PostIt.csproj" />
|
||||
</ItemGroup>
|
||||
|
|
@ -40,6 +44,7 @@
|
|||
<Compile Include="..\PostIt.Tests\AndroidCameraFrameRotationTestActivity.cs" Link="Tests\AndroidCameraFrameRotationTestActivity.cs" />
|
||||
<Compile Include="..\PostIt.Tests\AndroidPodcastBlurTestActivity.cs" Link="Tests\AndroidPodcastBlurTestActivity.cs" />
|
||||
<Compile Include="..\PostIt.Tests\AndroidLanCallTestActivity.cs" Link="Tests\AndroidLanCallTestActivity.cs" />
|
||||
<Compile Include="..\PostIt.Tests\AndroidWebRtcTestActivity.cs" Link="Tests\AndroidWebRtcTestActivity.cs" />
|
||||
<Compile Include="..\PostIt.Tests\AndroidScreenAwakeTestActivity.cs" Link="Tests\AndroidScreenAwakeTestActivity.cs" />
|
||||
</ItemGroup>
|
||||
<!-- This is an Avalonia app, not a MAUI app: exclude the .NET MAUI VS Code
|
||||
|
|
|
|||
164
src/PostIt/PostIt.Android/Services/AndroidWebRtcMedia.cs
Normal file
164
src/PostIt/PostIt.Android/Services/AndroidWebRtcMedia.cs
Normal file
|
|
@ -0,0 +1,164 @@
|
|||
using System;
|
||||
using System.Diagnostics;
|
||||
using System.Runtime.Versioning;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Android.Content;
|
||||
using Android.Runtime;
|
||||
using Avalonia.Android;
|
||||
using Avalonia.Controls;
|
||||
using Avalonia.Platform;
|
||||
using PostIt.Services;
|
||||
|
||||
namespace PostIt.Android.Services;
|
||||
|
||||
[SupportedOSPlatform("android26.0")]
|
||||
internal sealed class AndroidWebRtcMedia : WebRtcNativeMedia
|
||||
{
|
||||
private readonly CancellationTokenSource _lifetime = new();
|
||||
private readonly Peer _peer;
|
||||
private readonly Surface _local;
|
||||
private readonly Surface _remote;
|
||||
private readonly Task _poll;
|
||||
private readonly TaskCompletionSource _closed = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
private bool _disposed;
|
||||
|
||||
public override Control LocalVideo => _local;
|
||||
public override Control RemoteVideo => _remote;
|
||||
|
||||
public AndroidWebRtcMedia()
|
||||
{
|
||||
_peer = new Peer(MainActivity.Current ?? throw new InvalidOperationException("Activité Android indisponible."));
|
||||
_local = new Surface(_peer.GetView("localView"));
|
||||
_remote = new Surface(_peer.GetView("remoteView"));
|
||||
_poll = Task.Run(PollAsync);
|
||||
}
|
||||
|
||||
protected override async Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
|
||||
{
|
||||
var activity = MainActivity.Current ?? throw new InvalidOperationException("Activité Android indisponible.");
|
||||
await PodcastPermissions.EnsureAsync(activity, microphone: true, ct, notifications: false);
|
||||
if (!MainActivity.IsForeground)
|
||||
throw new InvalidOperationException("Revenez dans PostIt pour demarrer l'appel.");
|
||||
await Task.WhenAll(_local.Ready, _remote.Ready).WaitAsync(TimeSpan.FromSeconds(10), ct);
|
||||
await SendCommandAsync(StartCommand(configuration), ct);
|
||||
}
|
||||
|
||||
protected override Task SendCommandAsync(string json, CancellationToken ct)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
_peer.Command(json);
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
private async Task PollAsync()
|
||||
{
|
||||
var stats = Stopwatch.StartNew();
|
||||
var failed = false;
|
||||
try
|
||||
{
|
||||
while (!_lifetime.IsCancellationRequested)
|
||||
{
|
||||
for (var i = 0; i < 64 && _peer.Poll() is { } message; i++)
|
||||
{
|
||||
if (message == "{\"type\":\"closed\"}") _closed.TrySetResult();
|
||||
if (!failed)
|
||||
{
|
||||
try { ProcessMessage(message); }
|
||||
catch (Exception error)
|
||||
{
|
||||
failed = true;
|
||||
Fail(error);
|
||||
if (_disposed) _closed.TrySetException(error);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!_disposed && !failed && Configuration is not null && stats.Elapsed >= TimeSpan.FromSeconds(1))
|
||||
{
|
||||
_peer.Command("{\"command\":\"stats\"}");
|
||||
stats.Restart();
|
||||
}
|
||||
await Task.Delay(20, _lifetime.Token);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
|
||||
catch (Exception error) { Fail(error); _closed.TrySetException(error); }
|
||||
}
|
||||
|
||||
public override async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_disposed) return;
|
||||
_disposed = true;
|
||||
try
|
||||
{
|
||||
_peer.Command("{\"command\":\"close\"}");
|
||||
await _closed.Task.WaitAsync(TimeSpan.FromSeconds(10));
|
||||
}
|
||||
finally
|
||||
{
|
||||
_lifetime.Cancel();
|
||||
await _poll;
|
||||
_peer.Dispose();
|
||||
_lifetime.Dispose();
|
||||
Closed();
|
||||
}
|
||||
}
|
||||
|
||||
private sealed class Surface(global::Android.Views.View view) : NativeControlHost
|
||||
{
|
||||
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
public Task Ready => _ready.Task;
|
||||
protected override IPlatformHandle CreateNativeControlCore(IPlatformHandle parent)
|
||||
{
|
||||
_ready.TrySetResult();
|
||||
return new AndroidViewControlHandle(view);
|
||||
}
|
||||
}
|
||||
|
||||
// Only this small Java bridge crosses JNI. SDP and ICE callbacks stay in Java,
|
||||
// where the native SDK owns their lifetime and thread affinity.
|
||||
private sealed class Peer : IDisposable
|
||||
{
|
||||
private readonly Java.Lang.Class _type;
|
||||
private readonly Java.Lang.Object _instance;
|
||||
private readonly Java.Lang.Reflect.Method _command;
|
||||
private readonly Java.Lang.Reflect.Method _poll;
|
||||
|
||||
public Peer(Context context)
|
||||
{
|
||||
_type = Java.Lang.Class.ForName("fr.pschneider.postit.WebRtcPeer")!;
|
||||
using var contextType = Java.Lang.Class.FromType(typeof(Context));
|
||||
using var constructor = _type.GetConstructor(contextType);
|
||||
_instance = constructor!.NewInstance(context)!;
|
||||
using var stringType = Java.Lang.Class.FromType(typeof(Java.Lang.String));
|
||||
_command = _type.GetMethod("command", stringType)!;
|
||||
_poll = _type.GetMethod("poll")!;
|
||||
}
|
||||
|
||||
public global::Android.Views.View GetView(string method)
|
||||
{
|
||||
using var getter = _type.GetMethod(method);
|
||||
return getter!.Invoke(_instance)!.JavaCast<global::Android.Views.View>()!;
|
||||
}
|
||||
|
||||
public void Command(string message)
|
||||
{
|
||||
using var value = new Java.Lang.String(message);
|
||||
_command.Invoke(_instance, value);
|
||||
}
|
||||
|
||||
public string? Poll()
|
||||
{
|
||||
using var result = _poll.Invoke(_instance);
|
||||
return result?.ToString();
|
||||
}
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_command.Dispose();
|
||||
_poll.Dispose();
|
||||
_instance.Dispose();
|
||||
_type.Dispose();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -32,6 +32,7 @@ internal static class PodcastPermissions
|
|||
if (!await pending.Task.WaitAsync(ct))
|
||||
throw new UnauthorizedAccessException(notifications
|
||||
? "Autorisez la camera, les notifications et, si selectionne, le microphone pour diffuser."
|
||||
: microphone ? "Autorisez la camera et le microphone pour l'appel."
|
||||
: "Autorisez la camera pour la previsualisation.");
|
||||
}
|
||||
|
||||
|
|
|
|||
323
src/PostIt/PostIt.Android/Services/WebRtcPeer.java
Normal file
323
src/PostIt/PostIt.Android/Services/WebRtcPeer.java
Normal file
|
|
@ -0,0 +1,323 @@
|
|||
package fr.pschneider.postit;
|
||||
|
||||
import android.content.Context;
|
||||
import org.json.JSONArray;
|
||||
import org.json.JSONObject;
|
||||
import org.webrtc.*;
|
||||
import org.webrtc.audio.JavaAudioDeviceModule;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.*;
|
||||
|
||||
/** Keeps all SDK operations and asynchronous SDP callbacks on one executor. */
|
||||
public final class WebRtcPeer {
|
||||
private final Context context;
|
||||
private final EglBase egl = EglBase.create();
|
||||
private final SurfaceViewRenderer local;
|
||||
private final SurfaceViewRenderer remote;
|
||||
private final ExecutorService executor = Executors.newSingleThreadExecutor();
|
||||
private final ArrayBlockingQueue<String> events = new ArrayBlockingQueue<>(64);
|
||||
private final List<IceCandidate> pendingLocal = new ArrayList<>();
|
||||
private final List<IceCandidate> pendingRemote = new ArrayList<>();
|
||||
private PeerConnectionFactory factory;
|
||||
private PeerConnection peer;
|
||||
private JavaAudioDeviceModule audioModule;
|
||||
private VideoCapturer capturer;
|
||||
private SurfaceTextureHelper texture;
|
||||
private VideoSource videoSource;
|
||||
private AudioSource audioSource;
|
||||
private VideoTrack videoTrack;
|
||||
private AudioTrack audioTrack;
|
||||
private VideoTrack remoteTrack;
|
||||
private boolean localDescriptionSent;
|
||||
private boolean remoteDescriptionSet;
|
||||
private volatile boolean closed;
|
||||
private volatile boolean localRendered;
|
||||
private volatile boolean remoteRendered;
|
||||
|
||||
public WebRtcPeer(Context context) {
|
||||
this.context = context.getApplicationContext();
|
||||
local = new SurfaceViewRenderer(context);
|
||||
remote = new SurfaceViewRenderer(context);
|
||||
local.init(egl.getEglBaseContext(), new RendererCommon.RendererEvents() {
|
||||
public void onFirstFrameRendered() { localRendered = true; }
|
||||
public void onFrameResolutionChanged(int width, int height, int rotation) {}
|
||||
});
|
||||
local.setMirror(true);
|
||||
remote.init(egl.getEglBaseContext(), new RendererCommon.RendererEvents() {
|
||||
public void onFirstFrameRendered() { remoteRendered = true; }
|
||||
public void onFrameResolutionChanged(int width, int height, int rotation) {}
|
||||
});
|
||||
local.setEnableHardwareScaler(true);
|
||||
remote.setEnableHardwareScaler(true);
|
||||
}
|
||||
|
||||
public android.view.View localView() { return local; }
|
||||
public android.view.View remoteView() { return remote; }
|
||||
public String poll() { return events.poll(); }
|
||||
|
||||
private void enqueue(Runnable action) {
|
||||
try {
|
||||
executor.execute(() -> {
|
||||
try { action.run(); }
|
||||
catch (Exception failure) { error(failure.getMessage()); }
|
||||
});
|
||||
} catch (RejectedExecutionException failure) {
|
||||
if (!closed) throw failure;
|
||||
}
|
||||
}
|
||||
|
||||
public void command(String json) {
|
||||
enqueue(() -> {
|
||||
try {
|
||||
JSONObject message = new JSONObject(json);
|
||||
switch (message.getString("command")) {
|
||||
case "start": start(message.getJSONObject("configuration")); break;
|
||||
case "signal": signal(message.getJSONObject("signal")); break;
|
||||
case "stats": if (peer != null && !closed) peer.getStats(this::stats); break;
|
||||
case "close": close(); break;
|
||||
default: throw new IllegalArgumentException("Unknown WebRTC command");
|
||||
}
|
||||
} catch (Exception error) { error(error.getMessage()); }
|
||||
});
|
||||
}
|
||||
|
||||
private void emit(JSONObject value) {
|
||||
if (!events.offer(value.toString())) {
|
||||
events.clear();
|
||||
events.offer("{\"type\":\"error\",\"message\":\"WebRTC event queue overflow\"}");
|
||||
}
|
||||
}
|
||||
|
||||
private void error(String message) {
|
||||
try { emit(new JSONObject().put("type", "error").put("message", message)); }
|
||||
catch (org.json.JSONException impossible) { throw new IllegalStateException(impossible); }
|
||||
}
|
||||
|
||||
private void start(JSONObject configuration) throws Exception {
|
||||
if (factory != null || closed) throw new IllegalStateException("WebRTC already started or closed");
|
||||
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context)
|
||||
.setEnableInternalTracer(false).createInitializationOptions());
|
||||
audioModule = JavaAudioDeviceModule.builder(context).createAudioDeviceModule();
|
||||
factory = PeerConnectionFactory.builder().setAudioDeviceModule(audioModule)
|
||||
.setVideoEncoderFactory(new DefaultVideoEncoderFactory(egl.getEglBaseContext(), true, true))
|
||||
.setVideoDecoderFactory(new DefaultVideoDecoderFactory(egl.getEglBaseContext())).createPeerConnectionFactory();
|
||||
List<PeerConnection.IceServer> servers = new ArrayList<>();
|
||||
JSONArray configuredServers = configuration.getJSONArray("iceServers");
|
||||
boolean relay = configuration.getBoolean("relayOnly");
|
||||
for (int i = 0; i < configuredServers.length(); i++) {
|
||||
JSONObject item = configuredServers.getJSONObject(i);
|
||||
JSONArray urls = item.getJSONArray("urls");
|
||||
for (int j = 0; j < urls.length(); j++) {
|
||||
String url = urls.getString(j);
|
||||
if (relay && !url.startsWith("turn:") && !url.startsWith("turns:")) continue;
|
||||
PeerConnection.IceServer.Builder server = PeerConnection.IceServer.builder(url);
|
||||
if (!item.isNull("username")) server.setUsername(item.getString("username"));
|
||||
if (!item.isNull("credential")) server.setPassword(item.getString("credential"));
|
||||
servers.add(server.createIceServer());
|
||||
}
|
||||
}
|
||||
PeerConnection.RTCConfiguration rtc = new PeerConnection.RTCConfiguration(servers);
|
||||
rtc.sdpSemantics = PeerConnection.SdpSemantics.UNIFIED_PLAN;
|
||||
rtc.bundlePolicy = PeerConnection.BundlePolicy.MAXBUNDLE;
|
||||
rtc.rtcpMuxPolicy = PeerConnection.RtcpMuxPolicy.REQUIRE;
|
||||
rtc.iceTransportsType = relay ? PeerConnection.IceTransportsType.RELAY : PeerConnection.IceTransportsType.ALL;
|
||||
peer = factory.createPeerConnection(rtc, new Observer());
|
||||
if (peer == null) throw new IllegalStateException("Cannot create native WebRTC peer");
|
||||
Camera2Enumerator cameras = new Camera2Enumerator(context);
|
||||
String camera = configuration.getString("camera");
|
||||
capturer = cameras.createCapturer(camera, new CameraVideoCapturer.CameraEventsHandler() {
|
||||
public void onCameraError(String message) { error("Camera: " + message); }
|
||||
public void onCameraDisconnected() { if (!closed) error("Camera disconnected"); }
|
||||
public void onCameraFreezed(String message) { error("Camera frozen: " + message); }
|
||||
public void onCameraOpening(String name) {}
|
||||
public void onFirstFrameAvailable() {}
|
||||
public void onCameraClosed() { if (!closed) error("Camera closed unexpectedly"); }
|
||||
});
|
||||
if (capturer == null) throw new IllegalArgumentException("Camera unavailable");
|
||||
texture = SurfaceTextureHelper.create("PostIt-WebRTC-camera", egl.getEglBaseContext());
|
||||
videoSource = factory.createVideoSource(false);
|
||||
capturer.initialize(texture, context, videoSource.getCapturerObserver());
|
||||
videoTrack = factory.createVideoTrack("postit-video", videoSource);
|
||||
videoTrack.addSink(local);
|
||||
audioSource = factory.createAudioSource(new MediaConstraints());
|
||||
audioTrack = factory.createAudioTrack("postit-audio", audioSource);
|
||||
peer.addTrack(audioTrack, Collections.singletonList("postit"));
|
||||
peer.addTrack(videoTrack, Collections.singletonList("postit"));
|
||||
capturer.startCapture(640, 480, 24);
|
||||
if (configuration.getBoolean("caller")) peer.createOffer(new CreateDescription("offer"), new MediaConstraints());
|
||||
}
|
||||
|
||||
private abstract class SdpCallback implements SdpObserver {
|
||||
public void onCreateSuccess(SessionDescription description) {}
|
||||
public void onSetSuccess() {}
|
||||
public void onCreateFailure(String message) { error("SDP creation failed: " + message); }
|
||||
public void onSetFailure(String message) { error("SDP application failed: " + message); }
|
||||
}
|
||||
|
||||
private final class CreateDescription extends SdpCallback {
|
||||
private final String kind;
|
||||
CreateDescription(String kind) { this.kind = kind; }
|
||||
public void onCreateSuccess(SessionDescription description) {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
peer.setLocalDescription(new SdpCallback() {
|
||||
public void onSetSuccess() {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
try {
|
||||
emit(new JSONObject().put("type", "signal").put("signal",
|
||||
new JSONObject().put("kind", kind).put("sdp", description.description)));
|
||||
localDescriptionSent = true;
|
||||
for (IceCandidate candidate : pendingLocal) sendCandidate(candidate);
|
||||
pendingLocal.clear();
|
||||
} catch (Exception failure) { error(failure.getMessage()); }
|
||||
});
|
||||
}
|
||||
}, description);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private void signal(JSONObject signal) throws Exception {
|
||||
if (closed || peer == null) throw new IllegalStateException("WebRTC peer not running");
|
||||
String kind = signal.getString("kind");
|
||||
if (kind.equals("candidate")) {
|
||||
IceCandidate candidate = new IceCandidate(signal.getString("sdpMid"),
|
||||
signal.getInt("sdpMLineIndex"), signal.getString("candidate"));
|
||||
if (remoteDescriptionSet) addCandidate(candidate);
|
||||
else {
|
||||
if (pendingRemote.size() >= 256) throw new IllegalStateException("Too many pending ICE candidates");
|
||||
pendingRemote.add(candidate);
|
||||
}
|
||||
} else if (kind.equals("offer") || kind.equals("answer")) {
|
||||
peer.setRemoteDescription(new SdpCallback() {
|
||||
public void onSetSuccess() {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
remoteDescriptionSet = true;
|
||||
for (IceCandidate candidate : pendingRemote) addCandidate(candidate);
|
||||
pendingRemote.clear();
|
||||
if (kind.equals("offer")) peer.createAnswer(new CreateDescription("answer"), new MediaConstraints());
|
||||
});
|
||||
}
|
||||
}, new SessionDescription(kind.equals("offer") ? SessionDescription.Type.OFFER : SessionDescription.Type.ANSWER,
|
||||
signal.getString("sdp")));
|
||||
} else throw new IllegalArgumentException("Invalid WebRTC signal");
|
||||
}
|
||||
|
||||
private void addCandidate(IceCandidate candidate) {
|
||||
peer.addIceCandidate(candidate, new AddIceObserver() {
|
||||
public void onAddSuccess() {}
|
||||
public void onAddFailure(String message) { error("ICE candidate rejected: " + message); }
|
||||
});
|
||||
}
|
||||
|
||||
private void sendCandidate(IceCandidate candidate) {
|
||||
try {
|
||||
emit(new JSONObject().put("type", "signal").put("signal", new JSONObject()
|
||||
.put("kind", "candidate").put("candidate", candidate.sdp)
|
||||
.put("sdpMid", candidate.sdpMid).put("sdpMLineIndex", candidate.sdpMLineIndex)));
|
||||
} catch (Exception failure) { error(failure.getMessage()); }
|
||||
}
|
||||
|
||||
private void stats(RTCStatsReport report) {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
try {
|
||||
Map<String, RTCStats> all = report.getStatsMap();
|
||||
String selectedId = null, localType = null;
|
||||
long audioPackets = 0, videoFrames = 0;
|
||||
for (RTCStats item : all.values()) {
|
||||
Map<String, Object> values = item.getMembers();
|
||||
if (item.getType().equals("transport") && values.get("selectedCandidatePairId") instanceof String)
|
||||
selectedId = (String)values.get("selectedCandidatePairId");
|
||||
if (item.getType().equals("inbound-rtp")) {
|
||||
String media = String.valueOf(values.getOrDefault("kind", values.get("mediaType")));
|
||||
if (media.equals("audio")) audioPackets += number(values.get("packetsReceived"));
|
||||
if (media.equals("video")) videoFrames += number(values.get("framesDecoded"));
|
||||
}
|
||||
}
|
||||
RTCStats pair = all.get(selectedId);
|
||||
if (pair != null) {
|
||||
RTCStats candidate = all.get(String.valueOf(pair.getMembers().get("localCandidateId")));
|
||||
if (candidate != null) localType = String.valueOf(candidate.getMembers().get("candidateType"));
|
||||
}
|
||||
String state = peer.connectionState().toString().toLowerCase(Locale.ROOT);
|
||||
if (state.equals("connected") && localType == null) state = "checking";
|
||||
emit(new JSONObject().put("type", "status").put("status", new JSONObject()
|
||||
.put("connectionState", state).put("localCandidateType", localType)
|
||||
.put("audioPackets", audioPackets).put("videoFrames", videoFrames)
|
||||
.put("localVideoRendered", localRendered).put("remoteVideoRendered", remoteRendered)));
|
||||
} catch (Exception failure) { error(failure.getMessage()); }
|
||||
});
|
||||
}
|
||||
|
||||
private static long number(Object value) { return value instanceof Number ? ((Number)value).longValue() : 0; }
|
||||
|
||||
private final class Observer implements PeerConnection.Observer {
|
||||
public void onSignalingChange(PeerConnection.SignalingState state) {}
|
||||
public void onIceConnectionChange(PeerConnection.IceConnectionState state) {}
|
||||
public void onIceConnectionReceivingChange(boolean receiving) {}
|
||||
public void onIceGatheringChange(PeerConnection.IceGatheringState state) {}
|
||||
public void onIceCandidatesRemoved(IceCandidate[] candidates) {}
|
||||
public void onAddStream(MediaStream stream) {}
|
||||
public void onRemoveStream(MediaStream stream) {}
|
||||
public void onDataChannel(DataChannel channel) {}
|
||||
public void onRenegotiationNeeded() {}
|
||||
public void onIceCandidate(IceCandidate candidate) {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
if (localDescriptionSent) sendCandidate(candidate);
|
||||
else if (pendingLocal.size() < 256) pendingLocal.add(candidate);
|
||||
else error("Too many local ICE candidates");
|
||||
});
|
||||
}
|
||||
public void onTrack(RtpTransceiver transceiver) {
|
||||
MediaStreamTrack track = transceiver.getReceiver().track();
|
||||
if (track instanceof VideoTrack) {
|
||||
enqueue(() -> {
|
||||
if (closed) return;
|
||||
remoteTrack = (VideoTrack)track;
|
||||
remoteTrack.addSink(remote);
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private interface Cleanup { void run() throws Exception; }
|
||||
|
||||
private void release(Cleanup cleanup, List<String> failures) {
|
||||
try { cleanup.run(); }
|
||||
catch (Exception failure) { failures.add(failure.toString()); }
|
||||
}
|
||||
|
||||
private void close() {
|
||||
if (closed) return;
|
||||
closed = true;
|
||||
List<String> failures = new ArrayList<>();
|
||||
if (capturer != null) release(() -> capturer.stopCapture(), failures);
|
||||
if (remoteTrack != null) release(() -> remoteTrack.removeSink(remote), failures);
|
||||
if (videoTrack != null) release(() -> videoTrack.removeSink(local), failures);
|
||||
if (peer != null) {
|
||||
release(() -> peer.close(), failures);
|
||||
release(() -> peer.dispose(), failures);
|
||||
}
|
||||
if (videoTrack != null) release(() -> videoTrack.dispose(), failures);
|
||||
if (audioTrack != null) release(() -> audioTrack.dispose(), failures);
|
||||
if (capturer != null) release(() -> capturer.dispose(), failures);
|
||||
if (texture != null) release(() -> texture.dispose(), failures);
|
||||
if (videoSource != null) release(() -> videoSource.dispose(), failures);
|
||||
if (audioSource != null) release(() -> audioSource.dispose(), failures);
|
||||
if (factory != null) release(() -> factory.dispose(), failures);
|
||||
if (audioModule != null) release(() -> audioModule.release(), failures);
|
||||
new android.os.Handler(android.os.Looper.getMainLooper()).post(() -> {
|
||||
release(() -> local.release(), failures);
|
||||
release(() -> remote.release(), failures);
|
||||
release(() -> egl.release(), failures);
|
||||
if (!failures.isEmpty()) error("WebRTC cleanup failed: " + String.join("; ", failures));
|
||||
try { emit(new JSONObject().put("type", "closed")); }
|
||||
catch (Exception failure) { error(failure.getMessage()); }
|
||||
});
|
||||
executor.shutdown();
|
||||
}
|
||||
}
|
||||
2
src/PostIt/PostIt.Android/proguard-rules.pro
vendored
2
src/PostIt/PostIt.Android/proguard-rules.pro
vendored
|
|
@ -7,3 +7,5 @@
|
|||
-keepattributes *Annotation*
|
||||
-dontwarn org.videolan.libvlc.**
|
||||
-dontwarn org.videolan.medialibrary.**
|
||||
-keep class org.webrtc.** { *; }
|
||||
-keep class fr.pschneider.postit.WebRtcPeer { *; }
|
||||
|
|
|
|||
140
src/PostIt/PostIt.Tests/AndroidWebRtcTestActivity.cs
Normal file
140
src/PostIt/PostIt.Tests/AndroidWebRtcTestActivity.cs
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
#if ANDROID
|
||||
using System;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Net.Sockets;
|
||||
using System.Runtime.Versioning;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Android.App;
|
||||
using Android.Content;
|
||||
using Android.OS;
|
||||
using Avalonia.Controls;
|
||||
using Avalonia.Threading;
|
||||
using PostIt.Android;
|
||||
using PostIt.Android.Services;
|
||||
using PostIt.Services;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
using Path = System.IO.Path;
|
||||
|
||||
namespace PostIt.Tests;
|
||||
|
||||
[Activity(Name = "fr.pschneider.postit.WebRtcTestActivity", Exported = true)]
|
||||
[SupportedOSPlatform("android26.0")]
|
||||
public sealed class AndroidWebRtcTestActivity : Activity
|
||||
{
|
||||
protected override async void OnCreate(Bundle? state)
|
||||
{
|
||||
base.OnCreate(state);
|
||||
var result = Path.Combine(FilesDir!.AbsolutePath, "webrtc-test-result.txt");
|
||||
File.WriteAllText(result, "RUNNING");
|
||||
try
|
||||
{
|
||||
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(75));
|
||||
await RunAsync(Intent?.GetIntExtra("port", 0) ?? 0, timeout.Token);
|
||||
File.WriteAllText(result, "PASS\nnative WebRTC media and cleanup complete");
|
||||
}
|
||||
catch (Exception error) { File.WriteAllText(result, "FAIL\n" + error); }
|
||||
finally { Finish(); }
|
||||
}
|
||||
|
||||
private async Task RunAsync(int port, CancellationToken ct)
|
||||
{
|
||||
if (port is < 1024 or > 65535) throw new ArgumentOutOfRangeException(nameof(port));
|
||||
StartActivity(new Intent(this, typeof(MainActivity)).AddFlags(ActivityFlags.NewTask));
|
||||
while (!MainActivity.IsForeground || (Avalonia.Application.Current as App)?.View is null)
|
||||
await Task.Delay(100, ct);
|
||||
using var socket = new TcpClient();
|
||||
await socket.ConnectAsync("127.0.0.1", port, ct);
|
||||
using var stream = socket.GetStream();
|
||||
using var reader = new StreamReader(stream, Encoding.UTF8, false, 4096, leaveOpen: true);
|
||||
using var writer = new StreamWriter(stream, new UTF8Encoding(false), 4096, leaveOpen: true) { AutoFlush = true };
|
||||
using var output = new SemaphoreSlim(1);
|
||||
var callbackFailure = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
async Task EmitAsync(object value)
|
||||
{
|
||||
await output.WaitAsync(ct);
|
||||
try { await writer.WriteLineAsync(JsonSerializer.Serialize(value, new JsonSerializerOptions(JsonSerializerDefaults.Web))); }
|
||||
finally { output.Release(); }
|
||||
}
|
||||
void Emit(object value) => _ = EmitObservedAsync(value);
|
||||
async Task EmitObservedAsync(object value)
|
||||
{
|
||||
try { await EmitAsync(value); }
|
||||
catch (Exception error) { callbackFailure.TrySetException(error); }
|
||||
}
|
||||
var navigation = ((App)Avalonia.Application.Current!).View!.FindControl<NavigationPage>("NavRoot")
|
||||
?? throw new InvalidOperationException("Main navigation unavailable.");
|
||||
AndroidWebRtcMedia? media = null;
|
||||
var attached = false;
|
||||
try
|
||||
{
|
||||
await Dispatcher.UIThread.InvokeAsync(async () =>
|
||||
{
|
||||
media = new AndroidWebRtcMedia();
|
||||
var grid = new Grid { RowDefinitions = new RowDefinitions("*,*") };
|
||||
grid.Children.Add(media.LocalVideo);
|
||||
Grid.SetRow(media.RemoteVideo, 1);
|
||||
grid.Children.Add(media.RemoteVideo);
|
||||
await navigation.PushAsync(new ContentPage { Content = grid });
|
||||
attached = true;
|
||||
});
|
||||
var peer = media ?? throw new InvalidOperationException("Native UI unavailable.");
|
||||
peer.Signal += signal => Emit(new { type = "signal", signal });
|
||||
peer.StatusChanged += status =>
|
||||
{
|
||||
Emit(new { type = "status", status });
|
||||
if (status.LocalVideoRendered) Emit(new { type = "rendered", target = "local", size = 1_000 });
|
||||
if (status.RemoteVideoRendered) Emit(new { type = "rendered", target = "remote", size = 1_000 });
|
||||
};
|
||||
while (true)
|
||||
{
|
||||
var read = reader.ReadLineAsync(ct).AsTask();
|
||||
var completed = await Task.WhenAny(read, peer.Completion, callbackFailure.Task);
|
||||
if (completed != read)
|
||||
{
|
||||
await completed;
|
||||
throw new IOException("Native WebRTC stopped unexpectedly.");
|
||||
}
|
||||
var line = await read;
|
||||
if (line is null) throw new IOException("Test signaling disconnected.");
|
||||
using var document = JsonDocument.Parse(line);
|
||||
var root = document.RootElement;
|
||||
switch (root.GetProperty("command").GetString())
|
||||
{
|
||||
case "start":
|
||||
var configuration = root.GetProperty("configuration").Deserialize<WebRtcMediaConfiguration>(
|
||||
new JsonSerializerOptions(JsonSerializerDefaults.Web))!;
|
||||
var camera = AndroidLanCallMedia.GetDevices().Cameras.FirstOrDefault(c => c.Name == "Camera avant")
|
||||
?? throw new InvalidOperationException("Front camera required.");
|
||||
await Dispatcher.UIThread.InvokeAsync(() => peer.StartAsync(
|
||||
configuration with { Camera = camera.Value, Microphone = "microphone" }, ct));
|
||||
break;
|
||||
case "signal":
|
||||
var signal = root.GetProperty("signal").Deserialize<CallSignal>(
|
||||
new JsonSerializerOptions(JsonSerializerDefaults.Web))!;
|
||||
await peer.HandleSignalAsync(signal, ct);
|
||||
break;
|
||||
case "close":
|
||||
await Dispatcher.UIThread.InvokeAsync(async () => await peer.DisposeAsync());
|
||||
await EmitAsync(new { type = "closed" });
|
||||
return;
|
||||
default: throw new InvalidOperationException("Unknown test command.");
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception error)
|
||||
{
|
||||
await EmitAsync(new { type = "error", message = error.ToString() });
|
||||
throw;
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (media is not null) await Dispatcher.UIThread.InvokeAsync(async () => await media.DisposeAsync());
|
||||
if (attached) await Dispatcher.UIThread.InvokeAsync(async () => await navigation.PopAsync());
|
||||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
267
src/PostIt/PostIt.Tests/Native/webrtc_media_test.py
Normal file
267
src/PostIt/PostIt.Tests/Native/webrtc_media_test.py
Normal file
|
|
@ -0,0 +1,267 @@
|
|||
"""Real GStreamer/coturn regression. No camera, account or public service is used."""
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import queue
|
||||
import secrets
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
|
||||
ENGINE = Path(__file__).resolve().parents[2] / "PostIt/Services/WebRtc/gstreamer_peer.py"
|
||||
|
||||
|
||||
class AndroidPeer:
|
||||
"""Debug Activity over an adb-reversed, loopback-only test command socket."""
|
||||
def __init__(self):
|
||||
self.adb = [os.environ.get("POSTIT_ADB", "adb")]
|
||||
if os.environ.get("POSTIT_ANDROID_SERIAL"):
|
||||
self.adb += ["-s", os.environ["POSTIT_ANDROID_SERIAL"]]
|
||||
self.exited = threading.Event()
|
||||
self.reversed = []
|
||||
with socket.socket() as listener:
|
||||
listener.bind(("127.0.0.1", 0))
|
||||
listener.listen(1)
|
||||
listener.settimeout(35)
|
||||
port = listener.getsockname()[1]
|
||||
self.reverse(port)
|
||||
try:
|
||||
subprocess.run(self.adb + ["shell", "am", "start", "-f", "0x18000000", "-n",
|
||||
"fr.pschneider.postit/.WebRtcTestActivity", "--ei", "port", str(port)],
|
||||
check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, timeout=10)
|
||||
self.socket, _ = listener.accept()
|
||||
except BaseException:
|
||||
self.remove_reverses()
|
||||
raise
|
||||
self.input = self.socket.makefile("w", encoding="utf-8")
|
||||
self.output = self.socket.makefile("r", encoding="utf-8")
|
||||
self.stdin = self.input
|
||||
self.stdout = self.read()
|
||||
self.stderr = io.StringIO()
|
||||
|
||||
def reverse(self, port):
|
||||
subprocess.run(self.adb + ["reverse", f"tcp:{port}", f"tcp:{port}"], check=True,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=10)
|
||||
self.reversed.append(port)
|
||||
|
||||
def remove_reverses(self):
|
||||
for port in self.reversed:
|
||||
subprocess.run(self.adb + ["reverse", "--remove", f"tcp:{port}"], check=True,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=10)
|
||||
self.reversed.clear()
|
||||
|
||||
def read(self):
|
||||
try:
|
||||
yield from self.output
|
||||
finally:
|
||||
self.output.close()
|
||||
self.exited.set()
|
||||
|
||||
def poll(self):
|
||||
return 0 if self.exited.is_set() else None
|
||||
|
||||
def wait(self, timeout=None):
|
||||
if not self.exited.wait(timeout):
|
||||
raise subprocess.TimeoutExpired("Android test peer", timeout)
|
||||
self.socket.close()
|
||||
self.remove_reverses()
|
||||
return 0
|
||||
|
||||
def kill(self):
|
||||
self.socket.shutdown(socket.SHUT_RDWR)
|
||||
self.socket.close()
|
||||
|
||||
|
||||
class NativeWebRtcTests(unittest.TestCase):
|
||||
def run_pair(self, relay, unavailable=False, android=False, android_caller=False):
|
||||
processes = []
|
||||
events = queue.Queue()
|
||||
errors = []
|
||||
server = None
|
||||
server_log = None
|
||||
with tempfile.TemporaryDirectory(prefix="postit-turn-") as directory:
|
||||
try:
|
||||
servers = []
|
||||
if relay:
|
||||
binary = os.environ.get("POSTIT_TURN_SERVER", shutil.which("turnserver"))
|
||||
self.assertTrue(binary, "Install coturn or set POSTIT_TURN_SERVER")
|
||||
with socket.socket() as reservation:
|
||||
reservation.bind(("127.0.0.1", 0))
|
||||
port = reservation.getsockname()[1]
|
||||
secret = secrets.token_hex(32)
|
||||
username = str(int(time.time()) + 600) + ":native-test"
|
||||
password = base64.b64encode(hmac.new(secret.encode(), username.encode(), hashlib.sha1).digest()).decode()
|
||||
server_log = open(Path(directory) / "coturn.log", "w+")
|
||||
server = subprocess.Popen([
|
||||
binary, "-n", "--listening-ip=127.0.0.1", "--relay-ip=127.0.0.1",
|
||||
"--listening-port=" + str(port), "--min-port=52000", "--max-port=52100",
|
||||
"--realm=postit-native-test", "--use-auth-secret", "--static-auth-secret=" + secret,
|
||||
"--allow-loopback-peers", "--no-multicast-peers", "--no-tls", "--no-dtls",
|
||||
"--no-cli", "--relay-threads=1", "--pidfile=" + directory + "/turn.pid",
|
||||
"--log-file=stdout"
|
||||
], stdout=server_log, stderr=subprocess.STDOUT)
|
||||
deadline = time.monotonic() + 10
|
||||
while True:
|
||||
try:
|
||||
with socket.create_connection(("127.0.0.1", port), timeout=0.2):
|
||||
break
|
||||
except OSError:
|
||||
if time.monotonic() > deadline or server.poll() is not None:
|
||||
server_log.seek(0)
|
||||
self.fail("coturn failed to start: " + server_log.read()[-2000:])
|
||||
time.sleep(0.1)
|
||||
if unavailable:
|
||||
server.terminate()
|
||||
server.wait(timeout=5)
|
||||
servers = [{"urls": [f"turn:127.0.0.1:{port}?transport=tcp"],
|
||||
"username": username, "credential": password}]
|
||||
|
||||
def read(index, process):
|
||||
for line in process.stdout:
|
||||
try:
|
||||
message = json.loads(line)
|
||||
if message.get("type") == "frame":
|
||||
events.put((index, {"type": "rendered", "target": message["target"],
|
||||
"size": len(base64.b64decode(message["data"]))}))
|
||||
else:
|
||||
events.put((index, message))
|
||||
except Exception as error:
|
||||
errors.append(str(error))
|
||||
|
||||
def stderr(process):
|
||||
for line in process.stderr:
|
||||
errors.append(line.rstrip())
|
||||
|
||||
for index in range(2):
|
||||
if android and index == 1:
|
||||
process = AndroidPeer()
|
||||
if relay:
|
||||
process.reverse(port)
|
||||
else:
|
||||
process = subprocess.Popen([sys.executable, "-X", "faulthandler", "-u", str(ENGINE)],
|
||||
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE, text=True, bufsize=1)
|
||||
processes.append(process)
|
||||
threading.Thread(target=read, args=(index, process), daemon=True).start()
|
||||
threading.Thread(target=stderr, args=(process,), daemon=True).start()
|
||||
self.send(process, {"command": "start", "configuration": {
|
||||
"iceServers": servers, "relayOnly": relay,
|
||||
"caller": index == (1 if android_caller else 0),
|
||||
"camera": "test", "microphone": "test"}})
|
||||
stats = {}
|
||||
rendered = set()
|
||||
candidates = 0
|
||||
descriptions = set()
|
||||
started = time.monotonic()
|
||||
deadline = started + (12 if unavailable else 35)
|
||||
while time.monotonic() < deadline:
|
||||
self.assertTrue(all(p.poll() is None for p in processes),
|
||||
f"Native peer exited: {[p.poll() for p in processes]}, {errors[-20:]}")
|
||||
try:
|
||||
index, message = events.get(timeout=0.5)
|
||||
except queue.Empty:
|
||||
self.assertTrue(all(p.poll() is None for p in processes), str(errors[-10:]))
|
||||
continue
|
||||
kind = message["type"]
|
||||
if kind == "error":
|
||||
self.fail(message["message"] + "\n" + str(errors[-10:]))
|
||||
if kind == "signal":
|
||||
signal = message["signal"]
|
||||
if relay:
|
||||
candidate = signal.get("candidate")
|
||||
if candidate:
|
||||
tokens = candidate.split()
|
||||
self.assertEqual("relay", tokens[7])
|
||||
if "raddr" in tokens:
|
||||
self.assertIn(tokens[tokens.index("raddr") + 1], ("0.0.0.0", "::"))
|
||||
for line in (signal.get("sdp") or "").splitlines():
|
||||
if line.startswith("a=candidate:"):
|
||||
self.assertEqual("relay", line.split()[7])
|
||||
if line.startswith(("o=", "c=")):
|
||||
self.assertIn(line.split()[-1], ("0.0.0.0", "::", "127.0.0.1"))
|
||||
if signal["kind"] == "candidate":
|
||||
candidates += 1
|
||||
else:
|
||||
descriptions.add((index, signal["kind"]))
|
||||
self.send(processes[1 - index], {"command": "signal", "signal": signal})
|
||||
if kind == "rendered" and message["size"] > 100:
|
||||
rendered.add((index, message["target"]))
|
||||
if kind == "status":
|
||||
stats[index] = message["status"]
|
||||
if unavailable:
|
||||
self.assertNotEqual("connected", message["status"]["connectionState"])
|
||||
self.assertEqual(0, message["status"]["audioPackets"])
|
||||
self.assertEqual(0, message["status"]["videoFrames"])
|
||||
elif time.monotonic() - started >= 10 and len(stats) == 2 and len(rendered) == 4 and all(
|
||||
s["connectionState"] == "connected" and s["audioPackets"] > 100
|
||||
and s["videoFrames"] >= 100 for s in stats.values()
|
||||
):
|
||||
if relay:
|
||||
self.assertTrue(all(s["localCandidateType"] == "relay" for s in stats.values()))
|
||||
self.assertGreater(candidates, 0)
|
||||
print(f"\nMedia verified: relay={relay}, android={android}, "
|
||||
f"androidCaller={android_caller}, stats={stats}, rendered={len(rendered)}",
|
||||
flush=True)
|
||||
return
|
||||
if unavailable:
|
||||
caller = 1 if android_caller else 0
|
||||
self.assertEqual({(caller, "offer"), (1 - caller, "answer")}, descriptions)
|
||||
self.assertEqual({0, 1}, set(stats))
|
||||
self.assertTrue(all((index, "local") in rendered for index in range(2)))
|
||||
self.assertEqual(0, candidates, "TURN failure must never disclose direct candidates")
|
||||
self.assertFalse(any(target == "remote" for _, target in rendered))
|
||||
return
|
||||
self.fail(f"No bidirectional decoded/rendered media: {stats}, {rendered}, {errors[-12:]}")
|
||||
finally:
|
||||
for process in processes:
|
||||
if process.poll() is None:
|
||||
self.send(process, {"command": "close"})
|
||||
try:
|
||||
process.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
for stream in (process.stdin, process.stdout, process.stderr):
|
||||
stream.close()
|
||||
if server is not None and server.poll() is None:
|
||||
server.terminate()
|
||||
server.wait(timeout=5)
|
||||
if server_log is not None:
|
||||
server_log.close()
|
||||
|
||||
@staticmethod
|
||||
def send(process, message):
|
||||
process.stdin.write(json.dumps(message) + "\n")
|
||||
process.stdin.flush()
|
||||
|
||||
def test_direct_bidirectional_media(self):
|
||||
self.run_pair(False)
|
||||
|
||||
def test_relay_bidirectional_media_and_no_address_disclosure(self):
|
||||
self.run_pair(True)
|
||||
|
||||
def test_turn_unavailable_never_falls_back_to_direct(self):
|
||||
self.run_pair(True, unavailable=True)
|
||||
|
||||
@unittest.skipUnless(os.environ.get("POSTIT_TEST_ANDROID_WEBRTC") == "1", "Requires installed Debug APK and adb device")
|
||||
def test_android_linux_relay_bidirectional_media(self):
|
||||
for caller in (False, True):
|
||||
with self.subTest(android_caller=caller):
|
||||
self.run_pair(True, android=True, android_caller=caller)
|
||||
|
||||
@unittest.skipUnless(os.environ.get("POSTIT_TEST_ANDROID_WEBRTC") == "1", "Requires installed Debug APK and adb device")
|
||||
def test_android_turn_unavailable_never_falls_back_to_direct(self):
|
||||
self.run_pair(True, unavailable=True, android=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
|
@ -18,6 +18,7 @@
|
|||
<PackageReference Include="coverlet.collector" />
|
||||
<PackageReference Include="Avalonia.Headless" />
|
||||
<PackageReference Include="Avalonia.Headless.XUnit" />
|
||||
<PackageReference Include="Avalonia.Skia" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<FrameworkReference Include="Microsoft.AspNetCore.App" />
|
||||
|
|
|
|||
|
|
@ -7,10 +7,14 @@ namespace PostIt.Tests;
|
|||
|
||||
public class TestAppBuilder
|
||||
{
|
||||
public static AppBuilder BuildAvaloniaApp() =>
|
||||
AppBuilder.Configure<PostIt.App>()
|
||||
.UseHeadless(new AvaloniaHeadlessPlatformOptions
|
||||
{
|
||||
UseHeadlessDrawing = true
|
||||
});
|
||||
public static AppBuilder BuildAvaloniaApp()
|
||||
{
|
||||
var nativeMedia = System.Environment.GetEnvironmentVariable("POSTIT_TEST_NATIVE_WEBRTC") == "1";
|
||||
var builder = AppBuilder.Configure<PostIt.App>();
|
||||
if (nativeMedia) builder.UseSkia();
|
||||
return builder.UseHeadless(new AvaloniaHeadlessPlatformOptions
|
||||
{
|
||||
UseHeadlessDrawing = !nativeMedia
|
||||
});
|
||||
}
|
||||
}
|
||||
294
src/PostIt/PostIt.Tests/WebRtcCallViewModelTests.cs
Normal file
294
src/PostIt/PostIt.Tests/WebRtcCallViewModelTests.cs
Normal file
|
|
@ -0,0 +1,294 @@
|
|||
using Avalonia.Controls;
|
||||
using Avalonia.Headless.XUnit;
|
||||
using Avalonia.Media.Imaging;
|
||||
using Avalonia.Threading;
|
||||
using PostIt.Services;
|
||||
using PostIt.ViewModels.Chat;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
|
||||
namespace PostIt.Tests;
|
||||
|
||||
public sealed class WebRtcCallViewModelTests
|
||||
{
|
||||
private static readonly CallIceServer[] Turn = [new(["turn:localhost:3478"], "temporary", "password")];
|
||||
private const string Sdp = "v=0\r\no=- 1 1 IN IP4 127.0.0.1\r\nc=IN IP4 0.0.0.0\r\nm=video 9 UDP/TLS/RTP/SAVPF 96\r\n";
|
||||
|
||||
[AvaloniaTheory]
|
||||
[InlineData(true)]
|
||||
[InlineData(false)]
|
||||
public async Task Outgoing_waits_for_acceptance_and_passes_server_policy_to_native_engine(bool relay)
|
||||
{
|
||||
var signaling = new Signaling(relay);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
vm.AllowAddressDisclosure = !relay;
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
Assert.True(vm.Busy);
|
||||
Assert.Null(media.Configuration);
|
||||
signaling.Ready();
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
Assert.Equal(relay, media.Configuration!.RelayOnly);
|
||||
Assert.True(media.Configuration.Caller);
|
||||
Assert.Same(Turn, media.Configuration.IceServers);
|
||||
media.Emit(new("offer", Sdp: Sdp));
|
||||
await WaitAsync(() => signaling.Sent.Count == 1);
|
||||
signaling.Receive(new("answer", Sdp: Sdp));
|
||||
await WaitAsync(() => media.Received.Count == 1);
|
||||
await vm.HangupAsync();
|
||||
Assert.True(media.Disposed);
|
||||
Assert.Null(vm.LocalVideo);
|
||||
Assert.False(vm.Busy);
|
||||
Assert.Contains(signaling.Call.Id, signaling.EndedCalls);
|
||||
}
|
||||
|
||||
[AvaloniaFact]
|
||||
public async Task Incoming_relay_needs_acceptance_but_never_requires_IP_disclosure()
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
signaling.Invite();
|
||||
await WaitAsync(() => vm.Incoming);
|
||||
Assert.False(vm.AllowAddressDisclosure);
|
||||
Assert.Null(media.Configuration);
|
||||
await vm.AcceptCommand.ExecuteAsync(null);
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
Assert.False(media.Configuration!.Caller);
|
||||
media.Report(new("connected", "relay", 20, 20));
|
||||
await WaitAsync(() => vm.Status.Contains("IP masquee"));
|
||||
}
|
||||
|
||||
[AvaloniaFact]
|
||||
public async Task Direct_call_without_consent_ends_before_any_capture()
|
||||
{
|
||||
var signaling = new Signaling(false);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
Assert.Null(media.Configuration);
|
||||
Assert.False(vm.Busy);
|
||||
Assert.Contains("consentez", vm.Status);
|
||||
Assert.Single(signaling.EndedCalls);
|
||||
}
|
||||
|
||||
[AvaloniaTheory]
|
||||
[InlineData(false)]
|
||||
[InlineData(true)]
|
||||
public async Task Missing_TURN_or_policy_downgrade_never_creates_a_native_engine(bool downgrade)
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var created = false;
|
||||
await using var vm = new WebRtcCallViewModel(signaling, () =>
|
||||
{
|
||||
created = true;
|
||||
throw new InvalidOperationException("Should not create media");
|
||||
})
|
||||
{ Camera = new("Camera", "camera"), Microphone = new("Mic", "default"), Target = "bob", Room = "General" };
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
signaling.Ready(downgrade ? signaling.Call with { RelayOnly = false } : null,
|
||||
downgrade ? Turn : []);
|
||||
await WaitAsync(() => !vm.Busy);
|
||||
Assert.False(created);
|
||||
Assert.Single(signaling.EndedCalls);
|
||||
}
|
||||
|
||||
[AvaloniaTheory]
|
||||
[InlineData("hangup")]
|
||||
[InlineData("interrupted")]
|
||||
[InlineData("ended")]
|
||||
[InlineData("background")]
|
||||
[InlineData("dispose")]
|
||||
public async Task All_termination_paths_cancel_in_progress_native_startup(string reason)
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var media = new Media { DelayStartup = true };
|
||||
await using var vm = Configure(signaling, media);
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
signaling.Ready();
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
switch (reason)
|
||||
{
|
||||
case "hangup": await vm.HangupAsync(); break;
|
||||
case "interrupted": signaling.Disconnect(); break;
|
||||
case "ended": signaling.End(); break;
|
||||
case "background": Platform.SuspendPlayback(); break;
|
||||
case "dispose": await vm.DisposeAsync(); break;
|
||||
}
|
||||
await WaitAsync(() => media.Disposed);
|
||||
Assert.False(vm.Busy);
|
||||
Assert.Null(vm.RemoteVideo);
|
||||
Assert.True(media.StartupCancelled);
|
||||
}
|
||||
|
||||
[AvaloniaTheory]
|
||||
[InlineData("host")]
|
||||
[InlineData("srflx")]
|
||||
[InlineData(null)]
|
||||
public async Task Nonrelay_selected_path_is_rejected_and_media_is_stopped(string? candidateType)
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
signaling.Ready();
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
media.Report(new("connected", candidateType, 20, 20));
|
||||
await WaitAsync(() => media.Disposed);
|
||||
Assert.Contains("chemin ICE", vm.Status);
|
||||
}
|
||||
|
||||
[AvaloniaFact]
|
||||
public async Task Leaking_candidate_is_not_sent_and_fails_closed()
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
signaling.Ready();
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
media.Emit(new("candidate", Candidate: "candidate:1 1 UDP 1 192.0.2.1 1234 typ host",
|
||||
SdpMid: "0", SdpMLineIndex: 0));
|
||||
await WaitAsync(() => media.Disposed);
|
||||
Assert.Empty(signaling.Sent);
|
||||
Assert.Contains("candidat", vm.Status);
|
||||
}
|
||||
|
||||
[AvaloniaFact]
|
||||
public async Task Native_failure_is_visible_and_ends_the_server_call()
|
||||
{
|
||||
var signaling = new Signaling(true);
|
||||
var media = new Media();
|
||||
await using var vm = Configure(signaling, media);
|
||||
await vm.InviteCommand.ExecuteAsync(null);
|
||||
signaling.Ready();
|
||||
await WaitAsync(() => media.Configuration is not null);
|
||||
media.Fail();
|
||||
await WaitAsync(() => media.Disposed);
|
||||
Assert.Contains("native failure", vm.Status);
|
||||
Assert.Single(signaling.EndedCalls);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ host")]
|
||||
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ srflx")]
|
||||
[InlineData("candidate:1 1 UDP 1 192.0.2.1 1234 typ relay raddr 198.51.100.1 rport 1234")]
|
||||
public void Privacy_rejects_nonrelay_or_related_address_leaks(string candidate) =>
|
||||
Assert.Throws<InvalidOperationException>(() => WebRtcPrivacy.ValidateSignal(
|
||||
new("candidate", Candidate: candidate, SdpMid: "0", SdpMLineIndex: 0), true));
|
||||
|
||||
[Fact]
|
||||
public void Privacy_accepts_redacted_relay_and_rejects_SDP_address_leaks()
|
||||
{
|
||||
WebRtcPrivacy.ValidateSignal(new("candidate",
|
||||
Candidate: "candidate:1 1 UDP 1 192.0.2.1 1234 typ relay raddr 0.0.0.0 rport 0",
|
||||
SdpMid: "0", SdpMLineIndex: 0), true);
|
||||
WebRtcPrivacy.ValidateSignal(new("offer", Sdp: Sdp), true);
|
||||
Assert.Throws<InvalidOperationException>(() =>
|
||||
WebRtcPrivacy.ValidateSignal(new("offer", Sdp: Sdp.Replace("0.0.0.0", "198.51.100.1")), true));
|
||||
}
|
||||
|
||||
public static bool NativeMediaAvailable => OperatingSystem.IsLinux() &&
|
||||
Environment.GetEnvironmentVariable("POSTIT_TEST_NATIVE_WEBRTC") == "1";
|
||||
|
||||
[AvaloniaFact(Skip = "Requires Linux GStreamer/libnice and POSTIT_TEST_NATIVE_WEBRTC=1",
|
||||
SkipUnless = nameof(NativeMediaAvailable))]
|
||||
public async Task Linux_native_adapters_render_remote_bitmaps_and_receive_audio_in_both_directions()
|
||||
{
|
||||
await using var caller = new LinuxWebRtcMedia();
|
||||
await using var callee = new LinuxWebRtcMedia();
|
||||
var errors = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
|
||||
timeout.CancelAfter(TimeSpan.FromSeconds(30));
|
||||
void Forward(IWebRtcMedia destination, CallSignal signal) => Dispatcher.UIThread.Post(async () =>
|
||||
{
|
||||
try { await destination.HandleSignalAsync(signal, timeout.Token); }
|
||||
catch (Exception error) { errors.TrySetException(error); }
|
||||
});
|
||||
caller.Signal += signal => Forward(callee, signal);
|
||||
callee.Signal += signal => Forward(caller, signal);
|
||||
WebRtcMediaStatus? callerStatus = null, calleeStatus = null;
|
||||
caller.StatusChanged += status => callerStatus = status;
|
||||
callee.StatusChanged += status => calleeStatus = status;
|
||||
await callee.StartAsync(new([], false, false, "test", "test"), timeout.Token);
|
||||
await caller.StartAsync(new([], false, true, "test", "test"), timeout.Token);
|
||||
while (callerStatus is not { HasRemoteMedia: true } || calleeStatus is not { HasRemoteMedia: true } ||
|
||||
((Image)caller.RemoteVideo).Source is not Bitmap || ((Image)callee.RemoteVideo).Source is not Bitmap)
|
||||
{
|
||||
if (errors.Task.IsCompleted) await errors.Task;
|
||||
if (caller.Completion.IsCompleted) await caller.Completion;
|
||||
if (callee.Completion.IsCompleted) await callee.Completion;
|
||||
await Task.Delay(20, timeout.Token);
|
||||
}
|
||||
Assert.Equal(640, ((Bitmap)((Image)caller.RemoteVideo).Source!).PixelSize.Width);
|
||||
Assert.Equal(360, ((Bitmap)((Image)callee.RemoteVideo).Source!).PixelSize.Height);
|
||||
}
|
||||
|
||||
private static WebRtcCallViewModel Configure(Signaling signaling, Media media) => new(signaling, () => media)
|
||||
{ Camera = new("Camera", "camera"), Microphone = new("Mic", "default"), Target = "bob", Room = "General" };
|
||||
|
||||
private static async Task WaitAsync(Func<bool> condition)
|
||||
{
|
||||
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
|
||||
timeout.CancelAfter(TimeSpan.FromSeconds(5));
|
||||
while (!condition()) await Task.Delay(10, timeout.Token);
|
||||
}
|
||||
|
||||
private sealed class Media : IWebRtcMedia
|
||||
{
|
||||
private readonly TaskCompletionSource _completion = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
public Control LocalVideo { get; } = new Border();
|
||||
public Control RemoteVideo { get; } = new Border();
|
||||
public event Action<CallSignal>? Signal;
|
||||
public event Action<WebRtcMediaStatus>? StatusChanged;
|
||||
public Task Completion => _completion.Task;
|
||||
public WebRtcMediaConfiguration? Configuration { get; private set; }
|
||||
public bool Disposed { get; private set; }
|
||||
public bool StartupCancelled { get; private set; }
|
||||
public bool DelayStartup { get; init; }
|
||||
public List<CallSignal> Received { get; } = [];
|
||||
public async Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
|
||||
{
|
||||
Configuration = configuration;
|
||||
if (DelayStartup)
|
||||
{
|
||||
try { await Task.Delay(Timeout.Infinite, ct); }
|
||||
catch (OperationCanceledException) when (ct.IsCancellationRequested)
|
||||
{ StartupCancelled = true; throw; }
|
||||
}
|
||||
}
|
||||
public Task HandleSignalAsync(CallSignal signal, CancellationToken ct)
|
||||
{ Received.Add(signal); return Task.CompletedTask; }
|
||||
public void Emit(CallSignal signal) => Signal?.Invoke(signal);
|
||||
public void Report(WebRtcMediaStatus status) => StatusChanged?.Invoke(status);
|
||||
public void Fail() => _completion.TrySetException(new IOException("native failure"));
|
||||
public ValueTask DisposeAsync()
|
||||
{ Disposed = true; _completion.TrySetResult(); return ValueTask.CompletedTask; }
|
||||
}
|
||||
|
||||
private sealed class Signaling(bool relay) : IWebRtcCallSignaling
|
||||
{
|
||||
public event Action<CallSession>? Invitation;
|
||||
public event Action<CallAccepted>? Accepted;
|
||||
public event Action<CallEnded>? Ended;
|
||||
public event Action<Guid, CallSignal>? Signal;
|
||||
public event Action? Interrupted;
|
||||
public CallSession Call { get; } = new(Guid.NewGuid(), "General", "alice", "bob", relay,
|
||||
DateTimeOffset.UtcNow.AddMinutes(1));
|
||||
public List<CallSignal> Sent { get; } = [];
|
||||
public List<Guid> EndedCalls { get; } = [];
|
||||
public Task<CallSession> InviteAsync(string room, string user, CancellationToken ct = default) => Task.FromResult(Call);
|
||||
public Task<CallSession> InviteLanAsync(string room, string user, CancellationToken ct = default) =>
|
||||
throw new InvalidOperationException("Legacy LAN transport must not be used");
|
||||
public Task AcceptAsync(Guid callId, CancellationToken ct = default) { Ready(); return Task.CompletedTask; }
|
||||
public Task EndAsync(Guid callId, CancellationToken ct = default)
|
||||
{ EndedCalls.Add(callId); return Task.CompletedTask; }
|
||||
public Task SendAsync(Guid callId, CallSignal signal, CancellationToken ct = default)
|
||||
{ Sent.Add(signal); return Task.CompletedTask; }
|
||||
public void Invite() => Invitation?.Invoke(Call);
|
||||
public void Ready(CallSession? session = null, CallIceServer[]? servers = null) =>
|
||||
Accepted?.Invoke(new(session ?? Call, servers ?? Turn));
|
||||
public void Receive(CallSignal signal) => Signal?.Invoke(Call.Id, signal);
|
||||
public void End() => Ended?.Invoke(new(Call.Id, "ended"));
|
||||
public void Disconnect() => Interrupted?.Invoke();
|
||||
}
|
||||
}
|
||||
|
|
@ -19,6 +19,7 @@
|
|||
</Content>
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<EmbeddedResource Include="Services/WebRtc/gstreamer_peer.py" LogicalName="PostIt.WebRtc.gstreamer_peer.py" />
|
||||
<EmbeddedResource Include="postit-settings.json">
|
||||
<LogicalName>PostIt.postit-settings.json</LogicalName>
|
||||
</EmbeddedResource>
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ using Yavsc.Abstract.Chat.Calls;
|
|||
|
||||
namespace PostIt.Services;
|
||||
|
||||
public sealed class CallSignalingClient : ILanCallSignaling, IDisposable
|
||||
public sealed class CallSignalingClient : IWebRtcCallSignaling, IDisposable
|
||||
{
|
||||
private readonly HubConnection _connection;
|
||||
private readonly IDisposable[] _subscriptions;
|
||||
|
|
|
|||
32
src/PostIt/PostIt/Services/IWebRtcMedia.cs
Normal file
32
src/PostIt/PostIt/Services/IWebRtcMedia.cs
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
using System;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Avalonia.Controls;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
|
||||
namespace PostIt.Services;
|
||||
|
||||
public sealed record WebRtcMediaConfiguration(CallIceServer[] IceServers, bool RelayOnly,
|
||||
bool Caller, string Camera, string Microphone);
|
||||
|
||||
public sealed record WebRtcMediaStatus(string ConnectionState, string? LocalCandidateType,
|
||||
long AudioPackets, long VideoFrames, bool LocalVideoRendered = false, bool RemoteVideoRendered = false)
|
||||
{
|
||||
public bool HasRemoteMedia => ConnectionState == "connected" && AudioPackets > 0 && VideoFrames > 0;
|
||||
}
|
||||
|
||||
public interface IWebRtcMedia : IAsyncDisposable
|
||||
{
|
||||
Control LocalVideo { get; }
|
||||
Control RemoteVideo { get; }
|
||||
event Action<CallSignal>? Signal;
|
||||
event Action<WebRtcMediaStatus>? StatusChanged;
|
||||
Task Completion { get; }
|
||||
Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct);
|
||||
Task HandleSignalAsync(CallSignal signal, CancellationToken ct);
|
||||
}
|
||||
|
||||
public interface IWebRtcCallSignaling : ILanCallSignaling
|
||||
{
|
||||
Task<CallSession> InviteAsync(string room, string user, CancellationToken ct = default);
|
||||
}
|
||||
208
src/PostIt/PostIt/Services/LinuxWebRtcMedia.cs
Normal file
208
src/PostIt/PostIt/Services/LinuxWebRtcMedia.cs
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
using System;
|
||||
using System.Diagnostics;
|
||||
using System.IO;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Avalonia.Controls;
|
||||
using Avalonia.Media;
|
||||
using Avalonia.Media.Imaging;
|
||||
using Avalonia.Threading;
|
||||
|
||||
namespace PostIt.Services;
|
||||
|
||||
public sealed class LinuxWebRtcMedia : WebRtcNativeMedia
|
||||
{
|
||||
private readonly Image _local = new() { Stretch = Stretch.Uniform };
|
||||
private readonly Image _remote = new() { Stretch = Stretch.Uniform };
|
||||
private readonly SemaphoreSlim _input = new(1);
|
||||
private readonly CancellationTokenSource _lifetime = new();
|
||||
private readonly StringBuilder _errors = new();
|
||||
private readonly object _frameLock = new();
|
||||
private byte[]? _localFrame;
|
||||
private byte[]? _remoteFrame;
|
||||
private bool _frameScheduled;
|
||||
private Bitmap? _localBitmap;
|
||||
private Bitmap? _remoteBitmap;
|
||||
private Process? _process;
|
||||
private Task? _reader;
|
||||
private Task? _stderr;
|
||||
private string? _script;
|
||||
private bool _disposed;
|
||||
|
||||
public override Control LocalVideo => _local;
|
||||
public override Control RemoteVideo => _remote;
|
||||
|
||||
public static PodcastCaptureDevices GetDevices() => new(
|
||||
Directory.GetFiles("/dev", "video*").OrderBy(p => p).Select(p => new CaptureDevice(p, p)).ToArray(),
|
||||
[new("Microphone système (PulseAudio/PipeWire)", "default")]);
|
||||
|
||||
protected override async Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
|
||||
{
|
||||
if (!OperatingSystem.IsLinux()) throw new PlatformNotSupportedException("GStreamer requiert Linux.");
|
||||
_script = Path.Combine(Path.GetTempPath(), $"postit-webrtc-{Guid.NewGuid():N}.py");
|
||||
await using (var resource = typeof(LinuxWebRtcMedia).Assembly.GetManifestResourceStream("PostIt.WebRtc.gstreamer_peer.py")
|
||||
?? throw new InvalidOperationException("Adaptateur GStreamer introuvable."))
|
||||
await using (var file = new FileStream(_script, FileMode.CreateNew, FileAccess.Write, FileShare.None))
|
||||
await resource.CopyToAsync(file, ct);
|
||||
_process = new Process
|
||||
{
|
||||
StartInfo = new ProcessStartInfo("python3")
|
||||
{
|
||||
ArgumentList = { "-u", _script },
|
||||
UseShellExecute = false,
|
||||
RedirectStandardInput = true,
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
CreateNoWindow = true
|
||||
}
|
||||
};
|
||||
if (!_process.Start()) throw new InvalidOperationException("Démarrage du moteur GStreamer impossible.");
|
||||
_stderr = ReadErrorsAsync(_process);
|
||||
_reader = ReadAsync(_process);
|
||||
await SendCommandAsync(StartCommand(configuration), ct);
|
||||
}
|
||||
|
||||
protected override async Task SendCommandAsync(string json, CancellationToken ct)
|
||||
{
|
||||
await _input.WaitAsync(ct);
|
||||
try
|
||||
{
|
||||
if (_process is null || _process.HasExited)
|
||||
throw new InvalidOperationException("Le moteur GStreamer est arrêté. Vérifiez Python GI, GStreamer WebRTC et libnice.");
|
||||
await _process.StandardInput.WriteLineAsync(json.AsMemory(), ct);
|
||||
await _process.StandardInput.FlushAsync(ct);
|
||||
}
|
||||
finally { _input.Release(); }
|
||||
}
|
||||
|
||||
private async Task ReadErrorsAsync(Process process)
|
||||
{
|
||||
while (await process.StandardError.ReadLineAsync(_lifetime.Token) is { } line)
|
||||
{
|
||||
lock (_errors)
|
||||
{
|
||||
_errors.AppendLine(line);
|
||||
if (_errors.Length > 4096) _errors.Remove(0, _errors.Length - 4096);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async Task ReadAsync(Process process)
|
||||
{
|
||||
try
|
||||
{
|
||||
while (await process.StandardOutput.ReadLineAsync(_lifetime.Token) is { } line)
|
||||
{
|
||||
if (line.Length > 1500000) throw new InvalidOperationException("Trame native trop volumineuse.");
|
||||
ProcessMessage(line);
|
||||
}
|
||||
await process.WaitForExitAsync(_lifetime.Token);
|
||||
if (!_disposed)
|
||||
{
|
||||
if (_stderr is not null) await _stderr;
|
||||
string details;
|
||||
lock (_errors) details = _errors.ToString();
|
||||
throw new InvalidOperationException($"Le moteur GStreamer s'est arrêté ({process.ExitCode}). {details}");
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
|
||||
catch (Exception error) { Fail(error); }
|
||||
}
|
||||
|
||||
protected override void OnFrame(JsonElement frame)
|
||||
{
|
||||
var image = frame.GetProperty("data").GetBytesFromBase64();
|
||||
var target = frame.GetProperty("target").GetString();
|
||||
if (target is not ("local" or "remote"))
|
||||
throw new InvalidOperationException("Destination de trame WebRTC inconnue.");
|
||||
lock (_frameLock)
|
||||
{
|
||||
if (_disposed) return;
|
||||
if (target == "local") _localFrame = image;
|
||||
else _remoteFrame = image;
|
||||
if (_frameScheduled) return;
|
||||
_frameScheduled = true;
|
||||
}
|
||||
Dispatcher.UIThread.Post(DisplayFrames);
|
||||
}
|
||||
|
||||
private void DisplayFrames()
|
||||
{
|
||||
byte[]? local;
|
||||
byte[]? remote;
|
||||
lock (_frameLock)
|
||||
{
|
||||
_frameScheduled = false;
|
||||
local = _localFrame;
|
||||
remote = _remoteFrame;
|
||||
_localFrame = _remoteFrame = null;
|
||||
}
|
||||
if (_disposed) return;
|
||||
try
|
||||
{
|
||||
if (local is not null) Display(_local, local, ref _localBitmap);
|
||||
if (remote is not null) Display(_remote, remote, ref _remoteBitmap);
|
||||
}
|
||||
catch (Exception error) { Fail(error); }
|
||||
}
|
||||
|
||||
private static void Display(Image view, byte[] bytes, ref Bitmap? previous)
|
||||
{
|
||||
using var stream = new MemoryStream(bytes, writable: false);
|
||||
var image = new Bitmap(stream);
|
||||
view.Source = image;
|
||||
previous?.Dispose();
|
||||
previous = image;
|
||||
}
|
||||
|
||||
public override async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_disposed) return;
|
||||
_disposed = true;
|
||||
Exception? stopError = null;
|
||||
try
|
||||
{
|
||||
if (_process is { HasExited: false } process)
|
||||
{
|
||||
try { await SendCommandAsync("{\"command\":\"close\"}", CancellationToken.None); }
|
||||
catch (Exception error) when (error is IOException or InvalidOperationException)
|
||||
{
|
||||
stopError = error;
|
||||
}
|
||||
try { await process.WaitForExitAsync().WaitAsync(TimeSpan.FromSeconds(5)); }
|
||||
catch (TimeoutException)
|
||||
{
|
||||
process.Kill(entireProcessTree: true);
|
||||
await process.WaitForExitAsync();
|
||||
throw new TimeoutException("Le moteur GStreamer n'a pas terminé son arrêt ; processus interrompu.");
|
||||
}
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
_lifetime.Cancel();
|
||||
if (_reader is not null) await _reader;
|
||||
if (_stderr is not null)
|
||||
{
|
||||
try { await _stderr; }
|
||||
catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { }
|
||||
}
|
||||
await Dispatcher.UIThread.InvokeAsync(() =>
|
||||
{
|
||||
_local.Source = _remote.Source = null;
|
||||
_localBitmap?.Dispose();
|
||||
_remoteBitmap?.Dispose();
|
||||
});
|
||||
_process?.Dispose();
|
||||
if (_script is not null) File.Delete(_script);
|
||||
_input.Dispose();
|
||||
_lifetime.Dispose();
|
||||
Closed();
|
||||
}
|
||||
if (stopError is not null)
|
||||
throw new InvalidOperationException("La commande d'arret GStreamer a echoue.", stopError);
|
||||
}
|
||||
}
|
||||
|
|
@ -30,6 +30,8 @@ public static class Platform
|
|||
public static Func<IServiceProvider>? CreateServiceProvider { get; set; }
|
||||
public static Func<PodcastCaptureDevices>? GetLanCallDevices { get; set; }
|
||||
public static Func<string, int, ILanCallMedia>? CreateLanCallMedia { get; set; }
|
||||
public static Func<IWebRtcMedia>? CreateWebRtcMedia { get; set; }
|
||||
public static Func<PodcastCaptureDevices>? GetWebRtcDevices { get; set; }
|
||||
public static Func<PodcastCaptureDevices>? GetPodcastCaptureDevices { get; set; }
|
||||
public static Func<string?, CancellationToken, Task>? RequestPodcastPermissionsAsync { get; set; }
|
||||
public static Func<string, string?, string, CancellationToken, Task<IPodcastCapture>>? CreatePodcastCaptureAsync { get; set; }
|
||||
|
|
|
|||
358
src/PostIt/PostIt/Services/WebRtc/gstreamer_peer.py
Normal file
358
src/PostIt/PostIt/Services/WebRtc/gstreamer_peer.py
Normal file
|
|
@ -0,0 +1,358 @@
|
|||
import base64
|
||||
import json
|
||||
import queue
|
||||
import sys
|
||||
import threading
|
||||
from urllib.parse import quote, urlsplit
|
||||
|
||||
import gi
|
||||
|
||||
gi.require_version("Gst", "1.0")
|
||||
gi.require_version("GstApp", "1.0")
|
||||
gi.require_version("GstSdp", "1.0")
|
||||
gi.require_version("GstWebRTC", "1.0")
|
||||
from gi.repository import GLib, Gst, GstSdp, GstWebRTC
|
||||
|
||||
Gst.init(None)
|
||||
|
||||
|
||||
class Peer:
|
||||
def __init__(self):
|
||||
self.loop = GLib.MainLoop()
|
||||
self.pipeline = None
|
||||
self.rtc = None
|
||||
self.outgoing = queue.Queue(64)
|
||||
self.frames = {}
|
||||
self.frame_lock = threading.Lock()
|
||||
self.local_sent = False
|
||||
self.remote_set = False
|
||||
self.pending_local = []
|
||||
self.pending_remote = []
|
||||
self.mids = {}
|
||||
self.closed = False
|
||||
self.audio_packets = 0
|
||||
self.video_frames = 0
|
||||
self.writer = threading.Thread(target=self.write, daemon=True)
|
||||
self.writer.start()
|
||||
|
||||
def emit(self, message):
|
||||
try:
|
||||
self.outgoing.put_nowait(message)
|
||||
except queue.Full:
|
||||
raise RuntimeError("WebRTC event queue overflow")
|
||||
|
||||
def write(self):
|
||||
while True:
|
||||
message = self.outgoing.get()
|
||||
print(json.dumps(message, separators=(",", ":")), flush=True)
|
||||
if message["type"] == "closed":
|
||||
return
|
||||
|
||||
def fail(self, message):
|
||||
self.emit({"type": "error", "message": str(message)})
|
||||
if self.pipeline:
|
||||
self.pipeline.set_state(Gst.State.NULL)
|
||||
|
||||
def command(self, message):
|
||||
try:
|
||||
command = message["command"]
|
||||
if command == "start":
|
||||
self.start(message["configuration"])
|
||||
elif command == "signal":
|
||||
self.signal(message["signal"])
|
||||
elif command == "close":
|
||||
self.close()
|
||||
else:
|
||||
raise ValueError("Unknown WebRTC command")
|
||||
except Exception as error:
|
||||
self.fail(error)
|
||||
return GLib.SOURCE_REMOVE
|
||||
|
||||
def start(self, configuration):
|
||||
if self.pipeline is not None:
|
||||
raise RuntimeError("WebRTC already started")
|
||||
for plugin in ("webrtcbin", "nicesrc", "vp8enc", "opusenc", "jpegenc"):
|
||||
if not Gst.ElementFactory.find(plugin):
|
||||
raise RuntimeError("Missing GStreamer plugin: " + plugin)
|
||||
synthetic = configuration["camera"] == "test"
|
||||
camera = "videotestsrc is-live=true" if synthetic else "v4l2src name=camera"
|
||||
microphone = "audiotestsrc is-live=true" if synthetic else "pulsesrc name=microphone"
|
||||
self.pipeline = Gst.parse_launch(
|
||||
"webrtcbin name=rtc bundle-policy=max-bundle latency=100 "
|
||||
+ camera + " ! videoconvert ! videoscale ! videorate ! "
|
||||
"video/x-raw,width=640,height=360,framerate=15/1 ! tee name=local "
|
||||
"local. ! queue leaky=downstream max-size-buffers=2 ! "
|
||||
"vp8enc deadline=1 target-bitrate=900000 keyframe-max-dist=30 ! "
|
||||
"rtpvp8pay pt=96 ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! rtc. "
|
||||
"local. ! queue leaky=downstream max-size-buffers=1 ! videorate ! "
|
||||
"video/x-raw,framerate=10/1 ! jpegenc ! appsink name=local_frames "
|
||||
"emit-signals=true sync=false max-buffers=1 drop=true "
|
||||
+ microphone + " ! audioconvert ! audioresample ! audio/x-raw,rate=48000,channels=1 ! "
|
||||
"opusenc ! rtpopuspay pt=111 ! application/x-rtp,media=audio,encoding-name=OPUS,payload=111 ! rtc."
|
||||
)
|
||||
if not synthetic:
|
||||
self.pipeline.get_by_name("camera").set_property("device", configuration["camera"])
|
||||
if configuration["microphone"] != "default":
|
||||
self.pipeline.get_by_name("microphone").set_property("device", configuration["microphone"])
|
||||
self.synthetic = synthetic
|
||||
self.relay_only = configuration["relayOnly"]
|
||||
self.rtc = self.pipeline.get_by_name("rtc")
|
||||
self.ice = self.rtc.get_property("ice-agent")
|
||||
self.nice = self.ice.get_property("agent")
|
||||
self.nice.set_property("upnp", False)
|
||||
if configuration["relayOnly"]:
|
||||
self.rtc.set_property("ice-transport-policy", GstWebRTC.WebRTCICETransportPolicy.RELAY)
|
||||
self.nice.set_property("force-relay", True)
|
||||
for server in configuration["iceServers"]:
|
||||
for url in server["urls"]:
|
||||
if url.startswith(("turn:", "turns:")):
|
||||
parsed = urlsplit(url.replace(":", "://", 1))
|
||||
credentials = quote(server["username"], safe="") + ":" + quote(server["credential"], safe="")
|
||||
turn = parsed.scheme + "://" + credentials + "@" + parsed.netloc
|
||||
if parsed.query:
|
||||
turn += "?" + parsed.query
|
||||
if not self.rtc.emit("add-turn-server", turn):
|
||||
raise RuntimeError("Invalid TURN server configuration")
|
||||
elif not configuration["relayOnly"] and url.startswith("stun:"):
|
||||
self.rtc.set_property("stun-server", url.replace(":", "://", 1))
|
||||
elif not configuration["relayOnly"] and url.startswith("stuns:"):
|
||||
raise RuntimeError("GStreamer does not support STUN/TLS; configure STUN or TURN/TLS")
|
||||
self.pipeline.get_by_name("local_frames").connect("new-sample", self.frame, "local")
|
||||
self.rtc.connect("on-ice-candidate", self.candidate)
|
||||
self.rtc.connect("pad-added", self.incoming)
|
||||
bus = self.pipeline.get_bus()
|
||||
bus.add_signal_watch()
|
||||
bus.connect("message::error", self.bus_error)
|
||||
if configuration["caller"]:
|
||||
self.rtc.connect("on-negotiation-needed", self.offer)
|
||||
if self.pipeline.set_state(Gst.State.PLAYING) == Gst.StateChangeReturn.FAILURE:
|
||||
raise RuntimeError("Cannot start GStreamer camera/microphone pipeline")
|
||||
GLib.timeout_add(100, self.flush_frames)
|
||||
GLib.timeout_add(1000, self.stats)
|
||||
|
||||
def bus_error(self, bus, message):
|
||||
error, _ = message.parse_error()
|
||||
self.fail(error.message)
|
||||
|
||||
def offer(self, rtc):
|
||||
GLib.idle_add(self.create_description, "offer")
|
||||
|
||||
def promise(self, callback):
|
||||
def changed(promise, *_):
|
||||
try:
|
||||
reply = promise.get_reply()
|
||||
if reply and reply.has_field("error"):
|
||||
raise RuntimeError(reply.get_value("error").message)
|
||||
# The reply is owned by the promise until this callback returns.
|
||||
GLib.idle_add(callback, reply.copy() if reply is not None else None)
|
||||
except Exception as error:
|
||||
GLib.idle_add(self.fail, str(error))
|
||||
return Gst.Promise.new_with_change_func(changed, None, None)
|
||||
|
||||
def create_description(self, kind):
|
||||
if self.closed:
|
||||
return GLib.SOURCE_REMOVE
|
||||
def created(reply):
|
||||
try:
|
||||
description = reply.get_value(kind)
|
||||
sdp_message = description.sdp
|
||||
sdp = sdp_message.as_text()
|
||||
mids = {}
|
||||
index = -1
|
||||
for line in sdp.splitlines():
|
||||
if line.startswith("m="):
|
||||
index += 1
|
||||
elif line.startswith("a=mid:"):
|
||||
mids[index] = line[6:]
|
||||
def applied(_):
|
||||
try:
|
||||
self.mids = mids
|
||||
self.emit({"type": "signal", "signal": {"kind": kind, "sdp": sdp}})
|
||||
self.local_sent = True
|
||||
for index, candidate in self.pending_local:
|
||||
self.send_candidate(index, candidate)
|
||||
self.pending_local.clear()
|
||||
except Exception as error:
|
||||
self.fail(error)
|
||||
return GLib.SOURCE_REMOVE
|
||||
self.rtc.emit("set-local-description", description, self.promise(applied))
|
||||
except Exception as error:
|
||||
self.fail(error)
|
||||
return GLib.SOURCE_REMOVE
|
||||
self.rtc.emit("create-" + kind, None, self.promise(created))
|
||||
return GLib.SOURCE_REMOVE
|
||||
|
||||
def candidate(self, rtc, index, candidate):
|
||||
GLib.idle_add(self.local_candidate, index, candidate)
|
||||
|
||||
def local_candidate(self, index, candidate):
|
||||
if self.closed or not candidate:
|
||||
return GLib.SOURCE_REMOVE
|
||||
if self.local_sent:
|
||||
self.send_candidate(index, candidate)
|
||||
elif len(self.pending_local) < 256:
|
||||
self.pending_local.append((index, candidate))
|
||||
else:
|
||||
self.fail("Too many local ICE candidates")
|
||||
return GLib.SOURCE_REMOVE
|
||||
|
||||
def send_candidate(self, index, candidate):
|
||||
if self.relay_only:
|
||||
parts = candidate.split()
|
||||
if len(parts) < 8 or parts[6:8] != ["typ", "relay"]:
|
||||
self.fail("Relay-only ICE produced a non-relay candidate")
|
||||
return
|
||||
# RFC 8828: libnice exposes the base address in relay metadata.
|
||||
# Hide only that metadata; never rewrite SDP or the relay endpoint.
|
||||
for position in range(8, len(parts) - 1, 2):
|
||||
if parts[position] == "raddr":
|
||||
parts[position + 1] = "::" if ":" in parts[4] else "0.0.0.0"
|
||||
elif parts[position] == "rport":
|
||||
parts[position + 1] = "0"
|
||||
candidate = " ".join(parts)
|
||||
self.emit({"type": "signal", "signal": {
|
||||
"kind": "candidate", "candidate": candidate,
|
||||
"sdpMid": self.mids[index], "sdpMLineIndex": index}})
|
||||
|
||||
def signal(self, signal):
|
||||
if signal["kind"] == "candidate":
|
||||
if self.remote_set:
|
||||
self.rtc.emit("add-ice-candidate", signal["sdpMLineIndex"], signal["candidate"])
|
||||
elif len(self.pending_remote) < 256:
|
||||
self.pending_remote.append(signal)
|
||||
else:
|
||||
raise RuntimeError("Too many pending remote ICE candidates")
|
||||
else:
|
||||
result, sdp = GstSdp.SDPMessage.new_from_text(signal["sdp"])
|
||||
if result != GstSdp.SDPResult.OK:
|
||||
raise ValueError("Invalid remote SDP")
|
||||
kind = GstWebRTC.WebRTCSDPType.OFFER if signal["kind"] == "offer" else GstWebRTC.WebRTCSDPType.ANSWER
|
||||
description = GstWebRTC.WebRTCSessionDescription.new(kind, sdp)
|
||||
def applied(_):
|
||||
self.remote_set = True
|
||||
for candidate in self.pending_remote:
|
||||
self.rtc.emit("add-ice-candidate", candidate["sdpMLineIndex"], candidate["candidate"])
|
||||
self.pending_remote.clear()
|
||||
if signal["kind"] == "offer":
|
||||
self.create_description("answer")
|
||||
return GLib.SOURCE_REMOVE
|
||||
self.rtc.emit("set-remote-description", description, self.promise(applied))
|
||||
|
||||
def incoming(self, rtc, pad):
|
||||
if pad.get_direction() != Gst.PadDirection.SRC:
|
||||
return
|
||||
decoder = Gst.ElementFactory.make("decodebin")
|
||||
decoder.connect("pad-added", self.decoded)
|
||||
self.pipeline.add(decoder)
|
||||
decoder.sync_state_with_parent()
|
||||
if pad.link(decoder.get_static_pad("sink")) != Gst.PadLinkReturn.OK:
|
||||
GLib.idle_add(self.fail, "Cannot attach WebRTC decoder")
|
||||
|
||||
def decoded(self, decoder, pad):
|
||||
media = pad.get_current_caps().get_structure(0).get_name()
|
||||
if media.startswith("video/"):
|
||||
sink = Gst.parse_bin_from_description(
|
||||
"queue leaky=downstream max-size-buffers=2 ! videoconvert ! videoscale ! videorate ! "
|
||||
"video/x-raw,width=640,height=360,framerate=10/1 ! jpegenc ! "
|
||||
"appsink name=remote_frames emit-signals=true sync=false max-buffers=1 drop=true", True)
|
||||
sink.get_by_name("remote_frames").connect("new-sample", self.frame, "remote")
|
||||
elif media.startswith("audio/"):
|
||||
sink = Gst.parse_bin_from_description(
|
||||
"queue ! audioconvert ! audioresample ! "
|
||||
+ ("fakesink" if self.synthetic else "autoaudiosink") + " name=remote_audio sync=false", True)
|
||||
sink.get_by_name("remote_audio").get_static_pad("sink").add_probe(
|
||||
Gst.PadProbeType.BUFFER, self.audio_buffer)
|
||||
else:
|
||||
GLib.idle_add(self.fail, "Unsupported remote media")
|
||||
return
|
||||
self.pipeline.add(sink)
|
||||
sink.sync_state_with_parent()
|
||||
if pad.link(sink.get_static_pad("sink")) != Gst.PadLinkReturn.OK:
|
||||
GLib.idle_add(self.fail, "Cannot attach decoded media")
|
||||
|
||||
def audio_buffer(self, pad, probe):
|
||||
self.audio_packets += 1
|
||||
return Gst.PadProbeReturn.OK
|
||||
|
||||
def frame(self, sink, target):
|
||||
sample = sink.emit("pull-sample")
|
||||
if sample is None:
|
||||
return Gst.FlowReturn.EOS
|
||||
buffer = sample.get_buffer()
|
||||
image = buffer.extract_dup(0, buffer.get_size())
|
||||
if len(image) > 1024 * 1024:
|
||||
GLib.idle_add(self.fail, "Decoded image exceeds frame limit")
|
||||
return Gst.FlowReturn.ERROR
|
||||
with self.frame_lock:
|
||||
self.frames[target] = base64.b64encode(image).decode("ascii")
|
||||
if target == "remote":
|
||||
self.video_frames += 1
|
||||
return Gst.FlowReturn.OK
|
||||
|
||||
def flush_frames(self):
|
||||
if self.closed:
|
||||
return GLib.SOURCE_REMOVE
|
||||
with self.frame_lock:
|
||||
frames, self.frames = self.frames, {}
|
||||
for target, data in frames.items():
|
||||
# Preview/render frames are disposable, unlike signaling.
|
||||
if self.outgoing.qsize() < 4:
|
||||
self.emit({"type": "frame", "target": target, "data": data})
|
||||
return GLib.SOURCE_CONTINUE
|
||||
|
||||
def stats(self):
|
||||
if self.closed:
|
||||
return GLib.SOURCE_REMOVE
|
||||
def received(reply):
|
||||
if self.closed:
|
||||
return GLib.SOURCE_REMOVE
|
||||
try:
|
||||
all_stats = {reply.nth_field_name(i): reply.get_value(reply.nth_field_name(i))
|
||||
for i in range(reply.n_fields())}
|
||||
selected_id = None
|
||||
for item in all_stats.values():
|
||||
if isinstance(item, Gst.Structure) and item.has_field("selected-candidate-pair-id"):
|
||||
selected_id = item.get_value("selected-candidate-pair-id")
|
||||
pair = all_stats.get(selected_id)
|
||||
local = all_stats.get(pair.get_value("local-candidate-id")) if pair else None
|
||||
local_type = local.get_value("candidate-type") if local else None
|
||||
state = self.rtc.get_property("connection-state").value_nick
|
||||
if state == "connected" and local_type is None:
|
||||
state = "checking"
|
||||
self.emit({"type": "status", "status": {
|
||||
"connectionState": state, "localCandidateType": local_type,
|
||||
"audioPackets": self.audio_packets, "videoFrames": self.video_frames}})
|
||||
except Exception as error:
|
||||
self.fail(error)
|
||||
return GLib.SOURCE_REMOVE
|
||||
self.rtc.emit("get-stats", None, self.promise(received))
|
||||
return GLib.SOURCE_CONTINUE
|
||||
|
||||
def close(self):
|
||||
self.closed = True
|
||||
if self.pipeline:
|
||||
self.pipeline.set_state(Gst.State.NULL)
|
||||
self.emit({"type": "closed"})
|
||||
self.loop.quit()
|
||||
|
||||
|
||||
def main():
|
||||
peer = Peer()
|
||||
|
||||
def read():
|
||||
try:
|
||||
for line in sys.stdin:
|
||||
if len(line) > 65536:
|
||||
raise ValueError("WebRTC command exceeds size limit")
|
||||
GLib.idle_add(peer.command, json.loads(line))
|
||||
except Exception as error:
|
||||
GLib.idle_add(peer.fail, str(error))
|
||||
GLib.idle_add(peer.close)
|
||||
|
||||
threading.Thread(target=read, daemon=True).start()
|
||||
peer.loop.run()
|
||||
peer.writer.join(timeout=3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
78
src/PostIt/PostIt/Services/WebRtcNativeMedia.cs
Normal file
78
src/PostIt/PostIt/Services/WebRtcNativeMedia.cs
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
using System;
|
||||
using System.Text.Json;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Avalonia.Controls;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
|
||||
namespace PostIt.Services;
|
||||
|
||||
public abstract class WebRtcNativeMedia : IWebRtcMedia
|
||||
{
|
||||
protected static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
|
||||
private readonly TaskCompletionSource _completion = new(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
protected WebRtcMediaConfiguration? Configuration { get; private set; }
|
||||
public abstract Control LocalVideo { get; }
|
||||
public abstract Control RemoteVideo { get; }
|
||||
public event Action<CallSignal>? Signal;
|
||||
public event Action<WebRtcMediaStatus>? StatusChanged;
|
||||
public Task Completion => _completion.Task;
|
||||
|
||||
public async Task StartAsync(WebRtcMediaConfiguration configuration, CancellationToken ct)
|
||||
{
|
||||
if (Configuration is not null) throw new InvalidOperationException("Le moteur WebRTC est déjà démarré.");
|
||||
WebRtcPrivacy.ValidateConfiguration(configuration);
|
||||
Configuration = configuration;
|
||||
await StartCoreAsync(configuration, ct);
|
||||
}
|
||||
|
||||
protected abstract Task StartCoreAsync(WebRtcMediaConfiguration configuration, CancellationToken ct);
|
||||
protected abstract Task SendCommandAsync(string json, CancellationToken ct);
|
||||
public abstract ValueTask DisposeAsync();
|
||||
|
||||
public Task HandleSignalAsync(CallSignal signal, CancellationToken ct)
|
||||
{
|
||||
// Privacy constrains our own addresses, not a peer's public TURN address.
|
||||
WebRtcPrivacy.ValidateSignal(signal, false);
|
||||
return SendCommandAsync(JsonSerializer.Serialize(new { command = "signal", signal }, JsonOptions), ct);
|
||||
}
|
||||
|
||||
protected void ProcessMessage(string json)
|
||||
{
|
||||
using var document = JsonDocument.Parse(json);
|
||||
var root = document.RootElement;
|
||||
switch (root.GetProperty("type").GetString())
|
||||
{
|
||||
case "signal":
|
||||
var signal = root.GetProperty("signal").Deserialize<CallSignal>(JsonOptions)
|
||||
?? throw new InvalidOperationException("Signal natif vide.");
|
||||
WebRtcPrivacy.ValidateSignal(signal, Configuration!.RelayOnly);
|
||||
Signal?.Invoke(signal);
|
||||
break;
|
||||
case "status":
|
||||
var status = root.GetProperty("status").Deserialize<WebRtcMediaStatus>(JsonOptions)
|
||||
?? throw new InvalidOperationException("Statistiques WebRTC vides.");
|
||||
WebRtcPrivacy.ValidateStatus(status, Configuration!.RelayOnly);
|
||||
StatusChanged?.Invoke(status);
|
||||
break;
|
||||
case "error":
|
||||
throw new InvalidOperationException(root.GetProperty("message").GetString());
|
||||
case "closed":
|
||||
_completion.TrySetResult();
|
||||
break;
|
||||
case "frame":
|
||||
OnFrame(root);
|
||||
break;
|
||||
default:
|
||||
throw new InvalidOperationException("Événement natif WebRTC inconnu.");
|
||||
}
|
||||
}
|
||||
|
||||
protected virtual void OnFrame(JsonElement frame) =>
|
||||
throw new InvalidOperationException("Ce moteur ne transmet pas de trames décodées.");
|
||||
|
||||
protected void Fail(Exception error) => _completion.TrySetException(error);
|
||||
protected void Closed() => _completion.TrySetResult();
|
||||
protected static string StartCommand(WebRtcMediaConfiguration configuration) =>
|
||||
JsonSerializer.Serialize(new { command = "start", configuration }, JsonOptions);
|
||||
}
|
||||
73
src/PostIt/PostIt/Services/WebRtcPrivacy.cs
Normal file
73
src/PostIt/PostIt/Services/WebRtcPrivacy.cs
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
using System;
|
||||
using System.Linq;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
|
||||
namespace PostIt.Services;
|
||||
|
||||
public static class WebRtcPrivacy
|
||||
{
|
||||
public static void ValidateConfiguration(WebRtcMediaConfiguration configuration)
|
||||
{
|
||||
if (configuration.RelayOnly && !configuration.IceServers.Any(server =>
|
||||
server.Urls.Any(IsTurn) && !string.IsNullOrWhiteSpace(server.Username)
|
||||
&& !string.IsNullOrWhiteSpace(server.Credential)))
|
||||
throw new InvalidOperationException("Masquage IP : aucun relais TURN authentifié disponible. Aucun repli direct.");
|
||||
}
|
||||
|
||||
public static bool IsTurn(string url) =>
|
||||
url.StartsWith("turn:", StringComparison.Ordinal) || url.StartsWith("turns:", StringComparison.Ordinal);
|
||||
|
||||
public static void ValidateSignal(CallSignal signal, bool relayOnly)
|
||||
{
|
||||
if (signal.Endpoint is not null || signal.Kind is not ("offer" or "answer" or "candidate"))
|
||||
throw new InvalidOperationException("Signal WebRTC invalide.");
|
||||
if (signal.Kind == "candidate")
|
||||
{
|
||||
if (signal.Sdp is not null || signal.Candidate is not { Length: > 0 and <= 2048 } ||
|
||||
!signal.Candidate.StartsWith("candidate:", StringComparison.Ordinal) ||
|
||||
string.IsNullOrWhiteSpace(signal.SdpMid) || signal.SdpMid.Length > 256 ||
|
||||
signal.SdpMLineIndex is not (>= 0 and <= 16))
|
||||
throw new InvalidOperationException("Candidat ICE invalide.");
|
||||
}
|
||||
else if (signal.Sdp is not { Length: > 0 and <= 24576 } ||
|
||||
!signal.Sdp.StartsWith("v=0\r\n", StringComparison.Ordinal) ||
|
||||
signal.Candidate is not null)
|
||||
throw new InvalidOperationException("Description SDP invalide.");
|
||||
if (!relayOnly) return;
|
||||
if (signal.Candidate is { } candidate) ValidateCandidate(candidate);
|
||||
if (signal.Sdp is not { } sdp) return;
|
||||
var lines = sdp.Split("\r\n", StringSplitOptions.RemoveEmptyEntries);
|
||||
var candidates = lines.Where(l => l.StartsWith("a=candidate:", StringComparison.Ordinal)).ToArray();
|
||||
foreach (var line in candidates) ValidateCandidate(line[2..]);
|
||||
var relayAddresses = candidates.Select(l => l.Split(' ', StringSplitOptions.RemoveEmptyEntries)[4]).ToHashSet();
|
||||
foreach (var line in lines)
|
||||
{
|
||||
if (line.StartsWith("o=", StringComparison.Ordinal) ||
|
||||
line.StartsWith("c=", StringComparison.Ordinal) ||
|
||||
(line.StartsWith("a=rtcp:", StringComparison.Ordinal) && line.Contains(" IN ", StringComparison.Ordinal)))
|
||||
{
|
||||
var address = line.Split(' ', StringSplitOptions.RemoveEmptyEntries).Last();
|
||||
if (address is not ("0.0.0.0" or "::") &&
|
||||
!(line.StartsWith("o=", StringComparison.Ordinal) && address == "127.0.0.1") &&
|
||||
!relayAddresses.Contains(address))
|
||||
throw new InvalidOperationException("Masquage IP : le SDP expose une adresse non relayée.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static void ValidateCandidate(string candidate)
|
||||
{
|
||||
var parts = candidate.Split(' ', StringSplitOptions.RemoveEmptyEntries);
|
||||
if (parts.Length < 8 || parts[6] != "typ" || parts[7] != "relay")
|
||||
throw new InvalidOperationException("Masquage IP : un candidat non relayé a été produit.");
|
||||
for (var i = 8; i < parts.Length; i++)
|
||||
if (parts[i] == "raddr" && (i + 1 >= parts.Length || parts[i + 1] is not ("0.0.0.0" or "::")))
|
||||
throw new InvalidOperationException("Masquage IP : un candidat révèle son adresse associée.");
|
||||
}
|
||||
|
||||
public static void ValidateStatus(WebRtcMediaStatus status, bool relayOnly)
|
||||
{
|
||||
if (relayOnly && status.ConnectionState == "connected" && status.LocalCandidateType != "relay")
|
||||
throw new InvalidOperationException("Masquage IP : le chemin ICE sélectionné n'est pas confirmé comme relayé.");
|
||||
}
|
||||
}
|
||||
|
|
@ -56,11 +56,11 @@ namespace PostIt.ViewModels.Chat
|
|||
|
||||
public CallSignalingClient? Calls { get; private set; }
|
||||
|
||||
public bool SupportsLanCalls => Platform.CreateLanCallMedia != null ||
|
||||
public bool SupportsVideoCalls => Platform.CreateWebRtcMedia != null ||
|
||||
(OperatingSystem.IsLinux() && !OperatingSystem.IsAndroid());
|
||||
|
||||
[ObservableProperty]
|
||||
public partial LanCallViewModel? LanCall { get; set; }
|
||||
public partial WebRtcCallViewModel? VideoCall { get; set; }
|
||||
|
||||
[ObservableProperty]
|
||||
public partial string CurrentRoom { get; set; } = "General";
|
||||
|
|
@ -106,7 +106,7 @@ namespace PostIt.ViewModels.Chat
|
|||
|
||||
partial void OnSelectedRoomChanged(string? value)
|
||||
{
|
||||
if (LanCall != null) LanCall.Room = value ?? "";
|
||||
if (VideoCall != null) VideoCall.Room = value ?? "";
|
||||
FocusedMessages = value != null && _messagesByRoom.TryGetValue(value, out var msgs)
|
||||
? msgs : _systemMessages;
|
||||
FocusedUsers = value != null && _usersByRoom.TryGetValue(value, out var users)
|
||||
|
|
@ -160,7 +160,7 @@ namespace PostIt.ViewModels.Chat
|
|||
if (_transport == null || room == null) return;
|
||||
try
|
||||
{
|
||||
if (LanCall != null) await LanCall.HangupAsync();
|
||||
if (VideoCall != null) await VideoCall.HangupAsync();
|
||||
await _transport.LeaveAsync(room);
|
||||
JoinedRooms.Remove(room);
|
||||
_messagesByRoom.Remove(room);
|
||||
|
|
@ -210,11 +210,11 @@ namespace PostIt.ViewModels.Chat
|
|||
private async Task DisconnectCoreAsync()
|
||||
{
|
||||
Exception? callError = null;
|
||||
if (LanCall != null)
|
||||
if (VideoCall != null)
|
||||
{
|
||||
try { await LanCall.DisposeAsync(); }
|
||||
try { await VideoCall.DisposeAsync(); }
|
||||
catch (Exception error) { callError = error; }
|
||||
finally { LanCall = null; }
|
||||
finally { VideoCall = null; }
|
||||
}
|
||||
Calls?.Dispose();
|
||||
Calls = null;
|
||||
|
|
@ -269,10 +269,10 @@ namespace PostIt.ViewModels.Chat
|
|||
return;
|
||||
if (_connection != null)
|
||||
{
|
||||
if (LanCall != null)
|
||||
if (VideoCall != null)
|
||||
{
|
||||
await LanCall.DisposeAsync();
|
||||
LanCall = null;
|
||||
await VideoCall.DisposeAsync();
|
||||
VideoCall = null;
|
||||
}
|
||||
Calls?.Dispose();
|
||||
Calls = null;
|
||||
|
|
@ -296,8 +296,8 @@ namespace PostIt.ViewModels.Chat
|
|||
// Le hub est [Authorize] : on attache le Bearer OIDC.
|
||||
|
||||
Calls = new CallSignalingClient(_connection);
|
||||
if (SupportsLanCalls)
|
||||
LanCall = new LanCallViewModel(Calls, settings: _settings) { Room = SelectedRoom ?? "" };
|
||||
if (SupportsVideoCalls)
|
||||
VideoCall = new WebRtcCallViewModel(Calls, settings: _settings) { Room = SelectedRoom ?? "" };
|
||||
else
|
||||
Calls.Invitation += call => PostForCurrentSession(() =>
|
||||
Log($"Invitation audio-vidéo de {call.Caller} dans {call.RoomName} ; transport non disponible sur cette plateforme."));
|
||||
|
|
|
|||
364
src/PostIt/PostIt/ViewModels/Chat/WebRtcCallViewModel.cs
Normal file
364
src/PostIt/PostIt/ViewModels/Chat/WebRtcCallViewModel.cs
Normal file
|
|
@ -0,0 +1,364 @@
|
|||
using System;
|
||||
using System.Collections.ObjectModel;
|
||||
using System.Diagnostics;
|
||||
using System.Linq;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Avalonia.Controls;
|
||||
using Avalonia.Threading;
|
||||
using CommunityToolkit.Mvvm.ComponentModel;
|
||||
using CommunityToolkit.Mvvm.Input;
|
||||
using PostIt.Services;
|
||||
using Yavsc.Abstract.Chat.Calls;
|
||||
|
||||
namespace PostIt.ViewModels.Chat;
|
||||
|
||||
public partial class WebRtcCallViewModel : ObservableObject, IAsyncDisposable
|
||||
{
|
||||
private readonly IWebRtcCallSignaling _signaling;
|
||||
private readonly Func<IWebRtcMedia> _createMedia;
|
||||
private readonly Settings? _settings;
|
||||
private readonly SemaphoreSlim _events = new(1);
|
||||
private readonly CancellationTokenSource _lifetime = new();
|
||||
private CancellationTokenSource? _callLifetime;
|
||||
private CallSession? _call;
|
||||
private IWebRtcMedia? _media;
|
||||
private WebRtcMediaStatus? _mediaStatus;
|
||||
private Action<CallSignal>? _sendSignal;
|
||||
private Action<WebRtcMediaStatus>? _receiveStatus;
|
||||
private bool _caller;
|
||||
private bool _accepted;
|
||||
private bool _disposed;
|
||||
private string? _camera;
|
||||
private string? _microphone;
|
||||
|
||||
public ObservableCollection<CaptureDevice> Cameras { get; } = [];
|
||||
public ObservableCollection<CaptureDevice> Microphones { get; } = [];
|
||||
[ObservableProperty] public partial CaptureDevice? Camera { get; set; }
|
||||
[ObservableProperty] public partial CaptureDevice? Microphone { get; set; }
|
||||
[ObservableProperty] public partial bool AllowAddressDisclosure { get; set; }
|
||||
[ObservableProperty] public partial string Target { get; set; } = "";
|
||||
[ObservableProperty] public partial string Room { get; set; } = "";
|
||||
[ObservableProperty] public partial string Status { get; set; } = "Appel WebRTC : aucun appel.";
|
||||
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
|
||||
public partial bool Busy { get; set; }
|
||||
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
|
||||
public partial bool Incoming { get; set; }
|
||||
[ObservableProperty] public partial bool Preparing { get; set; }
|
||||
[ObservableProperty] public partial Control? LocalVideo { get; set; }
|
||||
[ObservableProperty] public partial Control? RemoteVideo { get; set; }
|
||||
public bool CanConfigure => !Busy || Incoming;
|
||||
|
||||
public WebRtcCallViewModel(IWebRtcCallSignaling signaling, Func<IWebRtcMedia>? createMedia = null,
|
||||
Settings? settings = null)
|
||||
{
|
||||
_signaling = signaling;
|
||||
_settings = settings;
|
||||
_createMedia = createMedia ?? Platform.CreateWebRtcMedia ?? (() => new LinuxWebRtcMedia());
|
||||
signaling.Invitation += OnInvitation;
|
||||
signaling.Accepted += OnAccepted;
|
||||
signaling.Signal += OnSignal;
|
||||
signaling.Ended += OnEnded;
|
||||
signaling.Interrupted += OnInterrupted;
|
||||
Platform.PlaybackSuspending += OnSuspending;
|
||||
if (settings is not null)
|
||||
{
|
||||
foreach (var device in settings.PodcastCameras) Cameras.Add(device);
|
||||
foreach (var device in settings.PodcastMicrophones.Where(d => !string.IsNullOrEmpty(d.Value)))
|
||||
Microphones.Add(device);
|
||||
RestoreSelection();
|
||||
}
|
||||
}
|
||||
|
||||
private void RestoreSelection()
|
||||
{
|
||||
Camera = Cameras.FirstOrDefault(d => d.Value == _settings?.PodcastCameraId) ?? Cameras.FirstOrDefault();
|
||||
Microphone = Microphones.FirstOrDefault(d => d.Value == _settings?.PodcastMicrophoneId)
|
||||
?? Microphones.FirstOrDefault();
|
||||
}
|
||||
|
||||
[RelayCommand]
|
||||
private void RefreshDevices()
|
||||
{
|
||||
try
|
||||
{
|
||||
if (!CanConfigure) throw new InvalidOperationException("Terminez l'appel avant de changer les peripheriques.");
|
||||
var devices = (Platform.GetWebRtcDevices ?? LinuxWebRtcMedia.GetDevices)();
|
||||
Cameras.Clear();
|
||||
Microphones.Clear();
|
||||
foreach (var device in devices.Cameras) Cameras.Add(device);
|
||||
foreach (var device in devices.Microphones.Where(d => !string.IsNullOrEmpty(d.Value))) Microphones.Add(device);
|
||||
RestoreSelection();
|
||||
Status = Cameras.Count == 0 ? "Aucune camera detectee."
|
||||
: "Peripheriques actualises. Le profil de masquage IP sera applique par le serveur.";
|
||||
}
|
||||
catch (Exception error) { Status = "Peripheriques indisponibles : " + error.Message; }
|
||||
}
|
||||
|
||||
private void SelectDevices()
|
||||
{
|
||||
if (Camera is null || Microphone is null)
|
||||
throw new InvalidOperationException("Une camera et un microphone sont requis.");
|
||||
_camera = Camera.Value;
|
||||
_microphone = Microphone.Value;
|
||||
if (_settings is not null)
|
||||
{
|
||||
_settings.PodcastCameraId = _camera;
|
||||
_settings.PodcastMicrophoneId = _microphone;
|
||||
try { _settings.Save(); }
|
||||
catch (Exception error) when (error is System.IO.IOException or UnauthorizedAccessException)
|
||||
{
|
||||
throw new InvalidOperationException("Sauvegarde des peripheriques impossible.", error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void CheckPolicy(CallSession session)
|
||||
{
|
||||
if (session.Transport != CallTransports.WebRtc)
|
||||
throw new InvalidOperationException("Le transport demande n'est pas WebRTC.");
|
||||
if (!session.RelayOnly && !AllowAddressDisclosure)
|
||||
throw new InvalidOperationException(
|
||||
"Activez le masquage IP dans votre profil ou consentez au partage IP pour cet appel.");
|
||||
}
|
||||
|
||||
[RelayCommand]
|
||||
private Task InviteAsync() => RunAsync(async () =>
|
||||
{
|
||||
if (Busy) throw new InvalidOperationException("Un appel est deja en cours.");
|
||||
SelectDevices();
|
||||
if (string.IsNullOrWhiteSpace(Target) || string.IsNullOrWhiteSpace(Room))
|
||||
throw new InvalidOperationException("Selectionnez une salle jointe et un autre utilisateur.");
|
||||
_callLifetime = CancellationTokenSource.CreateLinkedTokenSource(_lifetime.Token);
|
||||
_caller = true;
|
||||
Busy = true;
|
||||
_call = await _signaling.InviteAsync(Room, Target.Trim(), _callLifetime.Token);
|
||||
CheckPolicy(_call);
|
||||
Status = $"Invitation envoyee a {_call.Callee} ; attente du consentement.";
|
||||
});
|
||||
|
||||
[RelayCommand]
|
||||
private Task AcceptAsync() => RunAsync(async () =>
|
||||
{
|
||||
if (_call is null || !Incoming) throw new InvalidOperationException("Aucune invitation a accepter.");
|
||||
CheckPolicy(_call);
|
||||
SelectDevices();
|
||||
Incoming = false;
|
||||
_accepted = true;
|
||||
await _signaling.AcceptAsync(_call.Id, _callLifetime!.Token);
|
||||
Status = "Invitation acceptee ; preparation des medias.";
|
||||
});
|
||||
|
||||
[RelayCommand]
|
||||
public Task HangupAsync()
|
||||
{
|
||||
_callLifetime?.Cancel();
|
||||
return RunAsync(() => StopAsync("Appel termine.", true));
|
||||
}
|
||||
|
||||
private void OnInvitation(CallSession call) => Queue(async () =>
|
||||
{
|
||||
if (call.Transport != CallTransports.WebRtc || _call is not null)
|
||||
{
|
||||
await _signaling.EndAsync(call.Id, _lifetime.Token);
|
||||
Status = "Invitation refusee : transport incompatible ou appel deja en cours.";
|
||||
return;
|
||||
}
|
||||
_call = call;
|
||||
_caller = false;
|
||||
_callLifetime = CancellationTokenSource.CreateLinkedTokenSource(_lifetime.Token);
|
||||
Busy = Incoming = true;
|
||||
Status = $"{call.Caller} propose un appel dans {call.RoomName}. "
|
||||
+ (call.RelayOnly ? "Masquage IP : relais TURN obligatoire. " : "Mode direct : consentement au partage IP requis. ")
|
||||
+ "Acceptez ou refusez.";
|
||||
});
|
||||
|
||||
private void OnAccepted(CallAccepted accepted) => Queue(async () =>
|
||||
{
|
||||
if (_call?.Id != accepted.Session.Id) return;
|
||||
if (_media is not null || (!_caller && !_accepted) ||
|
||||
(_call.RelayOnly && !accepted.Session.RelayOnly))
|
||||
throw new InvalidOperationException("Acceptation ou politique WebRTC inattendue.");
|
||||
CheckPolicy(accepted.Session);
|
||||
_call = accepted.Session;
|
||||
Incoming = false;
|
||||
var configuration = new WebRtcMediaConfiguration(accepted.IceServers, _call.RelayOnly,
|
||||
_caller, _camera!, _microphone!);
|
||||
WebRtcPrivacy.ValidateConfiguration(configuration);
|
||||
_media = _createMedia();
|
||||
var id = _call.Id;
|
||||
var media = _media;
|
||||
_sendSignal = signal => Queue(async () =>
|
||||
{
|
||||
if (_call?.Id != id || !ReferenceEquals(_media, media)) return;
|
||||
WebRtcPrivacy.ValidateSignal(signal, _call.RelayOnly);
|
||||
await _signaling.SendAsync(id, signal, _callLifetime!.Token);
|
||||
});
|
||||
_receiveStatus = status => Queue(() =>
|
||||
{
|
||||
if (_call?.Id == id && ReferenceEquals(_media, media))
|
||||
{
|
||||
WebRtcPrivacy.ValidateStatus(status, _call.RelayOnly);
|
||||
_mediaStatus = status;
|
||||
}
|
||||
return Task.CompletedTask;
|
||||
});
|
||||
media.Signal += _sendSignal;
|
||||
media.StatusChanged += _receiveStatus;
|
||||
LocalVideo = media.LocalVideo;
|
||||
RemoteVideo = media.RemoteVideo;
|
||||
Status = _call.RelayOnly ? "Connexion WebRTC via TURN uniquement ; aucun repli direct."
|
||||
: "Connexion WebRTC ICE/STUN/TURN.";
|
||||
await media.StartAsync(configuration, _callLifetime!.Token);
|
||||
_ = MonitorAsync(id, media, _callLifetime.Token);
|
||||
});
|
||||
|
||||
private void OnSignal(Guid id, CallSignal signal) => Queue(async () =>
|
||||
{
|
||||
if (_call?.Id != id) return;
|
||||
if (_media is null) throw new InvalidOperationException("Signal recu avant le consentement.");
|
||||
WebRtcPrivacy.ValidateSignal(signal, false);
|
||||
await _media.HandleSignalAsync(signal, _callLifetime!.Token);
|
||||
});
|
||||
|
||||
private void OnEnded(CallEnded ended) => QueueStop(ended.CallId,
|
||||
() => StopAsync("Appel termine : " + ended.Reason, false));
|
||||
private void OnInterrupted() => QueueStop(null,
|
||||
() => StopAsync("Connexion SignalR interrompue ; medias arretes.", false));
|
||||
private void OnSuspending() => QueueStop(null,
|
||||
() => StopAsync("Application en arriere-plan ; appel termine.", true));
|
||||
|
||||
private void QueueStop(Guid? id, Func<Task> action) => Dispatcher.UIThread.Post(async () =>
|
||||
{
|
||||
if (id is not null && _call?.Id != id) return;
|
||||
_callLifetime?.Cancel();
|
||||
await RunAsync(() => id is null || _call?.Id == id ? action() : Task.CompletedTask);
|
||||
});
|
||||
|
||||
private async Task MonitorAsync(Guid id, IWebRtcMedia media, CancellationToken ct)
|
||||
{
|
||||
var progress = Stopwatch.StartNew();
|
||||
var received = false;
|
||||
long audio = 0, video = 0;
|
||||
try
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
ct.ThrowIfCancellationRequested();
|
||||
if (media.Completion.IsCompleted)
|
||||
{
|
||||
await media.Completion;
|
||||
throw new InvalidOperationException("Le moteur WebRTC s'est arrete.");
|
||||
}
|
||||
await Dispatcher.UIThread.InvokeAsync(() =>
|
||||
{
|
||||
if (_call?.Id != id || _mediaStatus is not { } status) return;
|
||||
if (status.ConnectionState is "failed" or "closed" or "disconnected")
|
||||
throw new InvalidOperationException("Connexion ICE interrompue ; aucun repli automatique.");
|
||||
if (!status.HasRemoteMedia) return;
|
||||
if (status.AudioPackets > audio && status.VideoFrames > video)
|
||||
{
|
||||
audio = status.AudioPackets;
|
||||
video = status.VideoFrames;
|
||||
progress.Restart();
|
||||
}
|
||||
received = true;
|
||||
Status = _call.RelayOnly ? "Audio-video WebRTC etabli via TURN (IP masquee)."
|
||||
: "Audio-video WebRTC etabli.";
|
||||
});
|
||||
if (progress.Elapsed > TimeSpan.FromSeconds(30))
|
||||
throw new TimeoutException(received ? "Reception audio-video interrompue."
|
||||
: "Aucun media distant. Verifiez ICE, TURN et les pare-feu ; aucun repli hors politique.");
|
||||
await Task.Delay(500, ct);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) when (ct.IsCancellationRequested) { }
|
||||
catch (Exception error)
|
||||
{
|
||||
Queue(() => _call?.Id == id && ReferenceEquals(_media, media)
|
||||
? FailAsync(error) : Task.CompletedTask);
|
||||
}
|
||||
}
|
||||
|
||||
private void Queue(Func<Task> action) => Dispatcher.UIThread.Post(async () => await RunAsync(action));
|
||||
|
||||
private async Task RunAsync(Func<Task> action)
|
||||
{
|
||||
await _events.WaitAsync();
|
||||
try
|
||||
{
|
||||
if (_disposed) return;
|
||||
Preparing = true;
|
||||
await action();
|
||||
}
|
||||
catch (OperationCanceledException) when (
|
||||
_lifetime.IsCancellationRequested || _callLifetime?.IsCancellationRequested == true) { }
|
||||
catch (Exception error) { await FailAsync(error); }
|
||||
finally { Preparing = false; _events.Release(); }
|
||||
}
|
||||
|
||||
private async Task FailAsync(Exception error)
|
||||
{
|
||||
try { await StopAsync("Echec de l'appel : " + error.Message, true); }
|
||||
catch (Exception cleanup)
|
||||
{
|
||||
Status = $"Echec de l'appel : {error.Message}. Nettoyage/signalisation : {cleanup.Message}";
|
||||
}
|
||||
}
|
||||
|
||||
private async Task StopAsync(string status, bool notify)
|
||||
{
|
||||
var call = _call;
|
||||
_call = null;
|
||||
Incoming = Busy = _accepted = false;
|
||||
_callLifetime?.Cancel();
|
||||
var media = _media;
|
||||
_media = null;
|
||||
_mediaStatus = null;
|
||||
try
|
||||
{
|
||||
if (media is not null)
|
||||
{
|
||||
media.Signal -= _sendSignal;
|
||||
media.StatusChanged -= _receiveStatus;
|
||||
await media.DisposeAsync();
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
LocalVideo = RemoteVideo = null;
|
||||
_sendSignal = null;
|
||||
_receiveStatus = null;
|
||||
_callLifetime?.Dispose();
|
||||
_callLifetime = null;
|
||||
Status = status;
|
||||
if (notify && call is not null)
|
||||
{
|
||||
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(5));
|
||||
await _signaling.EndAsync(call.Id, timeout.Token);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_disposed) return;
|
||||
_lifetime.Cancel();
|
||||
_callLifetime?.Cancel();
|
||||
await _events.WaitAsync();
|
||||
try
|
||||
{
|
||||
if (_disposed) return;
|
||||
_disposed = true;
|
||||
_signaling.Invitation -= OnInvitation;
|
||||
_signaling.Accepted -= OnAccepted;
|
||||
_signaling.Signal -= OnSignal;
|
||||
_signaling.Ended -= OnEnded;
|
||||
_signaling.Interrupted -= OnInterrupted;
|
||||
Platform.PlaybackSuspending -= OnSuspending;
|
||||
try { await StopAsync("Appel arrete.", true); }
|
||||
finally { _lifetime.Dispose(); }
|
||||
}
|
||||
finally { _events.Release(); }
|
||||
}
|
||||
}
|
||||
|
|
@ -104,17 +104,14 @@
|
|||
Content="Envoyer" />
|
||||
</Grid>
|
||||
|
||||
<Expander Grid.Row="3" Header="Appel direct IPv4/IPv6 (LAN/VPN ou IP publique)" IsVisible="{Binding SupportsLanCalls}" IsExpanded="True"
|
||||
<Expander Grid.Row="3" Header="Appel video WebRTC" IsVisible="{Binding SupportsVideoCalls}" IsExpanded="True"
|
||||
IsEnabled="{Binding IsConnected}" Margin="0,0,0,8">
|
||||
<ScrollViewer MaxHeight="520">
|
||||
<StackPanel DataContext="{Binding LanCall}" x:DataType="vm:LanCallViewModel" Spacing="8">
|
||||
<TextBlock Text="Android et Desktop Linux : IPv4 ou IPv6 globale/privee directement assignee a la machine (hors IPv6 link-local). Les pairs doivent pouvoir joindre leurs adresses. SRTP chiffre les medias ; le pair connait votre IP. Ouvrir le port UDP choisi et le suivant. Sans TURN ni traversee NAT. Casque recommande (pas d'annulation d'echo)."
|
||||
<StackPanel DataContext="{Binding VideoCall}" x:DataType="vm:WebRtcCallViewModel" Spacing="8">
|
||||
<TextBlock Text="Android et Linux : medias chiffres par WebRTC (DTLS-SRTP), ICE/STUN/TURN. Activez le masquage IP dans votre profil pour imposer un relais TURN aux deux participants, sans repli direct. La camera et le microphone ne demarrent qu'apres acceptation."
|
||||
TextWrapping="Wrap" />
|
||||
<WrapPanel IsEnabled="{Binding CanConfigure}" IsVisible="{Binding !RemoteVideo}">
|
||||
<Button Content="Actualiser les peripheriques" Command="{Binding RefreshDevicesCommand}" Margin="0,0,8,4" />
|
||||
<ComboBox Width="160" ItemsSource="{Binding Addresses}" SelectedItem="{Binding Address}"
|
||||
PlaceholderText="Adresse IPv4/IPv6 de cette machine" Margin="0,0,8,4" />
|
||||
<TextBox Width="80" Text="{Binding Port}" PlaceholderText="Port pair" Margin="0,0,8,4" />
|
||||
<ComboBox Width="190" ItemsSource="{Binding Cameras}" SelectedItem="{Binding Camera}"
|
||||
PlaceholderText="Camera" Margin="0,0,8,4">
|
||||
<ComboBox.ItemTemplate>
|
||||
|
|
@ -133,7 +130,7 @@
|
|||
</ComboBox>
|
||||
<TextBox Width="170" Text="{Binding Target}" PlaceholderText="Pseudo du pair" Margin="0,0,8,4" />
|
||||
</WrapPanel>
|
||||
<CheckBox Content="Je consens a partager mon adresse IP avec le pair."
|
||||
<CheckBox Content="Si le masquage IP est desactive, je consens a partager mon adresse IP avec le pair."
|
||||
IsChecked="{Binding AllowAddressDisclosure}" IsEnabled="{Binding CanConfigure}"
|
||||
IsVisible="{Binding !RemoteVideo}" />
|
||||
<WrapPanel>
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ public partial class ChatPage : ContentPage
|
|||
protected override async void OnDetachedFromVisualTree(VisualTreeAttachmentEventArgs e)
|
||||
{
|
||||
base.OnDetachedFromVisualTree(e);
|
||||
if (DataContext is ChatViewModel { LanCall: { } call })
|
||||
if (DataContext is ChatViewModel { VideoCall: { } call })
|
||||
await call.HangupAsync();
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue