feat: refund tracked storage usage on deletion and replacement
Persist per-file storage receipts, preserve refundability on rename, account for PDF replacements and permit deletion at the consumption limit. Update CGU, installation documentation and regression tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
parent
650e5f5e30
commit
ed8f41bd4a
17 changed files with 5636 additions and 65 deletions
|
|
@ -5,6 +5,8 @@
|
|||
|
||||
### Added
|
||||
|
||||
* [Yavsc/API/Blogs/PostIt] Refund tracked storage points when personal files are deleted or replaced, and when generated PDFs are replaced. Persist per-file accounting receipts, preserve them on rename, reject refunds for untracked legacy files, and allow deletion at the consumption limit without refunding API/CPU/transfer costs. Update CGU and installation instructions.
|
||||
|
||||
* [Yavsc/PostIt/PayPal] Add monthly Découverte (EUR 1 excluding VAT) and Premium (EUR 2 excluding VAT) resource subscriptions, charged at EUR 1.20 / EUR 2.40 including 20% VAT. Confirmed monthly PayPal payments erase up to one / three default quotas, carry two thirds of unused monthly credit without expiry, and preserve historical counters. Add persistent idempotent payment credits, refund reconciliation, authenticated Web subscription/cancellation pages, verified REST webhooks, PostIt links and balances, migration and updated CGU.
|
||||
* [Yavsc/PostIt/Subscriptions] Align new users' free quota with the existing 500 MiB database default; grant twice that standard quota for Discovery (1,000 MiB) and five times for Premium (2,500 MiB) during confirmed paid monthly periods, including their remainder after cancellation. Restore the user's base quota after expiry or refund without deleting files, refuse uploads beyond available space before writing, and expose the effective quota in profile and usage responses.
|
||||
* [PostIt/API/Blogs/Yavsc.Org] Compute cumulative resource consumption in configurable points per API call, recorded CPU second, MiB transferred and MiB added to storage, with backend-defined defaults, a default 1,000-point limit and per-user overrides. Serialize accounting and known-cost admission across PostgreSQL hosts, count streamed HTTP bodies, reject further consuming operations with HTTP 429, and check personal uploads before writing. Keep usage/session access available, display the indicator and balance in PostIt, and publish linked CGU explaining the rule and after-measurement enforcement.
|
||||
|
|
|
|||
10
INSTALL.md
10
INSTALL.md
|
|
@ -113,7 +113,15 @@ balance and subscription/payment tables without clearing existing counters.
|
|||
|
||||
Consumption is in **points**, independent of `EstimatedCompensation` and
|
||||
currency. Counters have no automatic reset. Storage means cumulative
|
||||
instrumented writes, not current occupancy; the historical CPU counter is
|
||||
instrumented writes less individually tracked bytes removed or replaced,
|
||||
not total filesystem occupancy. Replacements refund their tracked previous
|
||||
size and charge the new size. Legacy files without an individual receipt
|
||||
receive no refund; storage counters never become negative. Deletion remains
|
||||
available at the consumption limit, but API/transfer/CPU costs are not refunded.
|
||||
Apply the `RefundableResourceStorage` migration before deployment; its empty
|
||||
receipt table deliberately does not infer provenance from historical counters.
|
||||
Renames preserve receipts and repeated PDF generation replaces the previous
|
||||
tracked volume rather than charging the whole file again. The historical CPU counter is
|
||||
instrumented elapsed processing time, not exhaustive CPU profiling.
|
||||
|
||||
Authenticated HTTP operations are admitted atomically against known costs
|
||||
|
|
|
|||
|
|
@ -131,8 +131,12 @@ si une opération de gestion reçoit un refus de quota. L'administration
|
|||
charge également les usages avant ses autres outils et les conserve sur
|
||||
un refus HTTP 429, tant que le rôle administrateur est toujours présent.
|
||||
|
||||
L'indicateur cumulé est exprimé en points : `0,02 × appels API + secondes
|
||||
CPU enregistrées + 0,1 × Mio transférés + 0,3 × Mio ajoutés au stockage`.
|
||||
L'indicateur est exprimé en points selon les coefficients effectifs du
|
||||
backend, présentés dans les CGU. Le stockage correspond aux volumes
|
||||
instrumentés ajoutés, moins les volumes individuellement suivis supprimés
|
||||
ou remplacés. Les anciens fichiers sans suivi ne sont pas remboursés.
|
||||
La suppression reste disponible au seuil ; CPU, appels API et transferts
|
||||
ne sont pas remboursés. Les renommages conservent le suivi individuel.
|
||||
Le profil et les vues d'administration affichent le seuil effectif, le
|
||||
solde et l'atteinte du quota, indépendamment de l'estimation financière.
|
||||
La section serveur `ResourceUsage` configure les coefficients et les
|
||||
|
|
|
|||
|
|
@ -159,7 +159,6 @@ namespace Yavsc.ApiControllers
|
|||
uid,
|
||||
User.Identity?.Name ?? uid,
|
||||
cpuSeconds: (decimal)stopwatch.Elapsed.TotalSeconds,
|
||||
storageBytes: fo.Length,
|
||||
actorUserId: uid);
|
||||
|
||||
return File(fo.OpenRead(), "application/pdf");
|
||||
|
|
@ -185,12 +184,21 @@ namespace Yavsc.ApiControllers
|
|||
uid,
|
||||
User.Identity?.Name ?? uid,
|
||||
cpuSeconds: (decimal)stopwatch.Elapsed.TotalSeconds,
|
||||
storageBytes: renderedPdf?.Length ?? 0,
|
||||
actorUserId: uid);
|
||||
|
||||
logger.LogError(ex, "estimate {QueryId}: TeX/PDF render failed", queryId);
|
||||
return Problem("TeX render failed: " + ex.Message);
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (renderedPdf is not null)
|
||||
{
|
||||
renderedPdf.Refresh();
|
||||
if (renderedPdf.Exists)
|
||||
await ResourceUsageTracker.RecordStorageAsync(dbContext, uid, User.Identity?.Name ?? uid,
|
||||
"pdf/" + renderedPdf.FullName, renderedPdf.Length, CancellationToken.None);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Loads the estimate linked to a query with the navigation the
|
||||
|
|
|
|||
|
|
@ -98,6 +98,13 @@ namespace Yavsc.Blogs.Controllers
|
|||
return $"{subdir.TrimStart('/')}/{fileName}";
|
||||
}
|
||||
|
||||
static string StoragePath(string relativePath)
|
||||
{
|
||||
var root = Path.GetPathRoot(Path.GetFullPath("."))!;
|
||||
return "personal/" + Path.GetRelativePath(root,
|
||||
Path.GetFullPath(Path.Combine(root, relativePath.TrimStart('/', '\\')))).Replace('\\', '/');
|
||||
}
|
||||
|
||||
[HttpPost("{*subdir}")]
|
||||
public async Task<IActionResult> Post([FromServices] IResourceUsageService usage,
|
||||
[ValidRemoteUserFilePath] string subdir="", CancellationToken cancellationToken = default)
|
||||
|
|
@ -105,7 +112,15 @@ namespace Yavsc.Blogs.Controllers
|
|||
if (!ModelState.IsValid) return new BadRequestObjectResult(ModelState);
|
||||
var uid = User.GetUserId();
|
||||
var form = await Request.ReadFormAsync(cancellationToken);
|
||||
var anticipatedBytes = form.Files.Sum(f => f.Length);
|
||||
var anticipatedBytes = 0L;
|
||||
foreach (var file in form.Files)
|
||||
{
|
||||
var path = StoragePath(RelPath(subdir, AbstractFileSystemHelpers.FilterFileName(file.FileName)));
|
||||
var previous = await dbContext.ResourceStorageRecords.AsNoTracking()
|
||||
.Where(r => r.UserId == uid && r.Path == path)
|
||||
.Select(r => r.Bytes).SingleOrDefaultAsync(cancellationToken);
|
||||
anticipatedBytes = checked(anticipatedBytes + Math.Max(0L, file.Length - previous));
|
||||
}
|
||||
if (!await usage.TryAdmitAsync(uid, User.Identity?.Name ?? uid, 0m, 0m,
|
||||
anticipatedBytes, cancellationToken))
|
||||
{
|
||||
|
|
@ -136,6 +151,8 @@ namespace Yavsc.Blogs.Controllers
|
|||
|
||||
foreach (var f in form.Files)
|
||||
{
|
||||
var path = Path.Combine(destDir, AbstractFileSystemHelpers.FilterFileName(f.FileName));
|
||||
var oldLength = System.IO.File.Exists(path) ? new FileInfo(path).Length : 0L;
|
||||
var item = user.ReceiveUserFile(destDir, f);
|
||||
if (!item.QuotaOffense)
|
||||
{
|
||||
|
|
@ -170,12 +187,10 @@ namespace Yavsc.Blogs.Controllers
|
|||
item.ContentType = f.ContentType;
|
||||
}
|
||||
|
||||
await ResourceUsageTracker.RecordAsync(
|
||||
dbContext,
|
||||
uid,
|
||||
user.UserName,
|
||||
storageBytes: item.Length,
|
||||
actorUserId: uid);
|
||||
var newLength = System.IO.File.Exists(path) ? new FileInfo(path).Length : 0L;
|
||||
if (!item.QuotaOffense || newLength != oldLength)
|
||||
await ResourceUsageTracker.RecordStorageAsync(dbContext, uid, user.UserName,
|
||||
StoragePath(RelPath(subdir, item.FileName)), newLength, CancellationToken.None);
|
||||
|
||||
received.Add(item);
|
||||
_logger.LogInformation($"Received '{item.FileName}'.");
|
||||
|
|
@ -259,22 +274,24 @@ namespace Yavsc.Blogs.Controllers
|
|||
[HttpPost]
|
||||
[Route("/api/fsc/mvftd")]
|
||||
[Authorize()]
|
||||
public IActionResult MoveFile([FromBody] RenameFileQuery query)
|
||||
public async Task<IActionResult> MoveFile([FromBody] RenameFileQuery query)
|
||||
{
|
||||
if (!ModelState.IsValid) return new BadRequestObjectResult(ModelState);
|
||||
var uid = User.FindFirstValue(ClaimTypes.NameIdentifier);
|
||||
var uid = User.GetUserId();
|
||||
var user = dbContext.Users.Single(
|
||||
u => u.Id == uid
|
||||
);
|
||||
var info = user.MoveUserFileToDir(query.Id, query.To, siteSettings);
|
||||
if (!info.Done) return new BadRequestObjectResult(info);
|
||||
await ResourceUsageTracker.MoveStorageAsync(dbContext, uid, StoragePath(query.Id),
|
||||
StoragePath(Path.Combine(query.To ?? "", Path.GetFileName(query.Id))), CancellationToken.None);
|
||||
return Ok(new { moved = query.Id });
|
||||
}
|
||||
|
||||
[HttpPost]
|
||||
[Route("/api/fsc/mvf")]
|
||||
[Authorize()]
|
||||
public IActionResult RenameFile([FromBody] RenameFileQuery query)
|
||||
public async Task<IActionResult> RenameFile([FromBody] RenameFileQuery query)
|
||||
{
|
||||
if (!ModelState.IsValid) {
|
||||
var idvr = new ValidRemoteUserFilePathAttribute();
|
||||
|
|
@ -282,23 +299,30 @@ namespace Yavsc.Blogs.Controllers
|
|||
return this.BadRequest(new { id = idvr.IsValid(query.Id), to = idvr.IsValid(query.To), errors = ModelState });
|
||||
}
|
||||
_logger.LogInformation($"Valid move query: {query.Id} => {query.To}");
|
||||
var uid = User.FindFirstValue(ClaimTypes.NameIdentifier);
|
||||
var uid = User.GetUserId();
|
||||
var user = dbContext.Users.Single(
|
||||
u => u.Id == uid
|
||||
);
|
||||
try {
|
||||
if (Config.UserFilesOptions.FileProvider.GetFileInfo(Path.Combine(user.UserName, query.Id)).Exists)
|
||||
var destination = query.To;
|
||||
var root = Path.Combine(siteSettings.UserFilesRoot, user.UserName);
|
||||
if (System.IO.File.Exists(Path.Combine(root, query.Id)))
|
||||
{
|
||||
var result = user.MoveUserFile(query.Id, query.To, siteSettings);
|
||||
if (!result.Done) return new BadRequestObjectResult(result);
|
||||
}
|
||||
else {
|
||||
var dir = new DirectoryInfo(Path.Combine(root, query.To));
|
||||
if (dir.Exists) destination = Path.Combine(query.To, dir.Name);
|
||||
var result = user.MoveUserDir(query.Id, query.To, siteSettings);
|
||||
if (!result.Done) return new BadRequestObjectResult(result);
|
||||
}
|
||||
await ResourceUsageTracker.MoveStorageAsync(dbContext, uid, StoragePath(query.Id),
|
||||
StoragePath(destination), CancellationToken.None);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to rename user file {Path} for {UserId}", query.Id, uid);
|
||||
return new BadRequestObjectResult(
|
||||
new FsOperationInfo {
|
||||
Done = false,
|
||||
|
|
@ -310,7 +334,8 @@ namespace Yavsc.Blogs.Controllers
|
|||
}
|
||||
|
||||
[HttpDelete("{*id}")]
|
||||
public IActionResult RemoveDirOrFile ([ValidRemoteUserFilePath] string id)
|
||||
[ResourceUsageRelief]
|
||||
public async Task<IActionResult> RemoveDirOrFile ([ValidRemoteUserFilePath] string id)
|
||||
{
|
||||
if (!ModelState.IsValid) return new BadRequestObjectResult(ModelState);
|
||||
|
||||
|
|
@ -322,10 +347,13 @@ namespace Yavsc.Blogs.Controllers
|
|||
var result = user.DeleteUserDirOrFile(id, siteSettings);
|
||||
if (!result.Done)
|
||||
return new BadRequestObjectResult(result);
|
||||
await ResourceUsageTracker.RecordStorageAsync(dbContext, user.Id, user.UserName,
|
||||
StoragePath(id), 0L, CancellationToken.None);
|
||||
}
|
||||
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "Failed to delete user file {Path} for {UserId}", id, user.Id);
|
||||
return new BadRequestObjectResult(
|
||||
new FsOperationInfo {
|
||||
Done = false,
|
||||
|
|
|
|||
5221
src/Yavsc.Org/Migrations/20261108052221_RefundableResourceStorage.Designer.cs
generated
Normal file
5221
src/Yavsc.Org/Migrations/20261108052221_RefundableResourceStorage.Designer.cs
generated
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,34 @@
|
|||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace Yavsc.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class RefundableResourceStorage : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateTable(
|
||||
name: "ResourceStorageRecords",
|
||||
columns: table => new
|
||||
{
|
||||
UserId = table.Column<string>(type: "character varying(450)", maxLength: 450, nullable: false),
|
||||
Path = table.Column<string>(type: "character varying(1024)", maxLength: 1024, nullable: false),
|
||||
Bytes = table.Column<long>(type: "bigint", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_ResourceStorageRecords", x => new { x.UserId, x.Path });
|
||||
});
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropTable(
|
||||
name: "ResourceStorageRecords");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -3081,6 +3081,24 @@ namespace Yavsc.Migrations
|
|||
b.ToTable("Tags");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Yavsc.Models.ResourceStorageRecord", b =>
|
||||
{
|
||||
b.Property<string>("UserId")
|
||||
.HasMaxLength(450)
|
||||
.HasColumnType("character varying(450)");
|
||||
|
||||
b.Property<string>("Path")
|
||||
.HasMaxLength(1024)
|
||||
.HasColumnType("character varying(1024)");
|
||||
|
||||
b.Property<long>("Bytes")
|
||||
.HasColumnType("bigint");
|
||||
|
||||
b.HasKey("UserId", "Path");
|
||||
|
||||
b.ToTable("ResourceStorageRecords");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Yavsc.Models.ResourceUsageRecord", b =>
|
||||
{
|
||||
b.Property<long>("Id")
|
||||
|
|
|
|||
|
|
@ -36,7 +36,10 @@
|
|||
</li>
|
||||
<li>
|
||||
Le stockage est le volume cumulé ajouté par les opérations instrumentées (fichiers personnels et génération de PDF),
|
||||
pas l'espace disque actuellement occupé : suppression et remplacement ne remboursent pas de points.
|
||||
diminué des volumes comptabilisés supprimés ou remplacés : ces opérations remboursent les points de stockage associés.
|
||||
Un remplacement rembourse l'ancien volume suivi puis comptabilise le nouveau.
|
||||
Les anciens fichiers sans suivi individuel ne donnent pas de remboursement ; le compteur de stockage ne devient jamais négatif.
|
||||
Les points d'appels API, de CPU et de transfert ne sont pas remboursés.
|
||||
</li>
|
||||
<li>
|
||||
Le compteur historique « CPU » contient les durées enregistrées par les traitements instrumentés,
|
||||
|
|
@ -44,6 +47,7 @@
|
|||
Les ressources locales de PostIt, les appels directs entre appareils et le trafic WebSocket ne sont pas mesurés par cet indicateur HTTP.
|
||||
</li>
|
||||
</ul>
|
||||
<p>La suppression de fichiers reste autorisée même lorsque le seuil de consommation est atteint, afin de libérer du stockage.</p>
|
||||
<p>
|
||||
Le calcul utilise les valeurs décimales non arrondies ; l'arrondi de l'affichage ne décide pas du blocage.
|
||||
Cet indicateur n'est ni un prix, ni une facture, ni une estimation de compensation financière.
|
||||
|
|
|
|||
4
src/Yavsc.Server/Helpers/ResourceUsageReliefAttribute.cs
Normal file
4
src/Yavsc.Server/Helpers/ResourceUsageReliefAttribute.cs
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
namespace Yavsc.Server.Helpers;
|
||||
|
||||
[AttributeUsage(AttributeTargets.Method)]
|
||||
public sealed class ResourceUsageReliefAttribute : Attribute;
|
||||
|
|
@ -24,7 +24,14 @@ public sealed class ResourceUsageTelemetryMiddleware(
|
|||
|
||||
var userName = context.User.Identity?.Name ?? userId;
|
||||
var requestBytes = context.Request.ContentLength ?? 0L;
|
||||
if (!await usage.TryAdmitAsync(userId, userName, 1m, ToMb(requestBytes), 0,
|
||||
var relief = context.GetEndpoint()?.Metadata.GetMetadata<ResourceUsageReliefAttribute>() is not null;
|
||||
if (relief)
|
||||
{
|
||||
using var admissionScope = scopeFactory.CreateScope();
|
||||
await ResourceUsageTracker.RecordAsync(admissionScope.ServiceProvider.GetRequiredService<ApplicationDbContext>(),
|
||||
userId, userName, apiCalls: 1m, bandwidthMb: ToMb(requestBytes), actorUserId: userId);
|
||||
}
|
||||
else if (!await usage.TryAdmitAsync(userId, userName, 1m, ToMb(requestBytes), 0,
|
||||
context.RequestAborted))
|
||||
{
|
||||
logger.LogWarning("Resource quota refused a request for user {UserId}", userId);
|
||||
|
|
|
|||
|
|
@ -9,6 +9,55 @@ public static class ResourceUsageTracker
|
|||
{
|
||||
private static readonly SemaphoreSlim LocalGate = new(1, 1);
|
||||
|
||||
public static Task RecordStorageAsync(ApplicationDbContext db, string userId, string userName,
|
||||
string path, long bytes, CancellationToken cancellationToken = default)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(db);
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(userId);
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(path);
|
||||
ArgumentOutOfRangeException.ThrowIfNegative(bytes);
|
||||
return WithUserLockAsync(db, userId, async () =>
|
||||
{
|
||||
var stored = await db.ResourceStorageRecords.FindAsync([userId, path], cancellationToken);
|
||||
if (stored is not null) await db.Entry(stored).ReloadAsync(cancellationToken);
|
||||
var delta = bytes - (stored?.Bytes ?? 0L);
|
||||
if (stored is null && bytes > 0)
|
||||
db.ResourceStorageRecords.Add(new ResourceStorageRecord { UserId = userId, Path = path, Bytes = bytes });
|
||||
else if (stored is not null)
|
||||
{
|
||||
if (bytes == 0) db.ResourceStorageRecords.Remove(stored);
|
||||
else stored.Bytes = bytes;
|
||||
}
|
||||
await UpdateCoreAsync(db, userId, userName, 0m, delta, 0m, 0m,
|
||||
userId, Constants.ResourceUsageDefaultCurrency, null, 0, cancellationToken);
|
||||
await db.SaveChangesAsync(userId, cancellationToken);
|
||||
return true;
|
||||
}, cancellationToken);
|
||||
}
|
||||
|
||||
public static Task MoveStorageAsync(ApplicationDbContext db, string userId, string from, string to,
|
||||
CancellationToken cancellationToken = default)
|
||||
=> WithUserLockAsync(db, userId, async () =>
|
||||
{
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(from);
|
||||
ArgumentException.ThrowIfNullOrWhiteSpace(to);
|
||||
if (from == to) return true;
|
||||
var records = await db.ResourceStorageRecords.Where(r => r.UserId == userId
|
||||
&& (r.Path == from || r.Path.StartsWith(from + "/"))).ToListAsync(cancellationToken);
|
||||
foreach (var record in records)
|
||||
{
|
||||
db.ResourceStorageRecords.Remove(record);
|
||||
db.ResourceStorageRecords.Add(new ResourceStorageRecord
|
||||
{
|
||||
UserId = userId,
|
||||
Path = to + record.Path.Substring(from.Length),
|
||||
Bytes = record.Bytes
|
||||
});
|
||||
}
|
||||
await db.SaveChangesAsync(userId, cancellationToken);
|
||||
return true;
|
||||
}, cancellationToken);
|
||||
|
||||
public static async Task RecordAsync(
|
||||
ApplicationDbContext dbContext,
|
||||
string userId,
|
||||
|
|
@ -42,57 +91,64 @@ public static class ResourceUsageTracker
|
|||
ArgumentOutOfRangeException.ThrowIfNegative(anticipatedStorageBytes);
|
||||
|
||||
return await WithUserLockAsync(dbContext, userId, async () =>
|
||||
await UpdateCoreAsync(dbContext, userId, userName, cpuSeconds, storageBytes, apiCalls,
|
||||
bandwidthMb, actorUserId, currency, policy, anticipatedStorageBytes, cancellationToken), cancellationToken);
|
||||
}
|
||||
|
||||
private static async Task<ResourceUsageRecord?> UpdateCoreAsync(ApplicationDbContext dbContext,
|
||||
string userId, string userName, decimal cpuSeconds, long storageBytes, decimal apiCalls,
|
||||
decimal bandwidthMb, string? actorUserId, string currency, ResourceUsageOptions? policy,
|
||||
long anticipatedStorageBytes, CancellationToken cancellationToken)
|
||||
{
|
||||
var effectiveActor = string.IsNullOrWhiteSpace(actorUserId) ? userId : actorUserId;
|
||||
var now = DateTime.UtcNow;
|
||||
var storageMb = (decimal)storageBytes / (1024m * 1024m);
|
||||
var record = await dbContext.ResourceUsageRecords
|
||||
.Where(r => r.UserId == userId)
|
||||
.OrderByDescending(r => r.RecordedAtUtc).ThenByDescending(r => r.Id)
|
||||
.FirstOrDefaultAsync(cancellationToken);
|
||||
if (record is not null)
|
||||
await dbContext.Entry(record).ReloadAsync(cancellationToken);
|
||||
|
||||
if (policy is not null)
|
||||
{
|
||||
var effectiveActor = string.IsNullOrWhiteSpace(actorUserId) ? userId : actorUserId;
|
||||
var now = DateTime.UtcNow;
|
||||
var storageMb = (decimal)storageBytes / (1024m * 1024m);
|
||||
var record = await dbContext.ResourceUsageRecords
|
||||
.Where(r => r.UserId == userId)
|
||||
.OrderByDescending(r => r.RecordedAtUtc).ThenByDescending(r => r.Id)
|
||||
.FirstOrDefaultAsync(cancellationToken);
|
||||
if (record is not null)
|
||||
await dbContext.Entry(record).ReloadAsync(cancellationToken);
|
||||
var gross = record is null ? 0m :
|
||||
policy.Calculate(record.ApiCalls, record.CpuSeconds, record.BandwidthMb, record.StorageMb);
|
||||
var current = ResourceUsageCredit.NetConsumption(gross, record?.CreditPoints ?? 0m);
|
||||
var projected = ResourceUsageCredit.NetConsumption(gross + policy.Calculate(apiCalls, cpuSeconds, bandwidthMb,
|
||||
storageMb + (decimal)anticipatedStorageBytes / (1024m * 1024m)), record?.CreditPoints ?? 0m);
|
||||
var limit = policy.GetLimit(userId);
|
||||
if (current >= limit || projected > limit) return null;
|
||||
}
|
||||
|
||||
if (policy is not null)
|
||||
{
|
||||
var gross = record is null ? 0m :
|
||||
policy.Calculate(record.ApiCalls, record.CpuSeconds, record.BandwidthMb, record.StorageMb);
|
||||
var current = ResourceUsageCredit.NetConsumption(gross, record?.CreditPoints ?? 0m);
|
||||
var projected = ResourceUsageCredit.NetConsumption(gross + policy.Calculate(apiCalls, cpuSeconds, bandwidthMb,
|
||||
storageMb + (decimal)anticipatedStorageBytes / (1024m * 1024m)), record?.CreditPoints ?? 0m);
|
||||
var limit = policy.GetLimit(userId);
|
||||
if (current >= limit || projected > limit) return null;
|
||||
}
|
||||
|
||||
if (record is not null && apiCalls == 0m && cpuSeconds == 0m && bandwidthMb == 0m && storageBytes == 0)
|
||||
return record;
|
||||
|
||||
if (record is null)
|
||||
{
|
||||
record = new ResourceUsageRecord
|
||||
{
|
||||
UserId = userId,
|
||||
UserName = string.IsNullOrWhiteSpace(userName) ? userId : userName,
|
||||
Currency = currency,
|
||||
UserCreated = effectiveActor,
|
||||
DateCreated = now
|
||||
};
|
||||
dbContext.ResourceUsageRecords.Add(record);
|
||||
}
|
||||
|
||||
record.UserName = string.IsNullOrWhiteSpace(userName) ? record.UserName : userName;
|
||||
record.RecordedAtUtc = now;
|
||||
record.ApiCalls += apiCalls;
|
||||
record.CpuSeconds += cpuSeconds;
|
||||
record.BandwidthMb += bandwidthMb;
|
||||
record.StorageMb += storageMb;
|
||||
record.Currency = string.IsNullOrWhiteSpace(currency) ? record.Currency : currency;
|
||||
record.DateModified = now;
|
||||
record.UserModified = effectiveActor;
|
||||
|
||||
await dbContext.SaveChangesAsync(effectiveActor, cancellationToken);
|
||||
if (record is not null && apiCalls == 0m && cpuSeconds == 0m && bandwidthMb == 0m && storageBytes == 0)
|
||||
return record;
|
||||
}, cancellationToken);
|
||||
|
||||
if (record is null)
|
||||
{
|
||||
record = new ResourceUsageRecord
|
||||
{
|
||||
UserId = userId,
|
||||
UserName = string.IsNullOrWhiteSpace(userName) ? userId : userName,
|
||||
Currency = currency,
|
||||
UserCreated = effectiveActor,
|
||||
DateCreated = now
|
||||
};
|
||||
dbContext.ResourceUsageRecords.Add(record);
|
||||
}
|
||||
|
||||
record.UserName = string.IsNullOrWhiteSpace(userName) ? record.UserName : userName;
|
||||
record.RecordedAtUtc = now;
|
||||
record.ApiCalls += apiCalls;
|
||||
record.CpuSeconds += cpuSeconds;
|
||||
record.BandwidthMb += bandwidthMb;
|
||||
record.StorageMb = Math.Max(0m, record.StorageMb + storageMb);
|
||||
record.Currency = string.IsNullOrWhiteSpace(currency) ? record.Currency : currency;
|
||||
record.DateModified = now;
|
||||
record.UserModified = effectiveActor;
|
||||
|
||||
await dbContext.SaveChangesAsync(effectiveActor, cancellationToken);
|
||||
return record;
|
||||
}
|
||||
|
||||
public static async Task<T> WithUserLockAsync<T>(ApplicationDbContext dbContext, string userId,
|
||||
|
|
|
|||
|
|
@ -145,6 +145,7 @@ namespace Yavsc.Models
|
|||
builder.Entity<ApplicationUser>().HasMany<ChatConnection>(c => c.Connections);
|
||||
builder.Entity<ApplicationUser>().Property(u => u.Avatar).HasDefaultValue(Constants.DefaultAvatar);
|
||||
builder.Entity<ApplicationUser>().Property(u => u.DiskQuota).HasDefaultValue(Constants.DefaultFSQ);
|
||||
builder.Entity<ResourceStorageRecord>().HasKey(r => new { r.UserId, r.Path });
|
||||
builder.Entity<ApplicationUser>().HasAlternateKey(u => u.Email);
|
||||
builder.Entity<BlackListed>().HasOne<ApplicationUser>(bl => bl.User);
|
||||
builder.Entity<BlackListed>().HasOne<ApplicationUser>(bl => bl.Owner);
|
||||
|
|
@ -380,6 +381,7 @@ namespace Yavsc.Models
|
|||
public DbSet<Estimate> Estimates { get; set; }
|
||||
public DbSet<Signature> Signatures { get; set; }
|
||||
public DbSet<ResourceUsageRecord> ResourceUsageRecords { get; set; }
|
||||
public DbSet<ResourceStorageRecord> ResourceStorageRecords { get; set; }
|
||||
public DbSet<ResourceSubscription> ResourceSubscriptions { get; set; }
|
||||
public DbSet<ResourceSubscriptionPayment> ResourceSubscriptionPayments { get; set; }
|
||||
public DbSet<AccountBalance> BankStatus { get; set; }
|
||||
|
|
|
|||
13
src/Yavsc.Server/Models/ResourceStorageRecord.cs
Normal file
13
src/Yavsc.Server/Models/ResourceStorageRecord.cs
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
#nullable enable
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
|
||||
namespace Yavsc.Models;
|
||||
|
||||
public sealed class ResourceStorageRecord
|
||||
{
|
||||
[MaxLength(450)]
|
||||
public string UserId { get; set; } = "";
|
||||
[MaxLength(1024)]
|
||||
public string Path { get; set; } = "";
|
||||
public long Bytes { get; set; }
|
||||
}
|
||||
|
|
@ -136,6 +136,79 @@ public sealed class FileSystemApiTests : IClassFixture<BlogsWebServerFixture>
|
|||
"Expected the uploaded file on disk under the owner's personal root");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Same_size_replacement_is_admitted_near_the_limit_and_renaming_keeps_refundability()
|
||||
{
|
||||
_fixture.ResetDatabase();
|
||||
const string owner = "storage-replacement";
|
||||
_fixture.SeedUser(owner, u => u.DiskQuota = 10_000_000);
|
||||
using var http = NewClient(owner);
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
async Task Upload()
|
||||
{
|
||||
using var form = new MultipartFormDataContent();
|
||||
form.Add(new ByteArrayContent(new byte[1048576]), "file", "original.bin");
|
||||
using var response = await http.PostAsync("/api/v1/fs", form, ct);
|
||||
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
|
||||
}
|
||||
await Upload();
|
||||
using (var scope = _fixture.Services.CreateScope())
|
||||
{
|
||||
var policy = scope.ServiceProvider.GetRequiredService<Microsoft.Extensions.Options.IOptions<Yavsc.Settings.ResourceUsageOptions>>().Value;
|
||||
var cpu = (policy.DefaultLimit - policy.StorageMbWeight * 1.5m) / policy.CpuSecondWeight;
|
||||
await ResourceUsageTracker.RecordAsync(scope.ServiceProvider.GetRequiredService<ApplicationDbContext>(),
|
||||
owner, owner, cpuSeconds: cpu, cancellationToken: ct);
|
||||
}
|
||||
await Upload();
|
||||
using var moved = await http.PostAsync("/api/fsc/mvf",
|
||||
new StringContent("""{"id":"original.bin","to":"renamed.bin"}""", System.Text.Encoding.UTF8, "application/json"), ct);
|
||||
Assert.Equal(HttpStatusCode.OK, moved.StatusCode);
|
||||
using var deleted = await http.DeleteAsync("/api/v1/fs/renamed.bin", ct);
|
||||
Assert.Equal(HttpStatusCode.OK, deleted.StatusCode);
|
||||
using var verify = _fixture.Services.CreateScope();
|
||||
var db = verify.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
Assert.Equal(0m, (await db.ResourceUsageRecords.SingleAsync(r => r.UserId == owner, ct)).StorageMb);
|
||||
Assert.Empty(await db.ResourceStorageRecords.Where(r => r.UserId == owner).ToListAsync(ct));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Replacement_and_deletion_refund_tracked_storage_but_not_a_legacy_file()
|
||||
{
|
||||
_fixture.ResetDatabase();
|
||||
const string owner = "storage-refunds";
|
||||
_fixture.SeedUser(owner, u => u.DiskQuota = 10_000_000);
|
||||
using var http = NewClient(owner);
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
async Task Upload(int length)
|
||||
{
|
||||
using var form = new MultipartFormDataContent();
|
||||
form.Add(new ByteArrayContent(new byte[length]), "file", "refunded.bin");
|
||||
using var response = await http.PostAsync("/api/v1/fs", form, ct);
|
||||
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
|
||||
}
|
||||
await Upload(2 * 1048576);
|
||||
await Upload(1048576);
|
||||
using (var scope = _fixture.Services.CreateScope())
|
||||
{
|
||||
var db = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
Assert.Equal(1m, (await db.ResourceUsageRecords.SingleAsync(r => r.UserId == owner, ct)).StorageMb);
|
||||
Assert.Equal(1048576L, (await db.ResourceStorageRecords.SingleAsync(r => r.UserId == owner, ct)).Bytes);
|
||||
await ResourceUsageTracker.RecordAsync(db, owner, owner, cpuSeconds: 20000m, cancellationToken: ct);
|
||||
}
|
||||
using var deleted = await http.DeleteAsync("/api/v1/fs/refunded.bin", ct);
|
||||
Assert.Equal(HttpStatusCode.OK, deleted.StatusCode);
|
||||
Assert.False(File.Exists(PhysicalFile(_fixture, owner, "refunded.bin")));
|
||||
WritePhysicalFile(_fixture, owner, "legacy.bin", new byte[1048576]);
|
||||
using var legacy = await http.DeleteAsync("/api/v1/fs/legacy.bin", ct);
|
||||
Assert.Equal(HttpStatusCode.OK, legacy.StatusCode);
|
||||
using var verify = _fixture.Services.CreateScope();
|
||||
var final = verify.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
var usage = await final.ResourceUsageRecords.SingleAsync(r => r.UserId == owner, ct);
|
||||
Assert.Equal(0m, usage.StorageMb);
|
||||
Assert.Equal(20000m, usage.CpuSeconds);
|
||||
Assert.Empty(await final.ResourceStorageRecords.Where(r => r.UserId == owner).ToListAsync(ct));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Upload_is_refused_before_writing_when_known_storage_would_exceed_quota()
|
||||
{
|
||||
|
|
|
|||
|
|
@ -26,5 +26,8 @@ public sealed class ResourceUsageTermsTests
|
|||
Assert.Contains((Yavsc.Constants.DefaultFSQ * 2L).ToString(), html);
|
||||
Assert.Contains("quota disque", html);
|
||||
Assert.Contains("Découverte porte le quota disque", html);
|
||||
Assert.Contains("remboursent les points de stockage", html);
|
||||
Assert.Contains("suppression de fichiers reste autorisée", html);
|
||||
Assert.DoesNotContain("ne remboursent pas de points", html);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -256,6 +256,92 @@ public sealed class ResourceUsageTests
|
|||
.ResourceUsageRecords.ToListAsync(TestContext.Current.CancellationToken));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Tracked_storage_replacement_and_deletion_refund_only_their_original_volume()
|
||||
{
|
||||
using var provider = NewProvider();
|
||||
using var scope = provider.CreateScope();
|
||||
var db = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
await ResourceUsageTracker.RecordAsync(db, "alice", "alice", cpuSeconds: 5m, storageBytes: 1048576, cancellationToken: ct);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "personal/file", 2 * 1048576, ct);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "personal/file", 1048576, ct);
|
||||
var record = await db.ResourceUsageRecords.SingleAsync(ct);
|
||||
Assert.Equal(2m, record.StorageMb);
|
||||
var usage = scope.ServiceProvider.GetRequiredService<IResourceUsageService>();
|
||||
Assert.Equal(5.6m, (await usage.GetCurrentAsync("alice", "alice", ct)).Consumption);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "personal/legacy", 0, ct);
|
||||
Assert.Equal(2m, record.StorageMb);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "personal/file", 0, ct);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "personal/file", 0, ct);
|
||||
Assert.Equal(1m, record.StorageMb);
|
||||
Assert.Equal(5m, record.CpuSeconds);
|
||||
Assert.Equal(5.3m, (await usage.GetCurrentAsync("alice", "alice", ct)).Consumption);
|
||||
Assert.Empty(await db.ResourceStorageRecords.ToListAsync(ct));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Concurrent_deletions_do_not_refund_the_same_receipt_twice()
|
||||
{
|
||||
using var provider = NewProvider();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
using (var setup = provider.CreateScope())
|
||||
{
|
||||
var db = setup.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "file", 1048576, ct);
|
||||
await ResourceUsageTracker.RecordAsync(db, "alice", "alice", storageBytes: 1048576, cancellationToken: ct);
|
||||
}
|
||||
await Task.WhenAll(Enumerable.Range(0, 5).Select(async _ =>
|
||||
{
|
||||
using var scope = provider.CreateScope();
|
||||
await ResourceUsageTracker.RecordStorageAsync(scope.ServiceProvider.GetRequiredService<ApplicationDbContext>(),
|
||||
"alice", "alice", "file", 0, ct);
|
||||
}));
|
||||
using var verify = provider.CreateScope();
|
||||
Assert.Equal(1m, (await verify.ServiceProvider.GetRequiredService<ApplicationDbContext>()
|
||||
.ResourceUsageRecords.SingleAsync(ct)).StorageMb);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Repeated_pdf_storage_accounting_is_idempotent_and_renamed_files_remain_refundable()
|
||||
{
|
||||
using var provider = NewProvider();
|
||||
using var scope = provider.CreateScope();
|
||||
var db = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||
var ct = TestContext.Current.CancellationToken;
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "pdf/file", 1048576, ct);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "pdf/file", 1048576, ct);
|
||||
await ResourceUsageTracker.MoveStorageAsync(db, "alice", "pdf", "renamed", ct);
|
||||
Assert.Equal("renamed/file", (await db.ResourceStorageRecords.SingleAsync(ct)).Path);
|
||||
await ResourceUsageTracker.RecordStorageAsync(db, "alice", "alice", "renamed/file", 0, ct);
|
||||
Assert.Equal(0m, (await db.ResourceUsageRecords.SingleAsync(ct)).StorageMb);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Relief_request_runs_at_the_limit_but_still_counts_api_and_transfer_costs()
|
||||
{
|
||||
using var provider = NewProvider(o => o.DefaultLimit = 0.01m);
|
||||
using var scope = provider.CreateScope();
|
||||
var context = NewContext(scope.ServiceProvider);
|
||||
context.SetEndpoint(new Endpoint(_ => Task.CompletedTask,
|
||||
new EndpointMetadataCollection(new ResourceUsageReliefAttribute()), "delete"));
|
||||
var ran = false;
|
||||
var middleware = new ResourceUsageTelemetryMiddleware(
|
||||
async ctx =>
|
||||
{
|
||||
ran = true;
|
||||
await ctx.Response.Body.WriteAsync(new byte[1048576], TestContext.Current.CancellationToken);
|
||||
},
|
||||
provider.GetRequiredService<IServiceScopeFactory>(),
|
||||
NullLogger<ResourceUsageTelemetryMiddleware>.Instance);
|
||||
await middleware.InvokeAsync(context, scope.ServiceProvider.GetRequiredService<IResourceUsageService>());
|
||||
Assert.True(ran);
|
||||
var record = await scope.ServiceProvider.GetRequiredService<ApplicationDbContext>()
|
||||
.ResourceUsageRecords.SingleAsync(TestContext.Current.CancellationToken);
|
||||
Assert.Equal(1m, record.ApiCalls);
|
||||
Assert.Equal(1m, record.BandwidthMb);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Negative_counter_deltas_are_explicitly_rejected()
|
||||
{
|
||||
|
|
|
|||
Loading…
Reference in a new issue