Zaki/Password-Manager
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: sync ETag concurrency + fix chunk-transfer hang + sync overlay + auto-VACUUM

Browse Source 
Sync optimistic concurrency (ETag/If-Match)
- webdav GET captures the response ETag; PUT sends it back as If-Match so
  the server rejects (412) our write when another device changed the file
  between our pull and push. A 412 re-runs the whole pull→merge→push
  (bounded to 3) so the other device's changes are folded in instead of
  clobbered. Servers without ETags → empty etag → no If-Match → falls
  back to last-write-wins (no regression). onWebdavResult gained a 4th
  etag arg.

Chunked webdav PUT (big vaults no longer black-screen on sync)
- The whole encrypted snapshot base64'd into a single cmd://webdav/put URL
  blew past WebView2's cap → black screen once the vault grew (20MB of
  attachments). PUT bodies now stream through the file/chunk transport and
  commit via a new webdav/put-commit (reads the accumulated buffer).

Chunk-transfer hang fix (root cause of the stuck "Preparing…" sync)
- All chunked transfers (saveFile/writeFile/webdav PUT) share one
  reqId-keyed resolver. A resolved chunk's stale 30s timeout would later
  delete the CURRENT chunk's resolver and fire the wrong res(), leaving
  that chunk's await pending forever. Extracted a single _streamChunks()
  helper whose ack CLEARS the pending timeout, so resolvers stay strictly
  one-at-a-time. Also fixed _webdavCall referencing the Bridge-local cmd()
  from module scope (latent ReferenceError).

Sync busy overlay
- syncStatus() now drives the global busy overlay too, so a running sync
  blocks stray clicks (e.g. the auto-backup "Choose…" picker) and reads
  like the manual backup. The account-mismatch confirm hideBusy()s first
  so it's visible above the overlay.

Auto-VACUUM (reclaim space after deleting large attachments)
- SQLite never shrinks the file on DELETE, so deleting big attachments
  left vault.db bloated (35MB for 11 tiny entries). DB.CompactIfBloated
  VACUUMs when >20% of pages are free AND >~2MB is reclaimable — called on
  startup and after each attachment delete. A healthy small vault pays
  nothing. (Verified: 35MB → 695KB after the deletes.)

Rebuild: BuildAssets + F9 (UMainForm + PM.Database + PM.Handler.Attachments).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
r-zakarya
2026-07-04 00:02:25 +01:00
parent b367d031b5
commit ac909f4f09
6 changed files with 195 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
js/app.js
+104 -49
View File
@@ -16,6 +16,36 @@ const API = (location.pathname.indexOf('/password-manager/') === 0)
const prefResolvers = {};
const fileSaveResolvers = {};
const fileChunkResolvers = {};
// Stream a base64 string to Delphi in URL-sized pieces via cmd://file/chunk,
// each acked (onFileChunkAck) before the next is sent. Returns true when all
// pieces are buffered server-side, false on a stalled chunk. The ack CLEARS
// the pending timeout — without that, a resolved chunk's stale 30s timeout
// would later delete the CURRENT chunk's resolver and hang the transfer
// forever (only the last location.href navigation "wins" per event loop, so
// resolvers must be strictly 1-at-a-time and their timers torn down).
async function _streamChunks(reqId, b64, onProgress) {
const CHUNK = 1000000;
const total = Math.ceil(b64.length / CHUNK);
let done = 0;
for (let off = 0; off < b64.length; off += CHUNK) {
const piece = b64.slice(off, off + CHUNK);
const ok = await new Promise(res => {
const timer = setTimeout(() => {
if (fileChunkResolvers[reqId]) {
delete fileChunkResolvers[reqId];
res(false);
}
}, 30000);
fileChunkResolvers[reqId] = { resolve: res, timer };
window.location.href = 'cmd://file/chunk?reqId=' + encodeURIComponent(reqId) +
'&data=' + encodeURIComponent(piece);
});
if (!ok) return false;
done++;
if (typeof onProgress === 'function') onProgress(Math.round(done / total * 100));
}
return true;
}
let versionResolver = null;
let launchModeResolver = null;
const folderPickResolvers = {};
@@ -241,26 +271,8 @@ const Bridge = (() => {
});
}
return (async () => {
const totalChunks = Math.ceil(b64.length / CHUNK);
let done = 0;
for (let off = 0; off < b64.length; off += CHUNK) {
const piece = b64.slice(off, off + CHUNK);
const ackOk = await new Promise(res => {
fileChunkResolvers[reqId] = res;
cmd('cmd://file/chunk?reqId=' + encodeURIComponent(reqId) +
'&data=' + encodeURIComponent(piece));
setTimeout(() => {
if (fileChunkResolvers[reqId]) {
delete fileChunkResolvers[reqId];
res(false);
}
}, 30000);
});
if (!ackOk) return { ok: false, error: 'chunk transfer failed' };
done++;
if (typeof onProgress === 'function')
onProgress(Math.round(done / totalChunks * 100));
}
const ok = await _streamChunks(reqId, b64, onProgress);
if (!ok) return { ok: false, error: 'chunk transfer failed' };
return await new Promise(resolve => {
fileSaveResolvers[reqId] = resolve;
cmd('cmd://file/save-commit?reqId=' + encodeURIComponent(reqId) +
@@ -276,7 +288,11 @@ const Bridge = (() => {
},
onFileChunkAck(reqId) {
const r = fileChunkResolvers[reqId];
if (r) { delete fileChunkResolvers[reqId]; r(true); }
if (r) {
delete fileChunkResolvers[reqId];
if (r.timer) clearTimeout(r.timer);
r.resolve(true);
}
},
onFileSaveResult(reqId, ok, path, err) {
const r = fileSaveResolvers[reqId];
@@ -332,26 +348,8 @@ const Bridge = (() => {
});
}
return (async () => {
const totalChunks = Math.ceil(b64.length / CHUNK);
let done = 0;
for (let off = 0; off < b64.length; off += CHUNK) {
const piece = b64.slice(off, off + CHUNK);
const ackOk = await new Promise(res => {
fileChunkResolvers[reqId] = res;
cmd('cmd://file/chunk?reqId=' + encodeURIComponent(reqId) +
'&data=' + encodeURIComponent(piece));
setTimeout(() => {
if (fileChunkResolvers[reqId]) {
delete fileChunkResolvers[reqId];
res(false);
}
}, 30000);
});
if (!ackOk) return { ok: false, error: 'chunk transfer failed' };
done++;
if (typeof onProgress === 'function')
onProgress(Math.round(done / totalChunks * 100));
}
const ok = await _streamChunks(reqId, b64, onProgress);
if (!ok) return { ok: false, error: 'chunk transfer failed' };
return await new Promise(resolve => {
fileWriteResolvers[reqId] = resolve;
cmd('cmd://file/write-commit?reqId=' + encodeURIComponent(reqId) +
@@ -9889,15 +9887,45 @@ const SYNC_PREFS = {
};
let _webdavResolvers = {};
Bridge.onWebdavResult = function(reqId, status, payload) {
Bridge.onWebdavResult = function(reqId, status, payload, etag) {
const r = _webdavResolvers[reqId];
if (!r) return;
delete _webdavResolvers[reqId];
r({ status: status | 0, payload: payload || '' });
r({ status: status | 0, payload: payload || '', etag: etag || '' });
};
function _webdavCall(method, url, user, pwd, dataB64) {
// method: 'get'|'put'|'test'. opts.ifMatch → sent as If-Match on a put so
// the server rejects (412) a write when the remote changed since our pull.
function _webdavCall(method, url, user, pwd, dataB64, opts) {
opts = opts || {};
const reqId = 'dav_' + Date.now() + '_' + Math.random().toString(36).slice(2, 8);
const CHUNK = 1000000;
// Large PUT bodies (a big encrypted vault) can't ride in a single
// cmd:// URL — WebView2 caps it and the navigation blanks the page.
// Stream the base64 through the file/chunk transport, then commit via
// webdav/put-commit (which reads the accumulated buffer server-side).
if (method === 'put' && dataB64 && dataB64.length > CHUNK) {
return (async () => {
const ok = await _streamChunks(reqId, dataB64);
if (!ok) return { status: 0, payload: 'chunk transfer failed', etag: '' };
return await new Promise(resolve => {
_webdavResolvers[reqId] = resolve;
let q = 'cmd://webdav/put-commit'
+ '?reqId=' + encodeURIComponent(reqId)
+ '&url=' + encodeURIComponent(url)
+ '&user=' + encodeURIComponent(user || '')
+ '&pwd=' + encodeURIComponent(pwd || '');
if (opts.ifMatch) q += '&ifmatch=' + encodeURIComponent(opts.ifMatch);
window.location.href = q;
setTimeout(() => {
if (_webdavResolvers[reqId]) {
delete _webdavResolvers[reqId];
resolve({ status: 0, payload: 'timeout', etag: '' });
}
}, 60000);
});
})();
}
return new Promise(resolve => {
const reqId = 'dav_' + Date.now() + '_' + Math.random().toString(36).slice(2, 8);
_webdavResolvers[reqId] = resolve;
let q = 'cmd://webdav/' + method
+ '?reqId=' + encodeURIComponent(reqId)
@@ -9905,11 +9933,12 @@ function _webdavCall(method, url, user, pwd, dataB64) {
+ '&user=' + encodeURIComponent(user || '')
+ '&pwd=' + encodeURIComponent(pwd || '');
if (dataB64) q += '&data=' + encodeURIComponent(dataB64);
if (opts.ifMatch) q += '&ifmatch=' + encodeURIComponent(opts.ifMatch);
window.location.href = q;
setTimeout(() => {
if (_webdavResolvers[reqId]) {
delete _webdavResolvers[reqId];
resolve({ status: 0, payload: 'timeout' });
resolve({ status: 0, payload: 'timeout', etag: '' });
}
}, 60000);
});
@@ -10245,6 +10274,10 @@ function syncStatus(text) {
if (text) { el.textContent = text; el.style.display = ''; }
else { el.textContent = ''; el.style.display = 'none'; }
}
// Also drive the global busy overlay so a running sync blocks stray
// clicks (e.g. the auto-backup "Choose…" picker) and reads the same
// as the manual backup. Empty text = clear.
if (text) showBusy(text); else hideBusy();
// Lock the Sync/Test buttons while a run is in flight so the user
// can't double-click a second concurrent sync.
const active = !!text;
@@ -10254,14 +10287,15 @@ function syncStatus(text) {
});
}
async function runSyncNow() {
async function runSyncNow(_attempt) {
_attempt = _attempt || 0;
const cfg = await loadSyncConfig();
if (!cfg) return toast('Bridge not available', 'error');
if (!cfg.url) return toast('Sync not configured', 'warning');
if (!cfg.encPwd) return toast('Set the sync password first', 'warning');
if (!state.cryptoKey) return toast('Vault is locked', 'warning');
toast('Syncing…');
if (_attempt === 0) toast('Syncing…');
let merged = { added: 0, updated: 0, deleted: 0 };
// Fail-fast connectivity: hit the remote FIRST so a dead server /
@@ -10285,6 +10319,10 @@ async function runSyncNow() {
syncStatus('');
return toast('Pull failed: HTTP ' + pullResp.status, 'error');
}
// ETag of the version we just pulled — sent back as If-Match on the
// push so the server rejects our write if another device changed the
// file in between (optimistic concurrency, avoids lost updates).
const remoteEtag = pullResp.etag || '';
// Decrypt + cross-account guard BEFORE touching local state, so a
// wrong sync password or a foreign account aborts cleanly.
@@ -10300,6 +10338,9 @@ async function runSyncNow() {
}
if (remoteSnap && remoteSnap.username && state.username &&
remoteSnap.username !== state.username) {
// Drop the busy overlay so this confirm (z-index below it)
// is visible; a later syncStatus() re-shows it if we continue.
hideBusy();
const ok = await confirmDialog({
title: 'Different account on remote',
message: 'The remote snapshot belongs to <b>' +
@@ -10369,7 +10410,21 @@ async function runSyncNow() {
const bodyBytes = new TextEncoder().encode(JSON.stringify(container, null, 2));
syncStatus('Pushing…');
const r = await _webdavCall('put', cfg.url, cfg.user, cfg.pwd,
bytesToBase64(bodyBytes));
bytesToBase64(bodyBytes),
{ ifMatch: remoteEtag });
// 412 Precondition Failed = the remote changed since our pull
// (another device pushed). Re-run the whole pull→merge→push so we
// fold in their changes instead of clobbering them. Bounded to a
// few tries to avoid a livelock against a device syncing in a hot
// loop.
if (r.status === 412) {
syncStatus('');
if (_attempt < 3) {
toast('Remote changed — re-syncing…');
return await runSyncNow(_attempt + 1);
}
return toast('Sync gave up after repeated remote changes — try again', 'error');
}
if (!(r.status >= 200 && r.status < 300)) {
syncStatus('');
return toast('Push failed: HTTP ' + r.status, 'error');