El bloque [b2] ya vive en RCLONE_CONF (Infisical /backup-system); los 3
CronJobs vuelven a rclone-conf-infisical y se elimina el secret
out-of-band rclone-conf-b2.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- b2-crown-jewels.sh: sync diario 04:00 de las 2 copias mas recientes de
cada servicio irremplazable (todo menos uptime-kuma) a
b2:chemavx-k3s-backups/crown-jewels/ (~2GB, tier gratuito).
- reddit-intel.sh sube tambien a B2 (retencion 3d).
- verify.sh comprueba Mega (completo) + B2 (crown jewels).
- Config via Secret out-of-band rclone-conf-b2 (mega+b2, kubectl, NO git);
consolidar en Infisical RCLONE_CONF cuando se anada [b2] por UI (la
identity del operator es read-only por API).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
El k3s.yaml del host apunta a 127.0.0.1:6443 (inalcanzable desde el pod).
El script antiguo solo funcionaba porque su KUBECONFIG apuntaba a un
fichero inexistente y kubectl caia en fallback al SA. Se hace explicito:
unset KUBECONFIG + RBAC con apps/deployments,statefulsets get (para
kubectl exec deploy/...) y se retira el mount del kubeconfig.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- backup.sh: fix openclaw (tar-eaba un hostPath stub vacio, 104 bytes/dia
desde hace meses); ahora resuelve el PVC real via local-path.
- Nuevos servicios: ghost-en/zona-exclusion (VACUUM INTO consistente via
node del pod + tar del content), gitea (sqlite .backup + repos + conf,
sin packages 6.4G reconstruibles), researchowl (sqlite backup API via
python3), roswell-pg, umami-pg, infisical-pg, trilium, vaultwarden,
uptime-kuma (sqlite3 .backup), filebrowser-db, home-master (sin ~/.ssh
ni caches), k3s token + /etc/rancher/k3s (cadena de restauracion).
- Validacion por artefacto: gzip -t + tamano minimo; el job sale 1 si
falta algo -> alerta Grafana->Telegram existente.
- rclone-mega: de semanal a diario (03:30); --exclude reddit-intel/.
- Nuevo CronJob reddit-intel-backup en n97 (unico dato del worker fuera
de k3s; su cron local escribia en el mismo disco).
- verify.sh: lista completa (20 servicios + reddit-intel), diario,
ventana de frescura 3 dias (BusyBox-safe).
- backoffLimit 1 (retry recrearia artefactos y romperia la retencion).
- RESTORE.md: runbook con la cadena token->state.db->infisical-secrets.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
El pod fue OOMKilled el 2026-07-10 durante un research: 20 fuentes
concurrentes con PDFs grandes y pdfplumber superaron 1Gi. El scraper
se endurece en paralelo (cap PDF 15MB, contenido 300k chars), pero el
margen extra evita que un pico legítimo mate la tarea en memoria.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Regla homelab-backup-job-failed (kube_job_status_failed{namespace="backup-system"} > 0)
en el grupo homelab-infra; cubre backup, rclone-mega-backup y backup-verify.
Aplicado con kubectl replace + rollout restart de Grafana (namespace manual).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
App corre en docker compose en n97 (semana de calibración). El app-of-apps
no es recursivo: nada de este directorio se sincroniza hasta aplicar a mano
argocd-app.yaml. Checklist de activación en PROMOTION.md (imagen construida,
secrets en Infisical /reddit-intel, copia de la BD al PVC).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merge de feat/portfolio-polymarket-archived con las cifras finales de
cierre (12 trades, +$247.78 realized, 239k evaluaciones, 81 días).
El widget deja de consultar la API (apagada) y pasa a tarjeta estática
con enlace al case study.
Primer paso del apagado ordenado: se detiene el ciclo de evaluación con
postgres aún vivo, para que el dump final sea la foto exacta del cierre
sin ciclos a medias. Postgres y cronjobs se apagan en commits siguientes.
Se retiran las suscripciones on-sync-failed / on-health-degraded: durante
el apagado (F3) generarían ruido sin valor. No hay trigger 'informativo'
viable: escalar a 0 vía git deja la app Synced/Healthy. El resto de apps
conservan sus notificaciones.
Sustituye el widget live (fetch a polymarket.chemavx.xyz/api/summary) por
una tarjeta estática con las cifras finales del paper trading y enlace al
case study en el repo. La tarjeta de servicio apunta al repo de Gitea en
lugar del dominio, que desaparecerá con el apagado.
NO MERGEAR HASTA F2/F3: mientras el bot siga vivo, main mantiene el
widget live. Aplicar esta rama cuando se apague el bot (Fase 3 del plan
de decomisión).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Online-safe sqlite3 .backup at 03:00 Europe/Madrid, integrity check,
7-day retention. Data PVC mounted RW (WAL -shm requirement) but only read.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Tras el plan F1-F4 del scraper, el seed descubre hasta ~285 fuentes en
topics documentados y el cap de 150 descartaba la mitad (incluida la
recursión que encuentra los PDFs de archive.org). 200 cubre casi todos
los resultados directos + margen de recursión. Implica +2-4 min por
research de topic rico y céntimos más de scoring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extension login failed with 'No Bitwarden-Client-Version header provided'
after the Mac browser extension auto-updated. 1.36.0 supports the new
client login flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flip del flag que deja operativo el monitor F0-F2 ya desplegado en
b1c5bc73. Defaults: poll cada 6h, digest al primer TELEGRAM_ALLOWED_USERS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Corre python -m bot.outcomes a las 00:30 UTC (tras metrics-retention a
las 00:10): resoluciones UMA-finales de Gamma -> market_outcomes + reporte
de calibración. Solo-análisis: no toca tablas de trading. Idempotente por
market_id, seguro relanzarlo a mano. Misma imagen del bot; CI bumpa el tag
(cambio correspondiente en polymarket-bot/.gitea/workflows/ci.yml).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The theexclusionzone.com (Ghost EN) zone was never in DOMAINS, so favonia
never updated its origin A records. When the ISP rotated the public IP, the
EN records went stale and Cloudflare returned 522 (apex+www are proxied).
Add all three EN records to DOMAINS and use favonia 1.16.2's per-domain
PROXIED expression so apex+www stay orange while the wildcard and the
chemavx/zona zones stay grey, all from one DDNS instance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds www.zonadeexclusion.com to the Ingress (rules + tls hosts) so cert-manager
re-issues zona-exclusion-tls covering apex + www via HTTP-01, and a Traefik
redirectRegex Middleware that 301s www -> apex (ES canonical = apex). Mirrors the
EN redirect-to-www-https middleware, inverted. DNS for www was fixed in Phase 1
(stale explicit A removed; now resolves via the favonia wildcard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3c: n8n now sources the encryption key from n8n-secret-infisical (Infisical).
Remove the empty CreateOnly stub manifest and the now-dead ignoreDifferences
/data stanza so ArgoCD prunes the old out-of-band n8n-secret.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cut n8n over to the Infisical-synced encryption key (byte-identical, key never
changes -> credentials stay decryptable) and switch RollingUpdate -> Recreate
(485MB SQLite + WAL on RWO must not have two writers during a roll).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Out-of-band Secret -> Infisical homelab/prod /n8n. Parallel-name target
n8n-secret-infisical (Owner), passthrough of the single key encryption-key
(n8n's irrecoverable credentials-encryption root, lifted byte-identical).
Old secret stays live until decommission. ArgoCD-managed -> git->sync.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both AUTHENTIK_SECRET_KEY and AUTHENTIK_POSTGRESQL__PASSWORD (env) -> key
POSTGRES_PASSWORD now sourced from the Infisical-synced secret. Stateless re: DB
(media PVC only), DB-retry logic -> stay RollingUpdate, no Recreate flip.
Applied via kubectl (not ArgoCD-managed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Postgres-first cutover step. Already-initialized PG17 ignores POSTGRES_PASSWORD
on restart (role pw in pg_authid), so this byte-identical roll is benign.
StatefulSet RollingUpdate is ordered terminate-then-create (no two-postmaster
surge), so no Recreate flip. Applied via kubectl (not ArgoCD-managed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Out-of-band Secret -> Infisical homelab/prod /authentik. Parallel-name target
authentik-secrets-infisical (Owner), passthrough of the 2 LIVE keys
(AUTHENTIK_SECRET_KEY, POSTGRES_PASSWORD). The dead orphan key
AUTHENTIK_POSTGRESQL__PASSWORD is intentionally dropped. Old secret stays live
until decommission. Applied via kubectl (not ArgoCD-managed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cut the Deployment over to the Infisical-synced Secret and switch to Recreate
(SQLite/WAL on a RWO PVC must not have two writers during a roll). Applied via
kubectl (not ArgoCD-managed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Out-of-band Secret -> Infisical homelab/prod /vaultwarden. Parallel-name
target vaultwarden-secret-infisical (Owner), passthrough. Old vaultwarden-secret
stays live until decommission. Applied via kubectl (not ArgoCD-managed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Old out-of-band umami-secrets deleted (kubectl); the /data ignoreDifferences
stanza referencing it is now dead. Removed from both the registered app
manifest and the in-dir duplicate. umami-secrets-infisical is Owner-managed
by the operator, not a git-tracked Secret, so no ignore rule is needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DATABASE_URL + APP_SECRET now reference umami-secrets-infisical. Postgres
(3b-i) already healthy on the same secret; umami's wait-for-postgres
initContainer enforces ordering. Old umami-secrets stays live until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POSTGRES_PASSWORD now references umami-secrets-infisical. Adds strategy:
Recreate (single-replica RWO PVC — avoids two postmasters racing PGDATA
during a roll), permanent. Data dir already initialized, so the
byte-identical POSTGRES_PASSWORD is never re-applied on restart (benign).
umami still on old umami-secrets (both live in parallel = zero gap).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3a of the umami-secrets centralization. Adds umami-secrets-infisical synced
from homelab/prod /umami (3 lowercase-hyphenated keys, passthrough).
Parallel name — old out-of-band umami-secrets stays live until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3b of the centralization. All 5 secretKeyRefs now reference
researchowl-secrets-infisical (synced from /researchowl). Also adds
strategy: Recreate (single-replica RWO SQLite/WAL — avoids two pods
briefly sharing /data/researchowl.db during a roll), permanent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3a of the researchowl-secrets centralization. Adds researchowl-secrets-infisical
synced from homelab/prod /researchowl (5 lowercase-hyphenated keys, passthrough).
Parallel name — old out-of-band researchowl-secrets stays live until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Old out-of-band zona-exclusion-secrets deleted (kubectl); SMTP now sourced from
Infisical-backed zona-exclusion-secrets-infisical. Remove the obsolete /data
ignoreDifferences stanza from both app manifests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
InfisicalStaticSecret syncs shared /smtp-gmail -> zona-exclusion-secrets-infisical,
template-remapping UPPER_SNAKE -> hyphenated lowercase smtp-user/smtp-pass to match
the deployment secretKeyRef. Old secret stays live until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Old out-of-band ghost-en-smtp secret deleted (kubectl); SMTP now sourced from
Infisical-backed ghost-en-smtp-infisical. Remove the now-obsolete /data
ignoreDifferences stanza from both app manifests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mail__options__auth__user/pass secretKeyRef
ghost-en-smtp -> ghost-en-smtp-infisical
Recreate roll picks up the Infisical-synced credentials.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
InfisicalStaticSecret syncs shared /smtp-gmail -> ghost-en-smtp-infisical
(parallel name, passthrough SMTP_USER/SMTP_PASS). Old secret stays live until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
InfisicalStaticSecret syncs /gitea-runner -> gitea-runner-secret-infisical
(parallel name). Out-of-band: created via kubectl apply; commit is for record.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7th and final copy of the Telegram bot token leaving git. monitoring is
out-of-band (not ArgoCD-managed), so the live secret is pruned manually with
kubectl. Grafana already reads grafana-telegram-infisical (from /telegram).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Last plaintext secret leaving git; the 6th/last copy of the Telegram token.
ArgoCD prunes the live bot-secrets Secret (and its cleartext
last-applied-configuration annotation). Consumers already on bot-secrets-infisical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3c of migration #2: drop the Secret stanza from auth.yaml; the Middleware
stays and references filebrowser-auth-infisical (synced from homelab/prod
/filebrowser). ArgoCD prunes the old plaintext filebrowser-auth secret — the
bcrypt htpasswd leaves git. Completes migration #2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3b of migration #2: Middleware.spec.basicAuth.secret now references the
Infisical-managed secret. Secret stanza in auth.yaml left in place as fallback
until 3b is verified (401 w/o creds, 200 w/ creds), then removed in 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3a of migration #2: syncs FILEBROWSER_USERS from homelab/prod /filebrowser
via read-only infisical-operator/infisical-auth, template-remapped to the
lowercase 'users' key Traefik basic-auth requires, into parallel-name
filebrowser-auth-infisical. Old plaintext filebrowser-auth secret + the Traefik
Middleware untouched (repoint=3b, removal of the Secret stanza=3c).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3c: hooks now read telegram-notify-infisical (synced from homelab/prod
/telegram). Deleting these 4 manifests prunes the old plaintext secrets — the
bot token leaves git for the remaining 4 namespaces. Completes migration #1.
NOTE: polymarket-bot/bot-secrets still embeds the same token (migration #3).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3b: all 4 hooks' secretKeyRefs now read the Infisical-managed secret.
Old plaintext telegram-notify secrets stay as fallback until 3c.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 3a of telegram-notify migration: each syncs TELEGRAM_BOT_TOKEN/
TELEGRAM_CHAT_ID from shared homelab/prod /telegram via read-only
infisical-operator/infisical-auth, into parallel-name telegram-notify-infisical.
Old plaintext secrets + hooks untouched (repoint=3b, removal=3c).
polymarket-bot: standalone secret only; bot-secrets copy is migration #3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PostSync hook now reads telegram-notify-infisical (synced from
homelab/prod /telegram). Deleting this manifest prunes the old plaintext
secret — the bot token leaves git for researchowl. Security win.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both secretKeyRefs (TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID) now read the
Infisical-managed secret instead of the plaintext-in-git telegram-notify.
Old secret stays as fallback until 3c removes its manifest.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Syncs TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID from shared homelab/prod /telegram
via the read-only infisical-operator/infisical-auth. Targets parallel-name
telegram-notify-infisical; old plaintext telegram-notify stays live until the
hook is repointed (3b) and the old manifest removed (3c).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
api/bot: imagePullSecrets gitea-registry -> gitea-registry-infisical.
dashboard: add gitea-registry-infisical (previously had NO pull secret
despite pulling a private image — fixes latent ErrImagePull-on-reschedule
bug). Old gitea-registry stays as rollback net until decommission.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconstructs the dockerconfigjson pull secret from the shared /registry v3
fields (GITEA_REGISTRY_USERNAME/GITEA_REGISTRY_TOKEN) via the shared
infisical-operator/infisical-auth. Targets parallel-name
gitea-registry-infisical; old gitea-registry stays live until cutover.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
imagePullSecrets: gitea-registry -> gitea-registry-infisical (the
InfisicalStaticSecret-managed dockerconfigjson). Old gitea-registry stays
as rollback net until decommission.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconstructs the dockerconfigjson pull secret from the discrete
GITEA_REGISTRY_USERNAME/GITEA_REGISTRY_TOKEN fields at homelab/prod /registry,
via the shared infisical-operator/infisical-auth. Targets parallel-name
gitea-registry-infisical; old gitea-registry stays live until cutover.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 2: GitOps-managed shared auth layer in infisical-operator ns —
InfisicalConnection (https://infisical.chemavx.xyz) + InfisicalAuth
(kubernetes method, operator SA via TokenReview) + identity-ID Secret
(non-secret UUID; auth is via SA token, so safe in git). To be referenced
cross-namespace by per-app InfisicalStaticSecret CRs later. No real secret yet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 2 Step 2: dedicated SA infisical-token-reviewer bound to
system:auth-delegator (TokenReview), explicit long-lived SA token Secret
(k8s 1.34 doesn't auto-create). ArgoCD ignores /data so selfHeal won't
strip the controller-populated token. Token value never committed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The multi-source variant ($values git ref) caused the infisical app to
prune its own Application object on first sync (controller pruned
Application/argocd/infisical, orphaning the workloads). Switch to a
single-source Helm app with inline valuesObject (same chart 1.9.0, same
DB/Redis passwords) and drop the hand-written tracking-id annotation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deploy infisical-standalone 1.9.0 (Infisical v0.158.0) in a dedicated
'infisical' namespace via a multi-source ArgoCD app (chart + values).
Bundled Postgres (dedicated local-path PVC, 8Gi) + Redis. Ingress via
Traefik + cert-manager (letsencrypt-prod) at infisical.chemavx.xyz.
Crown-jewel secrets (ENCRYPTION_KEY, AUTH_SECRET, SITE_URL) are injected
out-of-band via the 'infisical-secrets' Secret (kubectl, IgnoreExtraneous),
NOT in git. Postgres/Redis passwords guard ClusterIP-only in-cluster
services holding ciphertext; acceptable in values per design review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dockerconfigjson held the old account password (now rotated/revoked)
in plaintext. Manage this image-pull secret out-of-band via kubectl
(like researchowl), so no registry credential lives in git going forward.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A ConfigMap volume is read-only at the FS level, so the Ghost entrypoint's
chown -R on content/ failed ("Read-only file system") and crash-looped.
Copy redirects.yaml onto the writable PVC with an initContainer instead;
still GitOps-sourced and refreshed on every pod start.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Recover ~454 GSC impressions stranded on a 404. Adds a ghost-en-redirects
ConfigMap (content/data/redirects.yaml) 301-redirecting the old slug
/pentagon-second-uap-declassification-may-2026-apollo-12-sandia/ to the
live /pentagon-uap-second-release-may-2026/. Mounted read-only via subPath
so it survives content PVC recreation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Same protection already given to polymarket-bot postgres and n8n: without
it, removing a PVC manifest from git (rename, multi-doc refactor) would
make ArgoCD destroy the volume and its data.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The real manifest is researchowl/argocd-app.yaml, self-managed inside the
app's own synced path; edits to this copy get silently reverted by selfHeal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
researchowl/argocd-app.yaml lives inside the app's own synced path, so it is
the source of truth: annotations applied only to the live object (or to the
argocd/ dump copy) get reverted by selfHeal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Notifications via generic webhook to the Telegram Bot API: the engine's
native telegram service only supports channels and negative group IDs,
so a positive private chat_id never worked ("chat not found").
- Only on-sync-failed and on-health-degraded triggers; on-sync-succeeded
dropped (CI already reports successful deploys).
- Subscribe annotations on polymarket-bot, researchowl and n8n.
- prune: true on polymarket-bot, guarded by Prune=false on the postgres
PVC so removing the manifest from git can never destroy the data.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
chemavxandClaude Fable 5