# BE System Design — Reconcile Token > Implementation source of truth. Human-facing review pages: [`index.html`](../../docs/features/reconcile-token/index.html). > Source draft: `docs/drafts/reconcile-token-draft.md`. > Schema reference: [`../social-schema-definition/be-system-design.md`](../../docs/archive/social-schema-definition/be-system-design.md) (v5). > Pipeline reference: [`../social-recording-process/be-system-design.md`](../../docs/features/social-recording-process/be-system-design.md) — the real-time path this feature reuses. > Open defects this feature closes or touches: [`../social-recording-refactor/recording-holes.md`](../../docs/features/social-recording-refactor/recording-holes.md) H-009 · H-013 · H-015. > Generator contract: [`../../../guides/social-generator-rubric.md`](../social-generator-rubric.md). ## 0. Scope & Context **Purpose.** Two operations that the real-time `token.flow` path cannot perform: 1. **Reconcile** — record tokens from a past time range that were never recorded at all (the stream only sees live events). 2. **Retry** — re-attempt social URLs that ended in a non-terminal failure state (the stream records each token exactly once and never returns). ### 0.1 The idempotency model — convergence This is the single idea the whole design rests on. Stated by the user: > Repeated invocation drives the system to **the maximum set processable at that moment**. Transient > failure is resolved by retry until that final state is reached. Changes in the outside world > (a deleted tweet, a moved metric) are **outside** this definition — once a URL is `ok`, calling > the function any number of times leaves its value unchanged. Three consequences, each load-bearing: | Consequence | Where it shows up | |---|---| | The unit of idempotency is the **URL**, not the token | `/reconcile` works on the token axis (tokens absent from our DB); `/retry` works on the URL axis (URLs not yet converged). Neither alone converges the system; together they do. | | `ok` is an **absorbing state** — never re-entered | Already the existing structure: rubric R-8 ("first creation is final") and H-008 (re-observation does not overwrite fields). This design does not introduce it, it depends on it. | | The decomposition axis for `FetchStatus` is **"can retry change this?"** | §1.2. `skipped_paid` and `unsupported` are *not* absorbing — enabling paid calls or shipping a generator makes them processable. A generator contract violation *is* absorbing — it fails identically forever. | ### 0.2 In scope | # | Deliverable | Boundary | |---|---|---| | 1 | `POST /internal/social-recording/reconcile` | Token axis. Records tokens absent from `tokens`; never touches ones already there. | | 2 | `POST /internal/social-recording/retry` | URL axis. Re-attempts non-converged URLs. Also the cron entry point (empty body → defaults). | | 3 | `FetchStatus` decomposition — `invalid` · `exhausted` | Vocabulary only. No new collection, no new field, no new index, no migration. | | 4 | `ApiKeyGuard` | The repo's first guard. Both endpoints trigger paid external calls. | | 5 | `SolTrackerApiSdk.getTokensByRange` | The repo could previously only ask about addresses it already knew. | | 6 | URL-scoped partial update on `tokens` | Closes H-015. Required by (2) — a whole-document `$set` would erase the other URLs. | | 7 | Entry-URL-scoped link replacement in `SocialGraphWriter.write` | Closes H-013. Makes retry produce no duplicate `token_links`. | ### 0.3 Out of scope Update Metrics (draft §"Update Metrics (TBD)") is explicitly excluded by user decision — see §7. ### 0.4 Non-functional requirements `[nfr-constraints] = none (platform defaults inherited)`. No scale, latency, security or availability target beyond what the system-level design already sets. ### 0.5 Cross-cutting aspects `[aspect-decisions]`, collected in Phase 1.6: - `[INCLUDED]` **Configurability** — §3.6. Signal: draft says "af count 기준도 설정가능하다"; `afGroupIds` fixed by env (user decision). - `[INCLUDED]` **Failure handling** — §1.2, §3.3, §5, §6. Signal: draft says "멱등성을 보장해야한다" and "실패한 토큰의 url 을 재시도한다"; two external dependencies. - `[INCLUDED]` **Overlap prevention (lock/lease)** — §3.5, §6.3. Signal: cron and the manual endpoint can hit the same token; the new URL-level entry point bypasses the existing token lock unless it takes it explicitly. - `[INCLUDED]` **Call boundary / abuse defence** — §2.3, §3.1. Signal: this repo has **no guard component at all** (`sol-tracker-test.controller.ts:11`), and `app.module.ts:26` warns that a guardless endpoint using our upstream key is an unauthenticated open proxy. These endpoints additionally spend money. - `[EXCLUDED]` Observability · Lifecycle · Access control & multi-tenancy · Data lifecycle · Cron catch-up — see §7. --- ## 1. Database Schema **No new collection. No new field. No new index. No migration script.** The only change is that one existing field's value domain widens by two. ```mermaid erDiagram TOKENS ||--o{ TOKEN_SOCIAL_URL : "embeds (array)" TOKENS ||--o| TOKEN_WEB : "embeds (single)" TOKENS ||--o{ TOKEN_LINKS : "joined by address" TOKEN_LINKS }o--|| CONTENTS : "object_id (polymorphic)" TOKEN_LINKS }o--|| ACCOUNTS : "object_id (polymorphic)" TOKEN_LINKS }o--|| VENUES : "object_id (polymorphic)" TOKENS { ObjectId _id PK string address UK string symbol date first_transfer_at string_array fingerprints TokenWeb web TokenSocialUrl_array social_urls date created_at date updated_at } TOKEN_SOCIAL_URL { string url FetchStatus status date attempted_at int attempts } TOKEN_WEB { string domain date domain_created_at date scan_time } TOKEN_LINKS { ObjectId _id PK string token_address FK string object ObjectId object_id FK string entry_url string platform string subtype int link_depth date created_at } CONTENTS { ObjectId _id PK string platform_key UK } ACCOUNTS { ObjectId _id PK string platform_key UK } VENUES { ObjectId _id PK string platform_key UK } ``` ### 1.1 Why no migration MongoDB is schemaless; the enum lives only in the TypeScript `FetchStatus` and mongoose's `@prop({ enum })` validator. Widening it requires zero database work. The repo has never been deployed, so no stored `error` row needs reinterpretation either — and even if one existed, the retry loop would hammer it up to `maxAttempts` and then freeze it as `exhausted`, which still converges. This is why `Lifecycle` is excluded (§7). ### 1.2 `FetchStatus` — 8 values, 5 absorbing / 3 retryable The decomposition axis is **"can a retry change this?"** — nothing else. | Value | Nature | Meaning | Why that nature | |---|---|---|---| | `ok` | absorbing | Collected; graph recorded. | The user's definition verbatim: once `ok`, repeated calls leave the value unchanged. Outside-world change is outside this definition. | | `not_found` | absorbing | Target does not exist (deleted, wrong handle). | Nothing we can do; retry returns the same answer. | | `blocked` | absorbing | Target-side condition — private, invite-only, IP block. | `fetchStatusOfError` already splits `TARGET_BLOCKED` for exactly this reason: it is not *our* condition. | | **`invalid`** ＋new | absorbing | The generator returned a response that **violates its contract** — what `SocialGraphWriter.plan()` throws on: empty `groups`, cyclic quoting, unreachable group, `parentRef` pointing at a contentless group. | **The absence of this value is H-009.** A contract violation fails **identically every time**, but it is currently lumped into `error`, so the retry cron would hammer a URL that can never open. Only a code fix changes it. | | **`exhausted`** ＋new | absorbing | `attempts` reached `maxAttempts`. | **It exists for the index** — see §1.4. If exhausted URLs stayed as `error`, the cron's only query axis would fill up with dead rows over time. | | `error` ~mod | retryable | External call failure · transient DB error. **Change: contract violations left this bucket.** | Both are changed by retrying. `attempts` increments; on reaching the cap it freezes as `exhausted`. | | `unsupported` | retryable | No generator for this platform yet. | Becomes processable the moment a generator ships. **Zero external calls, so `attempts` never increments** — therefore it can never become `exhausted`, which is correct: it should be retried forever. | | `skipped_paid` | retryable | Paid calls are off. | Becomes processable when `SOCIAL_RECORD_ENABLE_PAID` is turned on. Also zero calls. | **What the split actually buys.** The retry query becomes `{ 'social_urls.status': { $in: ['error','unsupported','skipped_paid'] } }` — a **pure index predicate**. Written as `$ne: 'ok'` it would be a negation, which cannot narrow an index range and degenerates into a full collection scan. **The cost — an absorbing state is a dead end by design.** The cron never looks at `invalid` or `exhausted` again. Fixing a generator bug does **not** auto-recover past `invalid` URLs. The only escape is a human calling `/retry` with `statuses=[invalid]` (§2.2). The same applies to `exhausted` after paid calls are enabled. ### 1.3 `tokens` | Field | Type | Required | Description | |---|---|---|---| | `_id` | ObjectId | ● | Surrogate key. | | `address` | string (UK) | ● | Mint address. Tokens are never reclassified or merged, so the address *is* the identity (explicit D-2 exception). Join target of `token_links.token_address`. | | `symbol` | string | ○ | Display copy; SoT is sol-alpha-finder-tracker. **The retry path reads this and passes it as `fetchOptions.tokenName`** — which is why retry needs no upstream call. | | `first_transfer_at` | date | ○ | Age anchor. Immutable, so copying is safe (D-4). | | `fingerprints` | string[] | ● | Flat `"kind:value"` array, default `[]`. **Direct reason this feature introduces partial update** — today `upsertByAddress`'s `$set: {...token}` blanks it (H-015). | | `web` | TokenWeb | ○ | Web-fingerprint block; no writer yet (v5 T-M3). Same victim of whole-document overwrite. | | `social_urls` | TokenSocialUrl[] | ● | Every social URL on this token plus each one's last outcome. **Change: element `status` domain widens to 8 values.** Array shape and fields unchanged. | | `created_at` | date | ● | "When we first saw this token" (v5 T-M4). | | `updated_at` | date | ● | Last modification. | **Embedded `social_urls[]` element (`TokenSocialUrl`)** | Field | Type | Required | Description | |---|---|---|---| | `url` | string | ● | Normalised canonical URL; only successfully normalised values enter (v5 rule ②). **Match key for the partial update.** | | `status` ~mod | FetchStatus | ● | Last outcome. Two values added (§1.2). **Sole basis for retry-target selection.** | | `attempted_at` | date \| null | ○ | Last time an external call was **actually made**; `null` if zero calls. Drives the cron's back-off. | | `attempts` | int | ● | External call count, default 0. **Sole basis for the `exhausted` transition.** Zero-call paths never increment, so they never exhaust. | ### 1.4 Indexes — unchanged, and why the existing four already cover the new access paths | Index | Kind | Why (logical reason) | |---|---|---| | `uniq_tokens_address (address)` | Unique | Single-document lookup + address uniqueness. **New role in this feature** — reconcile decides "which of these N addresses have we already seen" with one `find({address: {$in: […]}}, {address: 1})`. Calling `findByAddress` per address would make 100 queries out of 100 addresses. | | `idx_tokens_social_url_status ('social_urls.status')` | Multikey | **The retry cron's only query axis.** `{$in: […]}` is a set of equality conditions, so it rides the multikey index directly. `$ne: 'ok'` would not — a negation cannot narrow an index range. **This is precisely why `exhausted` is its own value**: if cap-reached URLs stayed `error`, this index would keep returning rows that can never open. | | `idx_tokens_fingerprints (fingerprints)` | Multikey | Fingerprint clustering. Unrelated to this feature. | | `idx_tokens_discovered (created_at DESC)` | Secondary | Recency sort. Unrelated. **Reconcile's time range does not use it** — the range is filtered against the tracker's `last_transfer_at`, not our `created_at` (§5.2). | ### 1.5 `token_links` — unchanged, including the deliberate absence of a unique index A unique index on `(token_address, object_id, entry_url)` would structurally destroy re-observation history, which schema-v5 Phase 1.6 decided against. Retry-induced duplication (H-013) is instead prevented **in the write path**: `write()` deletes that `(token_address, entry_url)`'s existing links and re-inserts (§6.4). The delete predicate `{token_address, entry_url}` rides the prefix of `idx_links_token_linked (token_address, created_at)` — `token_address` narrows to a small row set and `entry_url` is compared only within it. **Therefore no `entry_url` index is added.** --- ## 2. API Specification Two new endpoints. All paths sit under the global prefix `/api/v1`. The `internal/` prefix follows the most recent controller in the repo (`sol-tracker-test.controller.ts`), not the legacy double-`v1` shape of `notes`/`users`. ### 2.1 `POST /internal/social-recording/reconcile` — token axis Auth: `x-api-key`. **Request** | Field | Type | Req | Default | Description | |---|---|---|---|---| | `startDate` | date-time | ● | — | Range start. **Measured against the upstream's `last_transfer_at`**, not our `created_at`. | | `endDate` | date-time | ● | — | Range end. Must be after `startDate`. | | `minAfCount` | int ≥1 | ○ | `socialRecord.afInvestorThreshold` (3) | AF investor floor (`investorCount >= minAfCount`) — the same bar the live stream handler uses. | | `limit` | int 1–100 | ○ | 20 | Cap on **tokens recorded**. This does not determine response time; `deadlineMs` does. | | `deadlineMs` | int 1000–300000 | ○ | 45000 | Time cap, measured from request entry (includes the upstream range call). No further token is *started* past it. | **Response 200** — `data` = per-token results; `meta` = `{ scanned, filtered, alreadySeen, targeted, processed, remaining, stoppedBy }`. `alreadySeen` counts tokens skipped because a document already exists. ⚠️ It does **not** mean collection finished — since url-lock L-3 the document means "seen", and unconverged URLs may remain (those are `/retry`'s axis). `remaining > 0` means "call again with the same parameters". `stoppedBy ∈ {limit, deadline, drained}`. **Errors** — `400` validation (`startDate >= endDate`, out-of-range `limit`); `401` bad/missing key; `502` upstream range query finally failed (nothing is recorded — partial results are not returned). ### 2.2 `POST /internal/social-recording/retry` — URL axis Auth: `x-api-key`. Body optional; an external scheduler calls it with an empty body and gets the defaults, which *are* the cron behaviour. **No separate cron endpoint exists** — see §3.5. | Field | Type | Req | Default | Description | |---|---|---|---|---| | `statuses` | FetchStatus[] | ○ | `[error, unsupported, skipped_paid]` | Which URLs to re-attempt. **Putting `invalid` or `exhausted` here is the only escape from an absorbing state.** `ok` is rejected with 422. | | `tokenAddresses` | string[] | ○ | — | Narrow to specific tokens; for investigating one token. | | `limit` | int 1–200 | ○ | 50 | Cap on **URLs** (not tokens). | | `deadlineMs` | int 1000–300000 | ○ | 45000 | Time cap. **Applies to empty-body cron calls too — the only thing stopping the cron running unbounded.** | | `minIntervalMinutes` | int ≥0 | ○ | 60 | Skip URLs whose `attempted_at` is newer than this. **The only back-off mechanism.** `0` forces immediate retry. Zero-call URLs have `attempted_at = null` and always pass. | **Response 200** — `meta` = `{ candidateUrls, skippedByInterval, processed, converged, remaining, stoppedBy }`. `converged` staying at 0 across runs is the signal that retry is spinning uselessly; with Observability out of scope, it is the only such signal. **Errors** — `400` validation; `401`; `422` `STATUS_NOT_RETRYABLE` when `statuses` contains `ok` (re-opening a success breaks the convergence definition; if re-collection is genuinely wanted, that belongs to the re-observation policy H-007/H-008, not here). ### 2.3 Authentication — an escalation this feature is forced to make This repo has **no guard component**. Both endpoints trigger paid social calls, so leaving them unauthenticated reproduces the exact shape `app.module.ts:26` warns about, plus a bill. The alternative used for `SolTrackerTestApiModule` — excluding the module in prod — **is not available here**: reconcile exists to backfill production data, so removing the route in prod kills the feature. > **Phase 3 validation, conditional verdict (recorded, not overridden).** The strongest alternative > is a **network-level boundary** (private subnet / internal-only port), which costs zero application > code. `ApiKeyGuard` is the right answer **only if these endpoints are reachable from the public > internet**. That fact was not established during design. If the deployment is internal-only, the > guard is a pure addition and can be dropped. --- ## 3. Service & Class Design ```mermaid flowchart TD ApiKeyGuard["ApiKeyGuard
«Guard»"]:::added SocialReconcileController["SocialReconcileController
«Controller»"]:::added SocialReconcileService["SocialReconcileService
«Service»"]:::added SocialContractError["SocialContractError
«Error»"]:::added SocialRecordProcessor["SocialRecordProcessor
«Service»"]:::modified SocialGraphWriter["SocialGraphWriter
«Service»"]:::modified TokenRepository["TokenRepository
«Repository»"]:::modified TokenLinkRepository["TokenLinkRepository
«Repository»"]:::modified SolTrackerApiSdk["SolTrackerApiSdk
«External»"]:::modified RedlockSDK["RedlockSDK
«External · existing»"] ApiKeyGuard -->|uses| SocialReconcileController SocialReconcileController -->|uses| SocialReconcileService SocialReconcileService -->|uses| RedlockSDK SocialReconcileService -->|uses| SolTrackerApiSdk SocialReconcileService -->|uses| TokenRepository SocialReconcileService -->|uses| SocialRecordProcessor SocialRecordProcessor -->|uses| SocialGraphWriter SocialRecordProcessor -->|uses| RedlockSDK SocialGraphWriter -->|uses| TokenLinkRepository SocialGraphWriter -.->|depends| SocialContractError classDef added fill:#d4edda,stroke:#28a745,stroke-width:2px classDef modified fill:#fff3cd,stroke:#ffc107,stroke-width:2px ``` ### 3.1 Key members | Class | Change | Key methods / fields | |---|---|---| | `ApiKeyGuard` | NEW | `canActivate(context: ExecutionContext) boolean` | | `SocialReconcileController` | NEW | `reconcile(dto: ReconcileTokensReqDto) WrapperResWithMetaDto`, `retry(dto?: RetryUrlsReqDto) WrapperResWithMetaDto` | | `SocialReconcileService` | NEW | `reconcileByRange(command: ReconcileCommand) ReconcileOutcome`, `retryUnconverged(command: RetryCommand) RetryOutcome`, `private resolveFinalStatus(current: TokenSocialUrl, status: FetchStatus, attempted: boolean) FetchStatus`, `private selectRetryTargets(tokens: TokenEntity[], command: RetryCommand) RetryTarget[]`, `private deadlineReached(startedAt: number, deadlineMs: number) boolean` | | `SocialContractError` | NEW | Extends `Error`. Carries the violated-rule message plus `entryUrl`. | | `SocialRecordProcessor` | MODIFIED | **＋** `recordOneUrl(target: RetryTarget) TokenSocialUrlInput` · **~** `collectOne` (contract-violation branch) · **~** `fetchStatusOfError(error: unknown)` — signature changes from `code` to `error`, because a code alone cannot express a contract violation. `record` and `outcome` unchanged. | | `SocialGraphWriter` | MODIFIED | **~** `write` (entry-URL delete before link append) · **~** `plan` (throws `SocialContractError` instead of `Error`; checks and messages unchanged). `resolve*` unchanged. | | `TokenRepository` | MODIFIED | **＋** `findExistingAddresses(addresses: string[]) Set`, `findByUrlStatuses(statuses: FetchStatus[], addresses?: string[]) TokenEntity[]`, `recordSocialUrlResult(address: string, url: string, status: FetchStatus, attemptedAt: Date \| null) void`. Existing three unchanged. | | `TokenLinkRepository` | MODIFIED | **＋** `deleteByTokenAndEntryUrl(tokenAddress: string, entryUrl: string) number`. `appendMany` unchanged. | | `SolTrackerApiSdk` | MODIFIED | **＋** `getTokensByRange(params: TokenRangeQuery) TrackedToken[]`, `private transformTrackedToken(raw: RawTrackedToken) TrackedToken`. Existing three unchanged. | ### 3.2 Data Source Map Resolved **before** the class structure, because the chosen source determines the injected dependency. | Data | Candidate sources | Chosen + reason | Cost | |---|---|---|---| | Token list for a time range | ⓐ tracker `GET /api/v1/tokens` ⓑ our `tokens` ⓒ `token.flow` events | **ⓐ** — ⓑ by definition does not contain tokens we have never seen; ⓒ has no historical events left in the queue. | New external dependency; one new SDK method. **Time axis is the tracker's `last_transfer_at`**, not our `created_at`. | | AF investor count | ⓐ `investorCount` in the same response ⓑ event `totalInvestedAlphaFinders` ⓒ our DB | **ⓐ** — already in the response, zero extra calls. | Upstream already applies `investorCount > 1 && totalUsedInDollar > 1000` server-side, so lowering `minAfCount` below 2 changes nothing. | | A token's four social URLs | ⓐ already present in the `GET /tokens` response ⓑ re-fetch via `getTokenByAddress` | **ⓑ** — ⓐ would route the manual path through a *different DTO from a different endpoint* than the live-stream path, so the two could diverge. ⓑ reuses `record()` untouched, so no second code path exists to drift. | One extra upstream call per token. With `limit = 20`, twenty calls; in-house upstream, sub-second. | | Retry candidates (URL, status, attempts) | ⓐ our `tokens.social_urls[]` ⓑ tracker re-fetch | **ⓐ** — all of it is in our own document. | None. **Zero upstream calls on the retry path.** | | `tokenSymbol` | ⓐ our `tokens.symbol` ⓑ tracker re-fetch | **ⓐ** | Display copy, may be stale; it feeds the telegram fetcher's `tokenName`, so impact is minor. | ### 3.3 Failure handling — where `invalid` is decided `collectOne`'s catch must check **`SocialContractError` before `ExternalFetchError`**. Reversed, the earlier condition matches first and contract violations collapse back into `error`. ``` catch (error) { if (error instanceof SocialContractError) → FetchStatus.INVALID if (error instanceof ExternalFetchError && code === TARGET_BLOCKED) → FetchStatus.BLOCKED otherwise → FetchStatus.ERROR } ``` `attempted` is taken from what the generator reported, not inferred. A contract violation does not mean no external call happened — it means the response we *did* fetch broke the contract. ### 3.4 `exhausted` is decided in the service, not the processor `SocialRecordProcessor` does not know the cumulative `attempts` — its `outcome()` reports "how many calls did *this* invocation make". `SocialReconcileService.resolveFinalStatus` owns the cap: 1. If `status` is absorbing (`ok` · `not_found` · `blocked` · `invalid`), return it unchanged. 2. If `status` is retryable but `attempted === false`, return it unchanged. **This line is why `unsupported` and `skipped_paid` can never become `exhausted`.** 3. Otherwise compute `current.attempts + 1`; at or above `maxAttempts`, return `exhausted`. The read-then-decide is safe because the token lock is held throughout. ### 3.5 Locking — two layers | Layer | Key | Prevents | On failure | |---|---|---|---| | Job lock | `social-reconcile:reconcile` / `social-reconcile:retry` | Two runs of the **same batch kind** overlapping and selecting the same candidates. | `200` + `meta.skipped = 'already running'`. **Not `409`** — overlap is normal operation, and a scheduler receiving `409` would raise a false alarm. | | Token lock | `social-record:{address}` (existing key, reused) | A batch and the **live stream** touching the same token. `recordOneUrl` must take it; without that, the A-plan URL entry point bypasses the only serialisation the pipeline has. | Skip that URL; the next cycle picks it up. | Both use `tryLock` (immediate `null`), never `acquireLock` (which waits ~250 s then throws). Waiting is wrong: if a batch is already running it is already handling those targets. **`DistributedCronHelper` is deliberately not used.** It returns `false` unconditionally when `appEnv !== 'prod'`, which would make the manual API silently do nothing in dev and stage. **No cron-only endpoint exists.** Calling `POST /retry` with an empty body already produces exactly the cron behaviour, so the route serves both the operator and any scheduler. > ⚠️ **Reversed 2026-08-13 (`9cdf1c2`).** This paragraph used to continue: *"This repo has no > `@nestjs/schedule` and zero `@Cron` decorators; the sibling tracker repo's pattern is an > external scheduler calling an HTTP endpoint."* That is **no longer true.** `SocialRetryCron` > (`social-retry.cron.ts`) now runs in-process on `@Cron(EVERY_HOUR)` and calls > `SocialReconcileService.retryUnconverged()` **directly, not over HTTP** — it lives in > `SocialRecordingModule` so it shares the service's Redlock and repositories with the stream. > > The sentence above still holds: no cron-only *endpoint* was added. > > **What the dropped constraint was protecting.** With an external scheduler, "nobody calls it, > so it does not run" was the defence — non-prod environments simply had no scheduler. In-process > that defence is gone and only `ProductionOnlyCronHelper` remains (it returns `false` without > running the job when `appEnv !== 'prod'`). A `@Cron` that forgets that wrapper would fire in > local and stage, spending upstream calls and money. `social-reconcile.e2e-spec.ts` now pins > **that** invariant instead of the dependency's absence. ### 3.6 Configuration | Key | Env | Default | API-overridable | Note | |---|---|---|---|---| | `app.secretKey` | `APP_SECRET_KEY` | — (required) | no | Joi-required so an empty value cannot pass as "both empty, therefore equal". | | `socialRecord.afGroupIds` | `SOCIAL_RECORD_AF_GROUP_IDS` | — (required) | **no** | Upstream requires it; the caller has no business knowing which groups we watch. | | `socialRecord.maxAttempts` | `SOCIAL_RECORD_MAX_ATTEMPTS` | 5 | no | `exhausted` threshold. | | `socialRecord.reconcileLimit` / `retryLimit` | … | 20 / 50 | yes | Batch caps. | | `socialRecord.deadlineMs` | `SOCIAL_RECORD_DEADLINE_MS` | 45000 | yes | Time cap. | | `socialRecord.retryMinIntervalMinutes` | … | 60 | yes | Back-off. | | `socialRecord.reconcileLockTtlMs` / `retryLockTtlMs` | … | — | no | **Not a batch-duration bound (2026-08-13).** Both jobs now hold the lock via `tryUsingExtended`, which auto-extends while held, so the TTL only decides how long the key stays locked after a crash. | --- ## 4. Decision Log | # | Decision | Verdict | Alternative + trade-off | |---|---|---|---| | 1 | `invalid` for contract violations | **Strongly recommended** | Not splitting means burning `maxAttempts` external calls on a guaranteed failure — real money on paid platforms. | | 2 | `exhausted` promoted to a stored status | **Conditional** | Alternative: `$elemMatch { status ∈ …, attempts < N }` plus a compound multikey index. That makes `maxAttempts` **retroactive** — raising it auto-recovers stuck URLs with no manual step. The chosen design freezes the judgement into stored data, trading one index for one operational procedure. **Condition met** because the recovery path (`/retry statuses=[exhausted]`) is designed; it must survive as a real runbook entry. | | 3 | No unique index on `token_links`; delete-then-insert in the write path | **Strongly recommended** | A unique index destroys re-observation history (reverses a v5 decision); per-link upsert turns 2 round-trips into N. Atomicity rests on the token lock, and lock-TTL expiry (H-005) is the acknowledged residual risk. | | 4 | Synchronous + batch cap + re-invocation, no job-state collection | **Strongly recommended** (after revision) | Originally conditional: `limit = 20` had no basis, and at 4 s per URL a 20-token batch runs ~4 minutes — past a typical 60 s gateway timeout, at which point the caller cannot learn what was processed. **Resolved by adding `deadlineMs` + `stoppedBy` (§2).** Alternatives rejected: `202` + job collection (over-built for a manual ops endpoint); RabbitMQ hand-off (loses the `remaining` feedback loop and the job lock, and failures go silent — H-017). | | 5 | Re-fetch social URLs via `by-addresses` | **Strongly recommended** (after revision) | Reusing the `GET /tokens` payload saves N upstream calls but makes the manual path consume a different DTO from a different endpoint than the live path. Conditional on `limit` staying small; **`deadlineMs` now bounds it structurally.** | | 6 | `ApiKeyGuard` | **Conditional** | Network-level isolation costs zero application code and is stronger. Correct choice **only if the endpoints face the public internet** — undetermined at design time. Recorded in §2.3. | | 7 | No circuit breaker (keep existing stance) | **Strongly recommended** | Failure-rate sample is still zero, so no threshold has a basis, and multi-process workers inflate any process-local counter by the worker count. Batches create no back-pressure. | | 8 | Two lock layers | **Strongly recommended** | One layer alone leaves the other overlap open — they guard different things. | **Validation channels:** no `VALIDATE:` markers in the draft; no interactive flags raised; the eight items above were AI-identified as hard-to-reverse. WebSearch was not triggered (no named pattern/technology from the user, no NFR figures, no markers, no explicit request). --- ## 5. External API Integration ### 5.1 Integrations touched | Service · endpoint | Purpose | Auth | Role here | |---|---|---|---| | tracker `GET /api/v1/tokens` **＋new** | Token **list** for a period. | `X-API-Key` (existing SDK client attaches it) | The sole source of reconcile's candidate set; one call fixes the whole batch's candidates. | | tracker `POST /api/v1/tokens/by-addresses` | One token's metadata incl. four social URLs. | same | Inside the existing `record()`, once per token. Unchanged. | | Social platforms (Generator → Fetcher → SDK) | Content/account/venue collection. | per-platform keys | Once per URL on the retry path. Unchanged. | | Redis (redlock) | Job and token locks. | `redis.*` | Infrastructure; only `tryLock` is used. | ### 5.2 Error policy | Item | Value | Why | |---|---|---| | timeout | `TIMEOUT = 5000 ms` (existing constant; the new method shares the axios instance) | Cutting 30 s → 5 s is how the "no circuit breaker" decision is paid for: worst-case wait per call drops from ~93 s to ~18 s. | | retry | `RETRY_ATTEMPTS = 3`, backoff `1000ms × 2^i`; retryable only on `429` · `5xx` · `ECONNABORTED` · `ECONNREFUSED` | `400`/`401`/`403` give the same answer three more times. The new method goes through the same `withRetry` so the policy exists once. | | circuit breaker | **not adopted** | Zero failure sample; multi-process workers inflate local counters. This feature does not change that: reconcile is human-triggered, and retry's period is long enough to create no back-pressure. | | fallback | **none** | Token metadata has no substitute. An empty value would make "token with no links" indistinguishable from "lookup failed". Final failure surfaces as `502`. | | partial results | **not returned** for the range query; **isolated** at token and URL level | Different layers. A half-complete candidate set leaves the caller unable to know what is missing. But killing the batch on one token's failure discards every earlier success — and under convergence, what is skipped now is picked up next call. | | rate limit | no client-side cap; `limit` and `deadlineMs` bound call volume | With `limit = 20`, one reconcile makes at most 21 upstream calls. Reusing an existing bound beats inventing another number. | ### 5.3 Response mapping | External type | Internal type | Fields taken / dropped | |---|---|---| | `AlphaFinderInvestedToken` (`GET /tokens`) **＋new** | `TrackedToken` | **Taken:** `tokenAddress` · `tokenSymbol` · `investorCount` · `firstTransferAt` · `lastTransferAt`. **Dropped:** the four social URLs, `alphaScore`, `athMarketCap`, `rugScore` and every other metric. Dropping the social URLs is deliberate — see §3.2. | | `TokenDto` (`by-addresses`) | `TokenInfo` | Existing `transformTokenInfo`, all 18 fields. Unchanged. | ### 5.4 Two upstream facts that will be misread as our bug 1. **The upstream pre-filters.** `GET /tokens` applies `investorCount > 1 && totalUsedInDollar > 1000` server-side. Lowering `minAfCount` to 1 or 0 returns nothing extra. 2. **The period axis is `last_transfer_at`** — last trade, not creation, and not when we first saw it. An old token with recent trades lands in a recent range; a token that kept trading afterwards falls out of a past range. ### 5.5 Failure sequences ```mermaid sequenceDiagram participant Ctrl as SocialReconcileController participant Svc as SocialReconcileService participant Lock as RedlockSDK participant Sdk as SolTrackerApiSdk participant Up as tracker API Ctrl->>Svc: reconcileByRange(command) Svc->>Lock: tryLock(social-reconcile:reconcile) Lock-->>Svc: Lock Svc->>Sdk: getTokensByRange(...) Sdk->>Up: GET /api/v1/tokens (attempt 1) Up--xSdk: 503 Sdk->>Up: GET /api/v1/tokens (attempt 2, +1s) Up--xSdk: 503 Sdk->>Up: GET /api/v1/tokens (attempt 3, +2s) Up--xSdk: 503 Sdk-->>Svc: throw HttpException(502) Svc->>Lock: release() (finally) Svc-->>Ctrl: propagate Note over Ctrl,Up: zero DB writes — no record, no marker ``` No compensation is needed: nothing was written, and a read-only call leaves no upstream state. Worst-case wait ≈ 18 s. Re-invocation with the same parameters starts over safely. **Per-token failure is isolated.** If token B's `by-addresses` lookup finally fails, only B is counted as failed and the loop continues. B gets no `tokens` document — `upsertByAddress` runs only after the URL loop — so the next `/reconcile` picks it up again through the same set difference. > **Known gap, accepted.** If an address appears in `GET /tokens` but not in `by-addresses`, > `collect()` returns quietly and writes no document, so that address becomes a target on every > invocation forever. The two endpoints read different upstream sources, which makes the state > possible. This mirrors H-006, which was decided as "keep current behaviour". **Social call failure → status mapping** | Exception | Status | Picked up next cycle? | |---|---|---| | `SocialContractError` | `invalid` | **No** — absorbing. Only a code fix changes it; recovery is a manual `/retry statuses=[invalid]`. | | `ExternalFetchError` · `TARGET_BLOCKED` | `blocked` | **No** — target-side condition. | | `ExternalFetchError` · `CREDIT_EXHAUSTED` | `error` | **Yes**, until the cap. Also sets `context.creditExhausted`, skipping the remaining paid URLs **of this run only** — a process-wide switch would have no way back. | | `ExternalFetchError` · other | `error` | Yes, until the cap. | | Anything else (transient DB, etc.) | `error` | Yes, until the cap. | **Retry responsibility is split across layers.** The fetcher layer already retries — 3 attempts with 800 ms backoff for free sources (`RETRY_FREE`), **one attempt for paid ones** (`RETRY_NONE`, because retrying costs money). The fetcher handles a momentary wobble; this feature's cron handles conditions that change over time (paid switched on, a generator shipped, credit topped up). --- ## 6. Business Logic Flow ### 6.1 Manual sync (token axis) ```mermaid sequenceDiagram actor Op as Operator participant G as ApiKeyGuard participant C as SocialReconcileController participant S as SocialReconcileService participant L as RedlockSDK participant K as SolTrackerApiSdk participant R as TokenRepository participant P as SocialRecordProcessor Op->>G: POST /internal/social-recording/reconcile G->>C: x-api-key matches C->>S: reconcileByRange(command) S->>S: startedAt = now() S->>L: tryLock(social-reconcile:reconcile) L-->>S: Lock S->>K: getTokensByRange(start, end, afGroupIds) K-->>S: TrackedToken[] (scanned) S->>S: investorCount >= minAfCount (filtered) S->>R: findExistingAddresses(addresses) R-->>S: Set(already recorded) S->>S: set difference = targeted loop per token, sequential S->>S: limit reached? deadlineReached? -> break with stoppedBy S->>P: record(tokenAddress, tokenSymbol) P-->>S: void (recorded, or individually failed) end S->>L: release() (finally) S-->>C: ReconcileOutcome C-->>Op: 200 + meta(remaining, stoppedBy) ``` **Steps.** Auth first — a failed guard means zero external calls. The job lock is taken **before** the upstream call; reversed, two invocations would receive the same list and double-call upstream for the same tokens. One range query fixes the batch's candidates; everything after is local computation. `findExistingAddresses` **is** the idempotency check, in a single `$in` query. The loop is sequential: run in parallel, two tokens can create the same fan-out account simultaneously, and with no unique index on `contents` that becomes two rows. **No compensating transaction exists, because none is needed.** `record()` writes the marker (the `tokens` document) **last**. A token that dies mid-way has no document, and no document means it re-enters the next invocation's set difference. **Failure is simply the next attempt's input.** **Why there is no job-state collection.** `remaining > 0` tells the caller to call again; already- processed tokens now have documents and drop out of the set difference. **The progress state is the existence of the `tokens` document.** **Stopping conditions.** The check happens **between** tokens only. A token is an atomic unit — cutting mid-way would leave some URLs recorded with no marker, so the next invocation would redo the whole token and re-issue external calls for URLs that already succeeded. The price is that the real response can reach `deadlineMs` + one token's worst-case duration. ### 6.2 Retry (URL axis) — cron and manual share this path ```mermaid sequenceDiagram actor Sch as External scheduler participant C as SocialReconcileController participant S as SocialReconcileService participant L as RedlockSDK participant R as TokenRepository participant P as SocialRecordProcessor participant W as SocialGraphWriter participant LR as TokenLinkRepository Sch->>C: POST /retry (empty body -> defaults) C->>S: retryUnconverged(default command) S->>L: tryLock(social-reconcile:retry) L-->>S: Lock S->>R: findByUrlStatuses([error, unsupported, skipped_paid]) R-->>S: TokenEntity[] (candidateTokens) S->>S: flatten to (token,url) · minIntervalMinutes · limit loop per URL, sequential S->>S: deadlineReached? -> break with stoppedBy S->>P: recordOneUrl(target) P->>L: tryLock(social-record:{address}) L-->>P: Lock P->>W: write(graph, linkContext) W->>LR: deleteByTokenAndEntryUrl(addr, entryUrl) LR-->>W: deleted = n W->>LR: appendMany(links) P->>L: release() (finally) P-->>S: TokenSocialUrlInput(status, attempted) S->>S: resolveFinalStatus S->>R: recordSocialUrlResult(addr, url, finalStatus, attemptedAt) end S->>L: release() (finally) S-->>C: RetryOutcome(converged, stoppedBy) ``` **Zero upstream calls** — target URLs, statuses, attempt counts and symbol are all in our own document. URLs whose `attempted_at` is `null` always pass the back-off filter: they are the zero-call paths (`unsupported`, `skipped_paid`) and have no timestamp to back off from. **The URL is re-classified with `route()` rather than trusting a stored classification** — otherwise fixing a bug in the classification rules would never reach URLs already misclassified (v5 T-M1). **Partial update is a precondition, not an optimisation.** Using `upsertByAddress` would `$set` the whole document, blanking the other URLs' statuses plus `web` and `fingerprints` (H-015). The new repository method targets one array element via `arrayFilters`: ``` updateOne({ address }, { $set: { 'social_urls.$[el].status': status, 'social_urls.$[el].attempted_at': attemptedAt }, // only when attemptedAt is set $inc: { 'social_urls.$[el].attempts': 1 } }, // only when attemptedAt is set { arrayFilters: [{ 'el.url': url }] }) ``` ### 6.3 Lock contention ```mermaid sequenceDiagram participant A as cron call participant B as manual call participant S as SocialReconcileService participant L as RedlockSDK participant H as live TokenSocialRecordHandler participant P as SocialRecordProcessor A->>S: retryUnconverged() S->>L: tryLock(social-reconcile:retry) L-->>S: Lock acquired B->>S: retryUnconverged() (concurrent) S->>L: tryLock(social-reconcile:retry) L-->>S: null (held) S-->>B: return meta.skipped = already running H->>P: record(tokenX) — live event P->>L: tryLock(social-record:tokenX) L-->>P: Lock acquired S->>P: recordOneUrl(tokenX url) P->>L: tryLock(social-record:tokenX) L-->>P: null P-->>S: skipped S->>S: skip that URL, continue ``` Neither lock failure is an error. A job-lock miss returns `200` with `meta.skipped`; returning `409` would make a scheduler treat normal overlap as a failure. A token-lock miss skips one URL, and the next cycle takes it — under convergence, "not done this time" is not a loss. **~~Residual risk: lock TTL expiry.~~ Closed 2026-08-13.** The original text said a batch outliving its TTL would release the lock early and let two batches run together, and answered it by "setting the TTL above the worst-case batch duration, since `limit` and `deadlineMs` bound it." **That bound was wrong.** A single token holds its own lock via `tryUsingExtended`, which auto-extends, and URL-lock waiting happens inside that lifetime — up to `URL_LOCK_WAIT_MS x URL count` (180 s x 8 = about 24 minutes). `deadlineMs` only stops *starting* new tokens, so the worst case is not bounded by any constant, and the derived 180 s was far too short. Both jobs now take the lock with `tryUsingExtended` as well. The TTL no longer has to cover the run; it only decides how long the key stays locked if the process dies. Overlap caused by expiry is therefore gone, and genuine token-level contention surfaces as `meta.lockedOut` instead of being reported as success. ### 6.4 Partial failure and re-convergence ```mermaid sequenceDiagram participant S as SocialReconcileService participant P as SocialRecordProcessor participant W as SocialGraphWriter participant OR as Object Repos participant LR as TokenLinkRepository participant R as TokenRepository Note over S,R: cycle 1 — dies mid-write S->>P: recordOneUrl(url) P->>W: write(graph, ctx) W->>OR: account/content rows committed W->>LR: deleteByTokenAndEntryUrl -> 0 W->>LR: appendMany(links) fails LR--xW: write error W--xP: throw P-->>S: status=error, attempted=true S->>R: recordSocialUrlResult(url, error, at) Note over S,R: cycle 2 — the retry cleans up S->>P: recordOneUrl(url) P->>W: write(graph, ctx) W->>OR: same platform_key -> same rows W->>LR: deleteByTokenAndEntryUrl -> n deleted W->>LR: appendMany(links) succeeds P-->>S: status=ok S->>R: recordSocialUrlResult(url, ok, at) ``` There are no multi-document transactions (impossible without a replica set), so ordering is the only tool. **Compensation here is replacement, not rollback.** Nothing is undone at failure time; the next attempt swaps out that entry URL's links wholesale. With no undo step, there is no second failure point in the compensation itself. Rows and metrics are not cleaned up: they converge by `platform_key`, so rewriting produces the same row, and the metric repository records only the first point. **This is where H-013 closes.** Previously a retry kept appending the same `(token_address, object_id, entry_url)` and a reader had no basis to tell "observed again later" from "leftover of a failed attempt". What `deleteByTokenAndEntryUrl` can remove is *only* the leftovers of a failed attempt, because `ok` is absorbing — an already-successful URL never re-enters this path. > **Remaining gap.** `invalid` gets no cycle 2. It is absorbing, so it is not a retry candidate: the > rows and metrics cycle 1 committed stay, and links stay at zero. Recovery is a manual > `/retry statuses=[invalid]`, which then follows the cycle-2 path above. --- ## 7. Out of Scope (Explicit Exclusions) | Aspect | Why excluded | Revisit trigger | |---|---|---| | Observability | User decision in Phase 1.6; the draft says nothing about it. `meta.converged` and `meta.stoppedBy` are the only health signals, and they are pull-only. | The first time someone needs to know why retry stopped converging without reading response bodies by hand. | | Lifecycle (migration / rollback) | The `FetchStatus` change needs no database work (§1.1), and nothing has been deployed. | The first deployment — every enum or schema change after it needs a migration path. | | Access control & multi-tenancy | No tenant boundary and no role branching. The real concern was the call boundary, which is INCLUDED and handled in §2.3. | A second caller class appears that must not be able to trigger paid calls. | | Data lifecycle (TTL / retention) | No retention requirement stated. | `token_links` or exhausted `social_urls[]` growth becomes an operational concern. | | Cron catch-up policy | Under convergence the next period *is* the catch-up: candidates that were skipped remain candidates. Nothing separate to design. | Retry acquires a notion of ordering or priority, at which point "which ones were skipped" stops being derivable from status alone. | | Update Metrics (draft §3) | User decision: separate scope. Its "execution state" requirement is the one thing in this feature's neighbourhood that would need a state collection. | When periodic metric refresh is picked up; introduce the execution-state collection then, not before. | ### Carried-forward review item §2.3 records a conditional external-validation verdict the design proceeds under without resolving: `ApiKeyGuard` is the right mechanism **only if these endpoints face the public internet**. If the deployment is internal-only, a network boundary achieves the same protection with zero application code and the guard should be dropped. Resolve before implementation.