From a3817edfc70a293322486a94a5b0204ff16c76f9 Mon Sep 17 00:00:00 2001 From: Sergey Krylov Date: Tue, 16 Jun 2026 05:40:53 +0300 Subject: [PATCH] docs: add tbank broker portfolios design --- ...26-06-16-tbank-broker-portfolios-design.md | 384 ++++++++++++++++++ 1 file changed, 384 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-16-tbank-broker-portfolios-design.md diff --git a/docs/superpowers/specs/2026-06-16-tbank-broker-portfolios-design.md b/docs/superpowers/specs/2026-06-16-tbank-broker-portfolios-design.md new file mode 100644 index 0000000..2f27827 --- /dev/null +++ b/docs/superpowers/specs/2026-06-16-tbank-broker-portfolios-design.md @@ -0,0 +1,384 @@ +# T-Bank Broker Portfolios Design + +**Status:** Approved for implementation planning + +**Date:** 2026-06-16 + +**Owner:** MoexVibe + +## Context + +MoexVibe already supports manually managed virtual portfolios based on MOEX market data. The next +step is a read-only integration with T-Bank Invest so the application can show real brokerage +accounts, positions, cash balances, and operation history. + +The initial integration is personal and server-side. The backend reads one `T_BANK_TOKEN` from +`apps/backend/.env`. Later versions may replace this with per-user token storage, but the first +implementation must not require that larger secrets model. + +## External API Facts + +T-Bank Invest API is a gRPC API. REST is available as a proxy and WebSocket exists for clients that +need it, but gRPC is the primary protocol. Production endpoint is `invest-public-api.tbank.ru:443`; +sandbox endpoint is `sandbox-invest-public-api.tbank.ru:443`. + +Authorization is passed as gRPC metadata: + +```text +Authorization: Bearer +``` + +The relevant official services and methods are: + +| Need | T-Bank method | Notes | +| --- | --- | --- | +| Accounts | `UsersService/GetAccounts` | Filter by open brokerage accounts and IIS only. | +| Current portfolio valuation | `OperationsService/GetPortfolio` | Returns totals, positions, daily yield, expected yield. | +| Cash and settled positions | `OperationsService/GetPositions` | Returns money, blocked money, securities, futures, options. | +| Operation history | `OperationsService/GetOperationsByCursor` | Cursor pagination, limit up to 1000, operation type filters. | +| Instrument metadata | `InstrumentsService/GetInstrumentBy` | Resolve `instrument_uid`, ticker, class code, lot, name, ISIN. | + +The official docs recommend staying below 50 requests per second across accounts and tokens. The +documented unary limits include 100 requests per minute for the accounts service and 200 requests +per minute for the operations service. T-Bank also exposes rate limit response metadata such as +`x-ratelimit-limit`, `x-ratelimit-remaining`, and `x-ratelimit-reset`. + +Stream services exist for portfolio, positions, and operations updates. They are useful later, but +the first implementation will use unary reads plus local cache/sync because it is simpler, testable, +and less likely to hold unnecessary long-lived connections. + +References: + +- [T-Invest API intro](https://developer.tbank.ru/invest/intro/intro) +- [gRPC protocol](https://developer.tbank.ru/invest/intro/developer/protocols/grpc/) +- [Limits](https://developer.tbank.ru/invest/intro/intro/limits) +- [UsersService](https://developer.tbank.ru/invest/api/users-service) +- [OperationsService](https://developer.tbank.ru/invest/api/operations-service) +- [JS SDK](https://developer.tbank.ru/invest/sdk/faq_js) +- [Official proto contracts](https://opensource.tbank.ru/invest/invest-contracts) + +## Goals + +- Show real T-Bank brokerage accounts and IIS accounts in a separate broker portfolios area. +- Show current positions for each account: shares, bonds, other securities if returned, and cash. +- Show operation history for each account: buys, sells, commissions, taxes, coupons, dividends, + deposits, withdrawals, repayments, corrections, and other operation types returned by T-Bank. +- Avoid excessive calls to T-Bank through backend rate limiting, short-lived read cache, and durable + local operation sync. +- Keep the frontend isolated from T-Bank. The backend remains the only external API client. +- Publish human-readable architecture and usage documentation in `apps/docs`. + +## Non-Goals + +- No trading, order placement, transfers, or account funding in this phase. +- No Invest Box, DFA smart account, debit account, savings account, or money market fund account in + this phase. +- No per-user T-Bank token management in this phase. +- No always-on stream workers in the first version. +- No tax calculation engine beyond showing broker-provided tax operations. + +## Account Scope + +Use `UsersService/GetAccounts` and keep only: + +- `ACCOUNT_TYPE_TINKOFF` +- `ACCOUNT_TYPE_TINKOFF_IIS` + +Also keep only open accounts: + +- `ACCOUNT_STATUS_OPEN` + +Excluded account types: + +- `ACCOUNT_TYPE_INVEST_BOX` +- `ACCOUNT_TYPE_INVEST_FUND` +- `ACCOUNT_TYPE_DEBIT` +- `ACCOUNT_TYPE_SAVING` +- `ACCOUNT_TYPE_DFA` +- `ACCOUNT_TYPE_UNSPECIFIED` + +## Protocol Decision + +Use a thin backend gRPC integration instead of the REST proxy or a hard dependency on the official +JS SDK. + +### Why gRPC + +- It is the primary protocol of T-Bank Invest API. +- It gives direct access to unary and future stream APIs. +- It keeps request metadata, `x-tracking-id`, and rate-limit metadata visible at the transport + boundary. +- It fits the existing MoexVibe rule that backend is the only external data client. + +### Why Not REST First + +REST is useful for manual debugging and Swagger examples, but it is a proxy over the same service +surface. Building on REST would add translation overhead and make future stream support a separate +design. + +### Why Not SDK First + +The official JS SDK exists and can be useful as a reference. Current npm metadata for +`@tinkoff/invest-js` shows it already depends on gRPC libraries such as `@grpc/grpc-js`, +`@grpc/proto-loader`, `nice-grpc`, and `protobufjs`. Using it directly would hide transport details +that MoexVibe needs to own: rate limiting, metadata capture, test doubles, generated type updates, +and later token sourcing. The module boundary should still allow replacing the internal transport +adapter with SDK calls if that becomes clearly cheaper. + +## Backend Architecture + +Create a new feature module: + +```text +apps/backend/src/modules/tbank/ +├── tbank.module.ts +├── tbank.config.ts +├── tbank.controller.ts +├── services/ +│ ├── tbank-client.service.ts +│ ├── broker-accounts.service.ts +│ ├── broker-portfolio.service.ts +│ ├── broker-operations.service.ts +│ └── broker-instruments.service.ts +├── dto/ +│ ├── broker-account-response.dto.ts +│ ├── broker-portfolio-response.dto.ts +│ ├── broker-operation-query.dto.ts +│ └── broker-operation-response.dto.ts +└── mappers/ + ├── money.mapper.ts + ├── account.mapper.ts + ├── portfolio.mapper.ts + └── operation.mapper.ts +``` + +`TBankClientService` owns: + +- gRPC channel creation. +- `Authorization` metadata. +- optional `x-app-name` metadata, for example `ksv741.moex-vibe`. +- request timeout. +- retry for transient gRPC failures. +- service-level rate limiting. +- capturing `x-tracking-id` and rate-limit metadata for logs and error responses. + +Domain services depend on `TBankClientService`, not directly on generated gRPC clients. This keeps +the later switch from one server token to per-user token providers local to the client layer. + +## Public Backend API + +All endpoints require the existing JWT authentication. In the first version, any authenticated +MoexVibe user can technically read the same server-token broker data. Deployment must treat this as +single-user/admin-only until per-user token storage is implemented. + +| Endpoint | Method | Description | +| --- | --- | --- | +| `/api/v1/broker/accounts` | GET | List open T-Bank brokerage and IIS accounts. | +| `/api/v1/broker/accounts/:accountId/portfolio` | GET | Current account portfolio, positions, totals, cash. | +| `/api/v1/broker/accounts/:accountId/operations` | GET | Operation history with pagination and filters. | + +Operations query parameters: + +| Query | Type | Default | Notes | +| --- | --- | --- | --- | +| `from` | ISO datetime | start of current year | UTC in T-Bank request. | +| `to` | ISO datetime | now | UTC in T-Bank request. | +| `cursor` | string | absent | Passed to `GetOperationsByCursor`. | +| `limit` | integer | 100 | Clamp to `1..1000`. | +| `instrumentId` | string | absent | Supports figi, instrument UID, or `ticker_classCode`. | +| `operationTypes` | string list | absent | Optional T-Bank operation type enum names. | +| `state` | enum | `OPERATION_STATE_EXECUTED` | Default to executed operations for portfolio accounting views. | + +## Response Model + +Use JSON DTOs shaped for the frontend, not raw proto objects. + +Money values are normalized from T-Bank `MoneyValue { currency, units, nano }` into: + +```typescript +type BrokerMoney = { + currency: string; + units: string; + nano: number; + value: number; +}; +``` + +`units` remains a string to preserve the original int64 value. `value` is a convenience decimal for +display and charting. Financial calculations that require exact precision should use decimal helpers, +not floating-point arithmetic. + +Portfolio response shape: + +```typescript +type BrokerPortfolio = { + account: BrokerAccount; + totals: { + shares: BrokerMoney | null; + bonds: BrokerMoney | null; + etf: BrokerMoney | null; + currencies: BrokerMoney | null; + futures: BrokerMoney | null; + options: BrokerMoney | null; + structuredProducts: BrokerMoney | null; + dfa: BrokerMoney | null; + portfolio: BrokerMoney | null; + }; + yields: { + expectedPercent: number | null; + daily: BrokerMoney | null; + dailyPercent: number | null; + }; + cash: BrokerMoney[]; + blockedCash: BrokerMoney[]; + positions: BrokerPosition[]; + asOf: string; +}; +``` + +Operation response shape: + +```typescript +type BrokerOperationsPage = { + accountId: string; + items: BrokerOperation[]; + nextCursor: string | null; + hasNext: boolean; + asOf: string; +}; +``` + +Each operation keeps the original T-Bank enum in `type`, plus a MoexVibe display category: + +| Category | T-Bank operation examples | +| --- | --- | +| `trade` | `OPERATION_TYPE_BUY`, `OPERATION_TYPE_SELL`, margin/delivery variants | +| `income` | `OPERATION_TYPE_DIVIDEND`, `OPERATION_TYPE_COUPON`, repayments, overnight income | +| `tax` | `OPERATION_TYPE_TAX`, dividend tax, bond tax, progressive tax variants | +| `fee` | broker fee, service fee, margin fee, success fee, cash/out/advice/other fees | +| `transfer` | input, output, securities transfer, SWIFT/acquiring/multi transfers | +| `other` | unspecified and T-Bank operation types not yet categorized | + +Never drop unknown operation types. Store and display them as `other` with the original enum. + +## Operation Sync Strategy + +Phase 1 can read operations directly from T-Bank with a cache. Phase 2 should persist normalized +operations locally because operation history is the audit trail and should not depend on repeatedly +walking the same remote pages. + +Durable sync design: + +- Store one row per T-Bank operation item keyed by `(accountId, cursor)` when cursor is present. +- Also keep `operationId`, `parentOperationId`, `date`, `type`, `state`, `instrumentUid`, `figi`, + `ticker`, `classCode`, `payment`, `price`, `commission`, `yield`, `accruedInt`, `quantity`, + `quantityDone`, `raw`. +- Store sync state per account: last successful range, last cursor, last synced timestamp. +- Backfill historical data by date windows, for example one calendar year per run. +- Refresh a moving recent window, for example last 3 days, because broker operation IDs and parent + IDs can change according to the proto comments. +- Keep raw operation payload JSON for audit/debug while exposing normalized DTOs. + +## Cache Strategy + +Use existing backend `CacheService` for short-lived reads and add T-Bank-specific key prefixes. +Frontend TanStack Query should use matching or shorter stale times. + +| Data | Backend TTL | Reason | +| --- | --- | --- | +| Accounts | 1 hour | Account list changes rarely. | +| Portfolio totals and positions | 30-60 seconds | User-facing current view, should feel fresh but not spam T-Bank. | +| Cash/withdraw limits | 30-60 seconds | Similar freshness to positions. | +| Instrument metadata | 24 hours | Instrument name, lot, ISIN, UID are stable. | +| Operation page, recent window | 60-300 seconds | Useful before durable sync exists. | +| Operation historical pages | 24 hours or DB only | History is mostly immutable outside recent correction window. | + +Add environment variables: + +| Variable | Default | Description | +| --- | --- | --- | +| `T_BANK_TOKEN` | none | Server-side T-Bank Invest token. Required when integration is enabled. | +| `T_BANK_BASE_URL` | `invest-public-api.tbank.ru:443` | gRPC endpoint. | +| `T_BANK_APP_NAME` | `ksv741.moex-vibe` | Optional gRPC metadata. | +| `T_BANK_RATE_LIMIT_PER_SECOND` | `5` | Conservative local limiter across T-Bank calls. | +| `CACHE_TBANK_ACCOUNTS_TTL` | `3600` | Accounts cache TTL. | +| `CACHE_TBANK_PORTFOLIO_TTL` | `60` | Portfolio and positions cache TTL. | +| `CACHE_TBANK_OPERATIONS_TTL` | `300` | Recent operation page cache TTL. | +| `CACHE_TBANK_INSTRUMENT_TTL` | `86400` | Instrument metadata cache TTL. | + +The rate limiter must stay below documented public limits. On 429 or exhausted rate-limit metadata, +back off until reset when metadata is available. + +## Error Handling + +- Missing `T_BANK_TOKEN`: return a clear 503-style application error and mark integration + unavailable; do not crash the whole backend in development. +- Auth error from T-Bank: return 502/503 with a generic message; never echo token or full metadata. +- Account not found or excluded by type/status: return 404 from MoexVibe endpoints. +- T-Bank 429: return 429 or 503 with retry metadata when available. +- T-Bank transient gRPC errors: retry with small exponential backoff, then surface a typed upstream + error with `trackingId` if present. +- Mapping unknown enum values: preserve the original value and categorize as `other`. + +## Security + +- `T_BANK_TOKEN` must never be logged, returned in API responses, or committed. +- gRPC metadata logging must redact `Authorization`. +- The initial deployment is single-user by design. Before enabling access for multiple MoexVibe + users, add per-user encrypted token storage and authorization rules that bind each broker account + to its owner. +- The integration is read-only. Do not include order, stop-order, transfer, or pay-in clients in the + first module. + +## Frontend Product Shape + +Add a broker portfolios area separate from manually managed virtual portfolios: + +- List page: account cards for brokerage and IIS accounts, with total value, cash, daily change, and + last refresh time. +- Account detail page: tabs for `Позиции`, `Операции`, and later `Аналитика`. +- Positions table: instrument name, ticker, type, quantity, current price, current value, expected + yield, daily yield, blocked quantity. +- Cash section: available and blocked money by currency. +- Operations table: date, type/category, instrument, quantity, payment, commission, tax/income + indicators, status, expandable trade details. + +Do not merge real broker accounts into the existing manual `Portfolio` model. Keep them separate in +UI and backend API. Later, MoexVibe can add comparison views or import flows from broker operations +into virtual portfolios. + +## Documentation Deliverables + +Publish durable documentation in `apps/docs` during implementation: + +- `apps/docs/docs/backend/tbank-invest.md`: module architecture, API methods, env vars, error + handling, and read-only scope. +- Update `apps/docs/docs/backend/modules.md`: add `TBankInvestModule`. +- Update `apps/docs/docs/backend/configuration.md`: document `T_BANK_*` and cache TTL variables. +- Update `apps/docs/docs/backend/caching.md`: add T-Bank cache strategy. +- Update `apps/docs/docs/backend/portfolio.md`: distinguish manual portfolios from broker + portfolios. +- Add ADR `apps/docs/docs/adr/ADR-011-tbank-invest-grpc.md`: record gRPC over REST/SDK decision. +- Update `apps/docs/sidebars.ts` to include the new backend page and ADR. + +## Acceptance Criteria + +- Backend lists only open `ACCOUNT_TYPE_TINKOFF` and `ACCOUNT_TYPE_TINKOFF_IIS` accounts. +- Backend exposes current account portfolio with positions and cash without exposing raw token data. +- Backend exposes paginated operation history and includes buy, sell, tax, fee, coupon, dividend, + deposit, withdrawal, and unknown operation categories. +- T-Bank calls use a conservative rate limiter and cache strategy. +- Missing or invalid token produces a clear integration-unavailable error. +- Published docs in `apps/docs` describe architecture, configuration, caching, and the gRPC decision. +- Tests cover account filtering, money mapping, operation categorization, query validation, cache + keys, and upstream error mapping. + +## Implementation Phases + +1. Backend gRPC foundation and DTO mappers. +2. Read-only account and portfolio endpoints. +3. Operation history endpoint with cursor pagination. +4. Frontend broker account list and account detail pages. +5. Durable operation sync in database. +6. Optional stream workers for near-real-time refresh after unary sync is stable. +