docs: thêm modules-reference.md — tài liệu toàn bộ codebase

2026-07-03 13:25:48 +00:00
parent 85ca5c4581
commit 7af50cb397
1 changed files with 315 additions and 0 deletions
@@ -0,0 +1,315 @@
 # Trading Portal — Module Documentation
 > Tự động tạo bởi Hermes Agent ngày 2026-07-03.  
 > Cover toàn bộ codebase để team mới có thể tiếp tục phát triển.
 ---
 ## Backend Core (`backend/app/`)
 ### `config.py` — Application Settings
 - **Class:** `Settings(BaseSettings)` — Pydantic Settings, load từ `.env`
 - **Key configs:** `DATABASE_URL`, `JWT_PRIVATE_KEY_PATH`, `JWT_PUBLIC_KEYS_DIR`, `ENCRYPTION_KEY`, `CORS_ORIGINS`, `demo_user/pass`
 - JWT: RS256, access token 15 phút, refresh token 7 ngày
 ### `database.py` — Database Layer
 - **Engine:** `create_async_engine` (asyncpg), pool_size=60, max_overflow=20
 - **Session:** `async_session_factory` → `AsyncSession`
 - **Base:** `DeclarativeBase` cho tất cả ORM models
 - **`get_db()`:** Async generator, tự commit/rollback/close. Xử lý `InterfaceError` để tránh CPU spike
 - Statement timeout: 10s (API), 30s (Scheduler)
 ### `main_api.py` — FastAPI Entrypoint (User-facing)
 - Port 8001, JSON structured logging
 - Lifespan: auto-create tables, register WebSocket push listener
 - CORS middleware, RequestLoggingMiddleware
 - Exception handlers: `AppException`, `ValidationError` (UUID-safe), 404, 500
 - `/health` endpoint: uptime + DB latency
 - Bao gồm: `api_router` (REST) + `ws_router` (WebSocket candles)
 ### `main_scheduler.py` — Background Scheduler
 - No HTTP port — chạy độc lập, chỉ fetch candle + phân tích
 - BATCH_SIZE=25 symbols, SEMAPHORE=3 concurrent
 - Không import FastAPI → tiết kiệm RAM
 - Entrypoint: `python3 -m app.main_scheduler`
 ### `core/security.py` — Authentication & Encryption
 - **Password:** bcrypt hashing (sync + async via thread pool)
 - **JWT (RS256):** Multi-key rotation — private key để sign, public keys directory để verify. Token có `kid` header → thử matching key trước, fallback full scan
 - **`create_access_token()`** (15 phút), **`create_refresh_token()`** (7 ngày, có `jti`)
 - **`decode_token()`** thử tất cả public keys — cũ token vẫn valid sau rotation
 - **AES-256-CBC:** `encrypt_api_key()` / `decrypt_api_key()` cho API key exchange
 - **`generate_token_hash()`** SHA-256, **`generate_jti()`** UUID4
 ### `core/deps.py` — FastAPI Dependencies
 - `get_db_session()` → AsyncSession
 - `get_current_user()` → decode Bearer token, fetch User từ DB
 - `get_current_active_user()` → check is_active
 - `get_current_trader_user()` → role admin/trader
 - `get_current_viewer_user()` → any role
 - `get_current_admin_user()` → is_admin + role=="admin"
 ### `core/middleware.py` — RequestLoggingMiddleware
 - Log method, path, status, duration cho mọi request
 ### `core/exceptions.py` — Custom Exceptions
 - `AppException` (base), `InvalidCredentialsException`, `InvalidTokenException`, `ForbiddenException`, `NotFoundException`
 ---
 ## Models (`backend/app/models/`) — 10 bảng
 | Model | Bảng | Mục đích |
 |-------|------|----------|
 | **User** | `users` | id (UUID), username, email, password_hash, role (admin/trader/viewer), preferences (JSON), is_active, is_admin |
 | **Exchange** | `exchanges` | Tên sàn (binance, bybit, mexc, gate, bingx), fee config |
 | **Symbol** | `symbols` | symbol (BTC/USDT), exchange_id FK, active, market_type |
 | **Candle** | `candles` | symbol_id, timeframe (15m/1h/4h/1d), timestamp, OHLCV. **Partition by month** |
 | **Watchlist** | `watchlists` | user_id FK, symbol |
 | **ExchangeCredential** | `exchange_credentials` | user_id FK, exchange, api_key (AES encrypted), api_secret (AES encrypted) |
 | **RefreshToken** | `refresh_tokens` | user_id FK, token_hash (SHA-256), jti, expires_at, revoked |
 | **Signal** | `signals` | symbol, exchange, timeframe, signal_type (STRONG_BUY/SELL/BUY/CAUTION...), strength, price, indicators_snapshot (JSON), status |
 | **HypotheticalTrade** | `hypothetical_trades` | Paper trade — user_id, signal_id, symbol, direction, entry/exit_price, pnl, pnl_percent, status. Có decay PnL filter `ABS(pnl_percent) < 100` |
 | **RealTrade** | `real_trades` | Trade thật — user_id, exchange, symbol, side, amount, price, filled_amount, pnl, pnl_percent, order_id, status |
 | **AuditLog** | `audit_logs` | user_id, action, target_type, target_id, details (JSON), ip_address |
 ---
 ## API Routes (`backend/app/api/v1/`) — 13 modules
 | Router | Prefix | Endpoints chính |
 |--------|--------|-----------------|
 | **auth** | `/api/v1/auth` | POST register, login, refresh, logout, GET me |
 | **exchanges** | `/api/v1/exchanges` | GET list, GET config |
 | **symbols** | `/api/v1/symbols` | GET list, GET by exchange, GET top100 |
 | **watchlist** | `/api/v1/watchlist` | CRUD watchlist items per user |
 | **credentials** | `/api/v1/credentials` | CRUD API keys (encrypted), test connection |
 | **signals** | `/api/v1/signals` | GET list (filter by symbol/exchange/timeframe/status), GET stats |
 | **real_trades** | `/api/v1/trades` | GET list, GET stats (PnL, win rate), GET by id |
 | **orders** | `/api/v1/orders` | POST place order, GET open orders, DELETE cancel |
 | **strategies** | `/api/v1/strategies` | GET list 13 strategies + enabled/disabled status |
 | **backtest** | `/api/v1/backtest` | POST run backtest, GET results, GET symbols list |
 | **backtest_history** | `/api/v1/backtest-history` | CRUD saved backtest configs per user |
 | **alerts** | `/api/v1/alerts` | CRUD price alerts |
 | **analytics** | `/api/v1/analytics` | GET dashboard stats, PnL summary, win rate trend |
 | **audit** | `/api/v1/audit` | GET audit logs (admin only) |
 | **admin** | `/api/v1/admin` | User management (admin only) |
 ### WebSocket (`backend/app/api/ws/`)
 - **`candle_handler.py`** — `/ws/candles?symbols=BTC/USDT,ETH/USDT` — push real-time candles cho ChartContainer
 ---
 ## Services (`backend/app/services/`) — Business Logic
 ### `signal_service.py` (2328 dòng — file lớn nhất)
 - **Phát hiện tín hiệu:** Double BB + RSI, 13 thuật toán voting
 - **Signal pipeline:** fetch candle → compute indicators → vote → normalize → classify (STRONG/BUY/SELL/CAUTION)
 - **Hypothetical trades:** tự mở paper trade khi có signal, đóng khi opposing signal hoặc stop loss
 - **Cache:** win rates 5 phút, enabled strategies 5 phút
 ### `trade_executor.py` (368 dòng)
 - **`execute_signal_trade()`** — chỉ STRONG_BUY / STRONG_SELL mở trade thật
 - MAX_OPEN_TRADES=10, hybrid eviction (lỗ nhất trước, else FIFO)
 - Volatility filter: skip ATR > 8% hoặc < 0.5%
 - Reversal: STRONG signal đóng opposing trades
 ### `risk_manager.py` (203 dòng)
 - **DynamicKellySizer:** Fractional Kelly (25%), tính từ PnL stats thực
 - Kelly formula: `f* = (p × b - q) / b`
 - Min trade 5 USDT, max 500 USDT
 - **AdaptiveSLTPOptimizer:** 6 regime (trending/sideways/volatile/breakout/choppy/neutral) × SL/TP multipliers
 ### `signal_booster.py`
 - Win-rate boosting: vote × win_rate × 2.0
 - Exponential decay λ=0.05/ngày, half-life 14 ngày
 - MIN_TRADES=15 để đủ statistical significance
 - Correlation dampening: strategy cùng nhóm ÷ √count
 - Dynamic threshold: normalize score / √active_strategies
 - Cache TTL: 6 giờ
 ### `indicator_service.py`
 - **Tính indicators:** BB (20,2 + 20,1), RSI(14), MACD, SuperTrend, ATR, Ichimoku, OBV, StochRSI, MFI, Volume profile
 - **SMC indicators:** BOS (Break of Structure), CHoCH (Change of Character), Order Blocks, FVG (Fair Value Gap)
 - **Candlestick patterns:** 30+ mẫu (doji, hammer, engulfing, harami, morning star...)
 - **Multi-timeframe:** vote từ 15m, 1h, 4h
 - `compute_indicators()` trả về dict các indicator arrays + scalars
 ### `candle_service.py`
 - `get_indicators()` — fetch candles từ DB, compute indicators với pre-compute pattern
 - `safe_slice()` — xử lý None scalars trong indicator dicts
 - Backtest: pre-compute indicators trên toàn bộ dataset → slice trong loop (O(n) thay vì O(n²))
 ### `auth_service.py`
 - Register: hash password, tạo user, tạo refresh token
 - Login: verify password, tạo access + refresh token
 - Refresh: verify refresh token, tạo access token mới
 - Logout: revoke refresh token, update DB
 ### `ws_push_service.py`
 - `setup_push_listener()` — đăng ký WebSocket push cho real-time data
 - Push candles, signals, trades đến connected clients
 ### `notification_service.py`
 - `notify_user()` — gửi notification (Telegram/Slack) khi có trade
 - `notify_trade()` — thông báo trade opened/closed
 ### `audit_service.py`
 - `log_action()` — ghi audit log (user, action, target, details, IP)
 ---
 ## Exchange Integration (`backend/app/exchange/`) — 5 sàn
 | File | Exchange | Notes |
 |------|----------|-------|
 | `base.py` | Abstract base | Interface chung: fetch_candles, get_balance, place_order, cancel_order |
 | `binance.py` | Binance | ccxt.binance |
 | `bybit.py` | Bybit | ccxt.bybit |
 | `mexc.py` | MEXC | ccxt.mexc |
 | `gate.py` | Gate.io | ccxt.gate |
 | `bingx.py` | BingX | ccxt.bingx |
 | `factory.py` | Factory | `factory.get_exchange(name, api_key, secret)` → instance |
 | `rate_limiter.py` | Rate limiter | Token bucket, 10 req/s per exchange |
 | `types.py` | Types | OrderRequest, OrderResponse, Balance, CandleStick |
 ---
 ## Tasks (`backend/app/tasks/`)
 ### `candle_fetcher.py`
 - **`fetch_and_analyze()`** — main loop mỗi 5 phút
 - Fetch 25 symbols × 5 exchanges × 4 timeframes (batch)
 - Dedup: chỉ 1h timeframe phân tích tín hiệu
 - Semaphore 3 concurrent analyses
 - Chu kỳ đầy đủ ~95 phút cho 476 symbols
 ### `exchange_sync.py`
 - Sync symbols từ exchange config xuống DB
 - Cập nhật active/inactive status
 ### `stale_data_monitor.py`
 - Phát hiện symbols có candle cũ (>30 phút)
 - Auto retry fetch
 ---
 ## Scripts (`backend/scripts/`)
 | Script | Mục đích |
 |--------|----------|
 | `fetch_candles_cron.py` | Fetch candles định kỳ (cron job) |
 | `sync_candles.py` | Sync toàn bộ candles từ exchange |
 | `backtest.py` | Run backtest standalone |
 | `weekly_report.py` | Generate weekly trading report |
 | `train_xgboost_model.py` | Train XGBoost model từ historical data |
 | `select_top100_symbols.py` | Chọn top 100 symbols theo volume |
 | `export_symbols.py` | Export symbol list |
 | `backup_strategies.py` | Backup strategy configs |
 | `apply_top100.py` | Apply top 100 selection |
 ---
 ## Frontend (`frontend/src/`) — React + TypeScript + Redux
 ### Pages
 | Page | File | Chức năng |
 |------|------|-----------|
 | **Login** | `auth/LoginPage.tsx` | Login form (username/password), redirect sau login |
 | **Register** | `auth/RegisterPage.tsx` | Register form |
 | **Dashboard** | `dashboard/DashboardPage.tsx` | Chart + Order panel + Signal panel + Watchlist panel |
 | **Profile** | `profile/ProfilePage.tsx` | 6 tabs: Info, Keys, Sessions, Settings (13 strategies), History (Signal/Real), Backtest |
 | **Backtest** | `backtest/BacktestPage.tsx` | Backtest runner standalone |
 | **Alerts** | `alerts/AlertsPage.tsx` | Price alerts management |
 | **Analytics** | `analytics/AnalyticsPage.tsx` | PnL charts, win rate, trade history charts |
 | **Admin** | `admin/AdminPage.tsx` | User management (admin only) |
 | **AuditLog** | `AuditLogPage.tsx` | Audit log viewer (admin only) |
 ### Components
 | Component | File | Vai trò |
 |-----------|------|---------|
 | ChartContainer | `dashboard/ChartContainer.tsx` | TradingView-style chart với WebSocket real-time |
 | ChartToolbar | `dashboard/ChartToolbar.tsx` | Timeframe, indicator toggles, drawing tools |
 | OrderPanel | `dashboard/OrderPanel.tsx` | Place market/limit orders |
 | SignalPanel | `dashboard/SignalPanel.tsx` | Live signal feed |
 | WatchlistPanel | `dashboard/WatchlistPanel.tsx` | Symbol watchlist với giá real-time |
 | ErrorBoundary | `components/ErrorBoundary.tsx` | Catch React errors |
 | Skeleton | `components/Skeleton.tsx` | Loading skeleton placeholder |
 ### State Management (Redux)
 | Slice | File | State |
 |-------|------|-------|
 | **authSlice** | `auth/authSlice.ts` | user, token, isAuthenticated, loading |
 | **Store** | `app/store.ts` | Redux store config, middleware |
 ### API Services
 | Service | File | Endpoints |
 |---------|------|-----------|
 | apiService | `api/apiService.ts` | Axios instance, interceptors, token refresh |
 | signalApi | `api/signalApi.ts` | GET signals, GET stats |
 | realTradeApi | `api/realTradeApi.ts` | GET trades, GET stats, POST order |
 | watchlistApi | `api/watchlistApi.ts` | CRUD watchlist |
 | alertApi | `api/alertApi.ts` | CRUD alerts |
 | websocketService | `api/websocketService.ts` | WebSocket client, auto-reconnect |
 ### Translations
 | File | Ngôn ngữ |
 |------|----------|
 | `translations/vi.ts` | Tiếng Việt |
 | `translations/en.ts` | English |
 | `translations/index.tsx` | React context provider |
 ### Types (`types/trading.ts`)
 - `Signal`, `Trade`, `Candle`, `Exchange`, `Symbol`, `User`, `WatchlistItem`, `OrderRequest`, `BacktestConfig`, `Alert`, `AuditLogEntry`
 ---
 ## Host Scripts (`scripts/`)
 | Script | Mục đích |
 |--------|----------|
 | `health_check.py` | Docker + API health check (30 phút) |
 | `deploy-backend.sh` | Build + deploy backend container |
 | `deploy-frontend.sh` | Build + deploy frontend container |
 | `fetch_historical_3y.py` | Fetch 3 năm dữ liệu lịch sử |
 | `fetch_historical_v4.py` | Fetch historical v4 (optimized) |
 | `fetch_minimal.py` | Fetch tối thiểu |
 ---
 ## Docker Compose
 File: `/opt/data/trading-portal/docker-compose.yml`
 | Service | Image/Context | Port | Limit |
 |---------|--------------|------|-------|
 | `db` | postgres:16-alpine | 5432 | 2G / 1 CPU |
 | `backend-api` | ./backend | 8001 | 1G / 2 CPU |
 | `backend-scheduler` | ./backend | — | 3G / 4 CPU |
 | `frontend` | ./frontend | 3000:80 | 256M / 0.5 CPU |
 Networks: `trading-net` (bridge), `hermes-agent_hermes-net` (external)
 ---
 ## Key Constants Summary
 | Constant | Value | Location |
 |----------|-------|----------|
 | MAX_OPEN_TRADES | 10 | trade_executor.py |
 | MIN_TRADE_SIZE_USDT | 5 | trade_executor.py |
 | MAX_TRADE_SIZE_USDT | 500 | trade_executor.py |
 | BATCH_SIZE | 25 | main_scheduler.py |
 | SEMAPHORE | 3 | candle_fetcher.py |
 | MIN_TRADES | 15 | signal_booster.py |
 | DECAY_LAMBDA | 0.05/day | signal_booster.py |
 | CACHE_TTL | 6h | signal_booster.py |
 | PNL_CACHE_TTL | 1h | signal_booster.py |
 | MAX_ATR_PCT | 8% | trade_executor.py |
 | MIN_ATR_PCT | 0.5% | trade_executor.py |
 | MAX_HOLD_HOURS | 8 | signal_service.py |
 | TRAILING_PCT | 5% | per user prefs |
 | JWT_ACCESS_EXPIRE | 15 min | config.py |
 | JWT_REFRESH_EXPIRE | 7 days | config.py |
 | DB_POOL_SIZE | 60 | database.py |
 | DB_STATEMENT_TIMEOUT | 10s (API) / 30s (Scheduler) | database.py |