diff --git a/docs/modules-reference.md b/docs/modules-reference.md new file mode 100644 index 0000000..7769e49 --- /dev/null +++ b/docs/modules-reference.md @@ -0,0 +1,315 @@ +# Trading Portal — Module Documentation + +> Tự động tạo bởi Hermes Agent ngày 2026-07-03. +> Cover toàn bộ codebase để team mới có thể tiếp tục phát triển. + +--- + +## Backend Core (`backend/app/`) + +### `config.py` — Application Settings +- **Class:** `Settings(BaseSettings)` — Pydantic Settings, load từ `.env` +- **Key configs:** `DATABASE_URL`, `JWT_PRIVATE_KEY_PATH`, `JWT_PUBLIC_KEYS_DIR`, `ENCRYPTION_KEY`, `CORS_ORIGINS`, `demo_user/pass` +- JWT: RS256, access token 15 phút, refresh token 7 ngày + +### `database.py` — Database Layer +- **Engine:** `create_async_engine` (asyncpg), pool_size=60, max_overflow=20 +- **Session:** `async_session_factory` → `AsyncSession` +- **Base:** `DeclarativeBase` cho tất cả ORM models +- **`get_db()`:** Async generator, tự commit/rollback/close. Xử lý `InterfaceError` để tránh CPU spike +- Statement timeout: 10s (API), 30s (Scheduler) + +### `main_api.py` — FastAPI Entrypoint (User-facing) +- Port 8001, JSON structured logging +- Lifespan: auto-create tables, register WebSocket push listener +- CORS middleware, RequestLoggingMiddleware +- Exception handlers: `AppException`, `ValidationError` (UUID-safe), 404, 500 +- `/health` endpoint: uptime + DB latency +- Bao gồm: `api_router` (REST) + `ws_router` (WebSocket candles) + +### `main_scheduler.py` — Background Scheduler +- No HTTP port — chạy độc lập, chỉ fetch candle + phân tích +- BATCH_SIZE=25 symbols, SEMAPHORE=3 concurrent +- Không import FastAPI → tiết kiệm RAM +- Entrypoint: `python3 -m app.main_scheduler` + +### `core/security.py` — Authentication & Encryption +- **Password:** bcrypt hashing (sync + async via thread pool) +- **JWT (RS256):** Multi-key rotation — private key để sign, public keys directory để verify. Token có `kid` header → thử matching key trước, fallback full scan +- **`create_access_token()`** (15 phút), **`create_refresh_token()`** (7 ngày, có `jti`) +- **`decode_token()`** thử tất cả public keys — cũ token vẫn valid sau rotation +- **AES-256-CBC:** `encrypt_api_key()` / `decrypt_api_key()` cho API key exchange +- **`generate_token_hash()`** SHA-256, **`generate_jti()`** UUID4 + +### `core/deps.py` — FastAPI Dependencies +- `get_db_session()` → AsyncSession +- `get_current_user()` → decode Bearer token, fetch User từ DB +- `get_current_active_user()` → check is_active +- `get_current_trader_user()` → role admin/trader +- `get_current_viewer_user()` → any role +- `get_current_admin_user()` → is_admin + role=="admin" + +### `core/middleware.py` — RequestLoggingMiddleware +- Log method, path, status, duration cho mọi request + +### `core/exceptions.py` — Custom Exceptions +- `AppException` (base), `InvalidCredentialsException`, `InvalidTokenException`, `ForbiddenException`, `NotFoundException` + +--- + +## Models (`backend/app/models/`) — 10 bảng + +| Model | Bảng | Mục đích | +|-------|------|----------| +| **User** | `users` | id (UUID), username, email, password_hash, role (admin/trader/viewer), preferences (JSON), is_active, is_admin | +| **Exchange** | `exchanges` | Tên sàn (binance, bybit, mexc, gate, bingx), fee config | +| **Symbol** | `symbols` | symbol (BTC/USDT), exchange_id FK, active, market_type | +| **Candle** | `candles` | symbol_id, timeframe (15m/1h/4h/1d), timestamp, OHLCV. **Partition by month** | +| **Watchlist** | `watchlists` | user_id FK, symbol | +| **ExchangeCredential** | `exchange_credentials` | user_id FK, exchange, api_key (AES encrypted), api_secret (AES encrypted) | +| **RefreshToken** | `refresh_tokens` | user_id FK, token_hash (SHA-256), jti, expires_at, revoked | +| **Signal** | `signals` | symbol, exchange, timeframe, signal_type (STRONG_BUY/SELL/BUY/CAUTION...), strength, price, indicators_snapshot (JSON), status | +| **HypotheticalTrade** | `hypothetical_trades` | Paper trade — user_id, signal_id, symbol, direction, entry/exit_price, pnl, pnl_percent, status. Có decay PnL filter `ABS(pnl_percent) < 100` | +| **RealTrade** | `real_trades` | Trade thật — user_id, exchange, symbol, side, amount, price, filled_amount, pnl, pnl_percent, order_id, status | +| **AuditLog** | `audit_logs` | user_id, action, target_type, target_id, details (JSON), ip_address | + +--- + +## API Routes (`backend/app/api/v1/`) — 13 modules + +| Router | Prefix | Endpoints chính | +|--------|--------|-----------------| +| **auth** | `/api/v1/auth` | POST register, login, refresh, logout, GET me | +| **exchanges** | `/api/v1/exchanges` | GET list, GET config | +| **symbols** | `/api/v1/symbols` | GET list, GET by exchange, GET top100 | +| **watchlist** | `/api/v1/watchlist` | CRUD watchlist items per user | +| **credentials** | `/api/v1/credentials` | CRUD API keys (encrypted), test connection | +| **signals** | `/api/v1/signals` | GET list (filter by symbol/exchange/timeframe/status), GET stats | +| **real_trades** | `/api/v1/trades` | GET list, GET stats (PnL, win rate), GET by id | +| **orders** | `/api/v1/orders` | POST place order, GET open orders, DELETE cancel | +| **strategies** | `/api/v1/strategies` | GET list 13 strategies + enabled/disabled status | +| **backtest** | `/api/v1/backtest` | POST run backtest, GET results, GET symbols list | +| **backtest_history** | `/api/v1/backtest-history` | CRUD saved backtest configs per user | +| **alerts** | `/api/v1/alerts` | CRUD price alerts | +| **analytics** | `/api/v1/analytics` | GET dashboard stats, PnL summary, win rate trend | +| **audit** | `/api/v1/audit` | GET audit logs (admin only) | +| **admin** | `/api/v1/admin` | User management (admin only) | + +### WebSocket (`backend/app/api/ws/`) +- **`candle_handler.py`** — `/ws/candles?symbols=BTC/USDT,ETH/USDT` — push real-time candles cho ChartContainer + +--- + +## Services (`backend/app/services/`) — Business Logic + +### `signal_service.py` (2328 dòng — file lớn nhất) +- **Phát hiện tín hiệu:** Double BB + RSI, 13 thuật toán voting +- **Signal pipeline:** fetch candle → compute indicators → vote → normalize → classify (STRONG/BUY/SELL/CAUTION) +- **Hypothetical trades:** tự mở paper trade khi có signal, đóng khi opposing signal hoặc stop loss +- **Cache:** win rates 5 phút, enabled strategies 5 phút + +### `trade_executor.py` (368 dòng) +- **`execute_signal_trade()`** — chỉ STRONG_BUY / STRONG_SELL mở trade thật +- MAX_OPEN_TRADES=10, hybrid eviction (lỗ nhất trước, else FIFO) +- Volatility filter: skip ATR > 8% hoặc < 0.5% +- Reversal: STRONG signal đóng opposing trades + +### `risk_manager.py` (203 dòng) +- **DynamicKellySizer:** Fractional Kelly (25%), tính từ PnL stats thực +- Kelly formula: `f* = (p × b - q) / b` +- Min trade 5 USDT, max 500 USDT +- **AdaptiveSLTPOptimizer:** 6 regime (trending/sideways/volatile/breakout/choppy/neutral) × SL/TP multipliers + +### `signal_booster.py` +- Win-rate boosting: vote × win_rate × 2.0 +- Exponential decay λ=0.05/ngày, half-life 14 ngày +- MIN_TRADES=15 để đủ statistical significance +- Correlation dampening: strategy cùng nhóm ÷ √count +- Dynamic threshold: normalize score / √active_strategies +- Cache TTL: 6 giờ + +### `indicator_service.py` +- **Tính indicators:** BB (20,2 + 20,1), RSI(14), MACD, SuperTrend, ATR, Ichimoku, OBV, StochRSI, MFI, Volume profile +- **SMC indicators:** BOS (Break of Structure), CHoCH (Change of Character), Order Blocks, FVG (Fair Value Gap) +- **Candlestick patterns:** 30+ mẫu (doji, hammer, engulfing, harami, morning star...) +- **Multi-timeframe:** vote từ 15m, 1h, 4h +- `compute_indicators()` trả về dict các indicator arrays + scalars + +### `candle_service.py` +- `get_indicators()` — fetch candles từ DB, compute indicators với pre-compute pattern +- `safe_slice()` — xử lý None scalars trong indicator dicts +- Backtest: pre-compute indicators trên toàn bộ dataset → slice trong loop (O(n) thay vì O(n²)) + +### `auth_service.py` +- Register: hash password, tạo user, tạo refresh token +- Login: verify password, tạo access + refresh token +- Refresh: verify refresh token, tạo access token mới +- Logout: revoke refresh token, update DB + +### `ws_push_service.py` +- `setup_push_listener()` — đăng ký WebSocket push cho real-time data +- Push candles, signals, trades đến connected clients + +### `notification_service.py` +- `notify_user()` — gửi notification (Telegram/Slack) khi có trade +- `notify_trade()` — thông báo trade opened/closed + +### `audit_service.py` +- `log_action()` — ghi audit log (user, action, target, details, IP) + +--- + +## Exchange Integration (`backend/app/exchange/`) — 5 sàn + +| File | Exchange | Notes | +|------|----------|-------| +| `base.py` | Abstract base | Interface chung: fetch_candles, get_balance, place_order, cancel_order | +| `binance.py` | Binance | ccxt.binance | +| `bybit.py` | Bybit | ccxt.bybit | +| `mexc.py` | MEXC | ccxt.mexc | +| `gate.py` | Gate.io | ccxt.gate | +| `bingx.py` | BingX | ccxt.bingx | +| `factory.py` | Factory | `factory.get_exchange(name, api_key, secret)` → instance | +| `rate_limiter.py` | Rate limiter | Token bucket, 10 req/s per exchange | +| `types.py` | Types | OrderRequest, OrderResponse, Balance, CandleStick | + +--- + +## Tasks (`backend/app/tasks/`) + +### `candle_fetcher.py` +- **`fetch_and_analyze()`** — main loop mỗi 5 phút +- Fetch 25 symbols × 5 exchanges × 4 timeframes (batch) +- Dedup: chỉ 1h timeframe phân tích tín hiệu +- Semaphore 3 concurrent analyses +- Chu kỳ đầy đủ ~95 phút cho 476 symbols + +### `exchange_sync.py` +- Sync symbols từ exchange config xuống DB +- Cập nhật active/inactive status + +### `stale_data_monitor.py` +- Phát hiện symbols có candle cũ (>30 phút) +- Auto retry fetch + +--- + +## Scripts (`backend/scripts/`) + +| Script | Mục đích | +|--------|----------| +| `fetch_candles_cron.py` | Fetch candles định kỳ (cron job) | +| `sync_candles.py` | Sync toàn bộ candles từ exchange | +| `backtest.py` | Run backtest standalone | +| `weekly_report.py` | Generate weekly trading report | +| `train_xgboost_model.py` | Train XGBoost model từ historical data | +| `select_top100_symbols.py` | Chọn top 100 symbols theo volume | +| `export_symbols.py` | Export symbol list | +| `backup_strategies.py` | Backup strategy configs | +| `apply_top100.py` | Apply top 100 selection | + +--- + +## Frontend (`frontend/src/`) — React + TypeScript + Redux + +### Pages +| Page | File | Chức năng | +|------|------|-----------| +| **Login** | `auth/LoginPage.tsx` | Login form (username/password), redirect sau login | +| **Register** | `auth/RegisterPage.tsx` | Register form | +| **Dashboard** | `dashboard/DashboardPage.tsx` | Chart + Order panel + Signal panel + Watchlist panel | +| **Profile** | `profile/ProfilePage.tsx` | 6 tabs: Info, Keys, Sessions, Settings (13 strategies), History (Signal/Real), Backtest | +| **Backtest** | `backtest/BacktestPage.tsx` | Backtest runner standalone | +| **Alerts** | `alerts/AlertsPage.tsx` | Price alerts management | +| **Analytics** | `analytics/AnalyticsPage.tsx` | PnL charts, win rate, trade history charts | +| **Admin** | `admin/AdminPage.tsx` | User management (admin only) | +| **AuditLog** | `AuditLogPage.tsx` | Audit log viewer (admin only) | + +### Components +| Component | File | Vai trò | +|-----------|------|---------| +| ChartContainer | `dashboard/ChartContainer.tsx` | TradingView-style chart với WebSocket real-time | +| ChartToolbar | `dashboard/ChartToolbar.tsx` | Timeframe, indicator toggles, drawing tools | +| OrderPanel | `dashboard/OrderPanel.tsx` | Place market/limit orders | +| SignalPanel | `dashboard/SignalPanel.tsx` | Live signal feed | +| WatchlistPanel | `dashboard/WatchlistPanel.tsx` | Symbol watchlist với giá real-time | +| ErrorBoundary | `components/ErrorBoundary.tsx` | Catch React errors | +| Skeleton | `components/Skeleton.tsx` | Loading skeleton placeholder | + +### State Management (Redux) +| Slice | File | State | +|-------|------|-------| +| **authSlice** | `auth/authSlice.ts` | user, token, isAuthenticated, loading | +| **Store** | `app/store.ts` | Redux store config, middleware | + +### API Services +| Service | File | Endpoints | +|---------|------|-----------| +| apiService | `api/apiService.ts` | Axios instance, interceptors, token refresh | +| signalApi | `api/signalApi.ts` | GET signals, GET stats | +| realTradeApi | `api/realTradeApi.ts` | GET trades, GET stats, POST order | +| watchlistApi | `api/watchlistApi.ts` | CRUD watchlist | +| alertApi | `api/alertApi.ts` | CRUD alerts | +| websocketService | `api/websocketService.ts` | WebSocket client, auto-reconnect | + +### Translations +| File | Ngôn ngữ | +|------|----------| +| `translations/vi.ts` | Tiếng Việt | +| `translations/en.ts` | English | +| `translations/index.tsx` | React context provider | + +### Types (`types/trading.ts`) +- `Signal`, `Trade`, `Candle`, `Exchange`, `Symbol`, `User`, `WatchlistItem`, `OrderRequest`, `BacktestConfig`, `Alert`, `AuditLogEntry` + +--- + +## Host Scripts (`scripts/`) + +| Script | Mục đích | +|--------|----------| +| `health_check.py` | Docker + API health check (30 phút) | +| `deploy-backend.sh` | Build + deploy backend container | +| `deploy-frontend.sh` | Build + deploy frontend container | +| `fetch_historical_3y.py` | Fetch 3 năm dữ liệu lịch sử | +| `fetch_historical_v4.py` | Fetch historical v4 (optimized) | +| `fetch_minimal.py` | Fetch tối thiểu | + +--- + +## Docker Compose + +File: `/opt/data/trading-portal/docker-compose.yml` + +| Service | Image/Context | Port | Limit | +|---------|--------------|------|-------| +| `db` | postgres:16-alpine | 5432 | 2G / 1 CPU | +| `backend-api` | ./backend | 8001 | 1G / 2 CPU | +| `backend-scheduler` | ./backend | — | 3G / 4 CPU | +| `frontend` | ./frontend | 3000:80 | 256M / 0.5 CPU | + +Networks: `trading-net` (bridge), `hermes-agent_hermes-net` (external) + +--- + +## Key Constants Summary + +| Constant | Value | Location | +|----------|-------|----------| +| MAX_OPEN_TRADES | 10 | trade_executor.py | +| MIN_TRADE_SIZE_USDT | 5 | trade_executor.py | +| MAX_TRADE_SIZE_USDT | 500 | trade_executor.py | +| BATCH_SIZE | 25 | main_scheduler.py | +| SEMAPHORE | 3 | candle_fetcher.py | +| MIN_TRADES | 15 | signal_booster.py | +| DECAY_LAMBDA | 0.05/day | signal_booster.py | +| CACHE_TTL | 6h | signal_booster.py | +| PNL_CACHE_TTL | 1h | signal_booster.py | +| MAX_ATR_PCT | 8% | trade_executor.py | +| MIN_ATR_PCT | 0.5% | trade_executor.py | +| MAX_HOLD_HOURS | 8 | signal_service.py | +| TRAILING_PCT | 5% | per user prefs | +| JWT_ACCESS_EXPIRE | 15 min | config.py | +| JWT_REFRESH_EXPIRE | 7 days | config.py | +| DB_POOL_SIZE | 60 | database.py | +| DB_STATEMENT_TIMEOUT | 10s (API) / 30s (Scheduler) | database.py |