diff --git a/.env.example b/.env.example index ff67e75..3e5a54c 100755 --- a/.env.example +++ b/.env.example @@ -1,5 +1,9 @@ # --- Database --- DATABASE_URL=postgresql+asyncpg://trading:trading_secret@db:5432/trading_portal +# Production alternative: put the password in a Docker secret file instead +# of embedding it in DATABASE_URL above (overrides DATABASE_URL's password +# at startup). Leave unset for local dev. +# DB_PASSWORD_FILE=/run/secrets/db_password.txt # --- JWT (RS256) --- JWT_PRIVATE_KEY_PATH=/run/secrets/jwt_private_key.pem @@ -7,8 +11,11 @@ JWT_PUBLIC_KEY_PATH=/run/secrets/jwt_public_key.pem JWT_ACCESS_TOKEN_EXPIRE_MINUTES=15 JWT_REFRESH_TOKEN_EXPIRE_DAYS=7 -# --- Encryption (AES-256-CBC for API keys) --- +# --- Encryption (AES-256-GCM for API keys) --- ENCRYPTION_KEY= # 32-byte hex, generate with: openssl rand -hex 32 +# Production alternative: read the key from a Docker secret file instead +# (overrides ENCRYPTION_KEY above). Leave unset for local dev. +# ENCRYPTION_KEY_FILE=/run/secrets/encryption_key.txt # --- Server --- HOST=0.0.0.0 diff --git a/.gitignore b/.gitignore index 6d08533..a644aab 100755 --- a/.gitignore +++ b/.gitignore @@ -28,6 +28,14 @@ Thumbs.db docker-data/ pgdata/ +# Postgres backups (produced by scripts/backup_postgres.sh) +backups/ +*.sql.gz + +# Local dev tooling (machine-specific paths, not project code) +.claude/ +frontend/dev.cmd + # Logs *.log diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 40a5735..9b97751 100755 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -36,6 +36,7 @@ | `backend-api` | FastAPI | 1G / 2 CPU | REST API, WebSocket, auth | | `backend-scheduler` | Python async | 2G / 3 CPU | Fetch candle, phân tích tín hiệu, quản lý trade | | `db` | PostgreSQL 16 | 2G | Lưu trữ toàn bộ dữ liệu | +| `redis` | redis:7-alpine | 192M | Cache dùng chung giữa backend-api và backend-scheduler (win-rate, PnL stats) — có fallback in-memory nếu Redis down | | `nginx_proxy` | nginx:1.27-alpine | — | Reverse proxy SSL cho tất cả domain | | `certbot_ssl` | certbot/certbot | — | Auto-renew Let's Encrypt mỗi 12h | | `gitea` | gitea/gitea:latest | ~300MB | Git server tại git.dangloica.org | @@ -323,6 +324,17 @@ Script `/opt/data/scripts/health_check.py`: | `PROJECT_DIR` | `/opt/data/trading-portal` | | `DATABASE_URL` | postgresql+asyncpg://trading:***@db:5432/trading_portal | | `LOG_LEVEL` | INFO | + +### Secrets (Docker, tất cả nằm trong `/opt/data/trading-portal/secrets/`, bind-mount `:ro` vào `/run/secrets` cho `db`/`backend-api`/`backend-scheduler`) + +| File | Dùng bởi | Ghi chú | +|------|---------|---------| +| `jwt_private.pem` | backend-api, backend-scheduler | Ký JWT access/refresh token | +| `jwt_public_keys/*.pem` | backend-api, backend-scheduler | Verify JWT — hỗ trợ rotation nhiều key | +| `db_password.txt` | db, backend-api, backend-scheduler | Mật khẩu Postgres — `db` đọc qua `POSTGRES_PASSWORD_FILE`, backend đọc qua `DB_PASSWORD_FILE` (config.py tự ghép vào `DATABASE_URL`) | +| `encryption_key.txt` | backend-api, backend-scheduler | Khoá AES-256-GCM mã hoá API key sàn — đọc qua `ENCRYPTION_KEY_FILE` | + +> Không còn secret nào truyền qua plain env var (`${DB_PASSWORD}`, `${ENCRYPTION_KEY}`) trong `docker-compose.yml` — toàn bộ đã chuyển sang file, đồng nhất với cách JWT key đã làm từ trước. | Timezone display | UTC+7 (ICT) | --- @@ -341,6 +353,7 @@ Script `/opt/data/scripts/health_check.py`: - `compute_strategy_win_rates` có thể chạy 3 lần đồng thời khi cache hết hạn - Không nghiêm trọng — chỉ xảy ra 1 lần mỗi 6 giờ - Impact: redundant computation, không sai data +- (2026-07-03) `get_cached_rates()`/`get_pnl_stats()` giờ đọc qua Redis trước — chỉ `backend-scheduler` tính toán và ghi (`compute_strategy_win_rates` chạy trong scheduler), `backend-api` đọc chung kết quả thay vì mỗi process tự giữ cache riêng biệt. Race condition compute vẫn còn (nhiều lần gọi đồng thời trong cùng scheduler), nhưng không còn vấn đề "mỗi process thấy dữ liệu khác nhau" nữa. --- diff --git a/DEPLOYMENT.md b/DEPLOYMENT.md index 3f6ac7f..44404bb 100755 --- a/DEPLOYMENT.md +++ b/DEPLOYMENT.md @@ -366,3 +366,47 @@ psql -U trading -d trading -c " | PostgreSQL | 5432 | TCP | | Hermes Dashboard | 9119 | HTTP | | Hermes Gateway | — | HTTP/WS | + +## 9. Backup & Restore + +### 9.0 ⚠️ Migration bắt buộc trước khi deploy bản có secrets mới + +`docker-compose.yml` giờ yêu cầu 2 file secret mới — **service `db` sẽ không khởi động được nếu thiếu**: + +```bash +mkdir -p /opt/data/trading-portal/secrets +echo -n "" > /opt/data/trading-portal/secrets/db_password.txt +echo -n "" > /opt/data/trading-portal/secrets/encryption_key.txt +chmod 600 /opt/data/trading-portal/secrets/db_password.txt /opt/data/trading-portal/secrets/encryption_key.txt +``` + +> Nếu container `db` **đã có dữ liệu** với mật khẩu Postgres cũ, `db_password.txt` phải chứa **đúng mật khẩu cũ đó** (không phải mật khẩu mới) — đổi mật khẩu Postgres thật sự là việc khác (`ALTER USER trading WITH PASSWORD ...`), không tự động xảy ra chỉ vì đổi cách truyền secret. `encryption_key.txt` bắt buộc phải là **đúng `ENCRYPTION_KEY` đang dùng** — dùng sai giá trị sẽ khiến toàn bộ API key sàn đã lưu (mã hoá AES-GCM) không giải mã được. + +### 9.1 Backup định kỳ + +Script `scripts/backup_postgres.sh` chạy `pg_dump` bên trong container `trading-db`, nén gzip, lưu vào `backups/` (đã gitignore — không commit dump vào git), và tự xoá backup cũ hơn N ngày (mặc định 14). + +```bash +# Chạy tay +./scripts/backup_postgres.sh # giữ 14 ngày +./scripts/backup_postgres.sh 30 # giữ 30 ngày + +# Cron (khuyến nghị: mỗi ngày 3h sáng) +0 3 * * * /opt/data/trading-portal/scripts/backup_postgres.sh >> /var/log/trading-portal-backup.log 2>&1 +``` + +**Khuyến nghị**: đồng bộ thư mục `backups/` ra một nơi lưu trữ khác máy chủ (S3, rsync sang server khác...) — backup nằm cùng ổ đĩa với DB không chống được lỗi hỏng ổ cứng hay xoá nhầm cả server. + +### 9.2 Restore + +`scripts/restore_postgres.sh` — **thao tác phá huỷ**, luôn hỏi xác nhận trước khi chạy: + +```bash +./scripts/restore_postgres.sh backups/trading_portal_20260703_030000.sql.gz +``` + +Quy trình: dừng `backend-api`/`backend-scheduler` → restore vào DB tạm `trading_portal_restoring` → đổi tên DB cũ thành `trading_portal_old_` (giữ lại, không xoá ngay) → đổi tên DB mới phục hồi thành `trading_portal` → khởi động lại backend. Sau khi xác nhận restore đúng, xoá thủ công DB `_old_...` để giải phóng dung lượng. + +### 9.3 Test khôi phục định kỳ + +Backup không có giá trị nếu chưa từng thử restore. Khuyến nghị test restore vào một DB tạm (không phải production) ít nhất mỗi quý một lần để chắc chắn quy trình còn hoạt động đúng. diff --git a/backend/app/config.py b/backend/app/config.py index a9b4ba5..b132725 100755 --- a/backend/app/config.py +++ b/backend/app/config.py @@ -1,6 +1,12 @@ from __future__ import annotations from pydantic_settings import BaseSettings, SettingsConfigDict +from sqlalchemy.engine import make_url + + +def _read_secret_file(path: str) -> str: + with open(path, "r") as f: + return f.read().strip() class Settings(BaseSettings): @@ -8,6 +14,11 @@ class Settings(BaseSettings): # Database DATABASE_URL: str = "postgresql+asyncpg://trading:trading_secret@db:5432/trading_portal" + # If set, the password is read from this file (Docker secret) and used + # to override whatever password is embedded in DATABASE_URL. Keeps + # DB credentials out of plain env vars, consistent with how JWT keys + # are already handled. + DB_PASSWORD_FILE: str = "" # JWT JWT_PRIVATE_KEY_PATH: str = "/run/secrets/jwt_private.pem" @@ -18,6 +29,13 @@ class Settings(BaseSettings): # Encryption ENCRYPTION_KEY: str = "" + # If set, ENCRYPTION_KEY is read from this file (Docker secret) instead. + ENCRYPTION_KEY_FILE: str = "" + + # Redis (shared cache across backend-api / backend-scheduler processes). + # Optional: if unreachable, callers fall back to per-process in-memory + # caches (see app/core/redis_client.py). + REDIS_URL: str = "redis://redis:6379/0" # Server HOST: str = "0.0.0.0" @@ -39,5 +57,16 @@ class Settings(BaseSettings): extra="allow", ) + def model_post_init(self, __context) -> None: + if self.DB_PASSWORD_FILE: + password = _read_secret_file(self.DB_PASSWORD_FILE) + url = make_url(self.DATABASE_URL).set(password=password) + # SQLAlchemy's default str() masks the password with "***" — + # render_as_string(hide_password=False) is needed to get the + # real, usable connection string back. + self.DATABASE_URL = url.render_as_string(hide_password=False) + if self.ENCRYPTION_KEY_FILE: + self.ENCRYPTION_KEY = _read_secret_file(self.ENCRYPTION_KEY_FILE) + settings = Settings() diff --git a/backend/app/core/redis_client.py b/backend/app/core/redis_client.py new file mode 100644 index 0000000..9bf9478 --- /dev/null +++ b/backend/app/core/redis_client.py @@ -0,0 +1,74 @@ +"""Shared async Redis client, used to cache state across the backend-api and +backend-scheduler processes (e.g. signal_booster win-rate/PnL stats), which +are separate containers and can't share Python module-level globals. + +Every helper here degrades gracefully: if Redis is unreachable (not +configured for local dev, container down, network hiccup), callers get +``None``/a cache miss instead of an exception, and are expected to fall back +to a per-process in-memory cache. +""" +from __future__ import annotations + +import json +import logging +import time +from typing import Any, Optional + +from redis import asyncio as redis_asyncio + +from app.config import settings + +logger = logging.getLogger(__name__) + +_client: Optional["redis_asyncio.Redis"] = None +_failed_until: float = 0.0 # monotonic timestamp; skip Redis until past this +_RETRY_COOLDOWN_SECONDS = 30 + + +def _get_client() -> Optional["redis_asyncio.Redis"]: + """Lazily create the Redis client. Returns None while within the retry + cooldown after a recent failure, so a Redis outage costs at most one + slow connect attempt per cooldown window instead of stalling every call.""" + global _client + if time.monotonic() < _failed_until: + return None + if _client is None: + _client = redis_asyncio.from_url( + settings.REDIS_URL, decode_responses=True, socket_connect_timeout=2, + ) + return _client + + +def _mark_failed(exc: Exception, operation: str, key: str) -> None: + global _failed_until + logger.warning( + "Redis %s(%s) failed, falling back to in-memory cache for %ds: %s", + operation, key, _RETRY_COOLDOWN_SECONDS, exc, + ) + _failed_until = time.monotonic() + _RETRY_COOLDOWN_SECONDS + + +async def get_json(key: str) -> Any | None: + """Return the JSON-decoded value for ``key``, or None on miss/error.""" + client = _get_client() + if client is None: + return None + try: + raw = await client.get(key) + return json.loads(raw) if raw is not None else None + except Exception as exc: + _mark_failed(exc, "get_json", key) + return None + + +async def set_json(key: str, value: Any, ttl_seconds: int) -> None: + """Store ``value`` as JSON under ``key`` with an expiry. Silently no-ops + on failure — callers should already be maintaining their own in-memory + fallback, so a failed cache write here is not fatal.""" + client = _get_client() + if client is None: + return + try: + await client.set(key, json.dumps(value), ex=ttl_seconds) + except Exception as exc: + _mark_failed(exc, "set_json", key) diff --git a/backend/app/services/indicator_service.py b/backend/app/services/indicator_service.py index 6a03547..a8aa407 100755 --- a/backend/app/services/indicator_service.py +++ b/backend/app/services/indicator_service.py @@ -129,11 +129,15 @@ def rsi(prices: list[float], period: int = 14) -> list[Optional[float]]: avg_gain = sum(d for d in deltas[:period] if d > 0) / period avg_loss = abs(sum(d for d in deltas[:period] if d < 0)) / period + if avg_gain == 0 and avg_loss == 0: + # No price movement at all -- RSI is neutral by definition. + # (Previously this fell through to an `rs = 50.0` sentinel that + # was then run through the RSI formula, producing ~98.04 + # instead of the intended neutral 50.) + result.append(50.0) + continue if avg_loss == 0: - if avg_gain == 0: - rs = 50.0 # no movement - else: - rs = 100.0 + rs = 100.0 else: rs = avg_gain / avg_loss result.append(100.0 - (100.0 / (1.0 + rs))) @@ -1291,6 +1295,12 @@ def mfi(candles: list[dict], period: int = 14) -> list[Optional[float]]: pos_flow = 0.0 neg_flow = 0.0 for j in range(i - period, i): + if j == 0: + # No prior candle to compare against -- Python's negative + # indexing would otherwise wrap `typical_prices[j - 1]` + # around to the LAST candle in the series, spuriously + # injecting a bogus flow-direction comparison. + continue mf = raw_money_flow[j] if typical_prices[j] > typical_prices[j - 1]: pos_flow += mf diff --git a/backend/app/services/signal_booster.py b/backend/app/services/signal_booster.py index f634435..f5bbac1 100755 --- a/backend/app/services/signal_booster.py +++ b/backend/app/services/signal_booster.py @@ -15,14 +15,21 @@ from typing import Any from sqlalchemy import text from sqlalchemy.ext.asyncio import AsyncSession +from app.core import redis_client from app.database import async_session_factory logger = logging.getLogger(__name__) # ── Global cache ────────────────────────────────────────────────────────── +# Backed by Redis so backend-api and backend-scheduler (separate processes/ +# containers) share the same win rates instead of each computing/seeing +# their own. The in-memory dict below remains as a same-process fallback +# for whenever Redis is unreachable (e.g. local dev without a redis +# container) — see app/core/redis_client.py. _win_rate_cache: dict[str, float] = {} _last_cache_update: datetime | None = None _CACHE_TTL_SECONDS = 21_600 # 6 hours +_REDIS_KEY_WIN_RATES = "signal_booster:win_rates" # ── Strategy name normalisation ────────────────────────────────────────── # Maps entry_reason values stored in hypothetical_trades to canonical names. @@ -167,14 +174,15 @@ async def _compute_rates(db: AsyncSession) -> dict[str, float]: _win_rate_cache = rates _last_cache_update = datetime.now(timezone.utc) + await redis_client.set_json(_REDIS_KEY_WIN_RATES, rates, _CACHE_TTL_SECONDS) logger.info( "Computed win rates for %d strategies (decay=%.3f/day, min_trades=%d)", len(rates), DECAY_LAMBDA, MIN_TRADES, ) - + # Also refresh PnL stats for Kelly sizing await _refresh_pnl_stats(db) - + return rates except Exception: @@ -182,8 +190,13 @@ async def _compute_rates(db: AsyncSession) -> dict[str, float]: return dict(_win_rate_cache) or {} -def get_cached_rates() -> dict[str, float]: - """Return the current in-memory win-rate cache (may be stale or empty).""" +async def get_cached_rates() -> dict[str, float]: + """Return cached win rates: Redis first (shared across processes), then + falls back to this process's own in-memory cache (may be stale or + empty) if Redis is unavailable or has no data yet.""" + from_redis = await redis_client.get_json(_REDIS_KEY_WIN_RATES) + if from_redis: + return from_redis return dict(_win_rate_cache) @@ -258,14 +271,22 @@ def get_confidence( _pnl_stats_cache: dict[str, float] = {} _last_pnl_cache_update: float = 0.0 _PNL_CACHE_TTL = 3600 # 1 hour +_REDIS_KEY_PNL_STATS = "signal_booster:pnl_stats" -def get_pnl_stats() -> dict[str, float]: - """Return cached avg_win_pct / avg_loss_pct (sync, safe for async context). - - Cache is refreshed by the scheduler periodically via compute_strategy_win_rates. - Falls back to reasonable defaults (avg_win=3.0%, avg_loss=2.0%). +async def get_pnl_stats() -> dict[str, float]: + """Return cached avg_win_pct / avg_loss_pct. + + Cache is refreshed by the scheduler periodically via + compute_strategy_win_rates. Checks Redis first (shared across + backend-api/backend-scheduler processes), then this process's own + in-memory cache, then falls back to reasonable defaults + (avg_win=3.0%, avg_loss=2.0%). """ + from_redis = await redis_client.get_json(_REDIS_KEY_PNL_STATS) + if from_redis: + return from_redis + import time as _time now = _time.monotonic() if now - _last_pnl_cache_update < _PNL_CACHE_TTL and _pnl_stats_cache: @@ -294,7 +315,8 @@ async def _refresh_pnl_stats(db: AsyncSession) -> None: if row and row[0] and row[1]: _pnl_stats_cache = {"avg_win": float(row[0]), "avg_loss": float(row[1])} _last_pnl_cache_update = _time.monotonic() - logger.debug("PnL stats refreshed: avg_win=%.2f%%, avg_loss=%.2f%%", + await redis_client.set_json(_REDIS_KEY_PNL_STATS, _pnl_stats_cache, _PNL_CACHE_TTL) + logger.debug("PnL stats refreshed: avg_win=%.2f%%, avg_loss=%.2f%%", _pnl_stats_cache["avg_win"], _pnl_stats_cache["avg_loss"]) except Exception as e: logger.debug("Failed to refresh PnL stats: %s", e) diff --git a/backend/app/services/signal_scoring.py b/backend/app/services/signal_scoring.py new file mode 100644 index 0000000..ff71b57 --- /dev/null +++ b/backend/app/services/signal_scoring.py @@ -0,0 +1,541 @@ +"""Pure signal-scoring logic — the 13-algorithm voting system. + +Extracted out of `signal_service.py` (which mixes this scoring logic with +async DB/notification/trade-trigger orchestration) so the classification +math can be read, tested, and reasoned about independently of any database +or network side effects. Nothing in this module touches I/O. +""" + +from __future__ import annotations + +import logging +import math +from decimal import Decimal +from typing import Optional + +from app.services.signal_booster import ( + boost_score, + get_confidence as compute_booster_confidence, +) + +logger = logging.getLogger(__name__) + +# --------------------------------------------------------------------------- +# Signal type constants +# --------------------------------------------------------------------------- +STRONG_BUY = "STRONG_BUY" +BUY = "BUY" +STRONG_SELL = "STRONG_SELL" +SELL = "SELL" +CAUTION_LONG = "CAUTION_LONG" +CAUTION_SHORT = "CAUTION_SHORT" +SQUEEZE_ALERT = "SQUEEZE_ALERT" + +# --------------------------------------------------------------------------- +# Signal detection configuration +# --------------------------------------------------------------------------- +# Minimum distance from BB bounds to filter noise +MIN_BB_DISTANCE_PCT = Decimal("0.001") # 0.1% + + +def _get_bb_values(indicators: dict) -> dict[str, list[float]] | None: + """Extract Bollinger Band values from indicators dict.""" + bb = indicators.get("bollinger_bands") + if not bb: + return None + # Ensure all required keys exist + required = ["upper", "middle", "lower"] + if not all(k in bb for k in required): + return None + return bb + + +def _get_rsi_values(indicators: dict) -> list[float] | None: + """Extract RSI values.""" + rsi = indicators.get("rsi_14") + if not rsi or not isinstance(rsi, list) or len(rsi) == 0: + return None + return rsi + + +def _get_sma_values(indicators: dict) -> list[float] | None: + """Extract SMA values.""" + sma = indicators.get("sma_20") + if not sma or not isinstance(sma, list) or len(sma) == 0: + return None + return sma + + +def _detect_squeeze( + bb: dict[str, list[float]], + lookback: int = 10, +) -> bool: + """Detect Bollinger Band squeeze — narrowing of the band width. + + A squeeze occurs when the current band width is at the lower end + of the recent range, indicating low volatility before a breakout. + """ + upper = bb.get("upper", []) + lower = bb.get("lower", []) + if not upper or not lower or len(upper) < lookback or len(lower) < lookback: + return False + + # Band width = upper - lower + widths = [] + for i in range(max(0, len(upper) - lookback), len(upper)): + try: + widths.append(upper[i] - lower[i]) + except TypeError: + return False + if len(widths) < 3: + return False + + current_width = widths[-1] + # Check if upper_1 and lower_1 exist for squeeze detection + upper_1 = bb.get("upper_1", []) + lower_1 = bb.get("lower_1", []) + if upper_1 and lower_1 and len(upper_1) >= 1 and len(lower_1) >= 1: + try: + inner_width = upper_1[-1] - lower_1[-1] + outer_width = upper[-1] - lower[-1] + # Squeeze when inner bands are VERY narrow relative to outer (≤20%) + if outer_width > 0 and inner_width / outer_width < 0.2: + return True + except TypeError: + pass + + # Alternative: check if current width is at absolute minimum of longer lookback + min_width = min(widths) + if min_width > 0 and current_width <= min_width * 1.01: + return True + + return False + + +def _classify_signal_bb( + close_price: float, + bb: dict[str, list[float]], + rsi: list[float] | None, + sma: list[float] | None, +) -> tuple[Optional[str], Optional[str]]: + """Classify signal using Double BB + RSI only. + + Returns (signal_type, strength) or (None, None) if no signal. + Used as one vote in the combined 4-algorithm system. + """ + if len(bb.get("upper", [])) == 0 or len(bb.get("lower", [])) == 0: + return None, None + + upper_2 = bb["upper"][-1] + lower_2 = bb["lower"][-1] + upper_1 = bb.get("upper_1", [None])[-1] + lower_1 = bb.get("lower_1", [None])[-1] + middle = bb.get("middle", [None])[-1] + current_rsi = rsi[-1] if rsi and len(rsi) > 0 else 50 + current_sma = sma[-1] if sma and len(sma) > 0 else None + + # --- Price above BB 1σ Upper → Trend Long --- + if upper_1 is not None and close_price > upper_1: + if close_price > upper_2: + if current_rsi > 75: + return CAUTION_SHORT, "MODERATE" + # Only STRONG_BUY if RSI is firmly bullish AND SMA is above middle + if current_rsi > 60 and current_sma and middle and current_sma > middle: + return STRONG_BUY, "STRONG" + return BUY, "MODERATE" + else: + if current_rsi > 55: + return BUY, "MODERATE" + return BUY, "WEAK" + + # --- Price below BB 1σ Lower → Trend Short --- + if lower_1 is not None and close_price < lower_1: + if close_price < lower_2: + if current_rsi < 25: + return CAUTION_LONG, "MODERATE" + # Only STRONG_SELL if RSI is firmly bearish AND SMA is below middle + if current_rsi < 40 and current_sma and middle and current_sma < middle: + return STRONG_SELL, "STRONG" + return SELL, "MODERATE" + else: + if current_rsi < 45: + return SELL, "MODERATE" + return SELL, "WEAK" + + return None, None + + +def _classify_signal_combined( + close_price: float, + bb: dict[str, list[float]], + rsi: list[float] | None, + sma: list[float] | None, + macd_data: dict | None, + st_data: dict | None, + vol_data: list | None, + ichi_data: dict | None = None, + rsi_div: tuple = (None, None), + macd_div: tuple = (None, None), + smc_data: dict | None = None, + mtf_votes: list[tuple[Optional[str], Optional[str], float]] | None = None, + obv_data: list | None = None, + stoch_rsi_data: dict | None = None, + mfi_data: list | None = None, + fvg_data: dict | None = None, + candlestick_score: float | None = None, + rates: dict[str, float] | None = None, + enabled_strategies: list[str] | None = None, +) -> tuple[Optional[str], Optional[str], float, dict[str, float]]: + """Classify market state using 13-algorithm voting with win-rate boosting. + + Algorithms: + 1. Double BB + RSI + 2. MACD Crossover + 3. SuperTrend + 4. Volume Breakout + 5. Ichimoku Cloud + 6. Divergence Detection (RSI + MACD) + 7. 🌤️ Market Structure (SMC) — BOS, CHoCH, OB + 8. 🔄 Multi-Timeframe (15m + 1h + 4h) + 9. 📊 OBV (On-Balance Volume) Crossover + 10. 🔄 Stochastic RSI Crossover + 11. 💰 MFI (Money Flow Index) + 12. 🕯️ FVG (Fair Value Gap) + 13. 🕯️ Candlestick Patterns (30+ patterns) + + Each algorithm votes: BUY (+1/+2), SELL (-1/-2), or NEUTRAL (0). + If *rates* is provided, each strategy's raw score is boosted by its + historical win rate before the final classification. + + Returns (signal_type, strength, confidence, raw_scores) where + confidence is a 0-1 float and raw_scores is a dict of all 9 + algorithm scores for ML feature collection. + """ + # ── NaN/Inf guard: reject any invalid price before processing ── + if not math.isfinite(close_price) or close_price <= 0: + logger.warning("_classify_signal_combined: invalid close_price=%s, returning NEUTRAL", close_price) + return None, None, 0.0, {} + + # ── Special signals (override) ── + squeeze = _detect_squeeze(bb) + if squeeze: + return SQUEEZE_ALERT, "MODERATE", 0.5, {} + + # P2-2: Call _classify_signal_bb ONCE, reuse result for both + # early-return check AND the raw_scores vote + bb_type, bb_strength = _classify_signal_bb(close_price, bb, rsi, sma) + if bb_type in (CAUTION_LONG, CAUTION_SHORT): + return bb_type, "MODERATE", 0.5, {} + + # ── Collect per-strategy raw scores ── + raw_scores: dict[str, float] = { + "double_bb_rsi": 0.0, + "macd_crossover": 0.0, + "supertrend": 0.0, + "volume_breakout": 0.0, + "ichimoku": 0.0, + "divergence": 0.0, + "smc": 0.0, + "mtf": 0.0, + "obv": 0.0, + "stoch_rsi": 0.0, + "mfi": 0.0, + "fvg": 0.0, + "candlestick": 0.0, + } + + # 1. BB + RSI vote (reuse bb_type/bb_strength from above — P2-2) + if bb_type == STRONG_BUY: + raw_scores["double_bb_rsi"] = 2.0 + elif bb_type == BUY: + raw_scores["double_bb_rsi"] = 1.0 + elif bb_type == STRONG_SELL: + raw_scores["double_bb_rsi"] = -2.0 + elif bb_type == SELL: + raw_scores["double_bb_rsi"] = -1.0 + + # 2. MACD Crossover vote + if macd_data: + macd_line = macd_data.get("macd_line", []) + signal_line = macd_data.get("signal_line", []) + if len(macd_line) >= 2 and len(signal_line) >= 2: + m_curr = macd_line[-1] + m_prev = macd_line[-2] + s_curr = signal_line[-1] + s_prev = signal_line[-2] + if m_curr is not None and s_curr is not None and m_prev is not None and s_prev is not None: + if m_prev <= s_prev and m_curr > s_curr and m_curr > 0: + raw_scores["macd_crossover"] = 1.0 + elif m_prev >= s_prev and m_curr < s_curr and m_curr < 0: + raw_scores["macd_crossover"] = -1.0 + + # 3. SuperTrend vote + if st_data: + trend = st_data.get("trend", []) + if trend and len(trend) >= 1 and trend[-1] is not None: + raw_scores["supertrend"] = 1.0 if trend[-1] else -1.0 + + # 4. Volume Breakout vote (current bar only — avoid stale signals) + if vol_data and len(vol_data) >= 2: + vol_now = vol_data[-1] + if vol_now is True: + sma20 = sma[-1] if sma and len(sma) > 0 else None + if sma20 is not None and close_price > sma20: + raw_scores["volume_breakout"] = 1.0 + elif sma20 is not None and close_price < sma20: + raw_scores["volume_breakout"] = -1.0 + + # 4b. OBV (On-Balance Volume) Crossover vote + if obv_data and len(obv_data) >= 2: + obv_now = obv_data[-1] + obv_prev = obv_data[-2] + if obv_now is True: + raw_scores["obv"] = 1.0 + elif obv_now is False: + raw_scores["obv"] = -1.0 + elif obv_prev is True: + # One bar ago still counts for momentum + raw_scores["obv"] = 0.5 + elif obv_prev is False: + raw_scores["obv"] = -0.5 + + # 10. Stochastic RSI Crossover vote + if stoch_rsi_data: + k_line = stoch_rsi_data.get("k", []) + d_line = stoch_rsi_data.get("d", []) + if len(k_line) >= 2 and len(d_line) >= 2: + k_curr = k_line[-1] + k_prev = k_line[-2] + d_curr = d_line[-1] + if k_curr is not None and d_curr is not None and k_prev is not None: + # Oversold: %K < 20 and crossing above %D → BUY + if k_curr < 20 and k_prev <= d_curr and k_curr > d_curr: + raw_scores["stoch_rsi"] = 1.5 + # Overbought: %K > 80 and crossing below %D → SELL + elif k_curr > 80 and k_prev >= d_curr and k_curr < d_curr: + raw_scores["stoch_rsi"] = -1.5 + + # 11. MFI (Money Flow Index) vote + if mfi_data and len(mfi_data) >= 2: + mfi_curr = mfi_data[-1] + mfi_prev = mfi_data[-2] + if mfi_curr is not None and mfi_prev is not None: + # Oversold: MFI < 20 and turning up → BUY + if mfi_curr < 20 and mfi_curr > mfi_prev: + raw_scores["mfi"] = 1.5 + # Overbought: MFI > 80 and turning down → SELL + elif mfi_curr > 80 and mfi_curr < mfi_prev: + raw_scores["mfi"] = -1.5 + # Neutral divergence: MFI trending in opposite direction to close + elif mfi_curr < 30: + raw_scores["mfi"] = 0.5 + elif mfi_curr > 70: + raw_scores["mfi"] = -0.5 + + # 12. FVG (Fair Value Gap) vote — only when price is near the gap + if fvg_data: + fvg_type = fvg_data.get("type") + fvg_high = fvg_data.get("gap_high") + fvg_low = fvg_data.get("gap_low") + if fvg_type == "BULLISH" and fvg_high is not None and fvg_low is not None: + # Price near or inside bullish FVG → support → BUY + gap_mid = (fvg_high + fvg_low) / 2.0 + dist_pct = abs(close_price - gap_mid) / close_price * 100 + if dist_pct < 1.0: + raw_scores["fvg"] = 2.0 + elif dist_pct < 2.0: + raw_scores["fvg"] = 1.0 + # else: too far from gap → no vote (prevents noise) + elif fvg_type == "BEARISH" and fvg_high is not None and fvg_low is not None: + # Price near or inside bearish FVG → resistance → SELL + gap_mid = (fvg_high + fvg_low) / 2.0 + dist_pct = abs(close_price - gap_mid) / close_price * 100 + if dist_pct < 1.0: + raw_scores["fvg"] = -2.0 + elif dist_pct < 2.0: + raw_scores["fvg"] = -1.0 + # else: too far from gap → no vote + + # 13. Candlestick Patterns vote + if candlestick_score is not None and candlestick_score != 0.0: + raw_scores["candlestick"] = candlestick_score + + # 5. Ichimoku Cloud vote + if ichi_data: + tenkan = ichi_data.get("tenkan", []) + kijun = ichi_data.get("kijun", []) + senkou_a = ichi_data.get("senkou_a", []) + senkou_b = ichi_data.get("senkou_b", []) + if (tenkan and len(tenkan) >= 2 and kijun and len(kijun) >= 2 + and senkou_a and len(senkou_a) >= 2 and senkou_b and len(senkou_b) >= 2): + t_now = tenkan[-1] + k_now = kijun[-1] + t_prev = tenkan[-2] + k_prev = kijun[-2] + sa_now = senkou_a[-1] + sb_now = senkou_b[-1] + ichi_score = 0.0 + if t_now is not None and k_now is not None and t_prev is not None and k_prev is not None: + if t_prev <= k_prev and t_now > k_now: + ichi_score += 1.5 + elif t_prev >= k_prev and t_now < k_now: + ichi_score -= 1.5 + if sa_now is not None and sb_now is not None: + if sa_now > sb_now and close_price > sa_now: + ichi_score += 1.0 + elif sa_now < sb_now and close_price < sb_now: + ichi_score -= 1.0 + raw_scores["ichimoku"] = ichi_score + + # 6. Divergence Detection vote + div_score = 0.0 + for div_type, div_strength in [rsi_div, macd_div]: + if div_type == "BULLISH": + div_score += 2.0 if div_strength == "STRONG" else 1.0 + elif div_type == "BEARISH": + div_score -= 2.0 if div_strength == "STRONG" else 1.0 + raw_scores["divergence"] = div_score + + # 7. 🌤️ Market Structure (SMC) vote + smc_score = 0.0 + if smc_data: + bos = smc_data.get("bos") + choch = smc_data.get("choch") + trend = smc_data.get("trend", "NEUTRAL") + obs = smc_data.get("order_blocks", []) + + if bos == "BULLISH": + smc_score += 1.5 + elif bos == "BEARISH": + smc_score -= 1.5 + + if choch == "BULLISH": + smc_score += 1.5 + elif choch == "BEARISH": + smc_score -= 1.5 + + if trend == "BULLISH": + smc_score += 1.0 + elif trend == "BEARISH": + smc_score -= 1.0 + + if obs: + latest_close = close_price + bullish_obs = [ob for ob in obs if ob.get("type") == "BULLISH"] + bearish_obs = [ob for ob in obs if ob.get("type") == "BEARISH"] + + for ob in bullish_obs: + ob_high = ob.get("price_high", 0) + if 0 < ob_high and ob_high * 0.995 <= latest_close <= ob_high * 1.005: + smc_score += 1.0 + break + + for ob in bearish_obs: + ob_low = ob.get("price_low", 0) + if 0 < ob_low and ob_low * 0.995 <= latest_close <= ob_low * 1.005: + smc_score -= 1.0 + break + raw_scores["smc"] = smc_score + + # 8. 🔄 Multi-Timeframe vote (weighted) + mtf_score = 0.0 + if mtf_votes: + for sig_type, sig_strength, weight in mtf_votes: + if sig_type == STRONG_BUY: + mtf_score += 2.0 * weight + elif sig_type == BUY: + mtf_score += 1.0 * weight + elif sig_type == STRONG_SELL: + mtf_score -= 2.0 * weight + elif sig_type == SELL: + mtf_score -= 1.0 * weight + raw_scores["mtf"] = mtf_score + + # ── Apply enabled_strategies filter (zero-out disabled strategies) ── + if enabled_strategies is not None: + disabled = [s for s in raw_scores if s not in enabled_strategies] + if disabled: + logger.debug("Disabled strategies: %s", disabled) + for s in disabled: + raw_scores[s] = 0.0 + + # ── P1-16: Correlation dampening ── + # Strategies in the same group are highly correlated; dampen when + # multiple group members agree (same sign) to avoid overconfidence. + CORRELATION_GROUPS: list[list[str]] = [ + ["double_bb_rsi", "stoch_rsi", "mfi"], # Oscillator group + ["macd_crossover", "supertrend", "ichimoku"], # Trend group + ["volume_breakout", "obv"], # Volume group + ["divergence", "smc", "fvg", "candlestick"], # Pattern group + ] + for group in CORRELATION_GROUPS: + active = [(s, raw_scores[s]) for s in group if raw_scores[s] != 0.0] + if len(active) >= 2: + signs = [1 if v > 0 else -1 for _, v in active] + pos_count = sum(1 for s in signs if s > 0) + neg_count = sum(1 for s in signs if s < 0) + # Dampen: scale each strategy's score by 1/sqrt(count) + if pos_count >= 2: + dampen = 1.0 / (pos_count ** 0.5) + for strat, val in active: + if val > 0: + raw_scores[strat] = val * dampen + if neg_count >= 2: + dampen = 1.0 / (neg_count ** 0.5) + for strat, val in active: + if val < 0: + raw_scores[strat] = val * dampen + + # ── Apply win-rate boosting ── + boosted_scores: dict[str, float] = {} + for strategy, raw_score in raw_scores.items(): + boosted_scores[strategy] = boost_score(raw_score, strategy, rates) + + total_score = sum(boosted_scores.values()) + confidence = compute_booster_confidence(raw_scores, rates) + + # ── Dynamic threshold normalization ── + # Normalize total_score by sqrt(active_strategies) so that + # 3 strategies voting STRONG ≈ 10 strategies voting weak + active_count = sum(1 for v in boosted_scores.values() if v != 0.0) + if active_count > 1: + norm_factor = max(active_count ** 0.5, 1.0) + adjusted_score = total_score / norm_factor + else: + adjusted_score = total_score + + # ── Final classification from boosted score ── + # 🔧 Dynamic thresholds: STRONG needs effective 4.0, BUY/SELL needs 1.0 + if adjusted_score >= 4.0: + return STRONG_BUY, "STRONG", confidence, raw_scores + elif adjusted_score >= 1.0: + return BUY, "MODERATE", confidence, raw_scores + elif adjusted_score <= -4.0: + return STRONG_SELL, "STRONG", confidence, raw_scores + elif adjusted_score <= -1.0: + return SELL, "MODERATE", confidence, raw_scores + + return None, None, confidence, raw_scores + + +def _calculate_pnl( + entry_price: Decimal, + exit_price: Decimal, + direction: str, + quantity: Decimal, +) -> tuple[Decimal, Decimal]: + """Calculate absolute and percentage P&L.""" + if entry_price == 0: + return Decimal("0"), Decimal("0") + + # P1-15: Guard against divide-by-zero when quantity is 0 + if quantity is None or quantity <= 0: + return Decimal("0"), Decimal("0") + + if direction == "LONG": + pnl = (exit_price - entry_price) * quantity + else: + pnl = (entry_price - exit_price) * quantity + + pnl_percent = (pnl / (entry_price * quantity)) * Decimal("100") + return pnl, pnl_percent diff --git a/backend/app/services/signal_service.py b/backend/app/services/signal_service.py index d55adc4..2a80c9c 100755 --- a/backend/app/services/signal_service.py +++ b/backend/app/services/signal_service.py @@ -9,7 +9,6 @@ from __future__ import annotations import json import logging -import math import time as _time from datetime import datetime, timedelta, timezone from decimal import Decimal @@ -29,12 +28,7 @@ from app.models.symbol import Symbol from app.models.user import User from app.services.candle_service import get_indicators from app.services.indicator_service import atr as compute_atr -from app.services.signal_booster import ( - boost_score, - compute_strategy_win_rates, - get_cached_rates, - get_confidence as compute_booster_confidence, -) +from app.services.signal_booster import compute_strategy_win_rates from app.schemas.signal import SignalResponse, TradeResponse from app.core.security import decrypt_api_key from app.exchange.factory import factory as exchange_factory @@ -83,530 +77,33 @@ async def _get_cached_enabled_strategies(db: AsyncSession) -> list[str] | None: return _ENABLED_STRATEGIES_CACHE # --------------------------------------------------------------------------- -# Signal type constants +# Signal scoring — extracted to signal_scoring.py (pure, no I/O). Re-exported +# here so existing call sites/imports of `app.services.signal_service` +# continue to work unchanged. # --------------------------------------------------------------------------- -STRONG_BUY = "STRONG_BUY" -BUY = "BUY" -STRONG_SELL = "STRONG_SELL" -SELL = "SELL" -CAUTION_LONG = "CAUTION_LONG" -CAUTION_SHORT = "CAUTION_SHORT" -SQUEEZE_ALERT = "SQUEEZE_ALERT" +from app.services.signal_scoring import ( # noqa: E402 + BUY, + CAUTION_LONG, + CAUTION_SHORT, + MIN_BB_DISTANCE_PCT, + SELL, + SQUEEZE_ALERT, + STRONG_BUY, + STRONG_SELL, + _calculate_pnl, + _classify_signal_bb, + _classify_signal_combined, + _detect_squeeze, + _get_bb_values, + _get_rsi_values, + _get_sma_values, +) # P2-10: Cross-timeframe cooldown cache to prevent duplicate signals # Key: f"{symbol}:{exchange}:{timeframe}" → last signal timestamp _signal_cooldown: dict[str, float] = {} _COOLDOWN_SECONDS = 300 # 5 minutes between same-direction signals -# --------------------------------------------------------------------------- -# Signal detection configuration -# --------------------------------------------------------------------------- -# Minimum distance from BB bounds to filter noise -MIN_BB_DISTANCE_PCT = Decimal("0.001") # 0.1% - - -def _get_bb_values(indicators: dict) -> dict[str, list[float]] | None: - """Extract Bollinger Band values from indicators dict.""" - bb = indicators.get("bollinger_bands") - if not bb: - return None - # Ensure all required keys exist - required = ["upper", "middle", "lower"] - if not all(k in bb for k in required): - return None - return bb - - -def _get_rsi_values(indicators: dict) -> list[float] | None: - """Extract RSI values.""" - rsi = indicators.get("rsi_14") - if not rsi or not isinstance(rsi, list) or len(rsi) == 0: - return None - return rsi - - -def _get_sma_values(indicators: dict) -> list[float] | None: - """Extract SMA values.""" - sma = indicators.get("sma_20") - if not sma or not isinstance(sma, list) or len(sma) == 0: - return None - return sma - - -def _detect_squeeze( - bb: dict[str, list[float]], - lookback: int = 10, -) -> bool: - """Detect Bollinger Band squeeze — narrowing of the band width. - - A squeeze occurs when the current band width is at the lower end - of the recent range, indicating low volatility before a breakout. - """ - upper = bb.get("upper", []) - lower = bb.get("lower", []) - if not upper or not lower or len(upper) < lookback or len(lower) < lookback: - return False - - # Band width = upper - lower - widths = [] - for i in range(max(0, len(upper) - lookback), len(upper)): - try: - widths.append(upper[i] - lower[i]) - except TypeError: - return False - if len(widths) < 3: - return False - - current_width = widths[-1] - # Check if upper_1 and lower_1 exist for squeeze detection - upper_1 = bb.get("upper_1", []) - lower_1 = bb.get("lower_1", []) - if upper_1 and lower_1 and len(upper_1) >= 1 and len(lower_1) >= 1: - try: - inner_width = upper_1[-1] - lower_1[-1] - outer_width = upper[-1] - lower[-1] - # Squeeze when inner bands are VERY narrow relative to outer (≤20%) - if outer_width > 0 and inner_width / outer_width < 0.2: - return True - except TypeError: - pass - - # Alternative: check if current width is at absolute minimum of longer lookback - min_width = min(widths) - if min_width > 0 and current_width <= min_width * 1.01: - return True - - return False - - -def _classify_signal_bb( - close_price: float, - bb: dict[str, list[float]], - rsi: list[float] | None, - sma: list[float] | None, -) -> tuple[Optional[str], Optional[str]]: - """Classify signal using Double BB + RSI only. - - Returns (signal_type, strength) or (None, None) if no signal. - Used as one vote in the combined 4-algorithm system. - """ - if len(bb.get("upper", [])) == 0 or len(bb.get("lower", [])) == 0: - return None, None - - upper_2 = bb["upper"][-1] - lower_2 = bb["lower"][-1] - upper_1 = bb.get("upper_1", [None])[-1] - lower_1 = bb.get("lower_1", [None])[-1] - middle = bb.get("middle", [None])[-1] - current_rsi = rsi[-1] if rsi and len(rsi) > 0 else 50 - current_sma = sma[-1] if sma and len(sma) > 0 else None - - # --- Price above BB 1σ Upper → Trend Long --- - if upper_1 is not None and close_price > upper_1: - if close_price > upper_2: - if current_rsi > 75: - return CAUTION_SHORT, "MODERATE" - # Only STRONG_BUY if RSI is firmly bullish AND SMA is above middle - if current_rsi > 60 and current_sma and middle and current_sma > middle: - return STRONG_BUY, "STRONG" - return BUY, "MODERATE" - else: - if current_rsi > 55: - return BUY, "MODERATE" - return BUY, "WEAK" - - # --- Price below BB 1σ Lower → Trend Short --- - if lower_1 is not None and close_price < lower_1: - if close_price < lower_2: - if current_rsi < 25: - return CAUTION_LONG, "MODERATE" - # Only STRONG_SELL if RSI is firmly bearish AND SMA is below middle - if current_rsi < 40 and current_sma and middle and current_sma < middle: - return STRONG_SELL, "STRONG" - return SELL, "MODERATE" - else: - if current_rsi < 45: - return SELL, "MODERATE" - return SELL, "WEAK" - - return None, None - - -def _classify_signal_combined( - close_price: float, - bb: dict[str, list[float]], - rsi: list[float] | None, - sma: list[float] | None, - macd_data: dict | None, - st_data: dict | None, - vol_data: list | None, - ichi_data: dict | None = None, - rsi_div: tuple = (None, None), - macd_div: tuple = (None, None), - smc_data: dict | None = None, - mtf_votes: list[tuple[Optional[str], Optional[str], float]] | None = None, - obv_data: list | None = None, - stoch_rsi_data: dict | None = None, - mfi_data: list | None = None, - fvg_data: dict | None = None, - candlestick_score: float | None = None, - rates: dict[str, float] | None = None, - enabled_strategies: list[str] | None = None, -) -> tuple[Optional[str], Optional[str], float, dict[str, float]]: - """Classify market state using 13-algorithm voting with win-rate boosting. - - Algorithms: - 1. Double BB + RSI - 2. MACD Crossover - 3. SuperTrend - 4. Volume Breakout - 5. Ichimoku Cloud - 6. Divergence Detection (RSI + MACD) - 7. 🌤️ Market Structure (SMC) — BOS, CHoCH, OB - 8. 🔄 Multi-Timeframe (15m + 1h + 4h) - 9. 📊 OBV (On-Balance Volume) Crossover - 10. 🔄 Stochastic RSI Crossover - 11. 💰 MFI (Money Flow Index) - 12. 🕯️ FVG (Fair Value Gap) - 13. 🕯️ Candlestick Patterns (30+ patterns) - - Each algorithm votes: BUY (+1/+2), SELL (-1/-2), or NEUTRAL (0). - If *rates* is provided, each strategy's raw score is boosted by its - historical win rate before the final classification. - - Returns (signal_type, strength, confidence, raw_scores) where - confidence is a 0-1 float and raw_scores is a dict of all 9 - algorithm scores for ML feature collection. - """ - # ── NaN/Inf guard: reject any invalid price before processing ── - if not math.isfinite(close_price) or close_price <= 0: - logger.warning("_classify_signal_combined: invalid close_price=%s, returning NEUTRAL", close_price) - return None, None, 0.0, {} - - # ── Special signals (override) ── - squeeze = _detect_squeeze(bb) - if squeeze: - return SQUEEZE_ALERT, "MODERATE", 0.5, {} - - # P2-2: Call _classify_signal_bb ONCE, reuse result for both - # early-return check AND the raw_scores vote - bb_type, bb_strength = _classify_signal_bb(close_price, bb, rsi, sma) - if bb_type in (CAUTION_LONG, CAUTION_SHORT): - return bb_type, "MODERATE", 0.5, {} - - # ── Collect per-strategy raw scores ── - raw_scores: dict[str, float] = { - "double_bb_rsi": 0.0, - "macd_crossover": 0.0, - "supertrend": 0.0, - "volume_breakout": 0.0, - "ichimoku": 0.0, - "divergence": 0.0, - "smc": 0.0, - "mtf": 0.0, - "obv": 0.0, - "stoch_rsi": 0.0, - "mfi": 0.0, - "fvg": 0.0, - "candlestick": 0.0, - } - - # 1. BB + RSI vote (reuse bb_type/bb_strength from above — P2-2) - if bb_type == STRONG_BUY: - raw_scores["double_bb_rsi"] = 2.0 - elif bb_type == BUY: - raw_scores["double_bb_rsi"] = 1.0 - elif bb_type == STRONG_SELL: - raw_scores["double_bb_rsi"] = -2.0 - elif bb_type == SELL: - raw_scores["double_bb_rsi"] = -1.0 - - # 2. MACD Crossover vote - if macd_data: - macd_line = macd_data.get("macd_line", []) - signal_line = macd_data.get("signal_line", []) - if len(macd_line) >= 2 and len(signal_line) >= 2: - m_curr = macd_line[-1] - m_prev = macd_line[-2] - s_curr = signal_line[-1] - s_prev = signal_line[-2] - if m_curr is not None and s_curr is not None and m_prev is not None and s_prev is not None: - if m_prev <= s_prev and m_curr > s_curr and m_curr > 0: - raw_scores["macd_crossover"] = 1.0 - elif m_prev >= s_prev and m_curr < s_curr and m_curr < 0: - raw_scores["macd_crossover"] = -1.0 - - # 3. SuperTrend vote - if st_data: - trend = st_data.get("trend", []) - if trend and len(trend) >= 1 and trend[-1] is not None: - raw_scores["supertrend"] = 1.0 if trend[-1] else -1.0 - - # 4. Volume Breakout vote (current bar only — avoid stale signals) - if vol_data and len(vol_data) >= 2: - vol_now = vol_data[-1] - if vol_now is True: - sma20 = sma[-1] if sma and len(sma) > 0 else None - if sma20 is not None and close_price > sma20: - raw_scores["volume_breakout"] = 1.0 - elif sma20 is not None and close_price < sma20: - raw_scores["volume_breakout"] = -1.0 - - # 4b. OBV (On-Balance Volume) Crossover vote - if obv_data and len(obv_data) >= 2: - obv_now = obv_data[-1] - obv_prev = obv_data[-2] - if obv_now is True: - raw_scores["obv"] = 1.0 - elif obv_now is False: - raw_scores["obv"] = -1.0 - elif obv_prev is True: - # One bar ago still counts for momentum - raw_scores["obv"] = 0.5 - elif obv_prev is False: - raw_scores["obv"] = -0.5 - - # 10. Stochastic RSI Crossover vote - if stoch_rsi_data: - k_line = stoch_rsi_data.get("k", []) - d_line = stoch_rsi_data.get("d", []) - if len(k_line) >= 2 and len(d_line) >= 2: - k_curr = k_line[-1] - k_prev = k_line[-2] - d_curr = d_line[-1] - if k_curr is not None and d_curr is not None and k_prev is not None: - # Oversold: %K < 20 and crossing above %D → BUY - if k_curr < 20 and k_prev <= d_curr and k_curr > d_curr: - raw_scores["stoch_rsi"] = 1.5 - # Overbought: %K > 80 and crossing below %D → SELL - elif k_curr > 80 and k_prev >= d_curr and k_curr < d_curr: - raw_scores["stoch_rsi"] = -1.5 - - # 11. MFI (Money Flow Index) vote - if mfi_data and len(mfi_data) >= 2: - mfi_curr = mfi_data[-1] - mfi_prev = mfi_data[-2] - if mfi_curr is not None and mfi_prev is not None: - # Oversold: MFI < 20 and turning up → BUY - if mfi_curr < 20 and mfi_curr > mfi_prev: - raw_scores["mfi"] = 1.5 - # Overbought: MFI > 80 and turning down → SELL - elif mfi_curr > 80 and mfi_curr < mfi_prev: - raw_scores["mfi"] = -1.5 - # Neutral divergence: MFI trending in opposite direction to close - elif mfi_curr < 30: - raw_scores["mfi"] = 0.5 - elif mfi_curr > 70: - raw_scores["mfi"] = -0.5 - - # 12. FVG (Fair Value Gap) vote — only when price is near the gap - if fvg_data: - fvg_type = fvg_data.get("type") - fvg_high = fvg_data.get("gap_high") - fvg_low = fvg_data.get("gap_low") - if fvg_type == "BULLISH" and fvg_high is not None and fvg_low is not None: - # Price near or inside bullish FVG → support → BUY - gap_mid = (fvg_high + fvg_low) / 2.0 - dist_pct = abs(close_price - gap_mid) / close_price * 100 - if dist_pct < 1.0: - raw_scores["fvg"] = 2.0 - elif dist_pct < 2.0: - raw_scores["fvg"] = 1.0 - # else: too far from gap → no vote (prevents noise) - elif fvg_type == "BEARISH" and fvg_high is not None and fvg_low is not None: - # Price near or inside bearish FVG → resistance → SELL - gap_mid = (fvg_high + fvg_low) / 2.0 - dist_pct = abs(close_price - gap_mid) / close_price * 100 - if dist_pct < 1.0: - raw_scores["fvg"] = -2.0 - elif dist_pct < 2.0: - raw_scores["fvg"] = -1.0 - # else: too far from gap → no vote - - # 13. Candlestick Patterns vote - if candlestick_score is not None and candlestick_score != 0.0: - raw_scores["candlestick"] = candlestick_score - - # 5. Ichimoku Cloud vote - if ichi_data: - tenkan = ichi_data.get("tenkan", []) - kijun = ichi_data.get("kijun", []) - senkou_a = ichi_data.get("senkou_a", []) - senkou_b = ichi_data.get("senkou_b", []) - if (tenkan and len(tenkan) >= 2 and kijun and len(kijun) >= 2 - and senkou_a and len(senkou_a) >= 2 and senkou_b and len(senkou_b) >= 2): - t_now = tenkan[-1] - k_now = kijun[-1] - t_prev = tenkan[-2] - k_prev = kijun[-2] - sa_now = senkou_a[-1] - sb_now = senkou_b[-1] - ichi_score = 0.0 - if t_now is not None and k_now is not None and t_prev is not None and k_prev is not None: - if t_prev <= k_prev and t_now > k_now: - ichi_score += 1.5 - elif t_prev >= k_prev and t_now < k_now: - ichi_score -= 1.5 - if sa_now is not None and sb_now is not None: - if sa_now > sb_now and close_price > sa_now: - ichi_score += 1.0 - elif sa_now < sb_now and close_price < sb_now: - ichi_score -= 1.0 - raw_scores["ichimoku"] = ichi_score - - # 6. Divergence Detection vote - div_score = 0.0 - for div_type, div_strength in [rsi_div, macd_div]: - if div_type == "BULLISH": - div_score += 2.0 if div_strength == "STRONG" else 1.0 - elif div_type == "BEARISH": - div_score -= 2.0 if div_strength == "STRONG" else 1.0 - raw_scores["divergence"] = div_score - - # 7. 🌤️ Market Structure (SMC) vote - smc_score = 0.0 - if smc_data: - bos = smc_data.get("bos") - choch = smc_data.get("choch") - trend = smc_data.get("trend", "NEUTRAL") - obs = smc_data.get("order_blocks", []) - - if bos == "BULLISH": - smc_score += 1.5 - elif bos == "BEARISH": - smc_score -= 1.5 - - if choch == "BULLISH": - smc_score += 1.5 - elif choch == "BEARISH": - smc_score -= 1.5 - - if trend == "BULLISH": - smc_score += 1.0 - elif trend == "BEARISH": - smc_score -= 1.0 - - if obs: - latest_close = close_price - bullish_obs = [ob for ob in obs if ob.get("type") == "BULLISH"] - bearish_obs = [ob for ob in obs if ob.get("type") == "BEARISH"] - - for ob in bullish_obs: - ob_high = ob.get("price_high", 0) - if 0 < ob_high and ob_high * 0.995 <= latest_close <= ob_high * 1.005: - smc_score += 1.0 - break - - for ob in bearish_obs: - ob_low = ob.get("price_low", 0) - if 0 < ob_low and ob_low * 0.995 <= latest_close <= ob_low * 1.005: - smc_score -= 1.0 - break - raw_scores["smc"] = smc_score - - # 8. 🔄 Multi-Timeframe vote (weighted) - mtf_score = 0.0 - if mtf_votes: - for sig_type, sig_strength, weight in mtf_votes: - if sig_type == STRONG_BUY: - mtf_score += 2.0 * weight - elif sig_type == BUY: - mtf_score += 1.0 * weight - elif sig_type == STRONG_SELL: - mtf_score -= 2.0 * weight - elif sig_type == SELL: - mtf_score -= 1.0 * weight - raw_scores["mtf"] = mtf_score - - # ── Apply enabled_strategies filter (zero-out disabled strategies) ── - if enabled_strategies is not None: - disabled = [s for s in raw_scores if s not in enabled_strategies] - if disabled: - logger.debug("Disabled strategies: %s", disabled) - for s in disabled: - raw_scores[s] = 0.0 - - # ── P1-16: Correlation dampening ── - # Strategies in the same group are highly correlated; dampen when - # multiple group members agree (same sign) to avoid overconfidence. - CORRELATION_GROUPS: list[list[str]] = [ - ["double_bb_rsi", "stoch_rsi", "mfi"], # Oscillator group - ["macd_crossover", "supertrend", "ichimoku"], # Trend group - ["volume_breakout", "obv"], # Volume group - ["divergence", "smc", "fvg", "candlestick"], # Pattern group - ] - for group in CORRELATION_GROUPS: - active = [(s, raw_scores[s]) for s in group if raw_scores[s] != 0.0] - if len(active) >= 2: - signs = [1 if v > 0 else -1 for _, v in active] - pos_count = sum(1 for s in signs if s > 0) - neg_count = sum(1 for s in signs if s < 0) - # Dampen: scale each strategy's score by 1/sqrt(count) - if pos_count >= 2: - dampen = 1.0 / (pos_count ** 0.5) - for strat, val in active: - if val > 0: - raw_scores[strat] = val * dampen - if neg_count >= 2: - dampen = 1.0 / (neg_count ** 0.5) - for strat, val in active: - if val < 0: - raw_scores[strat] = val * dampen - - # ── Apply win-rate boosting ── - boosted_scores: dict[str, float] = {} - for strategy, raw_score in raw_scores.items(): - boosted_scores[strategy] = boost_score(raw_score, strategy, rates) - - total_score = sum(boosted_scores.values()) - confidence = compute_booster_confidence(raw_scores, rates) - - # ── Dynamic threshold normalization ── - # Normalize total_score by sqrt(active_strategies) so that - # 3 strategies voting STRONG ≈ 10 strategies voting weak - active_count = sum(1 for v in boosted_scores.values() if v != 0.0) - if active_count > 1: - norm_factor = max(active_count ** 0.5, 1.0) - adjusted_score = total_score / norm_factor - else: - adjusted_score = total_score - - # ── Final classification from boosted score ── - # 🔧 Dynamic thresholds: STRONG needs effective 4.0, BUY/SELL needs 1.0 - if adjusted_score >= 4.0: - return STRONG_BUY, "STRONG", confidence, raw_scores - elif adjusted_score >= 1.0: - return BUY, "MODERATE", confidence, raw_scores - elif adjusted_score <= -4.0: - return STRONG_SELL, "STRONG", confidence, raw_scores - elif adjusted_score <= -1.0: - return SELL, "MODERATE", confidence, raw_scores - - return None, None, confidence, raw_scores - - -def _calculate_pnl( - entry_price: Decimal, - exit_price: Decimal, - direction: str, - quantity: Decimal, -) -> tuple[Decimal, Decimal]: - """Calculate absolute and percentage P&L.""" - if entry_price == 0: - return Decimal("0"), Decimal("0") - - # P1-15: Guard against divide-by-zero when quantity is 0 - if quantity is None or quantity <= 0: - return Decimal("0"), Decimal("0") - - if direction == "LONG": - pnl = (exit_price - entry_price) * quantity - else: - pnl = (entry_price - exit_price) * quantity - - pnl_percent = (pnl / (entry_price * quantity)) * Decimal("100") - return pnl, pnl_percent - # ========================================================================= # Public API @@ -1319,8 +816,8 @@ async def _manage_trades( try: from app.services.risk_manager import DynamicKellySizer from app.services.signal_booster import get_cached_rates, get_pnl_stats - rates = get_cached_rates() - pnl_stats = get_pnl_stats() # real avg_win / avg_loss from DB + rates = await get_cached_rates() + pnl_stats = await get_pnl_stats() # real avg_win / avg_loss from DB kelly = DynamicKellySizer() # Estimate win rate and avg win/loss from cached booster rates overall_rate = rates.get("__all__", 0.5) diff --git a/backend/app/services/trade_executor.py b/backend/app/services/trade_executor.py index 416b60a..305f0c0 100644 --- a/backend/app/services/trade_executor.py +++ b/backend/app/services/trade_executor.py @@ -266,8 +266,8 @@ async def execute_signal_trade( try: from app.services.risk_manager import DynamicKellySizer from app.services.signal_booster import get_cached_rates, get_pnl_stats - rates = get_cached_rates() - pnl_stats = get_pnl_stats() + rates = await get_cached_rates() + pnl_stats = await get_pnl_stats() kelly = DynamicKellySizer() overall_rate = rates.get("__all__", 0.5) dir_rate = rates.get(f"__all___{signal_direction}", overall_rate) diff --git a/backend/requirements.txt b/backend/requirements.txt index c69a521..4ca60d3 100755 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -11,6 +11,7 @@ bcrypt==4.0.1 ccxt>=4.3 apscheduler==3.10.4 cachetools==5.5.0 +redis==5.0.8 structlog==24.4.0 httpx==0.27.0 websockets==13.0 diff --git a/backend/tests/test_config_secrets.py b/backend/tests/test_config_secrets.py new file mode 100644 index 0000000..462373a --- /dev/null +++ b/backend/tests/test_config_secrets.py @@ -0,0 +1,38 @@ +"""Tests for fix (o): DB_PASSWORD_FILE / ENCRYPTION_KEY_FILE support in +app/config.py, so DB password and encryption key can be sourced from Docker +secrets files (consistent with how JWT keys are already handled) instead of +plain env vars, while staying backward-compatible when the _FILE variants +are not set. +""" +from __future__ import annotations + +from app.config import Settings + + +def test_db_password_file_overrides_database_url_password(tmp_path): + pw_file = tmp_path / "db_password.txt" + pw_file.write_text("s3cr3t-from-file\n") + + settings = Settings( + DATABASE_URL="postgresql+asyncpg://trading:placeholder@db:5432/trading_portal", + DB_PASSWORD_FILE=str(pw_file), + ) + assert "s3cr3t-from-file" in settings.DATABASE_URL + assert "placeholder" not in settings.DATABASE_URL + + +def test_encryption_key_file_overrides_encryption_key(tmp_path): + key_file = tmp_path / "encryption_key.txt" + key_file.write_text("00" * 32 + "\n") + + settings = Settings(ENCRYPTION_KEY="", ENCRYPTION_KEY_FILE=str(key_file)) + assert settings.ENCRYPTION_KEY == "00" * 32 + + +def test_without_file_variants_plain_env_values_are_unchanged(): + settings = Settings( + DATABASE_URL="postgresql+asyncpg://trading:plain@db:5432/trading_portal", + ENCRYPTION_KEY="plain-key", + ) + assert settings.DATABASE_URL == "postgresql+asyncpg://trading:plain@db:5432/trading_portal" + assert settings.ENCRYPTION_KEY == "plain-key" diff --git a/backend/tests/test_indicator_service.py b/backend/tests/test_indicator_service.py index f24e86f..bd77b24 100644 --- a/backend/tests/test_indicator_service.py +++ b/backend/tests/test_indicator_service.py @@ -66,18 +66,15 @@ class TestRsi: result = rsi(prices, period=14) assert result[14] == pytest.approx(0.0) - def test_flat_prices_do_not_yield_neutral_50(self): - """Documents a discovered quirk (not fixed here — flagged for the - team to decide on): when there's truly zero price movement, the - code sets an internal `rs = 50.0` sentinel intending "neutral", but - that value is still run through the RSI formula - (100 - 100/(1+rs)), which maps rs=50 to RSI≈98.04, not the - conventionally-expected neutral RSI of 50. A perfectly flat run - (e.g. an illiquid pair or stablecoin) would misreport as - near-overbought instead of neutral.""" + def test_flat_prices_yield_neutral_50(self): + """Fix (q): zero price movement must report neutral RSI=50, not + ~98.04. Previously the code set an internal `rs = 50.0` sentinel + intending "neutral" but still ran it through the RSI formula + (100 - 100/(1+rs)), which maps rs=50 to RSI≈98.04 — misreporting a + perfectly flat run (illiquid pair, stablecoin) as near-overbought.""" prices = [10.0] * 16 result = rsi(prices, period=14) - assert result[14] == pytest.approx(100.0 - 100.0 / 51.0) + assert result[14] == pytest.approx(50.0) def test_insufficient_data_returns_all_none(self): result = rsi([1, 2, 3], period=14) @@ -191,27 +188,19 @@ class TestMfi: def test_overbought_when_no_negative_flow(self): # typical price strictly increasing -> every period contributes only # positive flow -> neg_flow == 0 -> MFI defined as 100.0. - # Index 4 (not 3) is asserted because index 3 is the very first - # computed value and hits the negative-indexing quirk below. candles = [candle(10 + i, 10 + i, 10 + i, volume=100) for i in range(5)] result = mfi(candles, period=3) assert result[4] == pytest.approx(100.0) - def test_first_computed_value_has_a_wraparound_indexing_quirk(self): - """Documents a discovered quirk (not fixed here — flagged for the - team to decide on): for the first computed MFI value in a series, - the loop compares `typical_prices[j-1]` with `j=0`, which in Python - wraps around to `typical_prices[-1]` (the LAST candle in the whole - series) instead of having no prior candle to compare against. This - spuriously injects one bogus flow-direction comparison. In practice - this only taints the single oldest computed value in a long series - (never the latest, which is what signal_service.py actually reads), - so real-world impact is negligible — but it is objectively wrong.""" + def test_first_computed_value_no_longer_wraps_around(self): + """Fix (r): the first computed MFI value in a series must not + compare typical_prices[0] against typical_prices[-1] (the LAST + candle) via Python's negative-index wraparound. With strictly + increasing prices and no real negative flow, the first computed + value must also be 100.0, same as later ones.""" candles = [candle(10 + i, 10 + i, 10 + i, volume=100) for i in range(5)] result = mfi(candles, period=3) - # Without the quirk this would also be 100.0 (strictly increasing, - # no real negative flow) — the quirk drags it down to ~69.7. - assert result[3] == pytest.approx(69.69696969696969) + assert result[3] == pytest.approx(100.0) class TestDetectMarketRegime: diff --git a/backend/tests/test_redis_client.py b/backend/tests/test_redis_client.py new file mode 100644 index 0000000..a11b47e --- /dev/null +++ b/backend/tests/test_redis_client.py @@ -0,0 +1,56 @@ +"""Tests for app/core/redis_client.py — the shared cache helper used to sync +state (win rates, PnL stats) between the backend-api and backend-scheduler +processes. Every path here must degrade gracefully when Redis is down, +since it's an optional cache, not a hard dependency. +""" +from __future__ import annotations + +from app.core import redis_client + + +class FakeRedis: + def __init__(self): + self.store: dict[str, str] = {} + + async def get(self, key): + return self.store.get(key) + + async def set(self, key, value, ex=None): + self.store[key] = value + + +class BrokenRedis: + async def get(self, key): + raise ConnectionError("redis unreachable") + + async def set(self, key, value, ex=None): + raise ConnectionError("redis unreachable") + + +async def test_set_then_get_json_roundtrip(monkeypatch): + fake = FakeRedis() + monkeypatch.setattr(redis_client, "_get_client", lambda: fake) + + await redis_client.set_json("k", {"a": 1}, ttl_seconds=60) + assert await redis_client.get_json("k") == {"a": 1} + + +async def test_get_json_returns_none_when_key_missing(monkeypatch): + monkeypatch.setattr(redis_client, "_get_client", lambda: FakeRedis()) + assert await redis_client.get_json("missing") is None + + +async def test_get_json_returns_none_when_client_unavailable(monkeypatch): + monkeypatch.setattr(redis_client, "_get_client", lambda: None) + assert await redis_client.get_json("k") is None + + +async def test_get_json_does_not_raise_when_redis_errors(monkeypatch): + monkeypatch.setattr(redis_client, "_get_client", lambda: BrokenRedis()) + assert await redis_client.get_json("k") is None + + +async def test_set_json_does_not_raise_when_redis_errors(monkeypatch): + monkeypatch.setattr(redis_client, "_get_client", lambda: BrokenRedis()) + # Must not raise -- callers treat cache writes as best-effort. + await redis_client.set_json("k", {"a": 1}, ttl_seconds=60) diff --git a/backend/tests/test_signal_booster_cache.py b/backend/tests/test_signal_booster_cache.py new file mode 100644 index 0000000..1996b7c --- /dev/null +++ b/backend/tests/test_signal_booster_cache.py @@ -0,0 +1,38 @@ +"""Tests for fix (l): signal_booster's win-rate/PnL caches read from Redis +first (shared across the backend-api and backend-scheduler processes), +falling back to this process's own in-memory cache when Redis has no data +or is unavailable. +""" +from __future__ import annotations + +from unittest.mock import AsyncMock + +from app.services import signal_booster + + +async def test_get_cached_rates_prefers_redis(monkeypatch): + monkeypatch.setattr(signal_booster.redis_client, "get_json", AsyncMock(return_value={"__all__": 0.7})) + monkeypatch.setattr(signal_booster, "_win_rate_cache", {"__all__": 0.1}) + + assert await signal_booster.get_cached_rates() == {"__all__": 0.7} + + +async def test_get_cached_rates_falls_back_to_in_memory_when_redis_empty(monkeypatch): + monkeypatch.setattr(signal_booster.redis_client, "get_json", AsyncMock(return_value=None)) + monkeypatch.setattr(signal_booster, "_win_rate_cache", {"__all__": 0.42}) + + assert await signal_booster.get_cached_rates() == {"__all__": 0.42} + + +async def test_get_pnl_stats_prefers_redis(monkeypatch): + monkeypatch.setattr(signal_booster.redis_client, "get_json", AsyncMock(return_value={"avg_win": 5.0, "avg_loss": 1.0})) + + assert await signal_booster.get_pnl_stats() == {"avg_win": 5.0, "avg_loss": 1.0} + + +async def test_get_pnl_stats_falls_back_to_defaults_when_nothing_cached(monkeypatch): + monkeypatch.setattr(signal_booster.redis_client, "get_json", AsyncMock(return_value=None)) + monkeypatch.setattr(signal_booster, "_pnl_stats_cache", {}) + monkeypatch.setattr(signal_booster, "_last_pnl_cache_update", 0.0) + + assert await signal_booster.get_pnl_stats() == {"avg_win": 3.0, "avg_loss": 2.0} diff --git a/backend/tests/test_signal_service_scoring.py b/backend/tests/test_signal_scoring.py similarity index 98% rename from backend/tests/test_signal_service_scoring.py rename to backend/tests/test_signal_scoring.py index a295e4e..d45ae11 100644 --- a/backend/tests/test_signal_service_scoring.py +++ b/backend/tests/test_signal_scoring.py @@ -1,5 +1,5 @@ """Tests for the pure scoring/classification helpers in -app/services/signal_service.py — the 13-algorithm voting core that decides +app/services/signal_scoring.py — the 13-algorithm voting core that decides BUY/SELL/STRONG signals. These functions take plain indicator dicts/lists and return classifications; no DB or network I/O involved. """ @@ -8,7 +8,7 @@ from __future__ import annotations import math from decimal import Decimal -from app.services.signal_service import ( +from app.services.signal_scoring import ( BUY, CAUTION_LONG, CAUTION_SHORT, diff --git a/docker-compose.yml b/docker-compose.yml index 152e77a..f4f01dc 100755 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -8,9 +8,10 @@ services: environment: POSTGRES_DB: trading_portal POSTGRES_USER: trading - POSTGRES_PASSWORD: ${DB_PASSWORD:-trading_secret} + POSTGRES_PASSWORD_FILE: /run/secrets/db_password.txt volumes: - pgdata:/var/lib/postgresql/data + - /opt/data/trading-portal/secrets:/run/secrets:ro ports: - "127.0.0.1:5432:5432" restart: unless-stopped @@ -28,6 +29,26 @@ services: networks: - trading-net + # ───────────────── Redis (shared cache: win-rate/PnL stats across api+scheduler) ───────────────── + redis: + image: redis:7-alpine + container_name: trading-redis + command: ["redis-server", "--maxmemory", "128mb", "--maxmemory-policy", "allkeys-lru"] + restart: unless-stopped + healthcheck: + test: ["CMD", "redis-cli", "ping"] + interval: 10s + timeout: 5s + retries: 5 + start_period: 10s + deploy: + resources: + limits: + memory: 192M + cpus: "0.5" + networks: + - trading-net + # ───────────────── FastAPI Backend (user-facing, no scheduler) ───────────────── backend-api: build: @@ -36,10 +57,14 @@ services: depends_on: db: condition: service_healthy + redis: + condition: service_healthy environment: - DATABASE_URL: postgresql+asyncpg://trading:${DB_PASSWORD:-trading_secret}@db:5432/trading_portal + DATABASE_URL: postgresql+asyncpg://trading:unused@db:5432/trading_portal + DB_PASSWORD_FILE: /run/secrets/db_password.txt PORT: 8001 - ENCRYPTION_KEY: ${ENCRYPTION_KEY} + ENCRYPTION_KEY_FILE: /run/secrets/encryption_key.txt + REDIS_URL: redis://redis:6379/0 CORS_ORIGINS: http://localhost,http://localhost:5173,http://localhost:3000 LOG_LEVEL: INFO JWT_PRIVATE_KEY_PATH: /run/secrets/jwt_private.pem @@ -75,9 +100,13 @@ services: depends_on: db: condition: service_healthy + redis: + condition: service_healthy environment: - DATABASE_URL: postgresql+asyncpg://trading:${DB_PASSWORD:-trading_secret}@db:5432/trading_portal - ENCRYPTION_KEY: ${ENCRYPTION_KEY} + DATABASE_URL: postgresql+asyncpg://trading:unused@db:5432/trading_portal + DB_PASSWORD_FILE: /run/secrets/db_password.txt + ENCRYPTION_KEY_FILE: /run/secrets/encryption_key.txt + REDIS_URL: redis://redis:6379/0 LOG_LEVEL: INFO JWT_PRIVATE_KEY_PATH: /run/secrets/jwt_private.pem JWT_PUBLIC_KEYS_DIR: /run/secrets/jwt_public_keys diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 53ad090..2cc1253 100755 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -18,11 +18,13 @@ "vite-plugin-pwa": "^1.3.0" }, "devDependencies": { + "@tailwindcss/vite": "^4.3.2", "@types/node": "^24.13.2", "@types/react": "^19.2.17", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^6.0.2", "oxlint": "^1.69.0", + "tailwindcss": "^4.3.2", "typescript": "~6.0.2", "vite": "^8.1.0" } @@ -2665,6 +2667,290 @@ "integrity": "sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==", "license": "MIT" }, + "node_modules/@tailwindcss/node": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/node/-/node-4.3.2.tgz", + "integrity": "sha512-yWP/sqEcBLaD8JuA6zNwxoYKr75qxTioYwlRwekj5Jr/I5GXnoJfjetH/psLUIv74cYTH2lBUEzBkinthoYcBg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jridgewell/remapping": "^2.3.5", + "enhanced-resolve": "5.21.6", + "jiti": "^2.7.0", + "lightningcss": "1.32.0", + "magic-string": "^0.30.21", + "source-map-js": "^1.2.1", + "tailwindcss": "4.3.2" + } + }, + "node_modules/@tailwindcss/oxide": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide/-/oxide-4.3.2.tgz", + "integrity": "sha512-z8ZgnzX8gdNoWLBLqBPoh/sjnxkwvf9ZuWjnO0l0yIzbLa5/9S+eC5QxGZKRobVHIC3/1BoMWjHblqWjcgFgag==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 20" + }, + "optionalDependencies": { + "@tailwindcss/oxide-android-arm64": "4.3.2", + "@tailwindcss/oxide-darwin-arm64": "4.3.2", + "@tailwindcss/oxide-darwin-x64": "4.3.2", + "@tailwindcss/oxide-freebsd-x64": "4.3.2", + "@tailwindcss/oxide-linux-arm-gnueabihf": "4.3.2", + "@tailwindcss/oxide-linux-arm64-gnu": "4.3.2", + "@tailwindcss/oxide-linux-arm64-musl": "4.3.2", + "@tailwindcss/oxide-linux-x64-gnu": "4.3.2", + "@tailwindcss/oxide-linux-x64-musl": "4.3.2", + "@tailwindcss/oxide-wasm32-wasi": "4.3.2", + "@tailwindcss/oxide-win32-arm64-msvc": "4.3.2", + "@tailwindcss/oxide-win32-x64-msvc": "4.3.2" + } + }, + "node_modules/@tailwindcss/oxide-android-arm64": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-android-arm64/-/oxide-android-arm64-4.3.2.tgz", + "integrity": "sha512-WHxqIuHpvZ5VtdX6GTl1Ik/Vp2YuN42Et+0CdeaVd/frQ9jAvGmvR8vLT+jk3e8/Q3x8kECB9+R17pgpp2BulA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-darwin-arm64": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-arm64/-/oxide-darwin-arm64-4.3.2.tgz", + "integrity": "sha512-GZypeUY/IDJW3877KeM+O67vbXr3MBnbtEL4aYhNErv/JWZhye2vGSWWG9tB6iiqR2MqRNkY8IOUy4NdSZV26w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-darwin-x64": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-darwin-x64/-/oxide-darwin-x64-4.3.2.tgz", + "integrity": "sha512-UIIzmefR6KO1sDU7MzRqAxC8iBpft/VhkGjTjnhoS6k7Z3rQ9wEgA1ODSiyH/tcSYssulNm4Ci3hOeK1jH7ccQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-freebsd-x64": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-freebsd-x64/-/oxide-freebsd-x64-4.3.2.tgz", + "integrity": "sha512-GN+uAmcI6DNspnCDwtOAZrTz6oukJnp337qZvxqCGLd3BHBzJpO0ZbTLRvJNdztOeAmTzewewGIMPb0tk2R4WA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm-gnueabihf": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm-gnueabihf/-/oxide-linux-arm-gnueabihf-4.3.2.tgz", + "integrity": "sha512-4ABn7qSbdHRwTiDiuWNegCyb5+2FJ4vKIKc3DmKrvAFw7MU1Lm11dIkTPwUaFdTzc7IsOpDbqBrlh0x6y36U/w==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm64-gnu": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-gnu/-/oxide-linux-arm64-gnu-4.3.2.tgz", + "integrity": "sha512-wDgEIGwoM8w8pufh9LVt1PahDgNdKXrLC2qfAnV3vAmococ9RWbxeAw4pxPttd/TsJfwjyLf90Dg1y9y8I6Emw==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-linux-arm64-musl": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-arm64-musl/-/oxide-linux-arm64-musl-4.3.2.tgz", + "integrity": "sha512-J5Nuk0uZQIiMTJj3LEx4sAA9tMFUoXQZFv1J6An+QGYe53HKRJuFDi0rpq/tuouCZeAbOBY3kQ6g8qeD4TUjtA==", + "cpu": [ + "arm64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-linux-x64-gnu": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-gnu/-/oxide-linux-x64-gnu-4.3.2.tgz", + "integrity": "sha512-kqCZpSKOBEJO4mz7OqWoofBZeXTAwaVGPj0ErAj7CojmhKpWVWVOnrt9dE8odoIraZq4oj3ausM37kXi+Tow8w==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "glibc" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-linux-x64-musl": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-linux-x64-musl/-/oxide-linux-x64-musl-4.3.2.tgz", + "integrity": "sha512-cixpqbh2toJDmkuCRI68nXA8ZxNmdK9Y+9v5h3MC3ZQKy/0BO8AWzlkWyRM7JAFSGBlfig4YVTPsK6MVgqz1uw==", + "cpu": [ + "x64" + ], + "dev": true, + "libc": [ + "musl" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-wasm32-wasi/-/oxide-wasm32-wasi-4.3.2.tgz", + "integrity": "sha512-4ec2Z/LOmRsAgU23CS4xeJfcJlmRg94A/XrbGRCF1gyU/zdDfRLYDVsS+ynSZCmGNxQ1jQriQOKMQeQxBA3Isw==", + "bundleDependencies": [ + "@napi-rs/wasm-runtime", + "@emnapi/core", + "@emnapi/runtime", + "@tybys/wasm-util", + "@emnapi/wasi-threads", + "tslib" + ], + "cpu": [ + "wasm32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/core": "^1.11.1", + "@emnapi/runtime": "^1.11.1", + "@emnapi/wasi-threads": "^1.2.2", + "@napi-rs/wasm-runtime": "^1.1.4", + "@tybys/wasm-util": "^0.10.2", + "tslib": "^2.8.1" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/@tailwindcss/oxide-win32-arm64-msvc": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.3.2.tgz", + "integrity": "sha512-Zyr/M0+XcYZu3bZrUytc7TXvrk0ftWfl8gN2MwekNDzhqhKRUucMPSeOzM0o0wH5AWOU49BsKRrfKxI2atCPMQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/oxide-win32-x64-msvc": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-x64-msvc/-/oxide-win32-x64-msvc-4.3.2.tgz", + "integrity": "sha512-QI9BO7KlNZsp2GuO0jwAAj5jCDABOKXRkCk2XuKTSaNEFSdfzqswYVTtCHBNKHLsqyjFyFkqlDiwkNbTYSssMQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 20" + } + }, + "node_modules/@tailwindcss/vite": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/@tailwindcss/vite/-/vite-4.3.2.tgz", + "integrity": "sha512-eHpMeX4JXfVNJDEcsouTeCBubJBTcTLigeaw/NTUW6PB5ATKKXdyonnXgTBX2VuRbjz1hjfz6C5XAhr52ImQXA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@tailwindcss/node": "4.3.2", + "@tailwindcss/oxide": "4.3.2", + "tailwindcss": "4.3.2" + }, + "peerDependencies": { + "vite": "^5.2.0 || ^6 || ^7 || ^8" + } + }, "node_modules/@trickfilm400/rollup-plugin-off-main-thread": { "version": "3.0.0-pre1", "resolved": "https://registry.npmjs.org/@trickfilm400/rollup-plugin-off-main-thread/-/rollup-plugin-off-main-thread-3.0.0-pre1.tgz", @@ -3290,6 +3576,20 @@ "integrity": "sha512-v/qV5aV5EUA2pGilzUCq5/eyOloZAqDZBu9UMBIzgPpLlprjSR6zswsWBTv0KpqxLGUAZEwhO95ZCt7srymNVA==", "license": "ISC" }, + "node_modules/enhanced-resolve": { + "version": "5.21.6", + "resolved": "https://registry.npmjs.org/enhanced-resolve/-/enhanced-resolve-5.21.6.tgz", + "integrity": "sha512-aNnGCvbJ/RIyWo1IuhNdVjnNF+EjH9wpzpNHt+ci/m9He9LJvUN8wrCcXjp9cWsGNAuvSpVFTx/vraAFQ8qGjQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "graceful-fs": "^4.2.4", + "tapable": "^2.3.3" + }, + "engines": { + "node": ">=10.13.0" + } + }, "node_modules/es-abstract": { "version": "1.24.2", "resolved": "https://registry.npmjs.org/es-abstract/-/es-abstract-1.24.2.tgz", @@ -4341,6 +4641,16 @@ "node": ">=10" } }, + "node_modules/jiti": { + "version": "2.7.0", + "resolved": "https://registry.npmjs.org/jiti/-/jiti-2.7.0.tgz", + "integrity": "sha512-AC/7JofJvZGrrneWNaEnJeOLUx+JlGt7tNa0wZiRPT4MY1wmfKjt2+6O2p2uz2+skll8OZZmJMNqeke7kKbNgQ==", + "devOptional": true, + "license": "MIT", + "bin": { + "jiti": "lib/jiti-cli.mjs" + } + }, "node_modules/js-tokens": { "version": "4.0.0", "resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz", @@ -5731,6 +6041,27 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/tailwindcss": { + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/tailwindcss/-/tailwindcss-4.3.2.tgz", + "integrity": "sha512-WtctNNSH8A9jlMIqxzuYumOHU5uGZyRv0Q5svQl+oEPy5w84YpBxdb7MdqyiSPQge5jTJ6zFQLq0PFygdccSBA==", + "dev": true, + "license": "MIT" + }, + "node_modules/tapable": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/tapable/-/tapable-2.3.3.tgz", + "integrity": "sha512-uxc/zpqFg6x7C8vOE7lh6Lbda8eEL9zmVm/PLeTPBRhh1xCgdWaQ+J1CUieGpIfm2HdtsUpRv+HshiasBMcc6A==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/webpack" + } + }, "node_modules/temp-dir": { "version": "2.0.0", "resolved": "https://registry.npmjs.org/temp-dir/-/temp-dir-2.0.0.tgz", diff --git a/frontend/package.json b/frontend/package.json index 8efc8db..3b52edf 100755 --- a/frontend/package.json +++ b/frontend/package.json @@ -20,11 +20,13 @@ "vite-plugin-pwa": "^1.3.0" }, "devDependencies": { + "@tailwindcss/vite": "^4.3.2", "@types/node": "^24.13.2", "@types/react": "^19.2.17", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^6.0.2", "oxlint": "^1.69.0", + "tailwindcss": "^4.3.2", "typescript": "~6.0.2", "vite": "^8.1.0" } diff --git a/frontend/src/components/ErrorBoundary.tsx b/frontend/src/components/ErrorBoundary.tsx index ac3ad67..800a4d3 100644 --- a/frontend/src/components/ErrorBoundary.tsx +++ b/frontend/src/components/ErrorBoundary.tsx @@ -29,21 +29,14 @@ export class ErrorBoundary extends React.Component { if (this.state.hasError) { if (this.props.fallback) return this.props.fallback; return ( -
-

⚠️ Something went wrong

-

+

+

⚠️ Something went wrong

+

{this.state.error?.message || 'An unexpected error occurred'}

diff --git a/frontend/src/components/Skeleton.tsx b/frontend/src/components/Skeleton.tsx index acbbd2a..af85f8f 100644 --- a/frontend/src/components/Skeleton.tsx +++ b/frontend/src/components/Skeleton.tsx @@ -16,32 +16,20 @@ export const Skeleton: React.FC = ({ count = 1, style = {}, }) => { - const baseStyle: React.CSSProperties = { + // width/height/borderRadius are runtime-computed props (can't be static + // Tailwind classes), so they stay inline; the shimmer gradient/animation + // itself is a shared, static style defined once in index.css. + const dynamicStyle: React.CSSProperties = { width, height, borderRadius, - background: 'linear-gradient(90deg, #2a2a3e 25%, #3a3a4e 50%, #2a2a3e 75%)', - backgroundSize: '200% 100%', - animation: 'skeleton-shimmer 1.5s ease-in-out infinite', marginBottom: count > 1 ? 8 : 0, }; - if (typeof document !== 'undefined' && !document.getElementById('skeleton-keyframes')) { - const styleEl = document.createElement('style'); - styleEl.id = 'skeleton-keyframes'; - styleEl.textContent = ` - @keyframes skeleton-shimmer { - 0% { background-position: -200% 0; } - 100% { background-position: 200% 0; } - } - `; - document.head.appendChild(styleEl); - } - return ( <> {Array.from({ length: count }).map((_, i) => ( -
+
))} ); @@ -52,7 +40,7 @@ interface DashboardSkeletonProps { } export const DashboardSkeleton: React.FC = ({ lines = 5 }) => ( -
+
{Array.from({ length: lines }).map((_, i) => ( diff --git a/frontend/src/features/admin/AdminPage.tsx b/frontend/src/features/admin/AdminPage.tsx index d89b585..933c7a5 100755 --- a/frontend/src/features/admin/AdminPage.tsx +++ b/frontend/src/features/admin/AdminPage.tsx @@ -30,6 +30,16 @@ interface ModalState { targetUser: User | null; } +const btnClass = 'min-h-9 touch-manipulation cursor-pointer whitespace-nowrap rounded-md border border-border-default bg-bg-hover px-4 py-1.5 text-xs text-text-primary'; +const activeBtnClass = 'min-h-9 touch-manipulation cursor-pointer whitespace-nowrap rounded-md border border-accent-blue bg-accent-blue px-4 py-1.5 text-xs text-white'; +const cellBtnClass = 'min-h-9 touch-manipulation cursor-pointer rounded border border-border-default bg-transparent px-2.5 py-[3px] text-[11px] text-accent'; +const dangerBtnClass = 'min-h-9 touch-manipulation cursor-pointer rounded border border-red/20 bg-transparent px-2.5 py-[3px] text-[11px] text-red'; +const inputClass = 'w-full rounded border border-border-default bg-bg-primary px-2 py-[5px] text-xs text-text-primary outline-none'; +const labelClass = 'mb-1 text-[11px] font-medium text-text-secondary'; +const thClass = 'px-3 py-1.5 text-left text-[11px] font-medium uppercase tracking-wide text-text-secondary'; +const tdClass = 'px-3 py-2 align-middle'; +const statCardClass = 'min-w-[100px] rounded-lg border border-border-default bg-bg-surface px-3.5 py-2 text-center'; + export default function AdminPage() { const navigate = useNavigate(); const dispatch = useAppDispatch(); @@ -303,48 +313,25 @@ export default function AdminPage() { if (!user?.is_admin) return null; if (initializing || isLoading) { return ( -
-
Loading admin panel...
+
+
Loading admin panel...
); } - // ── Styles ── - const btnStyle: React.CSSProperties = { - padding: '6px 16px', fontSize: 12, border: '1px solid #30363d', - borderRadius: 6, background: '#21262d', color: '#c9d1d9', cursor: 'pointer', - touchAction: 'manipulation', minHeight: 36, - }; - const activeBtn: React.CSSProperties = { ...btnStyle, background: '#1f6feb', color: '#fff', borderColor: '#1f6feb' }; - const cellBtn: React.CSSProperties = { - padding: '3px 10px', fontSize: 11, border: '1px solid #30363d', - borderRadius: 4, background: 'transparent', color: '#58a6ff', cursor: 'pointer', - touchAction: 'manipulation', minHeight: 36, - }; - const dangerBtn: React.CSSProperties = { ...cellBtn, color: '#f85149', borderColor: '#f8514933' }; - const badgeGreen: React.CSSProperties = { color: '#3fb950', fontSize: 12, fontWeight: 600 }; - const badgeRed: React.CSSProperties = { color: '#f85149', fontSize: 12, fontWeight: 600 }; - const inputStyle: React.CSSProperties = { - padding: '5px 8px', fontSize: 12, color: '#c9d1d9', background: '#0d1117', - border: '1px solid #30363d', borderRadius: 4, outline: 'none', width: '100%', boxSizing: 'border-box', - }; - const labelStyle: React.CSSProperties = { - fontSize: 11, color: '#8b949e', fontWeight: 500, marginBottom: 4, - }; - return ( -
+
{/* ═══ Navbar ═══ */} -