Commit Graph

27 Commits

Author SHA1 Message Date
Le 1c022264f5 Fix O(n^2) blowup and look-ahead leak in SMC/divergence backtest scoring
market_structure() (SMC) and detect_divergence() were each precomputed
once over the ENTIRE multi-year backtest range and reused unchanged for
every candle, so every candle's score could see results derived from
years of future price data — a look-ahead bug that inflated both
single-run backtest and walk-forward results, undermining the very
overfitting check walk-forward exists to provide. A prior fix bounded
this to a per-candle trailing window, which closed most of the leak but
still rescanned pivots from scratch on every candle (O(window) per
candle), too slow to enable 15m/30m walk-forward runs.

The real fix: pivot detection is itself a bounded rolling-window scan
(each position only depends on a few bars on either side), so it can be
precomputed once for the whole dataset just like BB/RSI/MACD. Per candle,
_compute_scores_series now just advances a monotonic pointer over
already-known pivots to whatever is causally confirmable as of that
candle — O(1) amortized across the whole run instead of O(window) or
O(n) per candle. Added an optional precomputed_pivots param to
detect_divergence() (backward compatible) to reuse this for RSI/MACD
divergence too.

Net effect: 16,000 candles went from 16.1s to 1.7s (confirmed empirically,
on top of an earlier ~10x from fixing the raw O(n^2)), and scaling stays
linear at 32,000 candles (3.2s). Walk-forward's timeframe options are now
15m/30m/1h/4h/1d (up from 1h/4h/1d) since 15m at the 3-year default
lookback now costs roughly 30s instead of 5+ minutes. Also wired
walk_forward.py's grid search to actually reuse one computed score series
across all 27 parameter combinations per fold (it was recomputing full
classification for every combination despite the scoring/threshold split
added earlier). 156 backend tests passing (3 new: causal-score regression,
pivot-detection-runs-once, order-block-window-bounded).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 10:42:32 +07:00
Le 56325113b3 Show all UI times in GMT+7 and unify the timeframe list across the app
Every displayed timestamp previously relied on either the viewer's
browser-local timezone (toLocaleString/toLocaleDateString/toLocaleTimeString)
or raw UTC ISO-string slicing (.slice(0,10)/.slice(5,16)) — both wrong for
a Vietnam-based system, and the string-slicing approach could show the
wrong calendar date entirely near the UTC/GMT+7 day boundary. Added
frontend/src/utils/dateTime.ts with formatVN* helpers that explicitly
render in Asia/Ho_Chi_Minh regardless of the viewer's machine, and applied
them across AdminPage, OrderPanel's live clock, SignalPanel, ProfilePage,
AuditLogPage, and BacktestPage (including the new walk-forward fold/history
dates) — 10 display sites total.

Also consolidated the timeframe list (15m/30m/1h/4h/1d/1w/1M), which had
drifted into 4 different copies across BacktestPage, ProfilePage,
ChartToolbar, and AlertsPage, into a single frontend/src/utils/timeframes.ts
source of truth. Extended Walk-Forward's timeframe options from 1h/4h to
1h/4h/1d, and fixed a latent backend bug where backtest_engine.py's
tf_minutes map was missing "1d", silently defaulting to 30 minutes for
any daily-timeframe backtest.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 09:31:48 +07:00
Le 625c2b3773 Add walk-forward backtest optimization to mitigate signal overfitting (item m)
Rolling train/test folds over 3 years of data auto-optimize the three
cheap-to-tune trading parameters (STRONG/BUY score thresholds, max hold
time) via grid search on each fold's train window, then evaluate purely
on the held-out test window. Stitching all out-of-sample results gives
an honest performance estimate uninflated by tuning against the same
data used to score it.

Split signal_scoring.py's expensive 13-algorithm scoring from its cheap
final threshold classification so grid search can replay many parameter
combinations without recomputing indicators each time. Moved the
backtest engine (fetch/precompute/simulate) out of the API layer into
app/services/backtest_engine.py so both /backtest/run and the new
walk-forward optimizer share one implementation instead of drifting
copies — same rationale as the earlier signal_service.py split (item h).

Also merges two long-diverged Alembic migration heads discovered while
adding the walk_forward_results table, so `alembic upgrade head` has a
single target again.

New: POST/GET/DELETE /walk-forward/* endpoints, a Walk-Forward tab on
the Backtest page (fold table, out-of-sample equity curve, run history).
19 new backend tests (153 total, all passing).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 09:15:21 +07:00
hanlap 95119b039e chore: bump redis from 5.0.8 to 8.0.1 2026-07-04 01:57:31 +00:00
hanlap cf0fedffd3 fix: add redis==5.0.8 to Dockerfile dependencies 2026-07-04 01:50:55 +00:00
Le 9de41ea92b Split signal scoring, add Redis cache, Tailwind design system, and fix UI coherence issues
Backend: extract pure scoring logic from signal_service.py into signal_scoring.py (h),
add Redis-backed win-rate/PnL caching with graceful degradation (l), add Postgres
backup/restore scripts (n), move DB/encryption secrets to Docker secrets pattern (o),
fix RSI flat-price bug and MFI wraparound index bug (q, r). 134 backend tests passing.

Frontend: consolidate all API calls onto shared apiFetch with auto token refresh (i),
wire AnalyticsPage to the real /analytics/dashboard endpoint instead of fake random
data (j), migrate all pages and shared components to a Tailwind CSS design system (k)
fixing 3 mismatched color palettes found along the way. UI review also found and fixed
missing mobile table scroll wrappers, non-stacking grids, and a missing nav/logout bar
on ProfilePage.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 08:32:31 +07:00
hanlap 4a130363ec fix: add missing async_session_factory import in main_scheduler 2026-07-04 00:28:36 +00:00
Le 8b37c74180 docs: dong su co (a) -- da rotate xong 4/4 mat khau Gitea + rewrite lich su thanh cong
Xac nhan: git log -p tren origin/master cho 0 lan xuat hien ca 4 mat khau
cu sau khi force-push lich su da rewrite. Cap nhat SECURITY_INCIDENT doc
va ARCHITECTURE.md phan anh trang thai da dong, ghi lai bai hoc quy trinh
cho lan sau (nen tam dung push vao nhanh dich TRUOC khi chuan bi rewrite,
khong phai sau khi chuan bi xong).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 23:02:39 +07:00
hanlap a015e89aff ci: fix git safe.directory permission 2026-07-03 15:57:22 +00:00
hanlap d4abeac6b1 ci: trigger test 2026-07-03 15:56:28 +00:00
hanlap 934a96c87c ci: fix deploy workflow - dùng host mode trực tiếp 2026-07-03 15:52:01 +00:00
hanlap c8adbc2cc7 ci: thêm Gitea Actions workflow auto deploy khi push master 2026-07-03 15:49:45 +00:00
Le 40ea6bb793 docs: xac nhan da rotate xong ca 4 mat khau Gitea, tao lai nhanh rewrite lan 3
Nhanh history-rewrite-2026-07-03 truoc do bi xoa ma chua force-push vao
master. Da tao lai (lan 3) dua tren tip moi nhat cua master (9d0a32c),
verify lai 0 lan xuat hien mat khau cu. Chi con cho ban force-push.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 22:47:52 +07:00
Le 9d0a32cdc7 docs: cap nhat SECURITY_INCIDENT (rewrite lich su xong) + theo_doi v4
- SECURITY_INCIDENT_GITEA_CREDENTIALS.md: rewrite lich su git da hoan
  tat va day len nhanh history-rewrite-2026-07-03, chi con cho force-push
  vao master (thao tac can ban tu chay)
- theo_doi_trading-portal_v4.md: fix (p), test indicator_service +
  signal_service async (122 test tong), phat hien (q)/(r), ghi nhan
  su co quy trinh (s) -- loi build TypeScript do replace_all khong khop
  het 2 vi tri thut le khac nhau, da duoc hanlap va kip thoi

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 22:32:24 +07:00
Le 06bc5ba29b test: them 40 pytest cho indicator_service.py + phan async cua signal_service.py
- test_indicator_service.py (33 test): sma, ema, rsi, bollinger_bands,
  macd, atr, vwap, obv, obv_signal, mfi, detect_market_regime. Phat hien
  2 quirk nho (chua fix, can quyet dinh cua team):
    (q) rsi() tra ~98 thay vi 50 khi gia hoan toan di ngang (rs=50 sentinel
        van bi dua qua cong thuc RSI thay vi tra thang 50)
    (r) mfi() bi wraparound index o diem tinh dau tien cua chuoi (j-1=-1),
        tac dong thuc te gan bang 0 vi signal_service chi doc mfi_data[-1]
- test_signal_service_async.py (7 test): close_stale_trades (time limit,
  stop loss, take profit, trailing stop) + expire_old_signals. Cac ham
  nay tu mo session rieng qua async_session_factory (khong nhan db lam
  tham so) nen test monkeypatch bien module-level nay sang SQLite in-memory.
- conftest.py: them fixture session_factory (async_sessionmaker thay vi 1
  session) + _UTCDateTime TypeDecorator de SQLite giu duoc tzinfo UTC qua
  round-trip (SQLite khong ho tro luu tz-aware datetime nhu Postgres).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 22:30:31 +07:00
Le 9a0d2ab220 fix: sua loi eviction dung nham gia cross-symbol trong trade_executor.py
Phat hien (p) khi viet test cho trade_executor: khi kiem tra hybrid
eviction, PnL cua TAT CA cac trade dang mo (o nhieu symbol khac nhau) bi
tinh bang current_price cua tin hieu dang xu ly, thay vi gia thuc cua
tung symbol. Fix bang cach lookup gia moi nhat theo tung
symbol/exchange/timeframe (batched query, cung pattern da dung dung trong
close_stale_trades), ap dung cho ca xep hang loser LAN gia dong lenh cuoi
cung.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 22:29:41 +07:00
hanlap 4ab2dfbe7c fix: thêm exchange prop cho OrderPanel (lỗi TypeScript build) 2026-07-03 14:57:28 +00:00
Le 1d18787579 docs: cap nhat trang thai rotate mat khau namcunguyen
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 21:51:44 +07:00
Le 3d32c5eb26 docs: them theo_doi_trading-portal v0-v3 -- nhat ky danh gia va fix du an
Ghi lai toan bo qua trinh: danh gia kien truc/uu-nhuoc diem ban dau (v0),
fix RBAC + order routing + DB password (v1), fix AES-GCM + CORS + huong
dan xu ly lo mat khau Gitea (v2), va bo test 81 cases cho phan logic
rui ro cao nhat (v3).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 21:35:40 +07:00
Le afbe4f4f15 test: them 81 pytest cho auth/orders/security/CORS/risk_manager/trade_executor/signal_service + CI
Backend truoc day chi co script goi httpx vao server dang chay that
(test_auth.py, test_full_api.py), khong phai pytest that. Them bo test
chay doc lap bang SQLite in-memory (khong can Postgres/Docker):

- test_rbac_deps.py: RBAC chain + regression-guard cho fix vai tro o /orders/place
- test_order_exchange_routing.py: routing dung san theo credential
- test_security_encryption.py: AES-GCM round-trip + tuong thich nguoc AES-CBC
- test_cors_config.py: CORS fail-closed khi thieu cau hinh
- test_risk_manager.py: Kelly sizing + SL/TP adaptive theo tung regime
- test_trade_executor.py: STRONG-only, dedup, reversal, volatility filter,
  hybrid eviction FIFO -- toan bo quy tac mo/dong trade
- test_signal_service_scoring.py: he thong cham diem 13 thuat toan

Them .gitea/workflows/backend-tests.yml chay pytest tu dong khi push/PR
dung vao backend/** (can Gitea Actions + runner da duoc bat tren instance).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 21:34:18 +07:00
Le b37fa1804e security: go mat khau Gitea plaintext khoi ARCHITECTURE.md
Bang mat khau 4 tai khoan Gitea (hanlap, nguyendao, tulinh, namcunguyen) da
bi lo trong lich su git ke tu commit 85ca5c4. Go khoi ban hien tai va them
huong dan xu ly day du (rotate mat khau + tuy chon rewrite lich su git) --
viec rotate mat khau that tren Gitea van can admin thuc hien thu cong.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 21:33:18 +07:00
Le 6c1edcda34 fix: vá lỗ hổng RBAC + hardcode sàn ở /orders/place, nâng cấp mã hoá và CORS
- orders.place_order: yêu cầu role trader/admin (get_current_trader_user)
  thay vì bất kỳ user đã đăng nhập nào — viewer không còn đặt được lệnh thật
- orders.place_order: resolve exchange theo OrderRequest.exchange thay vì
  hardcode "mexc", fallback về credential active gần nhất nếu không truyền
- security.py: mã hoá API key chuyển AES-256-CBC -> AES-256-GCM (có xác thực
  toàn vẹn), giữ đường giải mã cũ để credential đã lưu trước đây không hỏng
- main_api.py: CORS_ORIGINS rỗng -> deny-all thay vì fallback "*" (kèm
  allow_credentials=True là cấu hình nguy hiểm)
- docker-compose.yml: đồng bộ DB_PASSWORD giữa backend-api/scheduler và db
- frontend: OrderPanel/DashboardPage truyền kèm exchange đang chọn khi đặt lệnh

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-03 21:27:47 +07:00
hanlap 88ce9cdd2d docs: thêm chi tiết backend-core, backend-API, frontend docs 2026-07-03 13:28:20 +00:00
hanlap 7af50cb397 docs: thêm modules-reference.md — tài liệu toàn bộ codebase 2026-07-03 13:25:48 +00:00
hanlap 85ca5c4581 docs: thêm Gitea + Infrastructure vào kiến trúc hệ thống 2026-07-03 13:23:46 +00:00
hanlap 841b8e2271 Update .gitignore: exclude all secrets, logs, models, reports 2026-07-03 13:08:48 +00:00
hanlap 34a1e91541 Initial commit: Trading Portal - FastAPI + React + PostgreSQL 2026-07-03 13:08:22 +00:00