LLMs Evaluated for Technical Market Analysis and Trading

This research conducts a systematic evaluation of five prominent large language models (LLMs)—GPT-4 Turbo, Claude 3 Opus, Gemini 1.5 Pro, Llama 3 70B, and the specialized FinGPT—for their capabilities in technical market analysis. The evaluation framework covers four key tasks: recognizing candlestick patterns from OHLCV data, generating directional trading signals (BUY/SELL/HOLD), backtesting these signals through a simulated execution pipeline, and comprehending financial reports. Quantitative metrics like Sharpe ratio, maximum drawdown, and F1-score were used to assess performance. The study found that GPT-4 Turbo achieved the highest annualized return and Sharpe ratio among general-purpose models, while FinGPT showed competitive risk-adjusted performance due to its domain-specific fine-tuning. Both models surpassed a passive S&P 500 benchmark. However, the research also highlighted common failure modes across all LLMs, including numerical hallucination, context window limitations, and inconsistent performance in sideways markets, suggesting that robust deployment requires careful task decomposition and rigorous backtesting.