LLMs Evaluated for Technical Market Analysis in AI Trading

This paper presents a systematic evaluation of prominent large language models (LLMs) for their capabilities in technical market analysis within AI trading systems. Five models, including general-purpose LLMs like GPT-4 Turbo and domain-specialized FinGPT, were tested across tasks such as candlestick pattern recognition, generating buy/sell/hold signals, and backtesting signal quality. The evaluation used rigorous quantitative metrics like Sharpe ratio, maximum drawdown, and F1-score. Simulated backtesting revealed that GPT-4 Turbo achieved the highest annualized return and Sharpe ratio among general models, while FinGPT demonstrated strong risk-adjusted performance due to its financial domain fine-tuning. Both models surpassed a passive S&P 500 benchmark under the tested conditions. However, the study also identified consistent limitations across all LLMs, including numerical hallucination, context window constraints, and inconsistent performance in sideways markets. The findings suggest that while LLMs hold significant promise for AI trading, their robust deployment requires careful task decomposition, thorough backtesting, and domain-aware fine-tuning.