Perplexity Open-Sources WANDR Benchmark for AI Research Capabilities.
Summary
Perplexity has open-sourced WANDR, an internal benchmark used to measure the research capabilities of AI systems, which they claim contributes to their cost-effective and high-performing research harness.
Why it matters
Professionals in AI development and research can use this new open-source benchmark to rigorously evaluate their own models, potentially improving performance and efficiency.
How to implement this in your domain
- 1Download and integrate the WANDR benchmark into your AI model evaluation pipeline.
- 2Compare your model's research capabilities against established benchmarks using WANDR.
- 3Analyze WANDR's methodology to refine your internal evaluation strategies.
- 4Contribute to the open-source WANDR project with feedback or improvements.
Who benefits
Key takeaways
- Perplexity open-sourced WANDR, a benchmark for AI research capabilities.
- WANDR helps evaluate AI model performance and cost-efficiency.
- Open-sourcing promotes community-wide AI development and evaluation standards.
- AI developers can use WANDR to improve their models.
Original post by @AravSrinivas
"Perplexity has the best (both on cost and performance) deep and wide research harness in Computer. One of the contributing factors is strong internal evals and benchmarks. Today, we're open-sourcing WANDR, the benchmark we use internally for measuring research capabilities."
View on X
Originally posted by @AravSrinivas on X · view source
Want to go deeper?
Turn these trends into skills with Learnijoy's hands-on AI & tech courses.
Explore coursesMore in AI Research
Stanford Lab Releases Proto for Combining AI Biology Models
A Stanford lab, affiliated with the Arc Institute, has launched Proto, an open-source framework enabling researchers to combine various AI biology models that previously operated in isolation.
AI Models Overtuned to Agentsmd Cause Task Stalling
AI models are reportedly overtuned to `agentsmd` files, leading them to misinterpret outdated instructions and stall on tasks, as exemplified by an agent repeatedly refining stage zero of a multi-stage task. This highlights a self-inflicted prompt injection risk if `agentsmd` content is not properly managed.
Chai Discovery Secures $400M, Deploys with Pharma Giants.
Chai Discovery has successfully deployed its technology with major pharmaceutical companies Eli Lilly, Pfizer, and Novartis, and has secured an additional $400 million in funding. This significant development positions the company at the forefront of medical innovation.