#Benchmarks
Experiment: Can AI Models Catch All Bugs and Merge External Code into a Flawless Solution?
A practical test demonstrated why developers shouldn't blindly trust a single language model: four LLMs searched for seven bugs in a script, while seven others attempted to assemble a flawless version from external fixes.
Anthropic Debuts Claude Fable 5.1 as Frontier AI Race Accelerates
Anthropic has unveiled Claude Fable 5.1 with major performance gains and relaxed safety filters, kicking off a major wave of frontier AI releases alongside escalating regulatory debates and industry legal clashes.
Testing Qwen3.8-Flash-Next: Comparison with Production Models and the Importance of Multiple Runs
Recent benchmark tests of the Qwen3.8-Flash-Next architecture showed quality on par with Qwen3.8-27B alongside notable architectural innovations, though production adoption remains hindered by limited inference engine support.
Qwen 4 Architectural Breakthrough: Inside the Qwen3.8-Flash-Next Model
Alibaba has introduced Qwen3.8-Flash-Next, an architectural preview of the upcoming Qwen 4. The 180-billion-parameter model activates just 6 billion parameters per token and can run on 128 GB of system memory.
How to Make AI Agents Work Reliably: Lessons from Building Maritime Analyst Shippy
Engineers from the Allen Institute for AI shared the architectural challenges encountered while building the Shippy AI agent and explained why language models should not be trusted with executing strict technical algorithms.
OpenAI Releases ChatGPT Images 2.0 with Reasoning Mode, 2K Support, and Multilingual Text
OpenAI has introduced the gpt-image-2 model, bringing O-series reasoning logic to image generation, reliably rendering Cyrillic text, and taking the top spot on the LM Arena leaderboard.
Remote Labor Index Benchmark Shows Modern AI Agents Can Handle Only 2.5% of Real Freelance Tasks
Researchers from the Center for AI Safety and Scale AI have developed the RLI benchmark based on hundreds of real Upwork projects. Even leading language models and agents were able to successfully complete no more than 2.5% of the jobs.
Too Good to Be Bad: Study Shows Why AI Struggles to Convincingly Roleplay Villains
Researchers from Tencent and Sun Yat-sen University have found that built-in safety mechanisms prevent large language models from effectively roleplaying negative characters, replacing subtle manipulation with generic aggression.
GLM-5 Released: Zhipu AI's Open Model Reaches Top-Tier Performance in Agentic Coding
Zhipu AI and Tsinghua University have introduced GLM-5, a powerful open MoE model with 744 billion parameters capable of autonomously writing code, conducting deep web search, and tackling complex multi-step tasks on par with proprietary flagships.
ClawBench Benchmark Exposes AI Agent Weaknesses: Top Real-Web Success Rate Reaches Just 33%
The new ClawBench benchmark evaluated autonomous web agents on live websites. The best-performing model successfully completed only one-third of the practical tasks.
OpenAI Unveils First Custom AI Chip 'Jalapeño' to Challenge Nvidia on Inference
OpenAI has published the first performance benchmarks for Jalapeño, its in-house inference chip developed alongside Broadcom, reporting significant gains in speed and power efficiency over Nvidia hardware.