Human Votes Rank gpt-6-astra Top in Blind AI Web Design Test

How the study worked

HumanSignal, the company behind Label Studio, gave 10 models from 10 labs the same 10 design briefs earlier this month. The models were prohibited from using references, frameworks or web search. The aim, according to the company, was to understand the design sensibilities of the models themselves, including where they differ and converge, how they defend their design decisions, and how those designs align with human preferences.

For the human round, the team ran a blind, balanced round-robin in Label Studio. Every model faced every other model once for each brief, producing 450 head-to-head comparisons. With 10 votes per comparison, the study collected 4,500 votes from 29 annotators. Annotators saw two sites side by side without model names and picked the one they aesthetically preferred, or marked a tie.

The win rate gives a model a full point for each vote it won, half a point for each tie and nothing for a loss, divided by the number of votes it took part in.

The ranking

Gpt-6-astra led with a 64.3% win rate, followed by grok-4.6 at 62.7% and muse-spark-1.3 at 60.3%. Deepseek-v4-pro (56.2%), kimi-k3 (53.3%) and glm-5.3 (53.2%) made up the middle. Claude-sonnet-5 scored 49.1% and qwen3.8-2.4t 48.9%. Gemini-3.8-flash finished ninth at 32.8% and seed-2.0-code last at 19.1%.

HumanSignal also fit a Bradley-Terry model to the pairwise results, which estimates a single hidden strength for each model. According to the company, it returned the identical 1-to-10 order, which it takes as evidence that the ranking does not stem from scheduling or an unbalanced matchup. The schedule was balanced, so every model faced the same opponents the same number of times.

Two pairs are treated as ties: kimi-k3 and glm-5.3, separated by 0.1 percentage points, and claude-sonnet-5 and qwen3.8-2.4t, separated by 0.2.

Why one number can mislead

The company cautions against quoting the overall ranking alone. Seven different models finished first on at least one of the 10 briefs. Gpt-6-astra won two briefs outright, yet placed 8th on the medical-research-conference brief.

• glm-5.3 had the best average brief rank outside the top three, at 4.5, but finished last on the science-publication brief with a 24% win rate and ended 6th overall.
• claude-sonnet-5 never finished above 4th or below 9th on any brief, making it the most consistent model in the study. It still ranked 7th overall. In HumanSignal's words, consistency did not make it anyone's favorite.

For anyone choosing a model for a particular kind of design work, the company says results for similar briefs are more useful than the overall order.

A look at shared design choices

The interactive dashboard has four sections: the designs, the decisions, overall rankings and shared patterns. The designs section lets users browse the ten sites for each brief, with cost, generation time, the model's stated direction and the human evaluation. The decisions section compares typefaces, palettes and the reasoning each model gave.

The shared patterns section points to convergence. Inter appeared in 53 of 100 creative directions, IBM Plex Mono in 34 and Fraunces in 25. The dashboard also shows recurring color and layout choices. Results are downloadable as JSON, and the rankings and pilot methodology as CSV, so others can check the arithmetic.

HumanSignal says it will publish deeper dives into the most interesting results from the study in the coming weeks.

What it means for users

The study suggests that a single leaderboard position says little about how a model will perform on a specific design task. With the top model placing as low as 8th on one brief and a mid-table model winning others, teams picking a model for design work may get more from matching their own project type to the per-brief results. The data is public, so readers can inspect the votes and rationales themselves.