Model Value Comparisons Skewed by Determinism and Access Clients

A new study identifies two significant confounds that distort cross-model comparisons of language model "values" or dispositions. The first confound is "response determinism," which refers to how sharply a model commits to a forced choice. When measuring values with single-draw methods, differences in determinism can be mistaken for genuine value differences. The researchers developed a protocol using repeated, counterbalanced forced-choice measurements and a determinism index to separate true value divergence from variations in a model's certainty. The second confound is the "access harness" – the specific client or API used to interact with the model. The study demonstrated that the deployment client can substantially shift a model's value profile. For instance, one subscription CLI altered a model's profile significantly and even flipped responses on several items, making the model appear "softer" than when accessed via its raw API. This indicates that the client acts as a value-shaping layer, potentially making a base model compliant with forced choices it would otherwise refuse. Consequently, audits based on single-draw value distances are not only inflated by determinism but also skewed by the specific access method.