One of the most frequently stated criticisms of standardized tests is that they do not measure creativity and problem solving ability. Because they are typically presented in a multiple choice format, a teacher cannot, for example, see where a child went wrong when he or she failed to solve a math problem correctly, because no work is shown. In addition, critics feel that the questions are often too simplistic for children to fully demonstrate reading comprehension, critical thinking, and problem solving. Especially in the case of reading selections, there may not be a single "right" answer to a question, and learning why children pick the answers that they do might help educators to provide better support.

Critics who feel that tests are too simplistic believe that knowing information by rote does not always mean that information has been digested. For example, a student might be able to answer a question asking when Columbus came to the Americas, but may not understand the cultural and historical implications of the Age of Exploration, and the profound impact that Columbus had on global history. Test takers are not provided with space to process and explore issues because it would make them prohibitively expensive to administer and score.

Many critics are also concerned about bias in test taking. Certain cultural and language biases may be impossible to eradicate entirely from standardized tests, critics believe, meaning that some populations such as girls and minorities may be at a disadvantage. While test writing companies do their best to eliminate obvious bias and offensive language, it is a difficult thing to correct for entirely.