"Working within a system,” Bill Sedlacek replied when Ray Diffley and I asked him about what one criterion among the many on his non-cog list he would choose to evaluate applicants, if he could choose only one.

Our conversation took place after Sedlacek’s keynote presentation at the University of Southern California’s Rossiter Center for Enrollment Research, Policy and Practice annual conference “Attributes That Matter,” which Ray and I attended on behalf of the SSATB Think Tank.

In his landmark book, Beyond the Big Test, Sedlacek elaborates: “the applicant’s ability to understand the role of the system in life and to develop a method of assessing the cultural or racial demands of the system and respond accordingly and assertively.” This is one of the eight research-based traits this distinguished scholar has, over a lifetime, determined “present a method of improving assessments for all students and are particularly useful for nontraditional students.”

We should be clear here: Sedlacek is not arguing we should dismiss traditional cognitive assessment in the form of the SAT or its analogues. Like Sternberg and other scholars in this field, he is calling for a more balanced approach, a both/and proposition that is entirely aligned with the mission of this Think Tank. To quote him in what might be something of a thesis for Beyond the Big Test:

We do not need to ignore our current tests; what we need is to add some new measures that expand the potential we can derive from the assessment. The goal of using non-cognitive variables is not tosubstitute this approach for the cognitive focus more commonly employed in assessments, but to addto the range of attributes that we can consider in making the many judgments required of us all.” (Italics in original)

You can find a tremendous array of his research reports, most of them published in peer-review journals, and many of his non-cognitive research surveys, ready for implementation and improvisation, at his website, http://williamsedlacek.info/.

In addition to “working within a system,” what are the other seven research-supported noncog variables Sedlacek believes we should be assessing?

• Positive Self-Concept
• Realistic Self-Appraisal
• Preference for Long Term Goals
• Availability of Strong Support Person
• Leadership Experience
• Community Involvement
• Knowledge Acquired in a Field

For each, Sedlacek spells out further definition, and provides a set of positive and negative evidence – framed as inquiry questions – that admission officials can bring to reviewing an applicant. As he explains, admissions committees can look for this evidence in a wide variety of ways, and he breaks down each of them: in an administered survey, in applicant short essay answers, in interviews of applicants, in review of the full applicant materials. All of these approaches are fully explored in his comprehensive book, which is both a research volume and a working handbook.

Why is expanding assessment so essential? “Sed,” as he is known, offered several reasons, including one particularly relevant to the SSATB community: the topping-out phenomenon.

There is an increasing statistical problem of restriction of range—we’ve “topped out” on a lot of exams—we don’t know how to measure any better at the high ends of achievement. This is compounded by the problem of grade inflation, which is huge—average GPA has risen half a grade in the last decade, and it creates same problem, topping out on GPA. Now with so many students earning 4.0s, we’re taking it out to third decimals, which is really beyond what we can measure.

But Sed’s greatest passion, and as you hear him speak you hear this passion rooted in his personal experience in the 1960’s crusading for civil and equal rights, clearly is to better diversify our selective schools and universities.

Sedlacek is determined to combat what he labels the Three Musketeers problem in assessment: All for one and one for all. “If different groups have their own experiences and ways of presenting their attributes and abilities, it is unlikely that one can develop a single measure or test item that works equally for all.” Offering a diversity of assessment techniques allows us to enhance our perspective into the breadth of strengths required for success in our schools. For while one important area of aptitude might be less well demonstrated in an applicant, other areas more than compensate. Sedlacek implores us to differentiate our institutions and our learning programs by differentiating our array of assessments—particularly and especially when we consider how we supplement traditional cognitive evaluation with non-cog tools.

In our conversation over lunch, Sedlacek shared some additional thoughts with Ray and me about this work.What about fakeability/coachability on surveys, we asked? Yes, this is an issue always, he replied. But deep self-appraisal surveys are harder to fake and harder to coach than you might think, and research has demonstrated good validity and reliability for these tools. For one thing, students have to know how and when to admit their weaknesses on these surveys, because if you only boast about yourself you actually then get a lower score, and it is hard to coach kids to be able to do this.

How do you introduce these changes? Respecting your constituencies is key, "Sed" explained. It is so important for any school embarking on this to evaluate carefully, cautiously, and critically your audiences, listening to your key constituencies, including your admissions staff, alumni, board, and faculty, and then design a tool to match and meet their needs, concerns, and anxieties, rather than impose something upon them.

But, he went on, reiterating what was a key theme of his talk, be a leader. Yes, it is true that charting a new course to measure and evaluate and value noncognitive attributes can be a departure and can expose you to criticism and complaint, but that is what it means to be a leader in your field—and isn’t that a goal for most educational institutions?

Report from the field:

Bill McMahon from Thacher School (CA) writes me to share some developments in their application this year, adding a new challenge intended to better illuminate the creativity and individuality of applicants.

Dr. Seuss wrote: “Be who you are and say what you feel, because those who mind don't matter, and those who matter don't mind.” With this quote in mind, create an “I” poster that captures who you are at this moment in your life, and how you are different from everyone else. The poster should be on one side of an 8.5 X 11” sheet of paper. Please mail a hard copy to Thacher.

Bill explains that, “As you can guess, this project is right in the wheelhouse for many (but not all) 8th grade girls. Still thinking about a refined prompt/project that would engage boys and better play to their "stereotypical" processing and output strengths.

"In terms of the ‘I-poster’ and its impact on the admission process, it has not radically changed how we read individuals, but it definitely has added flavor. In particular, it has given enhanced insight into students' overall level of creativity, and who they are as individuals. (Towards the latter point, there seems to be less parental involvement than we might find on an essay.)” Bill also writes of an additional unintended consequence revealed by the challenge: “It also appears to be a decent measure relative to effort (given that some do a quick pencil sketch and others come up with beautifully designed 4-color works of art).”