Testing, Testing

Testing, Testing

Ensure the success of students with online assessment tools.

In order to produce learned, well-spoken alumni, IHEs nowadays need to assess the skills and abilities of students when they first walk onto campus. "No college wants to churn out people who don't know their subject," one software company president has said. IHEs already know the GPA and SAT score of a student, as well as what his or her teachers thought of them through letters of recommendation, but sometimes, freshman arrive with reading, writing, and math skills that are in need of remedial education.

That's where those pen-and-paper placement tests usually enter the equation. Now, however, paper tests are ceding to online assessments designed and powered by a handful of software companies.

Tom Ewing, spokesperson for Educational Testing Service (ETS), knows that college faculty members, who in recent years have been relegated to teaching essay-writing instead of delving into the world of ideas, decry students' writing abilities.

But, he says, "It's a problem not only in higher ed, but in business as well. Employers complain that a person out of college can't write, or perform even basic math. They are finding students are lacking in skills central to success."

At Penn State University, 12,000 freshmen are tested each fall in English and math, and sometimes even in chemistry. Students take online assessments in an unproctored environment using LXR*TEST, created by Logic eXtension Resources, a Georgetown, S.C., company that specializes in developing software tools for testing, certification, and licensure.

"The benefits are mostly for students," says Ralph Locklin, senior measurement specialist at the Schreyer Institute for Innovation in Learning at Penn State. "They can complete it at a place and time that they choose. In the past, tests were proctored and all students would be crammed together. Now, students can choose time or day. We take great pains to tell them to get a good fix on their ability."

Kirkwood Community College (Iowa), the state's third-largest college system behind Iowa State and the University of Iowa, cut class time by five minutes down to 50-minute class periods, so instructors welcomed Questionmark's Perception, according to Rich Edwards, professor of learning services. Questionmark, based in Stamford, Conn., created "Perception for Web," which is used to administer assessments using the internet or a school's intranet.

"It's a problem not only in higher ed, but in business
as well. Employers complain that a person out
of college can't write, or perform even basic math.
They are finding students are lacking in skills central
to success." -Tom Ewing, Educational Testing Service

"A lot of faculty find an advantage in online testing," says Edwards. "Tests are not given in class, so there is more time for instructors to teach."

He says online testing is even more beneficial for students. "Students get a week's window to go anytime to the testing center, and they get an immediate score."

Kirkwood, which serves 15,000 students in a seven-county area in eastern Iowa, found many advantages in using Perception, a system Edwards considers extremely secure and convenient. He says that students attend classes at several different campuses, so taking a day out of their schedule to congregate at the main campus for an assessment is not easy. Instead, students go to the main campus for testing at their convenience.

The community college system has used the untimed Perception tests since 1998. Kirkwood administered more than 7,000 Perception tests on its main campus in 2004.

Edwards listed a number of benefits for using Perception: It produces a more efficient and effective testing environment, creates a high degree of security, removes the fear of losing paper, obtains immediate scores, and provides a degree of quality control. Plus, "students like accessibility," he says.

Kirkwood also uses ACT's Compass to test incoming students on reading, writing, and math skills.

WebCT, a Lynnfield, Mass.-based provider of e-learning systems, helped the University of Alberta, Edmonton (Canada) pull off an efficient way to test incoming freshmen.

"They will have their name added to the course list, but will not be officially added until the placement test confirms their ability for math or English," says Barb Ross, chief operating officer of WebCT.

Ross says the system "saves wear and tear on the student and institution." It is beneficial for the students, because they will be placed in the appropriate math or English class. It is also good for the university, which can take a look at the level of achievement of incoming freshmen, and continually assess and improve its programs as they evaluate those results, she says.

John Hughes, vice president for academic affairs at The Master's College (Calif.), a Christian school, says that freshmen assessments help gather information on its 1,000 undergraduates.

However, size matters when it comes to testing, as revealed by some schools.

"Having large numbers of students creates different kinds of problems for administrators that smaller schools don't have," says Penn State's Locklin. Penn State tests more than 12,000 incoming freshmen each year, but, he says, "We're trying to be student-centered in everything we do."

However, in a community college setting like Kirkwood, with its generally open-door policy, Edwards says, "It becomes extremely important that students know what they need to improve on."

Although Kirkwood continually assesses students, Edwards considers it "more critical at the front end," when students are first admitted.

"You typically, philosophically, keep your doors open," he says.

Some software companies agreed that assessing students can create unwanted test anxiety. "You can worry students," says Questionmark President Eric Shepherd. "Ultimately, you want students to perform in a work environment. Some schools might overstress students."

And although incoming students are typically more web-savvy than ever, Melissa Anderson, a pedagogical advisor at Blackboard, a software company specializing in "e-education," says, "If [students] don't have experience doing online tests, they may be at a loss."

"The placement testing provides valuable information because it's standardized," says ETS' Ewing. "But the information is only useful when it's combined with other factors like a good AP score, SAT score, GPA, the personal essay, and the recommendation."

"We do have pitfalls," admits TMC's Hughes. "You add a whole extra layer of assessment that is not complementary to what students are studying and have no motivation to do it."

WebCT's Ross says the results of tests are not helpful to a university or to students if the test framework is inadequate. "As always, assessment is one of those issues that come down to the design of the assessment itself. The quality of the student's answer is only as good as the test's design."

"Student assessment tools are an integral part of the learning process, but it matters when or how you do it," says Kirkwood's Edwards. He adds that despite the licensing cost, assessments are necessary. "In the long run, it is a cost-efficient testing tool."

"Student assessment tools are an
integral part of the learning process,
but it matters when or how you do it."
-Rich Edwards, Kirkwood Community College

On the other hand, Penn State's Locklin doesn't think there is a particular downside to student assessments. "It's just another opportunity for us to find out a little more information about who our students are," he says.

As might be expected, the companies give each other very harsh critiques. Some think of themselves as highly specialized, focusing on security and analyzing students' results, while others claim to offer a broader experience by featuring chat sessions and class schedules.

Ross says WebCT integrates third-party software, like Questionmark, but indicates that students are able to track their results and can participate in chat sessions, even if they have yet to matriculate. "[WebCT] take students down different paths, and only release a certain question if they get above this number or group or by a certain date."

However, Shepherd says what makes Questionmark different is its enhancements. "We offer security, we lockdown PCs--even with open-book tests--and include sounds and videos," he says. "We offer a different presentation of questions and a statistical analysis, but present it in a consumable way for the student to understand."

On the other hand, Blackboard's Anderson says her company's product offers a "more robust assessment experience and ease of use."

Penn State's Locklin raves, "The really exciting thing about testing is new tests."

Locklin praised Questionmark, calling it an "exciting development." Questionmark is known in the industry for testing students using pictures, not just words, since not everybody learns in the same way. "You may be missing students that really need to see a picture," says Locklin, of word-based assessments.

"Things are much different now than they were five or 10 years ago," says Questionmark's Shepherd. "There are more tests, and people see the advantage of being tested. As our society matures, we see the benefits."

"It's that huge
freshman year
drop-off and
sophomore year
drop-off that
colleges are so
worried about."
-Matthew Greene,
Howard Greene & Assoc.

Shepherd anticipates plenty in the future for assessment software, such as better stimulus, videos, sounds, simulations, and better measurement of abilities.

"We don't want to show students a thousand statistics, and we don't want to merely show a pass or fail score," he says. "We want to show what the student did wrong and how they can improve on it. The student then sees the value of it."


Advertisement