Voting Theory and Web-based Decision Tasks:
The Stevens Assessment Center

by Arnold B. Urken

SmartChoice Technologies Corporation
Stevens Institute of Technology
http://copeland.smartchoice.com/
aurken@smartchoice.com

Abstract

This paper looks at voting, rating, and surveys as Web tools to define and articulate choices made collectively by any pre-defined public or private group. Specifically, the choices made collectively by public and private groups are dramatically affected by the procedures used to arrive at any given outcome. Voting theory, also known as "social choice" theory, provides a framework for analyzing the non-obvious relationship between individual choice and collective outcomes.

Web-based groups should employ procedures that will best present and define both the choices facing the group and determinations made by the group. This practice will make it possible to manage disagreements about the legitimacy of outcomes. Conflicts about the design and evaluation of voting procedures are related to the decision task and the values of the participants. Achieving consensus about task definition and evaluation criteria provides a basis for analyzing feasibility or optimality in designing and building Web tools. Although designing Web-based election systems seems like a straightforward task, unarticulated and entrenched interests are likely to impede rational analysis of feasibility and optimality. Initially, Web voting tools should be developed for relatively non-controversial tasks in which a consensus about goals and evaluation criteria can be defined. This work will enable participants to carry out a wide scope of complex tasks more effectively than they would otherwise be able to do. The experience in building and deploying tools will also educate citizens about choosing voting systems, thus providing a basis for more informed debate about election reform.

In order for Web voting to grow, it is important to be able to experiment with processing voting, rating, and annotation data under different procedures to enable us to develop standards for specific types of tasks. To promote experimentation, it is desirable to develop a public interface standard for choice transactions and a stylebook for voting, rating and survey analyses which could be adopted to guide designers and users of such tools.

An implementation of this approach using Sm@rtChoicetm choice processingtm software is discussed in the context of a Web-based educational assessment center at Stevens Institute of Technology. This discussion describes the range of individual and collective evaluation tasks that are being integrated under the aegis of the Stevens Assessment Center. The Center plans to conduct all course evaluations electronically beginning in the fall of 1997.

A suite of evaluation tools designed to support the redesign of the undergraduate engineering curriculum also will be deployed. This suite includes templates for skills or competencies such as engineering design, communication, ethical and social awareness, engineering analysis, and teamwork. The last template combines collective assessment of team performance, individual evaluation of the team experience, and peer evaluation of each team member.

The Stevens study describes how problems such as privacy and motivation have been handled to enable participants to accomplish objectives that they would not otherwise attain. This study also analyzes the conflict between traditional management paradigms for gauging group performance and the types of analysis that are enabled by choice processing.