How to evaluate science problem solving in a computerized learning environment? Construction of an analyzing scheme

Abstract

This paper describes the construction of a ‘computerized science problem solving’ scheme, which enables analysis and evaluation of the effectiveness of science problem-solving by junior high-school students working in a computerized learning environment. The scheme was based on observations of 187 students as they solved qualitative science problems taken from a specific computerized learning environment. Students were also interviewed before and after the problem solving. The scheme is presented on two levels. The large-scale comprises 11 main categories, each sub-divided into sub-categories to yield the detailed-level. The sub-categories were based on a repertoire of activities found in the observation protocols, and were approved by external judgement and a validation process. The detailed-level scheme enables evaluation and statistical analysis of the participants' problem-solving effectiveness, providing substantial evidence for the construct validity of the scheme, and demonstrating its potential as a valid analyzing and evaluative tool for computerized science problem solving.