Abstract
A Bayesian approach for simultaneous optimization of test-based decisions is presented using the example of a selection decision for a treatment followed by a mastery decision. A distinction is made between weak and strong rules where, as opposed to strong rules, weak rules use prior test scores as collateral data. Conditions for monotonicity of optimal weak and strong rules are presented. It is shown that under mild conditions on the test score distributions and utility functions, weak rules are always compensatory by nature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brouwer, U., & Vijn, P. (1979). Bayesiaanse schatter voor de correlatiecoëfficiënt in “restriction of range” (geval 1) [Bayesian estimators for the correlation coefficient with restriction of range (Case 1)].Tijdschrift voor Onderwijsresearch, 4, 281–290.
Chuang, D. T., Chen, J. J., & Novick, M. R. (1981). Theory and practice for the use of cut-scores for personnel decisions.Journal of Educational Statistics, 6, 129–152.
Cronbach, L. J., & Gleser, G. C. (1965).Psychological tests and personnel decisions (2nd ed.). Urbana, IL: University of Illinois Press.
Davis, C. E., Hickman, J., & Novick, M. R. (1973).A primer on decision analysis for individually prescribed instruction (ACT Technical Bulletin No. 17). Iowa City, IA: American Testing College Program.
de Gruijter, D. N. M., & Hambleton, R. K. (1984a). On problems encountered using decision theory to set cutoff scores.Applied Psychological Measurement, 8, 1–8.
de Gruijter, D. N. M., & Hambleton, R. K. (1984b). Reply to van der Linden's “Thoughts on the use of decision theory to set cutoff scores”.Applied Psychological Measurement, 8, 19–20.
Ferguson, T. S. (1967).Mathematical statistics: A decision theoretic approach. New York: Academic Press.
Gross, A. L., & Su, W. H. (1975). Defining a “fair” or “unbiased” selection model: A question of utilities.Journal of Applied Psychology, 60, 345–351.
Hambleton, R. K., & Novick, M. R. (1973). Toward an integration of theory and method for criterion-referenced tests.Journal of Educational Measurement, 10, 159–170.
Huynh, H. (1976). Statistical considerations of mastery scores.Psychometrika, 41, 65–79.
Huynh, H. (1977). Two simple classes of mastery scores based on the beta-binomial model.Psychometrika, 42, 601–608.
Huynh, H. (1980). A nonrandomized minimax solution for passing scores in the binomial error model.Psychometrika, 45, 167–182.
Huynh, H. (1982). A Bayesian procedure for mastery decisions based on multivariate normal test data.Psychometrika, 47, 309–319.
Huynh, H., & Perney, J. C. (1979). Determination of mastery scores when instructional units are hierarchically related.Educational and Psychological Measurement, 39, 317–323.
Keeney, R. L., & Raiffa, H. (1976).Decisions with multiple objectives: Preferences and value tradeoffs. New York: Wiley.
Kendall, M., & Stuart, M. A. (1979).The advanced theory of statistics. Volume 2: Inference and relationship. London: Charles Griffin & Co.
Lehmann, E. L. (1986).Testing statistical hypotheses (2nd ed.). New York: John Wiley and Sons.
Lindgren, B. W. (1976).Statistical theory (3rd ed.). New York: Macmillan.
Lord, F. M., & Novick, M. R. (1986).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Mellenbergh, G. J., & van der Linden, W. J. (1981). The linear utility model for optimal selection.Psychometrika, 46, 283–293.
Novick, M. R., & Lindley, D. V. (1979). Fixed-state assessment of utility functions.Journal of the American Statistical Association, 74, 306–311.
Novick, M. R., & Petersen, N. S. (1976). Towards equalizing educational and employment opportunity.Journal of Educational Psychology, 13, 77–88.
Roe, R. A. (1979). The correction for restriction of range and the difference between intended and actual selection.Educational and Psychological Measurement, 39, 551–560.
Sawyer, R. (1993, April).Decision theoretic models for validating course placement systems. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection discussion and tables.Journal of Applied Psychology, 23, 565–578.
van der Gaag, N. (1990).Empirische utiliteiten voor psychometrische beslissingen (Empirical utilities for psychometric decisions). Unpublished doctoral dissertation, University of Amsterdam, The Netherlands.
van der Linden, W. J. (1981). Using aptitude measurements for the optimal assignment of subjects to treatments with and without mastery scores.Psychometrika, 46, 257–274.
van der Linden, W. J. (1984). Some thoughts on the use of decision theory to set cutoff scores: Comment on de Gruijter and Hambleton.Applied Psychological Measurement, 8, 9–18.
van der Linden, W. J. (1987). The use of test scores for classification decisions with threshold utility.Journal of Educational Statistics, 12, 62–75.
van der Linden, W. J. (1990). Applications of decision theory to test-based decision making. In R. K. Hambleton & J. N. Zaal (Eds.),New developments in testing: Theory and applications (pp. 129–155). Boston: Kluwer.
van der Linden, W. J. (1995). Some decision theory for course placement. Manuscript submitted for publication.
van der Linden, W. J., & Mellenbergh, G. J. (1977). Optimal cutting scores using a linear loss function.Applied Psychological Measurement, 1, 593–599.
Vos, H. J. (1990). Simultaneous optimization of decisions using a linear utility function.Journal of Educational Statistics, 15, 309–340.
Vos, H. J. (1991). Simultaneous optimization of the aptitude treatment interaction decision problem with mastery scores. In M. Wilson (Ed.),Objective measurement: Theory into practice (Vol. 1, pp. 313–331). Norwood, NJ: Ablex.
Vos, H. J. (1993). Simultaneous optimization of selection-placement decisions followed by a mastery decision. In F. J. Maarse et al. (Eds.),Computers in psychology: Tools for experimental and applied psychology (Vol. 4, pp. 235–244). Lisse, The Netherlands: Swets & Zeitlinger.
Vos, H. J. (1994).Simultaneous optimization of test-based decisions in education. Unpublished doctoral dissertation, University of Twente, The Netherlands.
Vrijhof, B. J., Mellenbergh, G. J., & van den Brink, W. P. (1983). Assessing and studying utility functions in psychometric decision theory.Applied Psychological Measurement, 7, 341–357.
Author information
Authors and Affiliations
Additional information
The authors are indebted to Wilbert Kallenberg for his valuable comments and to Jan Gulmans for providing the data for the empirical example. The names of the authors are alphabetical; they are equally responsible for the contents of this paper.
Rights and permissions
About this article
Cite this article
van der Linden, W.J., Vos, H.J. A compensatory approach to optimal selection with mastery scores. Psychometrika 61, 155–172 (1996). https://doi.org/10.1007/BF02296964
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02296964