19.5 – Use Technology in Assessment and Evaluation
Purpose & Context
Career Development Professionals (CDPs) use reputable technology in the selection, administration, scoring, and interpretation of assessment and evaluation results. CDPs adhere to professional and ethical principles of privacy and confidentiality.
Effective Performance
Competent career development professionals must be able to:
- P1. Assess available technology-enabled assessment and evaluation tools, considering:
- Test publisher’s reputation
- Confidentiality of test results
- Security of data
- Accommodations available to meet needs of clients with disabilities
- P2. Select assessment and evaluation tools, considering:
- Test purpose and context
- Psychometric properties, e.g. validity, reliability, fairness
- Client characteristics
- Accommodation options
- P3. Evaluate own qualifications to determine if qualified to administer and interpret results:
- Contract administration and interpretation to qualified persons, where necessary
- P4. Obtain informed consent, e.g. for use of assessment and evaluation tools, for use of results
- P5. Select accommodations required, if appropriate
- P6. Prepare for assessment and evaluation session:
- Verify technology works
- Select accommodations, as required
- P7. Administer examination in compliance with test publisher’s requirements
- Explain how to use the technology
- Verify mode of testing is appropriate
- Allow client time to practice
- Monitor assessment, as appropriate
- Note any incident that may impact test results
- P8. Verify results:
- Recognize unusual or unexpected results, e.g. pattern across scales, inconsistencies with information
collected using other measures or interview findings - Report results that appear erroneous to the testing publisher
- Recognize unusual or unexpected results, e.g. pattern across scales, inconsistencies with information
- P9. Interpret results, considering:
- Technical documentation, e.g. limitations of test results, prescribed use and interpretation of scales, norm
or comparison groups - Psychometric characteristics, e.g. reliability, error of measurement, validity, fairness
- Purpose of the assessment and evaluation
- Technical documentation, e.g. limitations of test results, prescribed use and interpretation of scales, norm
- P10. Present results:
- Explain how to use results to inform decisions, e.g. career decision, developmental needs
- Describe norms, raw scores or derived score scales clearly, as applicable
- Advise client on benefits and limitations of test results and interpretations
- P11. Protect confidentiality of results, as required:
- Comply with legislation and regulations, e.g. Personal Information Protection and Electronic Documents Act
(PIPEDA)
- Comply with legislation and regulations, e.g. Personal Information Protection and Electronic Documents Act
Knowledge & Understanding
Competent career development professionals must know and understand:
- K1. Testing practice in the context area, e.g. assessment and evaluation measures appropriate for vocational and
career
planning purposes - K2. Testing principles and ethical use of tests, such as:
- Test construction, administration, scoring and interpretation
- Psychometrics and measurement, e.g. classical test theory
- Descriptive statistics, e.g. frequency distributions, descriptive statistics characterizing the normal curve,
measures of central tendency, measures of variation, indices of relationship - Scales, scores, and transformations, e.g. types of scales, types of scores, scale score equating, cut scores
- Reliability and measurement error
- Validity and meaning of test scores
- Fairness
- K3. Advantages and disadvantages of different types of assessment instruments, e.g. mental and physical ability tests,
achievement tests:- Knowledge
- Work sample or performance test
- Interest measures, e.g. career related interests
- Work and personal values measures
- Personality inventories, e.g. vocational-oriented or clinical
- Interviews
- Comprehensive career planning measures
- K4. Legal rights of test takers
- K5. Confidentiality of test materials and test information
- K6. Descriptive statistics, e.g. frequency distributions, measures of central tendency, measures of variation, indices
of relationships - K7. Scales, scores, and transformations, e.g. types of scales, types of scores, scale score equating
- K8. Reliability and measurement error
- K9. Validity and meaning of test scores
- K10. Fairness, i.e. ensuring test design, content and format do not result in biased test scores for particular groups
Contextual Variables
Competent career development professionals must be able to perform this competency in the following range of contexts:
Assessment and evaluation tools will be selected to meet the specific needs of each client.
Administration and interpretation of results might be delegated to authorized individual as per test publisher guidelines.
Glossary & Key References
Terms
Industry-specific terms contained in the standard defined here, where applicable.
Information Sources and Resources for Consideration
A Test-Taker’s Guide to Technology-based Testing. International Test Commission, 2010.
Context Rating Scales
Criticality
Q: What is the consequence of a professional being unable to perform this skill according to the standard?
Frequency
Q: How frequent and under what conditions is this skill performed?
Level of Difficulty
Q: Under routine circumstances, how would you rate the level of difficulty in performing this skill?
Time Required to Gain Proficiency
Q: What is the average length of time or number of repeated events that are minimally necessary for an individual to become proficient in performing the skill to the standard?
A career development professional requires a minimum experience of reporting assessment results to at least 40 clients with varying assessment needs.
Autonomy
Practitioners typically perform this competency without supervision, and alone.
Automation
It is unlikely that this competency will automate.