Common Metrics & Outcomes

In evaluation of CISE REU, goals and objectives inform the purpose of the program and expectations for participants (more specifically than the Recommended Indicators). These are measured by pre-defining the theoretical constructs and metrics to be employed. A construct is the subjective conceptual element, or variable that is being measured, for example, computing self-efficacy. A metric is the standard of measurement used to gauge the construct, such as a battery of survey items designed to measure computing self-efficacy. Simply put, a construct is a variable, i.e. something with differing value. Constructs are linked to outcomes in that outcomes (e.g. increased interest in graduate school) are measured by constructs (e.g. interest in graduate school scale).

Common CISE REU Constructs

Research Exposure: reading and discussing research, understanding research process, statistical analysis, ethics, presentation of research

• Knowledge acquisition – self report of research content knowledge gain
• Attitudes toward – like/dislike of research activities
• Activity (experience) – research projects undertaken, any planned outcomes as a result, like posters, products, papers

Computing: programming, math, problem solving, and interest in computing majors and computing research

• Knowledge acquisition – self report of computing content knowledge gain
• Attitudes toward – like/dislike of computing activities
• Commitment – student self reported intent to remain in major, plans for graduate research in computing, plans for computing career

Self-Efficacy: confidence in one’s ability

• Research Confidence level – belief in ability to conduct and engage in research activities
• Confidence level – belief in ability to conduct and engage in computing activities

Site Administration: overall site operations, including auxiliary activities

• Participant demographics – gender, ethnicity and underrepresented minority participant levels, as defined by glossary descriptions
• Participant satisfaction levels – enjoyment in program activities beyond research and computing, e.g. social or soft skill development activities
• Attribution – to what degree participants attribute the research experience to future plans
• Faculty & peer support – satisfaction with faculty assistance and group work

Additional Indicators are Encouraged:

• Help Seeking & Coping Behaviors (see Sample Surveys & Instruments)
• Faculty Relationships (see Sample Surveys & Instruments)
• Achievement Measures of Research and Computing Domains (see Sample Surveys & Instruments)
• Use evaluative faculty surveys of student ability in addition to self report survey
• Use knowledge tests of student ability in addition to self report survey
• Longitudinal Indicators: (see Longitudinal Guidelines & Recommendations)
• GPA
• GRE Score
• Graduation in Computing
• Plans/Enrollment in Computing Graduate School
• Career Placement in Computing
• To what degree do participants attribute REU to their future plans

Chart of Common Constructs, Metrics & Outcomes

*Survey items are being analyzed for scale development; references to specific scales and/or items will be forthcoming. Interview protocols are available in the Qualitative Methods section of the toolkit.