Assessment

University of the Rockies // Assessment

Assessment

University of the Rockies is committed to fostering student learning which includes assessment across multiple levels. Applying an assessment strategy that is valid, reliable, and supportive of the institutional mission remains a critical goal. This strategy includes clear alignment of institutional, programmatic, and course learning outcomes, as well as accounting for co-curricular activities, and non-academic services. Gathering, organizing, analyzing, documenting, and using these data in a broad range of meaningful ways ensures continuous improvement across the organization. Student performance is assessed at key points (i.e. “gates”) during the program. The following link provides an illustration of the University of the Rockies “Gates Model” of assessment.

UoR Gates Model

Assessment Methodologies

The University has developed an assessment methodology that is highly flexible, and capable of supporting a broad range of institutional assessment needs. It is a goal of our institution to gather data using techniques that are valid, reliable, and based on acceptable academic research and design standards. Data being collected must be both useful and appropriate for the student activity being measured. For example, student papers are assessed for evidence of learning by specially trained evaluators using rubrics that measure student achievement on specific criteria for each assignment.
Rubrics are tools used within our institution to support and enhance the student learning experience. They contain descriptions of key behaviors at different performance levels, and assist students in better understanding both how they are performing, as well as what they need to do in order to improve. Rubrics may contain standardized descriptions of behaviors, but may also be designed with customized descriptions for unique assignments within a course or program of study. In all cases, rubrics represent an initiative to extend consistent, professional feedback and evaluator guidance from our faculty to the valued students.

Assessment occurs in four main stages and through a series of processes as outlined in the Assessment Learning Loop. The main stages are 1) determining the need or targets, 2) customizing and conducting the assessment in service of measurement and learning, 3) reviewing and learning from the data as it supports University improvement, and 4) meta-assessment, which is review of the assessment process for improved evaluation in the future.

Assessment Learning Loop

The University of the Rockies Assessment Strategy emphasizes the goal to gather meaningful data in manageable and sustainable ways. The Assessment Learning Loop is the process we apply to direct and align our assessment activities in a way that supports the continuous improvement of university programs and practices, and a consistently sustainable, positive student experience.

Institutional Demographics

University of the Rockies embraces excellence, integrity, diversity, and service as components of its core value system. As we grow, we are confident that our student population will continue to reflect our commitment to these values. The following link to the Integrated Postsecondary Education Data System (IPEDS) provides access to the most current information about University of the Rockies, as well as an opportunity to compare us with a wide range of other institutions. 

Rockies IPEDS data

Learning Outcomes 

Institutional and Programmatic Learning Outcomes

Institutional Learning Objectives

Assessment Success Drivers

University of the Rockies is committed to learning on every level. As a community, University of the Rockies seeks to train and support students on the following key programmatic learning outcomes:

  • Ethics
  • Critical Thinking
  • Communication (Written, Oral, and APA Style)
  • Emotional Intelligence
  • Lifelong Learning
  • Diversity

There is a seventh competency that is woven into the skills and behaviors used to define the outcomes noted above. This seventh competency is "Professional Demeanor." The University of the Rockies model is "Practitioner Scholar," and professional demeanor is an important part of the development and growth of our students as they work through their programs.

Training to these targets, each student aligns with the University’s unique contribution to the field, which will result in robust and recognized practitioners participating in their social, geographical, business, and scholarly communities.

In the School of Professional Psychology, classes are predominately delivered on campus, and the faculty complete assessments of student development for the institutional learning outcomes identified above. The following are the results of those assessments performed in the classes during the August and October Terms of 2010, and the January and March Terms of 2011.

Holistic Assessment Data

Written communication skills are critically important to student success, and are targeted in our programmatic learning outcomes. What do we know about our students’ written communication skills, and how does the University support its students in the development of those skills?

University of the Rockies recognizes that effective writing is critically important to our students’ success. In order to better understand how well our students are performing in writing, we conduct “Holistic Assessments” on student work. In this case, examples of student work are comprehensively reviewed by specially trained assessors using University standards that are carefully mapped to specifically identified learning outcomes. We continue to review student work across our academic programs in order to expand and improve upon these efforts.
 
To support our students, University of the Rockies has developed and launched the Rockies Writing Center, with coaches on staff who are available to review student papers and give feedback that supports the development and application of essential writing skills.

The Writing Center is a web-based resource accessible through the Student Portal or the online classroom, and guides students as they develop a range of academic writing skills. It contains APA style guides, resources for developing thesis statements, tutorials for MS Office, and material on how to conduct effective research. The Writing Center is staffed by professional writing coaches who review student papers and provide feedback on grammar, clarity of language, and APA style.

Practicum Evaluation Assessment Data for the School of Professional Psychology

During their practicum, students practice key professional skills that they learned in their courses. How do students know if they are improving these skills and abilities in a practicum setting?

Doctoral students are required to complete a predetermined number of supervised practicum hours to develop experience in clinical skills and abilities in a formal work environment. Many of the students from the School of Professional Psychology engage in practica at the Rockies Counseling Center, where they provide clinical services to clients under the supervision of licensed clinical faculty.

During the term, each student meets with their Faculty Practicum Supervisor for feedback and training. At the end of the term, each student receives a formal evaluation of their clinical progress and development during that term.

The Supervising faculty use the evaluation forms to assess behaviors, skills, and abilities that both support good therapeutic activity and align with the School of Professional Psychology programmatic learning outcomes. As students move through their practicum experiences in sequence, there is a reflected increase in students’ skills and abilities as measured by their supervising faculty.

Examination for Professional Practice of Psychology (EPPP)

One significant measure of the effectiveness of an educational model is the ability and success of graduates to become accredited by the professional institutions that regulate their new careers. For a career in psychology, the Association of State and Provincial Psychology Boards (ASPPB) exam, more commonly known as the Examination for Professional Practice of Psychology (EPPP), is an important requirement and gating milepost. A passing score on this exam is a requirement for psychologist licensure by the licensing boards in all states. https://www.asppb.org/HandbookPublic/Reports/default.aspx?ReportType=EPPPPassingScore
At University of the Rockies, clinical PsyD students who graduated between 11/4/2003 and 3/7/2011 self-reported a 91.7% overall pass rate (based on 33 out of 36 students who reported taking and passing the EPPP Annual Alumni Survey 2011). Our clinical PsyD students further reported an 81% pass rate on their initial attempt on the exam (based on 21 out of 26 students who reported in an email survey conducted May/June 2011 that they passed the exam on their first attempt). University of the Rockies is not currently accredited by the American Psychological Association (APA) nor listed in a recognized national register such as ASPPB. For purposes of comparison with APA/Canadian Psychological Association (CPA)- accredited and non accredited schools, the data from our May/June 2011 email survey indicated that our students had an 84% pass rate when all attempts and passing scores were calculated (which is the methodology used to calculate the EPPP pass rate for all APA/CPA-accredited and non-accredited schools).

Our 84% pass rate (based on 26 passes out of 31 attempts) is higher than the 77% pass rate of APA/CPA-accredited schools and the 65% pass rate of non-designated/non-APA/CPA-accredited schools, as reported in the Psychology Licensing Exam Scores by Doctoral Programs published on the ASPPB website. https://www.asppb.net/files/public/ASPPBPsychExamScores3-19-10.pdf. These important statistics are yet another testament to our strong academic curriculum, the expertise and skill of our professors, and the hard work of our students.
 
Source 1: May/June email survey 
Source 2: APA/CAP-Accredited Schools and Non-Designated/Non-APA/CPA-Accredited Schools:https://www.asppb.net/files/public/ASPPBPsychExamScores3-19-10.pdf.
Because state licensure requirements vary widely and some of our graduates may not be able to obtain licensure in certain states, we strongly advise our students of their responsibility to research the requirements for licensure or certification in the states(s) in which they intend to practice, as each state has its own requirements for licensure.

Review of Program Effectiveness

An extended review of the assessment procedures aligns with the University’s intention to support its educational programs and plans for strengthening its quality moving forward. All programs at University of the Rockies undergo periodic reviews, conducted by a combination of both faculty and administration. Program reviews are designed to evaluate specific program goals, curriculum, resources, effectiveness, and congruence with the institutional mission. In an effort to ensure that programs meet the institutional mission and are effective in preparing students, the University engages in mapping across course, program, and institutional outcome levels. The mapping process is an important component in the development of our comprehensive assessment plan. Students are evaluated at key stages in their graduate education. Success in coursework and demonstration of mastering institutional outcomes are targeted goals throughout the program of study.

The Higher Learning Commission (HLC) Criteria for Accreditation

University of the Rockies is accredited by The Higher Learning Commission and a member of the North Central AssociationThe Higher Learning Commission utilizes criteria for accreditation to support regional schools in building and maintaining rigorous programs for their student body. These "Criteria for Accreditation" include criteria statements, core competencies, and specific examples of evidence. Further information about these guiding principles can be found at the HLC Website.

The Higher Learning Commission (HLC) Criteria for Accreditation

University of the Rockies is a private, independent graduate school offering the Master of Arts in Psychology (MA) as well as Doctor of Philosophy (PhD) and Doctor of Psychology (PsyD) degrees. For more information about our completion rates, the median debt of students who completed our programs, and other imoportant information, please visit our website at http://www.rockies.edu/pd.

University of the Rockies has also been granted Full Authorization status as a degree granting institution by the Colorado Commission on Higher Education (CCHE).

University of the Rockies is an associate member of the National Council of Schools and Programs of Professional Psychology.

Program and Course Descriptions

A comprehensive description of the programs and courses at University of the Rockies can be found at www.rockies.edu.

Transparency by Design

University of the Rockies is a member of the groundbreaking initiative called Transparency by Design. This initiative launched the College Choices for Adults website, which features program level content for prospective adult students seeking information about the programs and institutions offering online and distance education opportunities.

University of the Rockies Writing Center

The Rockies Writing Center provides an example of an improvement initiative that was supported by University assessment efforts. Internal research showed that students could benefit from extra support and access to effective written communication skill development resources and feedback. The online writing labs and access to writing coaches supported this critical need.

Waypoint Outcomes

University of the Rockies partners with Waypoint Outcomes™ LLC, a subsidiary of Bridgepoint Education, Inc., as a service provider of an online assessment tool, and a comprehensive platform for assessing and evaluating student work. University of the Rockies uses customized rubrics across an expanding range of programs to support student learning, and continues to use them for both holistic and in-course evaluation.


Assessment Resources

University of the Rockies employs best practices from the field of assessment and evaluation. Examples of these resources include:
Association of American Colleges and Universities (AAC&U) The Association of American Colleges and Universities is a consortium of member institutions that collaborate in support of educational improvement and reform. AAC&U develops and advances assessment practices that deepen, integrate, and demonstrate student learning. This website is an excellent source of assessment aids, rubrics, and other useful content: http://www.aacu.org/

Association of State and Provincial Psychology Boards Website: (2011) Retrieved from 
http://www.asppb.net/i4a/pages/index.cfm?pageid=3343.

Association of State and Provincial Psychology Boards Website: 2010 Psychology Licensing Exam Scores by Doctoral Program, pg 12, Retrieved from http://www.asppb.net/files/public/ASPPBPsychExamScores3-19-10.pdf.

Bloom’s Taxonomy

The revised Bloom’s Taxonomy is useful to support and inform curriculum development and assessment. As students learn more complex content and are encouraged to demonstrate their understanding and application of these ideas and practices, their level of professional ability evolves through the Bloom’s Taxonomy from "remembering" and "understanding," to "applying" and "analyzing," to "evaluating" and ultimately "creating" as they complete their final projects and dissertations.

2011 University of the Rockies, PsyD Clinical Specialization Alumni Survey and follow up results.

Assessment pageTitle:, pageUrl:, pageId:, pageDescription:,