Support efforts to define learning outcomes at the degree and certificate level
As states work to improve postsecondary attainment rates, leaders are supporting efforts to ensure that the credentials earned are of high quality and convey value to students and employers. By creating and using common frameworks that describe what a student should know and be able to do, institutions and employers can identify the learning represented by credentials. This transparency allows student learning towards these outcomes to be assessed, regardless of where the learning takes place—on or off campus. As institutions assess student learning, improvements in teaching can be made to ensure each student is achieving. This type of assessment has students working on projects with real world applications, which students can then share with employers to demonstrate their knowledge and skills. Additionally, by promoting a focus on learning outcomes rather than course numbers or credit hours, some states are seeking to improve transfer between institutions.
There are a number of national and multi-state efforts underway aimed at ensuring quality in different ways. Some of these efforts may be more appropriate for state-level leadership and others for institutional or system leadership. Systems and institutions are using learning outcomes and defined competencies in various ways. For example, some statewide systems and institutions are using learning outcomes to develop competency-based degree programs and to award credit through prior learning assessment. Others are using the outcomes to validate the quality of statewide transfer agreements and develop new assessments of student learning.
Degree Qualifications Profile
The Degree Qualifications Profile (DQP) is a framework that describes what students should know and be able to do once they earn a degree—associate, bachelor’s or master’s—regardless of the field of study. By defining the competencies that should be learned, the DQP emphasizes the integration of learning from various sources and the application of learning in different settings. The competencies are arranged within five areas of learning: broad and integrative knowledge; specialized knowledge; intellectual skills; applied and collaborative learning; and civic and global learning. Since first released in 2011, more than 500 colleges and universities have used the DQP, creating a wealth of resources to support implementation of clear learning outcomes, improved teaching, and assessment of student learning. This includes a newly developed assignment library where faculty can find student assignments and projects that are aligned with degree outcomes. The DQP has not only helped institutions improve student learning, but has also given students and their advisors a clearer understanding of learning pathways and has increased transparency for accreditors and state leaders.
Essential Learning Outcomes and VALUE
In its efforts to promote the benefits of a liberal arts education in the 21st century, the Association of American Colleges & Universities (AAC&U) developed a set of Essential Learning Outcomes that the organization and its members believe all students should understand, prepare for and achieve. Further, AAC&U has developed the Valid Assessment of Learning in Undergraduate Education (VALUE), a system of direct assessment of student learning that does not rely on standardized tests. Instead it focuses on ways to collect evidence of student learning that include a set of rubrics, expert judgment, student work and electronic portfolios. The Essential Learning Outcomes fit within the Degree Qualifications Profile and are often used as a starting place for institutions and systems to create shared understanding of degree outcomes.
In another effort to define learning outcomes, the U.S. Department of Labor Employment and Training Administration is working with business leaders and educators to create competency models that identify foundational and technical skills and competencies required for workplace success in a variety of critical industries. The competency models can be a resource for developing curriculum, certifications and assessments. The department has created a Competency Model Clearinghouse on its careeronestop.org website that provides a variety of resources related to competency models as well as the career ladders and lattices, which describe vertical and horizontal career pathways in various industries.
The growing number and variety of postsecondary credentials—including degrees, education certificates, occupational licenses, industry certifications, digital badges, and enhanced transcripts—present both opportunities and challenges. They offer students and workers different pathways to postsecondary education and job training, but there are questions about what many of them signify. Some represent acquisition of a clear and well-defined set of knowledge and skills. Others represent completion of programs that may not be well defined, and there is little clarity about their value or quality or how they connect to other programs and credentials.
Connecting Credentials is a collaborative effort spearheaded by Lumina Foundation with more than 80 co-sponsoring organizations working to improve the quality, transparency and connectivity of postsecondary credentials in the United States. As part of that effort, the Corporation for a Skilled Workforce and the Center for Law and Social Policy led an effort to develop a credentials framework that would provide a common set of competencies that would clarify the value of and relationship between the various types of credentials, including degrees as well as certificates, industry certifications, licenses, apprenticeships, badges, and other credentials. The Connecting Credentials framework builds off the Degree Qualifications Profile and other ongoing efforts to define student learning outcomes. It is also meant to support related efforts to reimagine the postsecondary credentialing system in the U.S.
A 2015 Connecting Credentials report from Lumina Foundation discusses these concerns as well as a reimagined system of connected and clearly defined credentials. The report indicates that there is widespread agreement that a new credential system should:
- Be easily understandable and based on competencies;
- Assure quality and represent the acquisition of competencies;
- Be updated and relevant to employer needs;
- Interconnect in ways that show clear pathways to careers and to additional credentials; and
- Enable comparisons as to the value of different credentials for different interests and needs.
In 2013, the National Institute for Learning Outcomes Assessment (NILOA) conducted a national survey about institutions’ current assessment activities and how the institutions were using evidence of student learning outcomes. Provosts or their designees at 1,202 regionally accredited, undergraduate-degree-granting, two- and four-year, public, private, and for-profit institutions in the U.S. provided responses. The survey found that by 2013, about 84 percent of all colleges and universities had adopted stated learning outcomes for all their undergraduates, an increase of 10 percent since 2009. In addition, there is significantly more assessment activity. Much of the impetus for the increase in assessment activity comes from postsecondary accrediting bodies and their changing expectations. However, institutions are increasingly interested in using assessments for internal program review and process improvement. Institutions more frequently report assessment results internally than to external audiences. In 2013, only about a third (35 percent) of campuses made assessment results publically available on their websites or in publications.
In September 2015, the Multi-State Collaborative to Advance Learning Outcomes Assessment (MSC)—led by the Association of American Colleges and Universities (AAC&U) and the State Higher Education Executive Officers (SHEEO)—released results of its pilot study, which gathered data from 59 institutions in nine public state systems. The study demonstrated that rubric-based assessment can be taken to scale and can produce valid findings with credible and actionable information about student learning. As part of the pilot study, researchers gathered more than 7,000 samples of student work produced for course assignments in students’ regular courses. Faculty members received training and then independently scored students’ work using common scoring rubrics—VALUE rubrics—that were developed and validated by faculty as part of AAC&U’s Liberal Education and America’s Promise (LEAP) initiative. The nine states participating in the Pilot Year of the MSC were Connecticut, Indiana, Kentucky, Massachusetts, Minnesota, Missouri, Oregon, Rhode Island, and Utah. The released results also include selected findings about students’ critical thinking, written communication, and quantitative reasoning skills.
Twelve states—Connecticut, Indiana, Hawaii, Kentucky, Maine, Massachusetts, Minnesota, Missouri, Oregon, Rhode Island, Texas, and Utah—will participate in the Demonstration Year phase of the project from September 2015 through August 2016. The work during the demonstration year will build on the efforts and assessments completed during the pilot year and will seek to provide additional information about the feasibility and sustainability of a common statewide model of assessment using actual student work.
Each of the following state examples is a policy solution crafted in response to the unique circumstances of the state in which it was formed. As a private foundation, Lumina does not support or oppose any legislation. Lumina provides educational information, nonpartisan research and analysis to advance Goal 2025.
The University System of Georgia (USG) was one of three universities and systems working with the American Association of State College and Universities (AASCU) to test the feasibility of using the DQP to help transform campuses into learning-centered institutions. More specifically, AASCU sought to support system and institution efforts to use the DQP to strengthen learning outcomes, develop assessments of learning outcomes in program areas, and facilitate transfer between two-year and four-year institutions. Two institutions within USG—Georgia State University (GSU) and Georgia Perimeter College (GPC)—wanted to explore the DQP competencies for associate and bachelor’s degrees to help facilitate transfer between their two institutions and support student success. The DQP effort built on an existing system wide initiative to create a common core curriculum. Specifically, leaders at the two institutions focused on competencies and assessment in three fields that are among the most common for students transferring between the two institutions—biology, psychology and criminal justice. They established a process for building a framework, developing protocols for assessment, testing to assess performance, and evaluating the process. The National Institute for Learning Outcomes Assessment produced a case study of the GSU and GPC effort. The main findings of the case study were that the DQP project helped increase faculty collaboration, expanded emphasis on learning outcomes, and offered opportunities to examine and improve supports for transfer student success.
LEAP States Initiative
The LEAP States Initiative is a multi-state collaborative that works with the LEAP Framework to raise the quality of college learning within and across states by advancing essential learning outcomes, high-impact practices and authentic assessment. AAC&U offers practical assistance, encourages innovation, and provides a national voice for the participating consortia, systems, states, and regions. There are currently 11 participating states: California, Indiana, Kentucky, Massachusetts, North Dakota, Oregon, Texas, Utah, Virginia, Washington, and Wisconsin. Each LEAP state forms its own collaborative at the state or system level, including representatives from two- and four-year and public and private institutions as well as P-16, civic and business leaders. In most of the participating states, a public higher education system or coordinating council provides leadership. Information about the participating entities in each state is available on the AAC&U LEAP States website.
The Indiana Commission for Higher Education’s Strategic Plan—Reaching Higher, Achieving More—was adopted in 2012 and calls for a student-centered, mission-differentiated and workforce-aligned higher education system in Indiana. The plan calls on the higher education community to focus on three key factors—completion, productivity, and quality. More specifically, under quality, the plan calls for institutions to work toward increased public transparency, innovative approaches to education with defined learning outcomes, and quality assessments.
Also in 2012, Indiana lawmakers enacted Senate Enrolled Act 182 (SEA 182), which called for the development of a competency-based Statewide Transfer General Education Core (STGEC) of at least 30 credit hours. A statewide leadership team, with representatives from public two- and four-year institutions, developed a framework and agreed upon six competencies, for which student learning outcomes would be developed. Faculty representatives from each institution met to agree upon the learning outcomes for each competency. Each state educational institution is required to offer a general education program of at least 30 credit hours, which addresses these statewide competencies and the associated learning outcomes. As of May 2013, any student who satisfactorily completes the STGEC at one Indiana institution and transfers to another Indiana institution will not have to repeat any of the core courses.
Since the adoption of the strategic plan and SEA 182, the commission has passed resolutions further demonstrating a commitment to learning outcomes and assessments. Through a resolution adopted in June 2013, Indiana joined the LEAP States initiative, a project led by the Association of American Colleges and Universities—that offers established learning outcomes and degree profiles that show what students should be mastering in higher education. The commission adopted another resolution in 2014 that endorsed recognition of competencies and prior learning.
The Massachusetts Department of Higher Education’s Vision Project provides the state’s public agenda for higher education and includes seven outcomes: college participation, college completion, student learning, workforce alignment, prepared citizens, elimination of disparities, and research. The student learning component includes achieving higher levels of student learning through better assessment and more extensive use of assessment results. The work is focused on strengthening campus-level assessment and finding ways to compare student learning among states. Massachusetts is participating in several initiatives as part of their efforts, including Advancing a Massachusetts Culture of Assessment (AMCOA), the AAC&U LEAP Initiative, and the AAC&U and State Higher Executive Officers-led Multi-State Collaborative to Advance Learning Outcomes Assessment.
AMCOA is a three-part initiative developed by 28 public institutions to help strengthen campus-level assessment, develop a system for statewide learning outcomes assessment, and form partnerships with other states to work together on a statewide assessment of student learning. The AMCOA team consists of faculty members and assessment professionals from each participating institution and has been meeting since 2011. Representatives from 22 public institutions worked together to develop an assessment model using the LEAP VALUE Rubrics for written communication, quantitative literacy, and critical thinking.
As a result of these various efforts within the state, Massachusetts led the effort to establish the Multi-State Collaborative, an effort among nine states, now led by SHEEO, to develop assessments of student learning that do not involve state-mandated standardized tests. The participating institutions—including 14 in Massachusetts—are using AAC&U’s LEAP Essential Learning Outcomes and VALUE rubrics to develop rubrics-based assessments that will allow for the collection of student performance data that can be compared across states and that help faculty assess and improve student learning. Fifty-nine institutions in the nine states completed a pilot study in 2015. Institutions in 12 states, including Massachusetts, have committed to continuing the initiative through a Demonstration Year from September 2015 through August 2016.
Southern New Hampshire University
Southern New Hampshire University (SNHU), a private, nonprofit institution, developed competency-based programs that have required it to define and assess learning outcomes. SNHU Business Administration faculty created a three-year bachelor’s degree program that integrates a competency-based curriculum composed of business and liberal arts requirements. The prescriptive program is divided into modules. Each semester culminates with a weeklong group project through which students demonstrate mastery of concepts and skills. The program has a 79 percent on-time graduation rate, and it saves students 25 percent on tuition and living expenses and saves the university 25 percent on educational delivery expenses.
SNHU also created College for America, an online, competency-based associate degree program in general studies with a business emphasis. The program is based on 120 competencies in nine clusters. To earn a degree, students must demonstrate mastery through a series of task-based assessments. Trained evaluators provide feedback within 48 hours, and students can revise and resubmit the tasks until they achieve mastery. The program costs students $2,500 per year and is self-paced and self-directed. Employer partners provide some tuition assistance, and the program is approved for Title IV federal financial aid.
- The Degree Qualifications Profile, Lumina Foundation
- The Degree Qualifications Profile: A Learning-Centered Framework for What College Graduates Should Know and Be Able to Do to Earn the Associate, Bachelor’s or Master’s Degree, Lumina Foundation, 2014
- The Degree Qualifications Profile: Case Studies of Institutions Using DQP and/or Tuning, National Institute for Learning Outcomes Assessment
- Examples of Good Assessment Practice, National Institute for Learning Outcomes Assessment
- Excellence in Assessment Designation, National Institute for Learning Outcomes Assessment
- Jamie P. Merisotis, President/CEO, Lumina Foundation, Opening keynote, CIC/DQP Consortium meeting, Indianapolis, August 1, 2013
- Leaders & Laggards: A State-by-State Report Card on Public Postsecondary Education, Transparency & Accountability (including measures of student learning outcomes), U.S. Chamber of Commerce, 2012
- The Lumina Degree Qualifications Profile: Implications for Assessment, National Institute for Learning Outcomes Assessment, 2013
- Scaling “Stackable Credentials”: Implications for Policy and Assessment, Center for Postsecondary and Academic Success at the Center for Law and Social Policy, March 2014
- Southern New Hampshire University: Integrated Competency-Based 3-Year Business Administration Degree, HCM Strategists
- Transparency Framework, National Institute for Learning Outcomes Assessment
- Voluntary Framework of Accountability
- Voluntary System of Accountability