The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in the following area: Academic
and Student Support Services.


Compliance Judgment:  In compliance

Narrative

Through ongoing assessment and analysis of learning outcomes of every aspect of academic and student support services, documentation of the analysis of assessment results to inform improvements in services, and institution-wide recognition of improvements through the ongoing use of assessment results, East Carolina University (hereafter, ECU or the university or the institution) is in compliance with SACS Comprehensive Standard 3.3.1.3.

This narrative includes sections that describe an ongoing assessment process inclusive of the following areas:

 

I.                     Organizational Approach

II.                  Academic and Student Support Services Compliance

III.                Improvements to Academic and Student Support Services Based on Assessment Results

IV.                Program Review

V.                  Alignment to ECU’s Strategic Directions

VI.                Continuous Improvement of the Assessment Process

 

Evidence of compliance is summarized in several tables which are introduced and fully elaborated upon in designated sections provided in the narrative:

 

Table 1: Assessment Units and Reports - Academic and Student Support Services

Table 2: Assessment Methods for Academic and Student Support Services

Table 3: Summary of Use of Results

Table 4: Alignment of Assessment Units to Strategic Directions

Table 5: Strategic Directions Stories

 

I.                     Organizational Approach


In keeping with its role as the fastest growing university in the UNC system and its emergence as a comprehensive doctoral institution, East Carolina University offers a broad array of academic and student support services. These services promote academic achievement as well as provide co-curricular and extracurricular engagement opportunities leading to a well-rounded experience for students. Each of the academic and student support services units at ECU establishes goals and outcomes, systematically assesses the extent to which these are attained, and uses the results to improve as described in detail below.

 

As shown in the ECU organizational chart, East Carolina University is organized into six major divisions (Academic Affairs, Student Affairs, Health Sciences, Research and Graduate Studies, University Advancement, and Administration and Finance). Each unit is led by a vice chancellor. Most academic and student support services are provided by the Divisions of Academic Affairs and Student Affairs, with a smaller number of units situated in the Divisions of Health Sciences, and Research and Graduate Studies. For each of these major university divisions, the process of setting goals, outcomes and measures, assessing progress and improving is institutionally standardized and described in detail in the Core Requirement 2.5 narrative.

 

Within each of these major divisions, the specific academic and student support services (ASSS) are provided by colleges or departments within those units. Each of these many units has a unique mission and goals contributing to attainment of East Carolina’s five strategic directions.

 

As described in the report for SACS CoreRequirement2.10, ECU provides academic and student support programs, services, and activities consistent with its mission (especially as articulated in ECU’s ECU's strategic directions) that promote student learning and enhance the development of its students. While individual units collected and analyzed data about support activities prior to 2009, there was no organized institution-wide assessment process. Formal assessment of academic and student support services began during the 2009-2010 academic year. While many units were easily classified as academic and student support service units, identification of such units embedded within colleges or administrative units with multiple purposes was an ongoing process through November 2011.  Housed in four divisions across the institution, 73 units have been designated as Academic and Student Support Services.

 

East Carolina University embraces a broad-based assessment process for evaluating institutional effectiveness by requiring each academic and student support services unit to define outcomes, perform an assessment of its programs or operations, and report results and improvements from this self-assessment on an annual cycle in TracDat, the institutional tracking system.  An annual Overview of Assessment Planning and Reporting document helped guide the work of units in developing and reporting assessment results and their use in improving services. ECU’s Academic and Student Support Services Working Group was established in September of 2010 to provide feedback to individual units with assessment plan design and implementation suggestions, and to evaluate the uses of assessment to improve performance.  In particular, attention is directed toward development of meaningful and measurable outcomes with corresponding measures and criteria for success. The assessment report feedback also includes a review of results and how those results are used for action steps such as decisions and recommendations that lead to improvements contributing to institutional effectiveness. The Academic and Student Support Services Working Group report its assessment activities through the Institutional Effectiveness Executive Council.  The Academic and Student Support Services Working Group is purposefully comprised of wide-ranging representation from varied campus units including those that are directly engaged in support programs, services, and activities.

 

The Academic and Student Support Services Working Group piloted a formal review of Student Affairs in 2009-2010 assessment reports in April 2011. Review rubrics and verbal feedback were shared with respective units by the chair of the working group in May 2011. During the summer of 2011, members of the working group evaluated the review protocol in coordination with the Administrative Support Services Working Group in order to build a sustainable process as well as to provide timely and useful feedback. Two members of the Academic and Student Support Services working group are assigned to each assessment report to evaluate the annual report utilizing the Rubric to Evaluate Unit Assessment Reports . These peer reviews are transformed into the Review of Assessment Plan Memorandum by the chair of the Academic and Student Support Services Working Group and sent to the respective unit representatives. Reviews of 2010-2011 unit assessment reports were initiated in October 2011 and completed by February 2012.

In addition to an Overview of Assessment Reporting, a set of question, called the ECU Assessment Plan and Report Quick Reference Guide for Support Units, was developed in fall 2011 to assist Academic and Student Support Services staff in their reporting efforts.  The questions were designed to provide a consistent approach to each component of the assessment report.  In the results and actions taken areas of the tracking system, units were encouraged to report their data collection findings along with the decisions they had made based upon the data.  When quality enhancement efforts were made, staff reported at a later date on the results of improvements or action steps as a follow-up.

II.                  Academic and Student Support Services Compliance

 

The 73 academic and student support services units are situated in four major university divisions as follows:

Division of Academic Affairs                                        = 36

Division of Health Sciences                                          =  9

Division of Research & Graduate Studies               =  1

Division of Student Affairs                                           = 27


Table 1: Assessment Unit
s and Reports- Academic and Student Support Services
identifies the Academic and Student Support Services Units at ECU and provides a link to each assessment report.

 

The Academic and Student Support Services audit report reveals the commitment to building a systemic culture of evidence and improvement. As of June 2012, the audit indicates that 100% (73) of the units possess assessment plans involving 408 outcomes and 602 measures. Each unit has chosen outcomes to measure based on their functions within ECU. In the institutional tracking system, academic and student support services staff have a list of six methods categories from which to choose as follows: annual reports, focus groups, interviews, observations, professional development, and surveys. The category selections are further explained by units with respect to the specific measurement approach and criteria for success. These approaches, in the collective, represent a broad range of assessment methods as illustrated in Table 2: Assessment Methods for Academic and Student Support Services.

 

Units that deliver services such as marketing products (SA Marketing), technology (SA Technology Services) support, and transportation (Student Transit) track service requests or user numbers and conduct client surveys with respect to satisfaction. Several of the methods are embedded within the university system of accounting and regulation, e.g. UNC Finance Improvement and Transformation (FIT) (Financial Aid), federal financial aid audits (Financial Aid), and university budget compliance regulations (SA Accounting). In other cases, inventories or reports are mandated by government regulations compliance or accrediting agencies (e.g. Disability Support Services and Federal Americans with Disabilities Act or Student Health and Accreditation Association for Ambulatory Health Care). Units involved in program delivery supporting student learning and development outcomes, (e.g., Campus Recreation and Wellness or Student Involvement and Leadership units, frequently employ observation of activities utilizing rubrics (demonstration of learning/development), review institutional data sets (grade point average, retention, graduation rates), and engage students in focused conversations (exploratory interviews and focus group). 

 

III.                Improvements to Academic and Student Support Services Based on Assessment Results

 

As highlighted previously, each academic and student support services unit engages in unit level processes to establish goals and outcomes, assess progress, and improve as needed. At the completion of each annual assessment cycle, units report on results for each outcome area and then pursue corresponding improvement actions, thereby “closing the loop” on annual assessment plans. In many cases, this closing the loop process serves to provide the direction or rationale for the following year’s assessment plan.  The assessment plan timeline is structured around an October 1 deadline for input of outcomes, measures, and criteria of success into TracDat followed by the annual reporting of results and actions deadline on August 1. Follow-up reporting typically occurs between the August 1 end of annual cycle reporting and finalizing the following year’s assessment cycle plan on October 1. 

 

Academic and Student Support Services units at East Carolina University have always prized accountability and improvement. Whether meeting enrollment or retention goals, increasing levels of student engagement, successfully innovating service delivery, or exceeding established student learning and development outcomes, academic and student support services offices met goals or actively worked to improve processes directed at attainment of intended outcomes. The Academic and Student Support Services audit report indicates that the 832 assessment results led to 853 actions taken to improve the academic and student support services enterprise at ECU. A representative sampling of how units in each of these major divisions has made improvements to their programs and services based on assessment results is provided in Table 3: Summary of Use of Results for Academic and Student Support Services Assessment Units.  The sample is representative based on the inclusion of assessment units from each of the four major Divisions in which academic and student support services are situated. The units selected represent the core academic and student support services functions at ECU.

 

ECU also participates in the National Survey of Student Engagement (NSSE), a widely used instrument that measures the degree of involvement of college undergraduates in a wide range of activities and experiences during their freshmen and senior college years. Specifically related to comprehensive standard 3.3.1.3, ECU has used the NSSE Toolkit to summarize NSSE results by Domain and Year related to Academic and Student Support Services with comparison to results from public institutions in the Southeast U.S. and from institutions of the same Carnegie Class. The report, NSSE Results by Domain and Year Related to Academic/Student Support Services and Institutional Effectiveness (SACS Principle 3.3.1.3) summarizes multi-year results of ECU first-year students and seniors who participated in the National Survey of Student Engagement (NSSE) in terms of experiences related to academic and student support services recognized as "good practices" in undergraduate education as the degree of engagement in such college activities and programs is one predictor of outcomes such as learning, personal development, and retention. Notably and affirmatively, the findings suggest that a majority of students have found ECU to provide the support needed to succeed academically throughout their college experience. Similarly, the majority of students consistently evaluate the quality of academic advising received as good or excellent throughout their college experience. The report also suggests that attention is warranted in connecting students and faculty outside of the classroom in enriching educational experiences as well as greater engagement in cultural or arts focused experiences. The NSSE serves as another important institutional assessment resource to measure and monitor student outcomes specific to academic and student support services.

 

IV.                Program Review

 

In addition to documented use of results of outcomes assessment in each academic and student support service area, programs in the Divisions of Academic Affairs and Student Affairs undergo program review in keeping with functional area standards outlined by the Council for the Advancement of Standards in Higher Education (CAS).  Structured non-academic reviews in the Division of Academic Affairs are managed by the Office of Academic Program Development and in the Division of Student Affairs by the Office for Assessment, Research, and Retention using the ECU Program Review Guidelines for Administrative and Support Units format since 2008.

Between 2008-2010, 27 Academic and Student Support Services units (12 in Academic Affairs and 15 in Student Affairs) completed program reviews. Some examples of improvements made based on the review are:

       In order to reach faculty and let them know of program offerings more effectively, the Office of Faculty Excellence revised their website, developed a systematic way of providing programming reminders weekly, and developed a new registration system.

       The North Carolina Early Math Placement Program initiated a study to examine the effects of program participation on remedial mathematics rates at NC colleges and universities.

       The Office of Academic Program Development through the Academic Program Development Collaborative Team consulted with the College of Business to develop an electronic budget template to be completed and submitted with degree program proposals.

       The Office of Catalog and Curriculum Development developed an email communication schedule to notify faculty, deans and department chairs when curricular changes receive official approval.

       A Director of Assessment for the Division of Student Affairs was hired, thereby facilitating development of comprehensive and coordinated assessment strategies in departmental units.

       Organizational analysis and restructuring in numerous units in Student Involvement and Leadership has facilitated new positions and hiring processes, thereby advancing capacity of units such as the Center for Student Leadership and Engagement and the Volunteer and Service-Learning Center.

       Significant master planning processes have been initiated and progress is underway for new student support services facilities on the Health Sciences campus as well as the East campus, which will involve Campus Recreation and Wellness, Student Involvement and Leadership, Campus Dining, and Dean of Student units such as Counseling and Student Develop. These new facilities will support the ongoing student population growth and diverse needs.

       Transfer student orientation, community college outreach (ECU Extends program) and prioritized resources have been directed toward enhancing recruitment, retention, and graduation support for transfer students.

A complete summary of improvements is included in each of the program review reports for the Divisions of Academic Affairs (Office of Faculty Excellence, North Carolina Early Math Placement Program, Office of Academic Program Development, Office of Catalog and Curriculum Development) and the Summary-Improvement Progress Report for 2009-2012 of external program reviews in the Division of Student Affairs.

V.      Alignment to ECU’s Strategic Directions

 

Academic and student support services units represent a diverse range of programs, services, and activities embedded in four university divisions. The common bond between all of these units is their commitment to supporting student success and contributing to achievement of ECU’s five strategic directions. Table 4: Alignment of Assessment Units to Strategic Directions – Academic and Student Support Services Units demonstrates how individual ASSS units support the university’s five strategic directions. A sampling of alignment and contributions to the five strategic directions is personified by stories generated by ECU’s News Services, found in Table 5: Strategic Directions Stories – Academic and Student Support Services Units.

VI.                Continuous Improvement of the Assessment Process

One issue emerging from the comprehensive examination of assessment plans, recording of results, and use of results to improve academic and student support services at ECU involves the quality of the overall intended outcomes integrated through the thoughtful use of results. For most of these units, 2009-2010 was the first year for a formal comprehensive assessment cycle that was outcomes driven. Prior to implementing an institution-wide approach, most units were goal-oriented in their evaluative processes. With experience, the quality of intended outcomes, of corresponding assessment plans, and the effective use of results has continued to improve. The Division of Student Affairs piloted a meta-review of unit assessment reports using a Meta-Assessment Criteria Checklist in summer 2012 to build a sustainable and criteria specific approach to improving the unit assessment process. Furthermore, implementation of the meta-self review is intended to support growth in assessment leadership and elevation of staff knowledge and empowerment with respect to co-creating a culture of evidence and improvement. The process was implemented as part of a divisional “closing the loop” meeting, Celebrating and Using Assessment Results for Improvement, thereby building an authentic assessment process as well. Staff members representing Institutional Planning, Assessment and Research also participated in the “closing the loop” meeting and meta-review process in support of building an institution-wide model.

 

Conclusion

East Carolina University has developed a culture of institutional effectiveness that is data-driven, systematic, and broad-based. A comprehensive institutional assessment audit completed in June 2012 illustrates the progression of systemic assessment development through the specific number and percentage of Academic and Student Support Services units completing reports between 2008 and 2012. The continuation of an institution-wide assessment cycle by units accompanied by completion of a peer review process and reporting audits on a regular, systematic basis ensure that new institutional units will automatically be included in institutional effectiveness efforts.

 

Documentation

 

Reference Title

Location

Academic and Student Support Services Audit

Audit ASSS June 2012

Academic and Student Support Services Working Group

Academic and Student Support Services Working Group  March 2012

Academic Program Development

Non-Academic Program Review

Carnegie Class

ECU NSSE 2012 Carnegie Class

Celebrating and Using Assessment Results for Improvement

Student Affairs closing the loop july 17 2012 powerpoint

Core Requirement 2.5

CoreRequirement2.5

Core Requirement 2.10

CoreRequirement2.10

Council for the Advancement of Standards in Higher Education (CAS)

CAS_Standards

 

ECU Assessment Plan and Report Quick Reference Guide for Support Units

Reference Guide Support Units

ECU Organizational Chart

ECU

ECU Program Review Guidelines for Administrative and Support Units

Non-Academic-Program-Review-Format

ECU’s Strategic Directions

ECU Tomorrow_ Our Strategic Directions

Meta-Assessment Criteria Checklist

Student Affairs Assessment Meta-Review Form

NSSE Results by Domain and Year Related to Academic/ Student Support Services and Institutional Effectiveness (SACS Principle 3.3.1.3)

NSSEbySACSprinc-3-3-1-3-(2012)

North Carolina Early Math Placement Program Review Report

Early Math external review and follow up

Office for Assessment, Research, and Retention

Office for Assessment, Research, and Retention

Office of Academic Program Development Program Review

APD external review and follow up

Office of Catalog and Curriculum Development Program Review

Catalog and Curriculum Dev external review and follow up

Office of Faculty Excellence Program Review Report

OFE external review and follow up

Overview of Assessment Planning and Reporting

Overview of Non-Academic Assessment Reporting for 2009-10

Review of Assessment Plan Memorandum

Academic and Student Support Services feedback template

Rubric to Evaluate Unit Assessment Reports

Rubric - Academic and Student Support Services (2)

Southeast U.S. Public Institutions

ECU NSSE 2012 Southeast Public

Summary-Improvement Progress Report for 2009-2012

External Review of Student Affairs Programs

Summary Progress Reports for Program Review in Student Affairs

SA_Program_Review_Progress_Report_20092010_Summary_Form1 (4)

Table 1: Assessment Units and Reports  -Academic and Student Support Services

Assessment Units Academic and Student Support Services

Table 2: Assessment Methods for Academic and Student Support Services

 

Assessment Methods ASSS April 27 2012

Table 3: Summary of Use of Results for Academic and Student Support Services Assessment Units

Summary of Use of Results Academic and Student Support Services Assessment Units

Table 4: Alignment of Assessment Units to Strategic Directions – Academic and Student Support Services Units

Alignment of Assessment Units to Strategic Directions - Academic and Student Support Services

Table 5: Strategic Directions Stories – Academic and Student Support Services Units

Strategic Directions Stories - Academic and Student Support Services