Undergraduate Program Review Policy and Procedures

Policy Category: 
Academic
Approved By: 
Senate
Effective Date: 
March 24, 1999 [Revised Spring 2000;23 September 2005]


TABLE OF CONTENTS


Objectives
The primary objective of Lakehead University's undergraduate program review process is to assess the quality of undergraduate programs at the University. The program reviews are intended to improve academic programs and to demonstrate accountability to prospective students and the public. The review process is designed to ensure objective and constructive assessments of the University's undergraduate programs, and to meet the following additional objectives:
        to evaluate the curricular and pedagogical policies and practices of the academic unit offering the program
        to recommend improvements regarding the administrative efficiency of the academic unit offering the program
        to ensure that programs are current with respect to developments in the discipline
        to identify areas of existing and emerging strengths and to identify areas of weakness or concern
        to assist the academic unit in future planning by clarifying academic objectives and the service provided to students.
 
Scope and Cycle of Reviews
The reviews are limited to undergraduate programs. The definition of a program follows that provided in the MET Operating Funds Distribution Manual: A YA program is defined as a sequence of courses or other units of study prescribed by an institution for the fulfilment of the requirements of a particular degree, diploma or certificate.  Programs are not necessarily co-terminus with academic organizational units, and joint and multidisciplinary programs will be reviewed in a way appropriate for Lakehead.
In practice, all of the undergraduate programs offered through an academic unit (used here to mean a Department, a School or a Faculty) will be reviewed at the same time.   The Vice-President (Academic) and Provost will ensure all programs listed under a unit will be reviewed.
In consultation with the appropriate Dean and the Program Coordinators of each Interdisciplinary Program and Access Programs, the Vice-President (Academic) and Provost will ensure that all interdisciplinary programs and access programs are reviewed. 
The full set of Lakehead's undergraduate programs will be reviewed over a ten-year cycle. Based on the information provided to all Ontario universities, the provincial Undergraduate Program Review Audit Committee's (UPRAC) cycle of audits shall not exceed seven (7) years. Lakehead University's first audit is scheduled for the Fall of 2000.
With respect to programs that might be developed in the future, the University will provide to the Auditors documentation on the institutional process for the implementation of new undergraduate programs following the UPRAC guidelines.

Externally Accredited Programs
Programs which undergo an external accreditation may wish to substitute
this accreditation process for the undergraduate program review of this
policy. This may be acceptable providing the university's own review
requirements are met.  Before the accreditation review takes place, the
Senate Academic Committee (or a standing subcommittee) will be provided
with a copy of the accreditation review template to compare with the
requirements of this policy. The committee will decide:

(i)  the accreditation review will be accepted as meeting all the criteria
for an internal review.  The program's accreditation documents and the
accreditation review reports will be submitted to SAC (or its standing
subcommittee, or
(ii) the accreditation review will be accepted as meeting most of the
criteria for an internal review.  The Senate Academic Committee (or its
standing subcommittee) will recommend some supplementary information to be submitted along with the program accreditation documents and the
accreditation review reports.  Depending on the nature and amount of
supplementary material needed, the Senate Academic Committee may establish a committee consisting of 3 internal reviewers, to receive the material and the reports, or
(iii) the accreditation process will not be sufficient to meet the
requirements of an internal review, and the process covered by this
policy, including external reviewers shall be followed.


Orientation and Review Stages
Each review will include the following six stages:
 
1.     A yearly meeting of the Faculty Deans and the Vice-President (Academic) and Provost, followed by meetings of the Faculty Deans and departments/units whose programs are to be reviewed.  Coordinating responsibility: Vice-President (Academic) and Provost in consultation with Deans / Deans in consultation with Chairs/Directors;
 
2.     A student survey to be administered by the Office of Institutional Analysis, in the year prior to each review - Appendix D.  Co-ordinating responsibility: Vice-President (Academic) and Provost in consultation with Office of Institutional Analysis.
         
  1. A self-appraisal of the program by faculty and students participating in the program.  Specific attention should be drawn to the introspective, creative, self-critical, and formative aspects of a quality review, as opposed to establishment of professional competence used to evaluate a review for accreditation purposes.   Co-ordinating responsibility: Dean in consultation with Chair/Director;
 
  1. Visits, evaluations and written reviews by two external reviewers and one internal reviewer.  Co-ordinating responsibility: Vice-President (Academic) and Provost, in consultation with Dean;
 
  1. A response to the reviews by the academic unit.  Co-ordinating responsibility: Dean in consultation with Chair/Director;
 
6.     A summary report (excluding all personal information) written by the Vice-President (Academic) and Provost summarizing the outcomes, findings and conclusions of the reviews. This summary report will be presented to the Senate and the Board of Governors of the University.
 
Authority Responsible for the Application of the Policy
The authority responsible for the application of the policy is the Vice-President (Academic) and Provost and the costs associated with the reviews will be covered by that office. The Vice-President, in consultation with the Faculty Deans, will prepare a schedule for the periodic review of all undergraduate programs in each faculty.
The Director of Institutional Analysis and Government Relations will provide the required institutional information and statistical data to conduct the reviews, and will aid in formulating the presentation of the material.
For academic units which are subject to other reviews, it is recommended that where feasible, the undergraduate review be conducted in the same year as these reviews. Also, an appropriate interval should occur between the undergraduate review and the other appraisal. In addition, the academic unit will be advised by the Vice-President (Academic) and Provost on the distinctions between the objectives of the undergraduate reviews and other appraisals to which it might be subject.


Procedures
Time-Lines and Identification of Reviewers
The Vice-President (Academic) and Provost will remind each academic unit by September 1 that a Self-Appraisal Report is due by June 30.
By June 30, the academic unit, using the form for the recommendation of consultants (see Appendix B), will submit to the Senate Academic Committee (SAC) through the Vice-President (Academic), the following:
  1. A list of at least six (6) Associate or Full Professors employed in other Universities who might serve as reviewers. The potential reviewers must not have any past or current formal affiliation with the unit or with members of the unit (supervisors/supervisees, co-authors, relatives, etc.);
 
  1. A list of four (4) tenured Associate or Full Professors at Lakehead from outside the academic unit whose programs are being reviewed.
 
Using the two lists of proposed reviewers, the standing sub-committee of SAC will select two external reviewers and one internal reviewer to evaluate the programs in the academic unit. All contact with the proposed reviewers will be by the Vice-President (Academic) and Provost or a delegate identified by the Vice-President (Academic) and Provost.  September 30.
 
 
STAGE I: SELF-APPRAISAL REPORT
The self-appraisal is intended to provide an objective assessment of past achievements, present strengths and weaknesses, and future plans of the academic unit and its programs.
The Self-Appraisal Report will contain three volumes. The information to be contained in each volume is indicated below. The report is to be submitted to the Vice-President (Academic) and Provost by June 30.
The Deans will have an opportunity to comment on the self-appraisal before it goes to the reviewers and may direct issues of their own to the reviewers.
Volume One
Academic Unit and Programs
       a brief history of the major academic achievements of the academic unit
       the academic objectives and mission of the academic unit
       the contributions of the academic unit to the mission, strategic plan and academic plan of the University
       recent or future changes in the academic objectives and mission of the academic unit
       recent and possible future curriculum changes (include strategies and schedule for future changes)
       the amount of service teaching for other academic units
       the curricular links between undergraduate and graduate programs (if graduate programs exist in the unit)
       the reports of recent accreditation or professional reviews, if appropriate
       a description of the administrative structures and processes within the unit (standing committees, ad hoc committees, how members are appointed, frequency of meetings)
       an assessment of the effectiveness of the administrative structures and processes
       evidence of the quality of teaching and advising in the unit
       evidence of research and scholarly activity in the unit
       the use of innovative or unique teaching programs and techniques
       the type and amount of professional service provided to the profession, the discipline of the community (as applicable)
       a summary of the strengths and weaknesses of the unit with respect to teaching, scholarly activity and service by the faculty.
       the results of the student survey and response of the unit.  See Appendix D - Undergraduate Program Review Student Survey
       where appropriate, the advice of others, such as representatives of industry, professional and practical training programs.
Students in the Programs
       the f.t.e. enrolment patterns over the past 7 years
       the average enrolment in honours and 3 year programs over the past 7 years
       the course registrations per f.t.e. faculty member
       the average class size for each year in the programs over the past 7 years
       the number of courses with labs or tutorials, by year level
       the requirements for an honours thesis and the number of students who have completed a thesis over the past 7 years
       an assessment of the quality of students in the programs. It is recognized that some indicators may vary across academic units. Indicators may include,
admission averages
average required to continue in the program
the rate of internal and number of external entrance scholarships per year
the rate and number of scholarships earned by continuing students
degree completion rates
time to completion (for full-time students)
number of students going on to second entry professional and graduate programs
number of co-op student placements
involvement of undergraduate students in research and scholarly activity.
awards received
 
  • academic units are invited to submit, where possible, data on the accomplishments and employment success of past graduates.
Program Support
       the number of support staff, and their roles and responsibilities
       the availability and use of physical resources for teaching and research
       Library support. The Chief Librarian will provide a report on the relevant information resources for the programs under review.
 
Analytical Assessment of the Program
In addition to the more quantitative aspects of the self-appraisal, this component of the program review is more introspective and self-examining, and should identify the strengths, weaknesses and needs of the program.  Specific questions to be addressed include:
1.     What are we trying to achieve with our curriculum and requirements?
2.     What standards are in place to assess our performance?
3.     How well are we performing with respect to those standards?
4.     What do we propose to do about whatever deficiencies we identify?
5.     How effectively have we addressed previously identified deficiencies?
Besides the questions listed above, the questions posed to the reviewers can provide a further evaluative assessment of the program under review.  Academic units are advised to refer to these questions listed in Stage II of the review policy, and to incorporate into this section of the self-appraisal their responses to those questions which best capture the qualitative aspects of the program.
Supplementary
       the academic unit should note any advice or directions sought from the reviewers.
Volume Two
This volume includes the most recent course outlines for the courses listed in the calendar (including objectives, assigned texts and readings, and the evaluation procedures) for the academic unit. The course outlines should indicate the last term in which the course was taught, the instructor and the enrolment.
Volume Three
This volume includes a current CV for each full-time member of the academic unit, using the CV format included in Appendix C of this policy. The CVs of part-time faculty members and adjuncts who contribute to the teaching and/or thesis supervision in the academic unit should be included as well.
 
STAGE II: REPORTS OF THE REVIEWERS
The Reviewers will review the Self-Appraisal Report submitted by the academic unit and may request any additional information through the Vice-President (Academic) and Provost. The reviewers will spend one or two days visiting the academic unit, and will meet with students, faculty and staff within the unit. In addition, the reviewers will meet with the Vice-President (Academic) and Provost, the Dean and Chair/Director responsible for the academic unit, the Dean of Graduate Studies and Research, the Chairs/ Directors of any collateral units (for joint or inter-departmental programs), as well as any member of the university that can provide pertinent information (e. g. the Chief Librarian, the Director of Computing Services, the Director of Co-op programming). Observations of selected classes or labs or seminars will be arranged, as will a tour of the Library if desired by the reviewers.
It is understood that with increased financial resources, many pressures and problem areas could be mitigated or eliminated. While the University continues to explore methods to increase external funding and to re-allocate funds internally, the reviewers are asked to formulate recommendations primarily in the context of the existing budgets of academic units.
The reviewers will submit their reports to the Vice-President (Academic) and Provost within six (6) weeks following the site visits. The reports will remain confidential to the Vice-President (Academic) and Provost, the relevant Dean and to the Chair/Director of the academic unit.  There shall be remuneration.
The following questions are intended to provide a general framework for the reviewers. The questions may be supplemented by others deemed appropriate by the reviewers for the program under review.
Program
  1. Is the program consistent with the general objectives of the Mission, Strategic Plan, and Academic Plan of Lakehead University?
 
  1. Does the program have clear educational objectives that are consistent with the standards, educational goals, and learning objectives of the degree?  Are the plans of the academic unit reasonably calculated to meet these standards, goals and objectives?
 
3.  Does the academic unit have effective and appropriate mechanisms in place for curriculum review and change?
 
4.  Are the program's structure and curriculum appropriate for meeting its learning objectives? Are the course offerings appropriate? Is there duplication? Are there courses in the program that should not be offered, or delivered differently? Are required courses offered regularly? Do other courses cycle on a regular and appropriate basis? Is there an appropriate balance between introductory and specialized courses?
 
5.  Requirements: Are three and four-year degree requirements appropriate to the program and the academic unit? Are requirements overly complex? Are the rationales for requirements presented intelligibly to students? Are the limitations on courses that can be taken outside of the academic unit appropriate?
 
6.  Are the modes of delivery employed by the program (including, where applicable, distance or online delivery) appropriate and effective in meeting the program's learning objectives?  Is there an appropriate balance between large lecture courses and smaller tutorials and seminars? Should (could) large section courses be divided into smaller sections, and should (could) the opposite be affected? Is the academic unit incorporating innovation in teaching/learning delivery? Are the non-lecture components of programs (labs, fieldwork, etc.) appropriate?
 
7.  How has the academic unit utilized technological and pedagogical innovation in its programs and imparted an international perspective?
 
8.  Are there curriculum areas that should be developed in the program?
 
  1. In reviewing the curriculum and patterns of course offerings, do you detect any areas in which offerings could be curtailed or modified in order to release faculty time?
 
Students
  1. Are the admission requirements appropriate and effective (i.e., do they adequately prepare students for the successful achievement of the program's learning objectives?)
 
  1. Are the applications for and registrations in the program consistent with the discipline in general?

 
  1. Is the level of achievement of students consistent with the educational goals for the program and the degree?  Is this level consistent with institutional standards?  Is this level sufficient to enable national and international student mobility?
 
  1. Are the time-to-graduation and graduation rates appropriate?
 
5.  Is the evaluation of student progress in the program appropriate?  (Are the methods used for the evaluation of student progress effective?  Does the program review its methods of evaluation to measure effectiveness?)
 
6.  Is the academic advising function in the academic unit sufficient and appropriate?
 
7.   Should enrolments be maintained, reduced or increased? If increased, how?
 
Faculty
  1. Is the match between faculty expertise and the curriculum as designed appropriate?
 
  1. Is the balance between full and part-time instructors appropriate?
 
  1. Are there immediate staffing needs in the academic unit?
 
  1. Are the mechanisms in place for conducting teaching evaluations, receiving feedback on them and reacting to them appropriate?
 
  1. What does the academic unit do to provide teaching support and professional support for faculty?  Are faculty provided adequate professional support for developing standards, educational goals, and learning objectives for their courses and programs?

 
  1. Is there a reasonable balance among teaching, research and university governance in the academic unit?
 
  1. Taking into account the faculty members' teaching, research and service responsibilities, how would you assess the workloads in the academic unit?
 
  1. With particular reference to teaching and research, what is your assessment of the quality of the faculty in the academic unit?
 
Resources
  1. How adequate is teaching and research support (for example, secretarial staff, computing support, technicians laboratory facilities and equipment, field equipment, for example) in this academic unit?
 
  1. How adequate is the library collection? Are there appropriate library services for the needs of the programs and the academic unit?
 
  1. Are the existing personnel and financial resources available to the academic unit being utilized appropriately and effectively to support and develop the educational goals and learning objectives of the program.
 
4.  Are there potential opportunities available to the academic unit for sharing resources with other units?
 
5.  Is the faculty complement sufficient to deliver the program?
 
 
Concluding Questions
What are the primary strengths and weaknesses of the programs under review? What are the most important steps the academic unit could take to improve the quality of its programs?
 
Recommendations
The reviewers are asked to formulate recommendations based on their findings.
 
 
STAGE III: RESPONSE TO THE REVIEWS
Within one month of receiving the reports of the three reviewers, the academic unit will submit a response to the Vice-President (Academic) and Provost. The response might include:
       Clarifications or corrections of statements in the reports of the reviewers.
 
       Whether and why the academic unit agrees or disagrees with the comments of the reviewers and their recommendations. How recommendations might be implemented should be indicated, as should recommendations that may not be considered for implementation, with attached rationales.
The Dean of the Faculty, in consultation with the appropriate Chair/Director, in which the program resides, shall be responsible for monitoring and implementation of accepted recommendations of reviewers.  Where a program straddles two or more Faculties, the responsibility will be with the Deans of the Faculties involved.
STAGE IV: VICE-PRESIDENT'S SUMMARY REPORT
A report that summarizes the findings and conclusions of the reviewers and the response of the academic unit will be prepared for the Senate Academic Committee by the Vice-President (Academic) and Provost in consultation with the Dean and the Chair/Director. The summary support will be generic in character, and will identify the general strengths and weaknesses in the program. Personal details and information will not be included. The report will include a statement of the strengths and weaknesses of the program, and the actions to be taken on the recommendations arising from the review. The summary report will be presented to the Senate and to the Board of Governors.
At the end of each yearly cycle of reviews, a standing sub-committee of the Senate Academic Committee will report to the Senate Academic Committee on the general effectiveness of Lakehead's review process with respect to reviews carried out that year.  The Vice- President (Academic) and Provost will be an ex-officio member of the standing sub-committee.  Suggested changes will be forwarded to the Senate Academic Committee for discussion, and on to Senate should changes be recommended by the Senate Academic Committee. 
Appendices
A. Summary of Schedule Dates
B. Resume for Proposed Reviewers/Consultants
C. CV Format
D. Student Self-Appraisal 
APPENDIX A
Summary of Schedule
Stage I
The Vice-President (Academic) and Provost reminds the academic unit of the forthcoming review. September 1
The academic unit submits Volumes One, Two and Three of the Self-Appraisal Report to the Vice-President (Academic) and Provost. June 30
The academic unit submits the names of the proposed external and internal reviewers to the Vice-President (Academic) and Provost. June 30
The Senate Academic Committee selects the external and internal reviewers, who are then invited to serve by the Vice-President (Academic) and Provost. September 30
Stage II
The reviewers visit the academic unit. The reviewers submit their reports within 6 weeks of the site visits. October 15-March 30
Stage III
The academic unit submits a response to the reviews to the Vice-President (Academic) and Provost. Within one month of following receipt of reviewers' reports.
Stage IV
The Vice-President (Academic) and Provost submits an annual report to Senate and the Board of Governors on the process, the reviews completed in the previous cycle, and summary reports on the individual reviews.  October - December.
 
  
APPENDIX B
NOTE:
The unit is requested not to approach consultants. The following information is to be supplied from personal knowledge or biographical sources.
 
PROGRAMME
1. NAME OF PROPOSED CONSULTANT:
 
2. RANK:
 
3. INSTITUTION:
 
4. DEGREES UNIVERSITY DISCIPLINE DATE
5. AREAS(S) OF SPECIALIZATION: (this should be related to those offered by the unit being reviewed)
6. Experience/expertise relevant to serving as a consultant (e.g. membership on editorial boards, administrative expertise, academic recognition).
7. Recent scholarly activity (cite 3 to 5 recent publications giving title, date, publication, and publisher, if a book).
9.     Previous affiliation with LU, if any, (e.g. visiting professor - give dates, internal consultant, former employee, any supervisor/student relationships with faculty members).

 
 
 
APPENDIX C
NOTE:
To be presented, in alphabetical order, in the same format for all faculty listed. Each CV should include, where relevant, the following information, organized in the following order:
 
a) NAME: with rank, status (tenured, contract, etc.)
b) DEGREES: designation, institution, department, year
c) EMPLOYMENT HISTORY: dates, rank/position, department, institution/firm
d) HONOURS: such as F.R.S., Governor General's Award, honorary degrees, etc.
e) SCHOLARLY and PROFESSIONAL ACTIVITIES: past 7 years only (e.g. executive and editorial positions but not memberships)
f) GRADUATE SUPERVISION: career number - Masters/Doctoral; completed/in progress
g) UNDERGRADUATE AND GRADUATE COURSES TAUGHT: past 7 years, by year
h) RESEARCH FUNDING: past 7 years only, by year, indicating source (major granting councils, industry, government, foundations, other); amount; purpose (research, travel, publication, etc.)
i) PUBLICATIONS:
i) Life-time summary (count) according to the following categories:
- books
- chapters in books
- papers in refereed journals
- papers in refereed conference proceedings
- technical reports
- abstract/posters and/or papers read at scholarly meetings
- others (e.g. workshops presented)
ii) Details for past 7 years (same categories as above)
 

NOTE:
For some faculty (e.g. in the performing arts) it may be more appropriate to list exhibitions/performances by year (for the past seven years), indicating the nature of the exhibition/performance (e.g. juried; local/national/international; public/competition; prizes awarded; and so forth).
 
 
 
APPENDIX D
Undergraduate Program Review
 Student Survey
 
The Department of (name of department) is currently undergoing an undergraduate program review.  Students majoring in (name of program) are asked to provide their opinions/comments on the program for use in this appraisal.  Your co-operation is appreciated.
For questions 1-12, please shade in the appropriate box on the Scantron score sheet
          A:       Strongly agree
          B:       Agree
          C:       Not sure/Not applicable
          D:       Disagree
          E:       Strongly disagree
 
  1. Does the program meet educational objectives that are consistent with the standards, educational goals and learning objectives of the degree when I enrolled in it?
 
  1. Is the program's structure appropriate for meeting its learning objectives?
 
  1. Is the program's curriculum appropriate for meeting its learning objectives?
 
  1. There is good balance between required and elective courses in the program.
 
  1. The workload in the courses in the program is too heavy.

 
  1. The workload in the courses in the program is too light.
 
  1. Are the modes of delivery employed by the program (including, where applicable, distance or online delivery) appropriate and effective in meeting the program's learning objectives?
 
  1. In general, the evaluation methods used to grade students in the program are appropriate.
 
  1. The library resources support the program.
 
  1. Computer-based resources support the program.
 
  1. The academic counselling available in the academic unit is good.
 
  1. The laboratory/practicum facilities and equipment support the program.
 
  1. Add any other comments you wish to make on the program on the reverse side of the Scantron score sheet.