Frequently Asked Questions about GE Assessment

Why do we have to do this?

Assessment of student learning in GE courses is mandated by the State. (Hopefully, you can structure assessments in your GE courses so that you obtain information about student learning that will be useful to you in improving the courses.)

What exactly does our campus need to do?

Assessment plans must be approved by GEAR, the General Education Assessment Review group. We must also report data on the percentage of students "exceeding", "meeting", "approaching" or "not meeting" each of the Board of Trustees GE objectives.

(The only data leaving the campus is a summary of the overall percentages for all students assessed.)

When is the assessment plan for my department due?

Generally, departments should submit new or revised assessment plans to the GE Board in late October, several months before the courses are liable to be assessed. Departments with courses in multiple categories may have more than one year in which their courses can be assessed.

One major exception to this rule concerns the three areas subject to strengthened campus-based assessment. Assessment plans are due for these areas by the end of November, 2005. This is true even if the actual assessments will be done years later. These three areas are MATH, COMP, and courses with our SI competency.

As our campus gains experience with this assessment process, a point will be reached where departmental assessment plans will usually be minor revisions of plans from three years before, possibly with some new GE courses added.

How often is GE assessment performed?

Each area is assessed in a three-year cycle. In most cases, this means that each course is assessed once every three years. The assessment schedule for our campus is:

Spring 2006: USST category, WRLD category, courses with the IL competency, courses in USST and WRLD with the SI competency.

Spring 2007: ART category, COMP category, FLNG category, MATH category, courses in ART, COMP, FLNG and MATH with the SI competency.

Spring 2008: HUM category, NSCI category, SSCI category, WEST category, courses in HUM, NSCI, SSCI and WEST with the SI competency.

(repeats in a three-year cycle)

How are students selected to have their work assessed?

We have some latitude on this. The requirements we must meet are: All students taking courses in the assessed area must have the same probability of being assessed. All courses and sections must be ready to have students in them assessed. At least 20% of all students enrolled in courses in the area must be assessed.

What is done in most areas is the following. Institutional Research selects entire sections at random to be assessed until the enrollment in assessed sections is at least 20% of the total enrollment in courses in the area. The assessments are done in the selected sections only. (Although all sections in the area must have plans in place by the start of the semester.)

How do students have their work assessed?

This is usually done based on particular assignments that have been designed to measure how well students are meeting the objectives of the area.

How are these assignments evaluated?

They are usually evaluated by the instructor, who refers to written documentation that clarifies what levels of performance constitute "exceeding", "meeting", "approaching", or "not meeting" each objective. The articulation of clear standards is important, since GEAR insists that there be a mechanism to insure inter-rater reliability.

What is inter-rater reliability, and why is it important?

GEAR requires that GE assessments be done so as to yield the best data possible. For this reason, GEAR wants to know what steps we are taking to minimize variability between how different instructors evaluate the assignments. Inter-rater reliability refers to the steps we take to insure that it does not matter who is evaluating an assignment. Some acceptable ways to insure inter-rater reliability are to use written rubrics or sample responses which those evaluating an assignment will refer to. Another useful method is to have a norming session to coordinate the standards of multiple raters. When evaluation in a discipline must be very subjective, it may be necessary for multiple raters to examine each assignment and for their ratings to be averaged.

What is a norming session?

A norming session would be done to match the standards that multiple raters would use. The idea is that most assignments would be evaluated by only one rater, but that the raters would have first conferred on the evaluation of some selected assignments. This initial conference where the raters compare their evaluations is a norming session.

Will data from the GE assessments be used to evaluate instructors?

No, this is not the purpose of the GE assessments. While instructors should use the assessments to gain insight into how well their students are learning, the data will not be used to evaluate the instructors. To insure that it is not used this way, all GE assessment data will be amalgamated before it is passed on. It should not be possible to recover information about individual instructors, once the data from several classes has been pooled together.

Institutional Research will take the raw data from instructors, and only release pooled data. (Departments will be given the pooled data for their own courses. Data will also be pooled across areas before it is sent to GEAR.) Alternatively, departments may pool the data for their own courses before passing it on to Institutional Research. (This is of course more work for those departments.)

Does my department have to do the GE assessments in the same way that other departments do?

No. With minor exceptions, each department is free to do them any way that satisfies GEAR. On the other hand, departments are encouraged to cooperate, since this should make the assessment process more efficient.

How does one pick which assignments to assess?

This depends on the objectives which are to be assessed. With luck, different parts or facets of a single assignment will serve to assess all objectives. Depending on your field, it may be best to assess an essay, a presentation, or particular questions on an exam.

What information is needed in a Departmental assessment plan?

This plan consists of a cover sheet, where one gives some departmental information and lists all the course/area combinations that might need to be assessed. Course assessment plans should be attached for all of the course/area combinations listed. Ideally, these attached plans would include all of their rubrics as well. It is acceptable to merely attach a rubric (or equivalent) for one course in each area, and to submit the remaining rubrics two weeks before the semester in which the courses are to be assessed.

What information is needed in a Course/Area assessment plan?

The course and area to be assessed.

How students should be selected to have their work assessed.

Which assignment(s) will be used to assess student work. To insure that all students selected have their work assessed, the assignment(s) should be required. Please be careful to pick assignments which will provide a good measure of how well students meet the objectives being assessed. GEAR plans to check for this quality, called face validity.

Attach a rubric (or equivalent) to show that the objectives will be assessed in a uniform way. (Instead of a rubric, a set of sample student responses may also be given. For each objective, these should have a sample response that "exceeds" the objective, a sample response which "meets" the objective, one which "approaches" it, and one which "does not meet" it.) (Other approaches may also work--consult the GE Board.)

Even having a rubric leaves some items open to interpretation. State how your department will insure that the rubric is used uniformly. (You should demonstrate that you are making a reasonable effort to insure inter-rater reliability.)

What is a rubric?

In the context of GE assessment, a rubric is a document that spells out the features which student work should have if it is to "exceed", "meet", "approach", or "not meet" each objective (or part of objective). Rubrics are often presented as tables, with rows for each objective, and columns for " exceed", "meet", "approach", and "does not meet".

Which objectives should be assessed in an area?

The problem is that one area will have several sets of objectives. One set is given by the SUNY Board of Trustees (BoT). Another set consists of the official New Paltz objectives for the category or competency corresponding to the area. Still another set of objectives would be those for the course objectives of the particular course being assessed.

Our campus must report percentages "exceeding", "meeting", etc the Board of Trustees objectives, so it would be best to assess these objectives. Since all three sets of objectives do correspond to each other, it is also possible to assess one of the other sets, and then to convert percentages for that set into the percentages we must report. (Consult the GE Board for details.)

What are the Board of Trustees objectives?

They are the statements of what students are expected to be able to do in GE courses in each area. Our campus must report percentages “exceeding”, “meeting”, “approaching” and “not meeting” each of them. They are stated for each of the ten areas at the following link, which is adapted from:

http://www.sysadm.suny.edu/provost/generaleducation/course-guidelines-v2.pdf

What areas of General Education are we required to assess?

There are twelve areas in all. Areas such as the Diversity category and the Ethical Reflection competency are local to our campus, and are not subject to this GE assessment.

Some areas are subject to Strengthened Campus-Based Assessment, or SCBA. Our campus will perform assessments in these areas in a manner aligned with the approved SCBA rubrics, which are available at: http://www.cortland.edu/gear/news.html#rubricsdrafts

The twelve areas are:

Mathematics (our MATH category) --subject to SCBA

Natural Sciences (our NSCI category)

Social Sciences (our SSCI category)

American History (our USST category)

Western Civilization (our WEST category)

Other World Civilizations (our WRLD category)

Humanities (our HUM category)

The Arts (our ART category)

Foreign Language (our FLNG category)

Basic Communication (essentially our COMP category) --parts are subject to SCBA

Critical Thinking (courses with our Systematic Inquiry competency) --subject to SCBA

Information Management (courses with our Information Literacy competency)

My course has the SI competency. Will it need to do both the Critical Thinking assessment and the assessment for its category?

The course will need to have assessment plans in place for both assessments. This is in keeping with the GEAR requirement that every course have a chance of being assessed in each area it qualifies for.

However, no course need do more than one assessment at a time. We are permitted to only assess 20% of all students enrolled in courses in a GE area. This is usually done by having Institutional Research pick sections at random until at least 20% of the total number of students are in selected sections. In practice, Institutional Research can always do random selections for both areas in a way that insures that no section is picked for both assessments.

(It would be possible for everyone teaching courses in an entire category to agree to combine Critical Thinking assessments with category assessments. The GE Board recommends against this because of the logistical problems involved.)

Do I need to submit a GE assessment plan for courses which won’t be offered in Spring?

Yes. This is because GEAR requires that all courses be “ready to be assessed”.

The form itself is not much work to fill out, and only requires that one specify some details about how the assessment will be performed. The more labor-intensive part of the process is the development of a rubric (or equivalent), which specifies in more detail how student work will be evaluated. This rubric is not due until two weeks before the start of the semester in which the course is liable to be assessed.

If a course is not offered in a given semester, then the rubric would not need to be submitted. However, we do ask that each department submit at least one sample rubric in each area with its departmental assessment plan.