Darby

It’s May. Time to reflect back on a (hopefully) successful academic year. When things start to quiet down, it’s a great time to reflect on your assessment before getting wrapped up in the planning for the next year. We have to set aside time for it, or else we get sucked into putting out the fires and handling the immediate (which is not always the most important).

What does reflection look like? It’s really a process of dialoging with oneself (or with others) to gain additional perspectives, think about our values and beliefs, and put that in the larger context in which we operate. This gives us clarity to change (Jay & Johnson, 2002).

How do you start? One of the easiest “formulas” I have seen for reflection is the “What? So What? Now What?” process. The “What?” is a description of what happened, the “So What?” examines the significance of what happened in context, and the “Now What?” looks forward to what you will do differently.

Let’s look at a simple example.

What? You did a pre/during/post rubric with the executive officers in the organization you advise to gauge their leadership skills growth. Students used the rubric to self-evaluate, and you also completed a rubric on each student at the end of September, beginning of January, and April. The rubric represented several of the institution’s undergraduate learning outcomes. Most students indicated growth in the areas of communication, critical thinking, and personal responsibility (ethical leadership). Students did not grow in the area of working collaboratively (teamwork, considering different points of view, supporting a shared goal). Not only did the students not score highly on the rubric in April, you also observed their group conflict and lack of cohesion to fulfill the group’s mission.

So What? This is significant because students need to be able to work together now (in classes and co-curricular activities) and in the future when they have a career where they will work with others. The organization, its audience, and its members suffer when the executive officers are not on the same page and working together. Because this is also an expectation from the university, you know students should be developing this skill before they leave the university.

Now What? As you think about the new officers who have been elected for the fall, you know you might need to be a little more engaged with them at the beginning. Working with the president-elect, you plan a retreat that will focus on group development, understanding different viewpoints, and setting organizational goals. You will also continue to use the rubric to evaluate student performance and observation to evaluate organizational performance.

Block some time on your calendar to review the results of your formal and informal assessment(s) from the past year. Maybe make an appointment with a colleague to talk through your reflections. Set a timeline to implement potential changes in the new academic year. If you don’t do it, you will be doing everyone, including yourself, a disservice.

Reference

Jay, J. K., & Johnson, K. L. (2002). Capturing complexity: A typology of reflective practice for teacher education. Teaching and Teacher Education, 18, 73-85.

It’s April 1^st. I admit, I’m not much for April Fool’s jokes, but that got me thinking about assessment failures (and potentially looking like an assessment fool). I thought it might put people at ease to hear about some of the mistakes I/we have made over the years. Fortunately, we are quick learners and only make big mistakes one time (knock on wood).

 

One of the most memorable was the time we accidently sent an electronic survey to students that appeared to come from the university president. A setting had been changed at the university level in the software system we were using, and no one caught it until the survey came out. In the big scheme of things, the survey content was really benign but not something that the president would really send out. As soon as we found out about it, we were on the phone to the Vice President’s Office so they could communicate up. Not surprisingly, we had a fairly high response rate (so it’s true that your email invitations should come from a recognizable person)!

 

We are all about helping people try new assessment methods. One department had an event outside, so it would be virtually impossible for people to card swipe or sign in. We had the technology to create a QR code that participants could use to basically use their phone to scan the code that would open up the assessment. The department posted the QR codes around the event. Unlike the example above where we had a high response rate, this time we had a 0% response rate! Not one person scanned the QR code (we checked to make sure the technology was working correctly). Lesson learned: students don’t use QR codes. The department was disappointed and embarrassed, so we talked about how they could get feedback from the staff who worked the event as a means of assessment. I gave them lots of kudos for trying something new.

 

Technology is a great thing and has allowed us to personalize survey invitations. At the same time, Excel is not always the best way to keep accurate data. In this case, our client deleted a cell in the spreadsheet, moving all the other cells in that column up one. We did not catch onto that until it was too late. The result is that some of our “personalized” emails went to the wrong person (e.g., Eric’s email said “Howdy Susan!”). That also interfered with the demographic data we had preloaded into the database. Assessment is only as good as the data that gets fed into it.

 

I hope this makes you feel better about any assessment mistakes you have made or will make. The moral of the story is to fix the problem as best you can as soon as you identify it, make note of how to prevent it from happening again, and chuckle at yourself when it’s over.

February is known for Valentine’s Day and celebrating relationships. I hope that you will celebrate your relationship with assessment. Some of you are thinking, “I certainly don’t have a loving relationship with assessment. I’m not sending chocolate or even a greeting card.” Thinking beyond the nuts and bolts of assessment toward the end result of assessment, you might actually have a positive relationship.

Think about the purpose of assessment. I emphasize that assessment is about improvement and doing the best for the students (or staff, clients, users, etc.) we work with. We are doing assessment to provide the best possible experiences for the people we care about. We build the best facilities so students can study, live, exercise, build relationships, and more. We provide programs, services, and activities where students can learn, grow, and connect with others. We coordinate leadership and employment opportunities so students can learn and apply skills and get feedback from peers, supervisors, and advisors.

What do those things have in common? It’s our desire to give students the best experience possible, to encourage their learning, satisfaction, and connection. If we didn’t do assessment, how would be know that we were not living up to that standard? We wouldn’t. We might keep doing the same thing, not knowing that it was not beneficial to the people we care about. It’s all about open communication.

Assessment is like having that “relationship check in” talk. You might be nervous and not know where to start. You might be afraid of what people are going to tell you (you want to be the perfect partner, don’t you?), you think you might not be able to meet their expectations….That’s totally reasonable. But, wouldn’t you rather know how to improve rather than continuing to do something annoying or unproductive?

Even when you “break up” (students graduate, staff move to other positions, etc.), you want people to say positive things about you. You want them to recommend you to others. You want students to be better prepared for their next relationship: employment, grad school, next leadership position, etc. Others will be able to see the positive influence you had when they start a relationship with the student. You want to know how you can be better in the next relationship.

Feeling a little more positive about assessment? I hope so. If not, Student Life Studies can be your assessment relationship mediator. We can help you ask useful questions and interpret what others say, so you can be the best you can be. Feel free to contact us at sls@tamu.edu or (979) 862-5624. We are always here to help.

I hear a lot of staff and graduate students talk about “proving” the worth of a program or “proving” student learning took place or having to “prove” that student affairs has value on a college campus. I appreciate the effort and sentiment behind the word, but I also caution folks about making claims they cannot always support. Word choice is important.

Student affairs assessment can be messy. Working with people, mostly college students who are still developing and growing, is messy (literally and figuratively!). Students do not arrive on campus as a clean slate—they bring their own experiences, cultures, beliefs, skills, and motivations. They have various experiences while they are here, so no two students have the exact same college experience. Even within the Division of Student Affairs, students may have similar opportunities, but not the same outcomes. Students may or may not do the following: live on campus, be treated by Student Health Services, join and/or lead a student organization, face discipline for making poor choices, attend a program, work out at the Rec Center….The possibilities and combinations are endless.

What does that have to do with proving things through assessment? We have to be careful about attributing student learning/success to any one intervention or program. Yes, in general, students who live on campus their first year are more academically successful. But, does that mean that every student who lives on campus will be academically successful? Unfortunately, no. In a sense, a few students will disprove the idea that residence life causes academic success.

Another example would be asking students to rate themselves on communication skills after attending a two-hour program. How sure are you that any increase in their scores can be attributed solely to attending the program? What were their communication skills before they attended the program? And, for something like communication skills, were students given any opportunity to practice or demonstrate those skills to be observed and rated by someone who has expertise in that area? So, does attending a communication skills program prove that students improved? Not necessarily. If that topic is important to us, should we keep offering it? Probably. Can the assessment of the program tell us about improvements we can make? Yes.

If you don’t use the word “prove,” what should you use? I like “provide evidence of” or “supports” or “indicates.” While they are not as forceful, they provide a foundation and a demonstration of something in a narrow context. Words are important to communicate correctly to a variety of audiences.

P. S. In my opinion, we need to stop worrying about proving our worth in higher education. Accept that we have value, and take opportunities to share assessment results that show that.

If you need any assessment assistance, please contact Student Life Studies at (979) 862-5624 or sls@tamu.edu.