From the Fall Program for Freshmen (FPF) to the Office of Planning and Analysis (OPA), Noam Manor has seen the daily impact data can have on our work at Berkeley. Whether it is for program improvement or professional development, data activism can open many doors.
The idea of Data Activism suggests that anyone can proactively use data to plan ahead and make more informed decisions. Does this idea resonate with you?
NM: Yes, absolutely. When I worked as an advisor at the Fall Program for Freshmen (FPF), I had a very good sense of how to help individual students. But there are times when we need to step back and look beyond specific circumstances and consider broader questions pertaining to students or programs as a whole. In the process of moving from the particular to the general, we also need to move from our own impressions and experience to see a bigger and more objective picture and one that can be tracked over time. That’s precisely when data proactivism is so beneficial.
Can you give an example of how you used data to support an informed decision?
NM: Reading & Composition courses at FPF used to have an enrollment limit of 34 students. Over the years, we heard from students and instructors that a course that relies so heavily on classroom discussion would benefit from a smaller class size.
When this topic started coming up in student and instructor surveys, we wanted to study it a little more closely before proposing changes. We knew anecdotally that R&C courses on campus had a limit of 17 students, but now that we wanted to do something about it, we had to be able to quantify the difference.
As advisors, we had in-depth knowledge of FPF course scheduling and enrollment management. We were, therefore, in an ideal position to carefully study R&C trends. We looked at FPF enrollment data for the preceding decade and gained a better understanding of student demand and the availability of instructors.
The Cal Answers curriculum dashboards were not available when we started our evaluation, so we used the Online Schedule of Classes (including the archive) and manually tallied R&C enrollment limits and scheduling practices.
The program's leadership had to consider other aspects as well, such as alignment with L&S guidelines for R&C, academic HR policies, and the instructional budget. The findings were presented to our Dean, who agreed to a one-year pilot. The enrollment limits for R&C courses were all set to 17!
At the end of that year, we used survey results, enrollment data, and grade trends to start evaluating the outcomes. Instructors praised the change, and preliminary grade analysis suggested a relationship between smaller class size and higher R&C GPAs.
The Cal Answers curriculum dashboards had become available at that point, making it very easy to look comparatively at years of data relating to average class size, grading trends, student demand, and wait list patterns.
The lower R&C enrollment limits are now the norm, making those FPF courses aligned with campus practices and offering FPF students a comparable experience. But the outcome assessment is still ongoing. A hallmark feature of effective evaluation is its iterative nature, helping you assess where your program is and track how it may be evolving.
The work done by FPF advisors to pick up on this issue and analyze enrollment data initiated a process that resulted in a significant curricular change benefiting students and instructors alike. The data we collected helped us advocate for the cause. By the same token, data doesn’t replace advisors’ intuition and vast insight into the student experience, but it can help to connect the dots and be easily shared with others.
Do you think engaging in data activism has provided opportunities for your own professional development on campus?
NM: Definitely! First of all, I always made sure to reach out to colleagues on campus, both advisors and analysts, when I had questions. I learned a lot and developed a network of experts I could call upon. In turn, they became familiar with my work too, and we started collaborating on various projects. For example, I was invited to present on program evaluation at a NACADA conference with a panel of advisors and analysts. Our presentation won the Best of the Region award, and we were invited to repeat it at the national conference.
The FPF program leadership recognized the value of this kind of work, and my job responsibilities were formally changed to include analysis and evaluation, in addition to, not instead of, advising. Being an engaged advisor helped me understand the data better. Likewise, understanding the data made me a better advisor, and I was able to advise students with more confidence.
I also started regularly attending sessions of the Cal Assessment Network (CAN), where I met even more colleagues and learned (and continue to learn) new skills.
I developed a reputation on campus for being committed to and knowledgeable about program evaluation and improvement, which eventually led to my transition to the Office of Planning & Analysis (OPA), where I now work to help academic departments prepare for their own program reviews. So you could certainly say that being a data activist at FPF provided major professional development opportunities.
What advice might you give to Berkeley advisors looking to dive into assessment in their own units?
NM: The Advising Council Fellows Program is now accepting applications for its second cohort. This program was developed specifically for advisors without formal experience to start doing evaluation and assessment work in their departments. I served as a mentor for the first cohort, and it was obvious that everyone made important progress.
I also strongly recommend regularly using Cal Answers, an online analytic tool with a lot of data useful for advising programs. All advisors already have access to it, and there are great video tutorials and training materials. OPA analysts also hold Cal Answers office hours, and that's a great way to get help.
While local surveys can be very instructive, extensive survey results are already available on the OPA website. Reviewing existing survey results for your department is a great way to start.
Perhaps my most important advice is to begin with a small project, looking at a topic you are already very knowledgeable about. Use Cal Answers or survey results to verify what you already know, or you may be surprised and learn something new.
Next time someone asks you a question about the student experience or enrollment issues in your department, look at it as an opportunity to get started, and know that there are now great tools and support to help you answer such questions quickly and confidently.