The osprey is the mascot for the University of North Florida (UNF), Jacksonville. So our quality assurance program is aptly named "Osprey Eyes." This program, simply put, acts as the "eyes" of the students in identifying service excellence as well as needed improvements.
While surveys are good at capturing the perception of customers, Osprey Eyes relies on student reviews that serve as snapshots of particular experiences on campus. The reviews are based on our student reviewers’ face-to-face interactions; website assessments; and telephone contacts with various offices, units, and services.
Far-reaching Reviews
Since launching the program in the spring of 2014, we have conducted reviews of all campus food venues and most services, ranging from parking and transportation to the IT help desk, from the theater box office to the golf complex.
This past summer we recruited an incoming freshman to evaluate the new student orientation program. Then in the fall, we added reviews of other areas: the library, student health services, wellness center, university police department, and residential housing.
Typically, we schedule an eight-week review period to coincide with the start of classes in the fall, followed by another eight-week assessment period about halfway through the spring semester.
UNF conducts about 500 reviews during each eight-week period. In addition to capturing the customer experience at these different times of the year, we send different reviewers in the spring and fall to ensure a range of perspectives.
The 15 UNF students hired as Osprey Eyes reviewers for the current academic year were all identified through the university’s online recruitment system. We look for a variety of levels (freshmen through seniors) as well as a range of academic majors.
Before undertaking any assignments, the student reviewers attend a two-hour training that covers the elements and expectations of customer service at UNF. The session provides instructions on how to complete and submit the online review form, and tips for writing a constructive review.
Since the reviews are shared with unit managers, we want the students to be honest but also as professional as possible. For example, we ask them to avoid sarcasm and to offer solutions to any perceived problems, rather than simply complain about a product or service without suggesting constructive comments.
We also ask the reviewers to maintain secrecy about their jobs, rather than walking around with a checklist or conspicuously taking notes. Our campus staff know that the university’s services will be reviewed by students, but staff don’t know when and by whom. Some units try to guess the reviewer’s identity—but the finance and administration division never tells.
Gathering Data
In the fall and spring, each student reviewer receives a weekly schedule specifying what unit to review, when to begin the review, and a budget for expenses. The reviewer’s budget might include, for example, $7 for lunch at one of the campus restaurants or $5 for a purchase at the bookstore. Another day, the allocation might be $10 for dinner and $6 for a smoothie at the wellness center. Mixed into the schedule are visits to campus units that don’t require a purchase, such as career services and student financial services.
Each Osprey Eyes reviewer conducts 10 to 15 reviews per week. We pay $10 per hour—which is higher than most on-campus jobs—and reimburse all authorized purchases via direct deposit after submission of a receipt. Within 30 minutes of completing an assignment, the reviewers submit an online form that explains the details of their visit, website assessment, or phone conversation.
Assessing the Experience
Using the checklist-like form, the secret shoppers rate various aspects of their customer experience and then answer three open-ended questions. One question asks for the specific scenario used—such as checking for financial holds, getting help with a resume, having a virus removed from a laptop, or riding the campus shuttle. Another question asks them to identify the activities in which the venue is performing well and should continue. The final question requests recommendations for areas the unit could improve upon. We also ask reviewers to assign a letter grade, from A+ through F, to the interaction.
After each eight-week review period, we analyze the collected data and prepare summary reports, which are shared with each unit’s leadership team.
For consistency and benchmarking purposes, we use the same criteria to compare performance from one semester to the next: the assigned letter grade and the answer (yes, no, or somewhat) to the question, "Did the way you were treated make you feel like a priority?" The latter ties directly to UNF’s mission to make students its first priority.
Team Effort
Because the Osprey Eyes program focuses on the positives of a customer experience, not just the negatives, it has been well-received on campus. Division leaders are always interested in knowing the top-rated venues or services (interestingly, parking services at UNF tend to make the top 10 list).
We also ask the units about any changes they made between review periods. Reviewers’ comments and suggestions have prompted numerous changes, such as restocking vending machines more frequently, improving signage on campus, using standardized telephone greetings, increasing the number of employees at some food service locations, counseling staff about inappropriate language, relocating equipment for greater efficiency, and modifying business hours.
Initially, units were expected to cover their portion of the program’s costs. With budgets already stretched, this was not ideal. UNF’s leadership felt so strongly about the program’s benefits that they changed the funding structure. Starting with the 2015–16 academic year, Osprey Eyes is "university funded." The communications and training department, within the division of administration and finance, administers the program and its $25,000 annual budget.
Maintaining program consistency is an ongoing challenge. Numerous details must be exactly replicated during each term to maintain program integrity. This is accomplished by project checklists, timelines, tracking sheets, and good documentation.
As we have expanded the areas that our shoppers reviewed, additional challenges have arisen. Most venues use a similar review process but some require distinctive considerations. For example, academic advising and new student orientation are unique and, therefore, require logistical coordination and individualized review forms.
Osprey Eyes is not a mechanism to "spy on" or "catch" employees misbehaving. Instead, the program is a team effort to improve interpersonal communication and interactions between customer-facing staff and university students. Osprey Eyes turns data into information, information into insight, insight into action, and action into improved customer experiences for UNF’s 16,500 students.
SUBMITTED BY Alison Cruess, assistant director for communications and training, University of North Florida, Jacksonville.