New Article Series Dissects Data Analytics
One of the priorities of NACUBO’s Strategic Blueprint is to empower the chief business officer to facilitate the integration of analytics. Carefully woven into this priority are notions of leadership, facilitation, and collaboration. This article is the first in a series that describes the numerous ways that institution leaders can apply this strategic process to decision making of all kinds.
Imagine yourself in a conference room with colleagues from areas across campus—faculty, leadership, student services, development, facilities, and finance. The discussion is lively, and colored Post-it notes are everywhere. The university has a happy problem: As a result of society’s emphasis on STEM education and related academic offerings, enrollment and retention are at an all-time high.
The university is bursting at the seams and needs more academic and residential space. The only negative seems to be that the campus is landlocked within the confines of a small city, and adding space is constrained by both campus boundaries and building-height limitations.
The meeting participants have been tasked with identifying and prioritizing options for short- and long-term solutions to the space issues. You are a new hire at this institution and a new member of this team, and are surprised that the discussion is open, friendly, creative, and wide-ranging. How can this be, and what is different at this institution than at those you’ve served in the past?
Finally, it hits you: The difference is data! Accurate, reliable, relevant, and timely data. And data that represent a shared value to be used by those in attendance. Ideas are presented, variables are identified, estimates are tweaked, projections are evaluated, and uncertainties are understood.
You observe that the faculty participants represent different colleges within the university, and that the enrollment, financial, and space data are broken down by individual college and shared freely with all participants. The development representatives are sharing nonconfidential donor information and interests, to identify funding opportunities. The CFO has provided details of the funds saved for a rainy day, broken down by amounts that are committed to projects and amounts that may be available. And facilities has shared an up-to-date space inventory and maintenance schedule, showing the transition of space over time and projections for when buildings will be renovated.
Again, you ask yourself, what has permitted this free flow of information? Are you dreaming, and is this nirvana or is this truly possible?
An Organizational Emphasis
One of the priorities of NACUBO’s Strategic Blueprint is to empower the chief business officer to facilitate the integration of analytics. Carefully woven into this priority are notions of leadership, facilitation, and collaboration.
Asked to underscore these goals, speakers from institutions at NACUBO’s November 2017 Managerial Analysis and Decision Support (MADS) workshop assured all in attendance that, through collaboration and ingenuity, the scenario at Nirvana University is indeed possible.
Presenters at the conference representing institutions of many disparate missions, sizes, and resources—Clemson University, Delaware State University, Emory University, New York Institute of Technology, Rutgers, Stetson University, and SUNY—shared their successes in applying analytics and data-informed decision making to solve some of their most intractable issues.
- Crucial counsel. Several key themes emerged across all the institutions’ stories.
- Data are available and shared across colleges, departments, and organizational structures.
- Committees include representation from all the university functions.
- Successful data-informed decision making is supported by the president and senior leadership team.
- Data governance is important, but so is achieving quick success. Proceeding along parallel paths to develop the structure, while gathering and analyzing data to identify solutions, is possible.
- Useful data can be found in places other than the conventional sources and administrative systems.
- Analytics software can help, but may be the least important aspect of the effort.
Data-informed versus data-driven approaches. Another key concept expressed in many of the presentations was the contrast between data-driven and data-informed approaches. In her opening keynote address, Belva White, chief business, analytics, and operations officer, at Emory University’s Goizueta Business School, expressed the difference this way, "In a data-driven process, data lead the decision-making process. In a data-informed approach, data are used to form a hypothesis, the limitations of data are understood, and other factors are also considered."
The institutions represented at MADS described data-informed approaches that proceed along these general lines:
- Describe an issue.
- Ask a question.
- Develop a hypothesis.
- Identify the relevant data and relationships to prove or disprove the hypothesis.
- Analyze the results.
- Repeat the process until an actionable path emerges.
This is recommended over a process that starts with developing a theory and then finding the data to confirm the theory.
Angela Henderson, director, and Resche Hines, assistant vice president, respectively, of Institutional Research and Effectiveness, Stetson University, shared their experience in gathering data from all sources to understand more about the university’s 40 percent growth in undergraduates between 2010 and 2013. The objective: discern how to best maintain enrollment levels and retain students. As Hines noted: "Data analytics is first about ‘learning’—to provide a context for what is happening at the institution."
To achieve their goal, Henderson and Hines compiled data on enrollment, retention and attrition, graduation, movement through majors, grade distribution, credit hours, outcomes and employment data, faculty, general education, national student engagement data, peer benchmarks, athletics, and advancement. They then developed interactive visualizations and shared the resulting reports at dean- and department-level meetings to increase awareness about how Data patterns can be used for decision making.
A subsequent benefit was the ability to identify and target previously unidentified at-risk students. Improved understanding of the characteristics of the nonretained students led to (1) better monitoring and planning at the college level to retain students in their majors, (2) increased outreach by advisers to at-risk students, and (3) a change to the scholarship model.
Illustrating driven versus informed. If we go back to our meeting at Nirvana University, we’ll recall that leaders identified a shortage of classroom space. One participant posited that the administrative offices had expanded to take over space used for seminar and discussion groups, causing those sessions to then take over conventional classroom space. This suggestion was based on anecdotal experiences, in which one dean’s office had used a student seminar room for a collegewide planning process. Several others asked for data to show the conversion of student meeting space into employee meeting space, to confirm this theory.
The chief data officer (CDO) encouraged the group to think more broadly, and suggested an alternate possibility. He wondered about patterns of converting space into administrative offices, and if class scheduling times had anything to do with the space issue. He offered a hypothesis that the shortage may be caused by the convergence of class times into the middle of the day, (i.e., more classes scheduled between 10 a.m. and 3 p.m.) and fewer scheduled earlier and later in the day.
The CDO suggested gathering data regarding all possible classroom and meeting spaces available during the last three years, looking at how the use of space has changed over time (as captured in the space inventory), and matching it up with the classroom scheduling software and the scheduled use of each space.
Thinking more broadly helped the committee learn that a few seminar rooms had been repurposed to project spaces and had not been returned to available seminar spaces when the projects were completed. However, based on the distribution of classes, committee members also learned that one of the primary causes of the shortage was not less space, but more demand for the existing space during the middle of the day. Had they focused only on proving the theory that classroom space had been converted to administrative space, they would have missed the more significant cause for the demand on academic space.
Achieving Nirvana at Your Institution
The genesis of data-informed decision making at institutions is as varied as the institutions themselves, as evidenced by participants on a data governance panel: Mike Gower, executive vice president for finance and administration, and university treasurer, Rutgers University; Mark Hampton, vice president for enrollment and enterprise analytics, New York Institute of Technology; Eileen McLoughlin, senior vice chancellor for finance and chief financial officer, The State University of New York; and Belva White, chief business, analytics and operations officer, Emory University’s Goizueta Business School. Charles Tegen, associate vice president, finance, Clemson University, facilitated the panel.
Although the reasons for getting started were varied, the panelists described common, critical components for establishing a robust data governance structure, including:
- Standards for data use, tools, and access.
- Consensus on data definitions, vocabulary, labels, and formulas.
- Identification of data trustees, data stewards, and data consumers. (See table, "Data Governance Responsibilities.")
- Clearly delineated responsibilities for data accuracy and accountability.
- Emphasis on data as a valuable institutionwide resource.
- An optimal balance between accuracy and agility.
The methods for achieving these objectives also vary, and schools may take many different paths. However, based on the information shared by the conference presenters, schools that need a place to start have resources upon which to draw and do not need to reinvent the wheel. Data governance policies and processes are accessible on institution websites and provide structure and ideas for initial conversations.
A Web search of "university data governance policies" returned numerous policies in place at both public and independent institutions—many contained similar content (distilled below).
Data governance structures and roles. Policies surveyed include an overarching principle that the institution "owns" the data. Hence, institutions have moved away from describing any of the roles as "data owners." Instead, roles are commonly arranged in either a three- or four-tiered structure, using terms commonly associated with fiduciary responsibilities and stewardship.
The titles may vary but the roles and responsibilities are generally described as shown in the table.
In addition, of the university policies surveyed, many have data governance groups that are institutionwide committees comprising members with the data "steward" roles; and many have someone in charge, such as a chief data officer or director of data management. Responsibilities of the chief data officer often include coordinating the activities related to data management.
Data user responsibilities. A few of the policies surveyed establish standards for all employees who use data, requiring them to bring problems and suggestions for improvement to the data steward or equivalent. Establishing this requirement seems to confirm one of the most common issues cited as a barrier to data-informed decision making—the lack of accurate data. It appears that clearly stating employee responsibilities is one of many approaches to addressing the issue of accuracy.
Panelists essentially confirmed researched observations and offered a few other suggestions. Clemson’s Charles Tegen suggested changing the culture by emphasizing that data are a resource of the institution, similar to human and financial resources. "We would not allow others to consume or harm these resources inappropriately, and we need to steward data resources in the same way." New York Institute of Technology’s Hampton commented that delivering results quickly can demonstrate how data can be used effectively, and showing the consequences of inaccurate data, can help improve data accuracy. "The keys to a strong data environment are like a three-legged stool, in which data are trusted and available, data are desired, and a technological infrastructure exists to support them."
Friday afternoon measurement method. During her keynote, Emory’s White explained a process described in the Harvard Business Review known as the Friday Afternoon Measurement Method (FAM). Managers review 100 transactions a week, count the total of error-free records and calculate the data quality score (percent of data created correctly). Error-free records are assigned a positive value of $1 per record, while records with errors are assigned a negative value of $10 per error. The results are used to illustrate to transaction processors the cost/benefit of entering data correctly.
Kathy Dively, executive director of strategy and analytics at Clemson, described the university’s requirement to certify data sources, and that only certified data can be loaded into the decision-support environment. Dively and her team collaborate with the functional experts, data stewards, and data trustee for each subset of university data. They identify the questions and information that are most commonly requested, the fields and definitions that are most commonly used to provide the information, and the frequency of any data changes (and how often data should be extracted). They also verify that the data represents what it is intended to represent, and use job descriptions to establish the rules for granting access to the data. As the data are loaded from each administrative system, the team establishes the linkages that tie the information together, such as room scheduling information between the student system and the facilities inventory; or the accounting string (chart of accounts) values to which compensation expenses are charged, linking the human resources and financial systems.
Decision-support and metadata tools. While many software vendors will highlight the need for the newest, shiniest dashboards and database tools, presenters at the conference emphasized that the tools are the least important factor in establishing data-informed decision making at your campus. Many data analysis tools exist, including open source software such as "R" and "Python." Others used cloud-based compilation software to store the data from many systems, making it easier for campus users to access comprehensive information about students, majors, finances, and so forth, eliminating the need to navigate between and among systems.
In the survey of data governance policies, tools to capture data definitions and other metadata include "data cookbook" and use of "wikis" supported by collaboration software such as Confluence. Several conference attendees also reported the same.
The key takeaways were that tools are unable to fix bad data, and that the external look is the easiest part of a project. More important than the tools used in the process is concentrating on (1) getting the data right through a combination of clearly defined roles and responsibilities, (2) emphasis on data as a valued institutional resource, and (3) effective use of information to assist decision makers.
Get Started, Look Forward
Success stories abound at institutions that have found ways to leverage existing data, identify an employee or two with analytical and organization skills, use faculty with data science skills, offer opportunities to students with technical talents, convene representatives of each functional area who understand the data from their respective systems, and deliver a few high-profile results.
The momentum that comes from preliminary success can help sustain what can be more contentious and time-consuming conversations to establish data governance policies, structures, and standards.