What Universities Can Learn from Big Data – Higher Education Analytics

By Bill Schmarzo July 2, 2014

There is a growing interest by institutions of higher education to take advantage of Big Data to improve student performance and raise teacher/professor effectiveness, while reducing administrative workload. Student performance data is increasingly being captured as part of software-based and online classroom exercises and testing. This data can be augmented with behavioral data captured from sources such as social media, student-professor meeting notes, blogs, student surveys, and so forth.

There is also a wealth of public data on, as well as many open data initiatives at institutions of higher learning across the country. Higher educational institutions can benchmark their student, professor and curriculum performance against like universities, yielding yet new insight into potential for improvement.

Higher Education Analytics

Higher education institutions have numerous student engagement points that can benefit from Big Data—from initial profiling all the way through to alumni giving. (See Figure 1).

Student engagement lifecycle

Figure 1: Student Engagement Lifecycle

Let’s review a sample of Big Data-powered applications for higher education institution:

  • Student Acquisition. Use historical performance and demographics data of current and former students to create profiles of applicants most likely to enroll—then augment with social media data to score the institution’s sentiment scores. Employ graphic analysis to examine current and prospective students’ social networks to identify first-level friends that may be potential new students.
  • Student Course Major Selection. Think “ for Students,” where a first-year college student’s high school performance and aptitude tests are compared to former student profiles to recommend a possible curriculum and major. Create detailed profiles based upon high school performance, areas of interest captured in both survey and social media, and aptitude test results. Compare those profiles to profiles on courses and majors to find the right match. Integrate external data regarding future workforce skills demands and salaries to help students make informed decisions on a major and minor.
  • Student Performance Effectiveness. Monitor ongoing student test performance and compare to 1) previous tests results as well as 2) clusters of similar students. Integrate social media data and teacher notes to create a more detailed profile on the student’s behaviors and propensities. Develop student- and class-specific recommendations, such as individual or small group tutoring, supplemental learning materials in “problem” subject areas, or even changes in classes or majors (see Figure 2).
Student progress flow analysis example

Figure 2: Student Performance Effectiveness

  • Student Work Groups. Leverage cohort analysis for groups of students that can collaborate inside and outside class to improve individual performance. The analytics allows the teacher to see cohort assignment, factors, and reasons for the assignment, and allow teacher overrides. The analytics could “reshuffle” cohort assignment based on planned design elements and/or random factor; teacher to record observations (to be blended with objective cohort performance) after reshuffle to update the data set (see Figure 3).
Cohort effects on student grouping

Figure 3: Student Cohorts Analysis

  • Student Retention. Combine previous analytics and scores including Student Performance Effectiveness and Student Work Groups, coupled with individual demographic, financial and social data to 1) score the likelihood of attrition, and 2) deliver recommendations that allow the institution to make a decision on whether to try to retain this student. Deliver and measure the effectiveness of specific recommendations—based upon the success of previous interventions. Empower teachers to make their own recommendations, which can be monitored for results and applied in future retention intervention recommendations.
  • Teacher Effectiveness. Measure and fine-tune teacher performance. While some institutions may be limited here, those that have the freedom to measure performance can benefit from insight into an individual teacher’s effectiveness when compared similar teachers. Performance can be measured by subject matter, number of students, student demographics, student behavioral classifications, student aspirations, and a number of other variables to ensure that the teacher is matched to the right classes and students to ensure the best experience for teachers and students alike (see Figure 4).
Caveats of teachers

Figure 4: Teacher Effectiveness Analysis

  • Student Lifetime Value/Booster Effectiveness. Plan ahead with respect to potential giving levels for both current students and alumni. Understanding the likelihood to recommend and current or future earnings/wealth potential can all be major factors in profiling, targeting, and messaging to optimize alumni giving. Take advantage of these insights for the early identification of future boosters and uncovering Booster Top Performance predictors.
  • Student Advocacy. Leverage graphic analysis to examine a student’s social network and score/monitor the likelihood to recommend (LTR) and triage specific areas of the college experience that are better or worse for a particular student. Use this data to create a Student Advocacy score that can be leveraged in the Student Acquisition (targeting a happy student’s friends), Student Retention (flagging changes that can be a precursor to retention problems), Student Performance Effectiveness (flagging changes that can be a precursor to classroom performance problems), and Student Lifetime Value apps.
  • Bookstore Effectiveness. Use retail industry best practices to improve bookstore profitability using analytics-driven applications like merchandising effectiveness and textbook inventory optimization. Below are some links to blogs that I’ve written on Big Data in the traditional retail industry that may be applicable in the higher education institution world:

−        Big Data In Traditional Retail – Part I
−        Big Data In Traditional Retail – Part II
−        Retail Store Manager Actionable Dashboard

Leveraging Higher Education Open Source Data

One of the beauties of doing analysis in the higher education space is the bevy of data that universities and state governments capture and share about university performance. Let’s use Texas as an example, as they provide not only the raw data, but also some rudimentary analytics to interested parties. Let’s start with Texas Public Education Information Resources, which provides a portal where one can select the types and amount of data you want to pull into your analysis.

Texas public education information resource

Figure 5: TPIER Portal

Here’s a sample report titled “Higher Education Graduation – Statewide by Degree Level, Gender and Ethnicity.” This report looks at number of graduates by degree, cut by key demographics (see Figure 6). Note: selecting the link actually runs Crystal Report (a Business Intelligence tool) to generate the full report that includes the graphic in Figure 6.

Sample TPIER report

Figure 6: Sample TPIER Report

The data from this report can be integrated with your data to create benchmarks against which you can compare your own performance and identify potential areas of operational and educational performance. Below is another example of publicly available data from the University of New Mexico Provost’s Analytics Dashboard. The analytics are part of the University of New Mexico’s UNM 2020 initiative. As an example, this report compares the 8-, 6- and 4-year graduation rates of a select group of universities.

4 year graduation rate: 2010-2011

Figure 7: University of New Mexico Provost Dashboard


There are a multitude of opportunities for universities and colleges to make use of Big Data to improve student and professor satisfaction and performance. And although there is competition among universities in recruiting students, all can benefit from sharing data, analytics, and best practices in areas such as student performance effectiveness, student retention, teacher retention, and more. In a world where education holds the greatest potential to drive quality-of-life improvements, there are countless opportunities for educational institutions to collaborate and raise the fortunes of students, teachers, and society as a whole.

About Bill Schmarzo

Read More

Share this Story
Join the Conversation

Our Team becomes stronger with every person who adds to the conversation. So please join the conversation. Comment on our posts and share!

Leave a Reply

Your email address will not be published. Required fields are marked *

3 thoughts on “What Universities Can Learn from Big Data – Higher Education Analytics