S1:E12 John Felushko – Using existing data to answer new questions

Tyler Jacobson  0:00

Welcome to LabChats, a podcast from the team at LabStats. I’m Tyler Jacobson, your host for today’s episode. Each week, we’ll sit down with technology leaders in higher education to get the latest buzz and insights while we discuss current events, trends, problems and solutions. Now let’s get into it.

Tyler Jacobson  0:19  

Joining us today we have John Felushko, who is the Product Manager here at LabStats. And the reason that I wanted to have you join us, John was you have done an extensive amount of speaking with universities and colleges about how they’re using data and the value they’re getting out of data. I wanted to get some of the insights that you’ve received [from] that. Let’s start with a big vision of: What should we be doing with the data that universities are collecting? 

John Felushko  0:52  

Whatever you want! Data is a tool that helps us make better decisions. And the more we have of it, the more easily accessible it is, the more it’s combined, the more interesting questions we can have. And the more quickly, we can see the impacts of our decisions and make the decisions. So we have this concept in software of failing fast. The idea is…really being humble as decision makers and saying, “We don’t know what the impact of our decisions are going to be when we make a change. We don’t know what’s going to happen. So let’s make sure we’re watching it using the scientific method, having a hypothesis and seeing what changes.”

Tyler Jacobson  1:40  

So basically treating every change that you make in the infrastructure, everything you do on the university as an experiment, and then capturing what impact it has. 

John Felushko  1:51  

Absolutely. That’s been proven in many different businesses across a lot of time, to help you get to your goals more quickly, at a lower cost with less risk.

Tyler Jacobson  2:06  

One of the things that I have often had conversations with [school staff about is] they would like to use data to make data-driven decisions. However, it costs a lot to collect the data. And so a logical first step is, what are they collecting now? And how are they using it now?

John Felushko  2:26  

Yeah, I think when I talk to schools…schools have a huge range of their data literacy, both between organizations and across organizations. But a lot of people are incredibly surprised about how much data exists in their organization already, going back for [a long time], and how easy it is with modern tools, to combine it and provide the context they need to make decisions.

Tyler Jacobson  2:54  

What data streams are they collecting? Now, give me some examples of what they have to work with without any additional expense. 

John Felushko  3:01  

Sure. So you have your student management system. So you have a lot of information, first of all on your students, right, and your students and how they’re doing and how they’re performing are tied to many, many of the goals that learning organizations have. You have your financial system. [It] contains all the records of what you spent, all the bills that got paid. You have your feed from your learning management system that tells you about how students are interacting with resources. You have your library’s computer system that tells you who checked out and who looked into and who accessed what resources and when. Those things were on every campus I’ve ever talked to. And they provide a huge amount of data. And they can be combined with stuff you probably don’t think is related, like your network system. Who logged in? What’s our login rate per Wi-Fi? How many devices are logged in? That tells you a huge amount of information about who’s where in space. What spaces are getting used? And I’ve seen these combined. I was at a campus last year where these were combined…to know whether students were in class or not. Without any new tools. Just new analysis of the data that they already had.

Tyler Jacobson  4:22  

Some of those data streams that you had mentioned, seem like they’re very department specific. For instance, the library data, how is that valuable outside of the library? And how do you get it to people that are outside the library in a usable manner?

John Felushko  4:42  

So, [a] really simple question a lot of people have is: “What students are succeeding?”, “What [are] students going to do well?” and “What students [aren’t doing] well?”. For a lot of programs, there’s a pretty strong association between what library resources the students are accessing and how they perform in the course. So you can use library data as a leading indicator of students performance in their program, if they are in history or political science, and then they’re not showing up in the library system on a regular basis and accessing stuff from the library on a regular basis, [that’s] a pretty good indicator [that] they’re not going to get their research papers done.

Tyler Jacobson  5:28  

How can a university share that data interdepartmentally as well as up and down the chain of administration.

John Felushko  5:35  

So there’s a lot of great tools out there. Generally, they’re called business intelligence tools, BI tools, I think, because they have the word business in them. And intelligence. People think they’re the specialist domain of analysts, but I’ve been in a lot of schools where they’re being used across the organization, I was at a major customer that in their IT organization had over 50 databases, streaming into their business intelligence system. And when I first saw that, I thought, well, this is years of work. This is a huge amount of effort. This is beyond my capability to understand. And going out and playing with those tools, we found that they’re very, very easy to use. So the most popular one in our research universities is Power BI, the next most popular is Tableau. If you’ve interacted with your state, or country’s COVID dashboards, in the last year or so you’ve probably interacted with a Tableau dashboard, or a Power BI dashboard. And every single school I’ve been at, in the last two years, has had those capabilities somewhere on campus and specialists [or] analysts for the summer on campus. And very often they’re paid for already. The[se tools are] free to add users [and] free to download.

Tyler Jacobson  6:58  

So how would somebody identify who on campus is their data analyst, because a lot of the people that we speak with, they’re working with the data within their department, what would be the first step for them to get some of this bigger picture data analysis?

John Felushko  7:18  

Just start talking. If you ask…when I get a couple people in a room, and I ask, “Where’s an analyst?”. Usually somebody knows. Very often they report to the CIO, or the CFO. There’s some department somewhere in your school that has a bunch of analysts. There’s some of the most posted jobs we see from universities. They’re out there. But you don’t need them. Honestly, you can just go download, figure out what your school’s using. If you’re a Microsoft brand school, go download Power BI onto your desktop, start playing with it. And there are links to most major applications built into Power BI. Like our devs here, say, it’s mostly a matter of knowing what to Google.

Tyler Jacobson  8:03  

How easy is it for departments to share their data streams with each other?

John Felushko  8:10  

if you have the permissions…and this is where the politics of your organization work out, but if you have the permissions? Minutes. It’s minutes to get the data into systems. So an example would be: “I want to understand how people use my computer labs who aren’t logged into the machines…How many people are in my computer labs that aren’t logged into computers?” I can take my data from something like LabStats that has a record of everybody who logs into computers. And then I talk to my network guys and figure out where the data exists for logging into WiFi. Most places I’ve talked to have what’s called an API, an “application programmer interface”. It’s a place you can access that data online and Power BI and Tableau have hooks to those. And you can get the list of who logged into the Wi Fi in line, who log on to your lab computers. And when and by comparing those two sets of data, you can see how many more people were in your computer labs than are logged into your machines.

Tyler Jacobson  9:22  

How does that help? What additional insights is that giving? 

John Felushko  9:26  

Well, we see very often when we go to a conventionally set up computer labs, [we see] rows of desks, for example. You know, next to machines [and] next to each other, that all the seats are in use or close to all the seats are in use. But less than half the machines are in use because people are working as groups together on projects. You might be getting a very false impression from only looking at one data source of how many people are using that space. And if you find out: “Hey, wow, there’s like…for every log in to our machine, there are six or seven devices logged into the local wireless network in that space, we know that the average student has…2.7, something like that wireless devices with them at any given time. So that number, you can kind of divide that by two and a half or whatever, and say, okay: this many people, we have 90 people logged into the wireless in that lab, we only have 20, people using computers, oh, there’s a bunch of groups in there. And when you go to say, reorganize that space, you can reorganize that space [so] each computer is at a separate desk, or you have a bunch more [space per computer]. And we’ve seen this on a lot of campuses, where there’s station computers with big screens, and four or five chairs, around the perimeter of the lab. So people can come in and do group projects on those machines. Some schools call those pods, for example. And by looking at that kind of data that supports how many pods doing, how many group projects spaces do I need? Maybe I don’t need more machines, I need those same machines spread out over more space. So I can accommodate more group work. Here’s the data that supports that. And that’s really simple, just combining two sets of data, your wireless login with your LabStats, you could then further combine that with your student management system data to figure out who’s using it. One of the things we see anecdotally is that [it] tends to be a much higher percentage of foreign students, first generation students and disadvantaged students using those public spaces and those computers to do work. Because they might be living in dorms, they might not have big computers and big screens accessible in the home. If one of the goals for your school is to help those groups of students succeed, it’s really great data to have so your student management system can be tied to the log in counts, the WiFi accounts and figure out who’s using what spaces.

Tyler Jacobson  12:07  

With the transfer of information or departmental managers using this data in the same way? For instance, the Director of IT. What types of data is he using? And what questions is he asking for that, versus the CIO? Are they using the same data sets and asking different questions? 

John Felushko  12:28  

Very often they’re using the same data sets. They’re asking very different questions that need the data structured in very different ways. I regularly hear about CIOs who don’t want to open anything. Or if they do want to open something, they only want to open Power BI. Everything else reports up to them in Power BI or Tableau and they have a dashboard. And a lot of our customers, for example, in the IT world found that the best way to communicate with the CIO is to build a custom dashboard in whatever tool he’s currently using. That seems like a huge operation, but honestly, it’s a few days worth of work once you have access to the data. And he probably wants to see the same report structured the same way over and over and over again. You can also set up a lot of these BI tools to do things like make a PDF of that graphic, and email it out to them on a regular basis. So they didn’t have to open anything. In their email every Monday morning or every first of the month. Here’s a snapshot of what’s happening on these systems [and] what’s changed. They can go and take that to their meeting.

Tyler Jacobson  13:34  

Are a lot of the administrators using these databases and these BI tools as their primary source of information? How’s that information flow from different levels of management?

John Felushko  13:45  

We hear totally different things. We have a couple of CIOs that we’ve worked with who are in the data itself, asking questions, using it as an ad hoc reporting tool to ask really sophisticated questions all the time. I talked to one, one of our most successful customers who was able to save millions of dollars in their IT infrastructure. And she would just go home every night and run report after report after report. And then we run into other people and they’re like: “My boss wants a printout on paper.” And we run into the full range, right? It depends on your institution and the personalities you’re dealing with. But somebody at some level is going to be asking interesting questions, doing ad hoc stuff [and] combining data with all kinds of different sources.

Tyler Jacobson  14:30  

So how easy is it to transfer that you had mentioned, some of them are going to be printing things out and sending that. Obviously, that’s handled natively on your device… to hardware that you create a hardcopy and send that over. Aside from that, how easy is it to share some of these dashboards with other organizations and configure them according to what they need without giving them things that are irrelevant?

John Felushko  14:59  

So all of the ones I’m familiar with all the BI tools I’m familiar with have web versions, where you can publish or natively create reports, dashboards, metrics, alerts on a website. And as long as you have the permissions and pay the licensing, and it’s usually a few dollars a month, people can just view that on a website. Send them a link, “Here’s your custom dashboard on a website, check it any time refresh the data, anytime you want.”

Tyler Jacobson  15:27  

So you mentioned a few dollars a month: What are the total costs to institute something like this interdepartmentally?

John Felushko  15:34  

It really varies by how much data you have. It can be as cheap as free. With huge amounts of data, it can be 10s of 1000s of dollars a month for billions of rows of data. A lot of schools, big schools, big institutions, they’re already paying those resource costs for handling billions of rows of data, because that’s what your student management system needs. That’s what your financial system needs. You’re adding one more thing? Very little marginal cost. Adding a few 100 million more rows of data in most of these systems? Next to free.

Tyler Jacobson  16:15  

So being able to understand what students are using for multiple angles…I can see the value in that. How does that impact the students? With all of this data, [it] sounds like it’s going to the administration? Why would the students care?

John Felushko  16:33  

Well, the kind of first and most obvious thing is there are so many resources on campus that are underutilized. That students don’t know exist. Until you know what the outcomes are and the usage is, and specifically who’s not using the resources you’re already paying for…this is the thing we see time and time and time again. When you understand who’s using and not using resources, you know how to target the information, the educational campaigns, the technological resources to get to people. Because it doesn’t matter what you have. In any domain, whether that’s computers, technology, services, grants, tutoring, food pantries, any of it. Unless the people who need it, know it’s available. And the first step in targeting that, those campaigns and changing the behavior of students is finding out who’s using and not using the resources so you can target it.

Tyler Jacobson  17:46  

If they’re not using it, is it because they’re unaware of it? Or is it because it’s [not] needed?

John Felushko  17:51  

So that depends on…you know, that’s where the data really comes in handy. Let’s say I’m finding that in a particular program…there’s 30% of my students [that] aren’t using the resource. Say it’s public CAD machines. Machines with CAD in an engineering course. You can start slicing that data by reporting the data from your student management system and say, “Okay, this group that’s not using public resources, what’s the common thing between them? Is it mostly students from a single instructor? Is there an instructor who doesn’t know these resources exist? And isn’t telling [students]? Is it some demographic? First year, second year, third year, or first generation, second generation, third generation students or local students versus international students? What are the groups that are finding out about this resource?” By looking at resource usage, combined with your student management systems data, you can figure out who’s not having it. And it might be as simple as: “You know, what? Instructor Jones? New instructor doesn’t know about this?” Let’s go have an intervention with Instructor Jones, and say, “Hey, I wanted to let you know that there’s a 24 hour lab for engineering students with CAD software. Let’s take you to it. Let’s give you some slides to talk about in your next lecture to your students and say, ‘Hey, we want to let you know that there’s these things.” And that simple problem…how do you educate your part time and adjunct instructors into what resources exist on your campus? Is something that’s really difficult to solve through just mass communication and blasting it out. But if you can look at the result, whose students aren’t accessing the resources, you can target your efforts where it’s needed.

Tyler Jacobson  19:49  

Are there any cases where any of this data would be valuable for the students directly?

John Felushko  19:55  

Yeah, absolutely. Wouldn’t you love to know, when you were a student, what successful students do? Wouldn’t you like to know what a successful strategy is? So you’re not wasting your time making stuff up? So I’ve seen some really interesting work, where tools like this were used to analyze the difference in study habits and resource access between students who got A’s, B’s, C’s, and D’s. And then when they intervene through their student management system with the students, they were able to say, “Hey, very early in the semester, you are not accessing these resources on a regular [basis].” And whatever it is: tutoring, [the] library, technology…And say “Hey…we don’t have any grades for you. But your pattern of behavior isn’t the same as the students who historically get A’s. You might want to consider coming to the library more often and accessing the software more often, getting into your classes more often.” Whatever that information is. And that’s proven to be an extremely successful way of changing students behavior, before they’ve seen their first grade. And the goal is to help them succeed.

Tyler Jacobson  21:15  

In my household, we have several college aged-residents,

John Felushko  21:21  

Congratulations.

Tyler Jacobson  21:23  

While we also have a couple of high school age as well, so we have a very interesting mix of perspectives. Now, when I talk to the college attending, when they talk about what they want to know about what’s going on on campus. A lot of it is: “I just want to know how to find what I need. I want to know where it’s at, I want to know what’s going on on campus.” Basically, if I need something, I want to know how and where to get it. And if it’s available, and it’s everything from parking, laundry service, cafeteria menus, computer labs, where’s the software, they need class schedules, there’s just this list of things that they want. That in a lot of cases is extremely challenging for them to get, because it’s available in 30 different places.

John Felushko  22:17  

Right. 

Tyler Jacobson  22:17  

So what options do schools have as far as notifying students and what resources are available?

John Felushko  22:24  

So we’re seeing a lot of people push data to a single student-focused app. That’s a very common solution. And that allows data from a whole bunch of different sources to be layered in such a way that it appears to be one interface to students. So we see stuff like Canvas maps, that as you zoom into them show you rooms, and when you click on the room, say: “Hey, do you want to book this room?” Or these are the computers in this room, or this is the software available in this space. We’re seeing those kinds of things with directions and hours and notifications built into a map on a Canvas app. That’s extremely common. Again, those apps are powered by these API’s that all of your systems are using. And API’s allow systems for multiple places to talk to each other. LabStats, for example, supports with our API’s, supports bringing information about any kind of real-time usage of your computer system into those maps and calendars. So that people can see what’s being used or not used. All kinds of, you know, lots of booking software, library software, all kinds of other resource software.

Tyler Jacobson  23:30  

And that makes sense because as I’ve talked to people about our mobile app, and our ability to do computer lab mapping…it’s a great added feature. But it’s not the solution. Because relaying that to the students is problematic, because in a mobile app, you may have a mobile app that’s limited to just computer software utilization, which is a hard sell to get students to download onto their phone. 

John Felushko  24:01  

Yup.

Tyler Jacobson  24:01  

So if you have a campus app that integrates five or six different channels of information, they’re far more likely to download it and even beyond that, they’re more likely to use it on a regular basis.

John Felushko  24:13  

Yeah, there’s a real split. You know, sort of Gen X and the older millennials tend to be Google-focused, go to a general purpose browser and google it people younger than that are much more likely to use apps on their mobile devices as their primary way of answering questions. And there’s just a mindset shift that’s happened.

Tyler Jacobson  24:40  

So combination of possibly putting it on the website, as well as having a robust mobile app would bridge between the two different thought processes.

John Felushko  24:51  

 The two different technology usage patterns, we could call it? Yeah.

Tyler Jacobson  24:56  

Okay. Is there anything else that hasn’t been covered as far as big data on campus and the value as well as overcoming that sense of that data being siloed in one area or another?

John Felushko  25:14  

I think the big takeaway for me is I’ve learned about these things is just realizing that whatever data exists, there’s a way to combine it with all the other data that exists. So virtually any question I can answer. Any question I can ask, there’s data out there that I can get at. And as I’ve learned about these tools, and gotten familiar with them over the last few years, my confidence level in my ability to answer any question I have, has gone much higher. So I have to ask whether it’s worth it. I still have to ask whether it’s worth the effort to find the answer. Do I really want to find the answer? But I know that if I do, I can, it’s technically possible. And it’s pretty straightforward.

Tyler Jacobson  26:03  

Well, I think that plays into conversations that sound like we both [have] had with people is: some things are just not high enough priority to put a lot of effort into. Others, you’re afraid you know the answer (laughs), and having it in print in front of you could be something that prevents you from being able to ignore it any further. 

John Felushko  26:26  

Right. 

Tyler Jacobson  26:27  

And then also, my impression is there’s a lot of different siloing, between departments, and bridges of communication between different layers of administration, these different data sets coming in from very different directions. Compiling them all is going to be partially the solution for really cooperating from different levels of administration and across different departments, where traditionally, they have operated as their own universe.

John Felushko  27:01  

Yup. Yeah. And I’d say, you know, my method…it’s different for everybody, but my method is go: “Okay, what are we trying to accomplish? What questions do I have? [What questions] do I need to answer to be able to decide on a course of action?” And very often, my first question is “What have we done before, and did it work?” 

Tyler Jacobson  27:25  

Defining what you mean, on “Did it work?”

John Felushko  27:28  

Did it work?

Tyler Jacobson  27:29  

…is a very key piece of that puzzle. 

John Felushko  27:29  

Right. And then once I’ve done that, now, I know I can go…the data probably exists somewhere. And with a little bit of self-education, a little bit of reaching out to people better at analyzing than me. I’m able to get to those answers pretty fast and painlessly.

Tyler Jacobson  27:46  

I just want to say, John, thank you very much for all of your efforts. I know that you’ve spent a lot of time over the last year to not only make LabStats data better, but also make sure that people have the ability to utilize it in the ways that they are using data and incorporating it into other things. And I know that our customers have appreciated that. And it’s going to be key for them to integrate LabStats data as one component of many different data sets that they are leveraging to truly make data-driven decisions.

John Felushko  28:21  

Thank you. And when we should be making that easier and easier as time goes on.

Tyler Jacobson  28:24  

Thank you for joining us.

Tyler Jacobson  28:27  

That’s all for today’s episode of LabChats. Be sure to subscribe so you’ll be notified when a new LabChats episode is posted each week. We’ll see you next time.