How to Guess Less

Sometimes you have to guess: you’re constrained by time or resources, and you just need to move forward with a decision. But it’s 2020. Why are we still guessing on measurable things that have a huge impact on how we work and how students succeed? 

If you’re still guessing on any of the questions listed below, drop everything and finish reading this post:

• Where should we deploy this software?
• Which students need access to these machines?
• How will students find the technology resources they’re paying for?
• Do we have enough accessibility resources to be compliant?
• How can we keep faculty and staff happy?
• Where can we find more space to schedule classes on campus?

Full disclosure, we’ve had to guess sometimes at LabStats. Crazy right? How could a data company make non-data-driven decisions? It’s tough, but we’re actively trying to guess less.

1 – Get Data

Start by collecting every piece of data you can. Even if you know you won’t have time to review the data until next semester or next year, gather it now so when the time comes you’ll have a semester or year’s worth of data to start with. You’re never going to say: “we have too much data” or “we don’t need a historical baseline.”

LabStats automatically collects usage data on hardware, software and users on every machine where the client is installed. That’s a lot of data, so you’ll need to configure LabStats to reflect how your school is set up.

Configure the following items:

1. Track Applications – By default, LabStats tracks a list of local and web-based applications, but you’ll need to assign any additional software or websites you’d like to track.
2. Set up Groups –  Groups act as a framework for your data in space. Assign machines to groups based on location (ie: classroom, lab) to help with discoverability and reporting.
3. Define Schedules – Schedules are a framework for your data in time. Usage will look very different for times when class is in session and times when it’s an open lab, so setting up schedules will help you differentiate between the two.
4. Create Tags – Tags help you track anything else that’s important to you – collaboration space, department, major, accessibility feature, etc.

Once LabStats is configured to reflect your unique environment, you can move on to the next step.

2 – Define Values and Goals

It’s important to understand your institution’s values and goals. 

If you’re having trouble defining goals, or connecting school-wide value to your everyday work, consider how your work is evaluated. What type of accomplishment would merit recognition, more funding, a bigger team, or a promotion?

In our research with colleges and universities around the world, we’ve come across three common goals:

1. Financial efficiency such as saving money, finding waste and achieving optimal usage percentages.
2. Improving student experience by reducing complaints and understanding lead indicators of student success.
3. Reducing risk by maintaining compliance with rules and regulations.

Defining what’s important to your school now, and what you want to achieve builds valuable context. Without context, data is meaningless. 

3 – Get Buy-in

If you don’t consider your team, department or organization to be data-driven, you can still use data to spark change. Data is never the end goal, it’s just a tool. 

Even in the most data-centric schools, people don’t care about the data itself, they care about the impact it has on their lives and those around them.

So what do you and your team care about? What does leadership care about?

• Saving time and effort
• Being seen as an innovator 
• Helping students succeed 
• Having budget for something else 
• Reducing risk 
• Reducing student or faculty complaints
• Increasing job security
• Meeting compliance regulations
• Managing staff efficiently

If you can show people how data can help them achieve their goals, they will see data as a tool, an ally in achieving their own goals.

What if people see data as a threat?

Data scares the stuffing out of a lot of people. Some may fear that usage data will prove that their resources aren’t getting used, and then they’ll lose funding.

But that’s not always the case. We recently met an accessibility team that was afraid if they started tracking the usage of accessible stations and software, leadership would radically cut back on those resources. 

What we’ve learned working with hundreds of schools is that low usage doesn’t automatically mean that the appropriate course of action is to remove the resource. A computer, lab or software application that doesn’t get used could be due to a wide range of factors including: 

• A discoverability problem:
  • The basement lab has the newest computers but never gets used. Rather than closing the lab, implement LabMaps and LabFind to help students find the computers.
• Competition with BYOD: 
  • There are 20 computers in the Student Center, but students just plop their laptops on the desk in front of the provided computers. You could clear out those desktops and transfer the budget to create more lounge areas and charging stations.
• Wrong resource:
  • Your statistics department might be providing 20 computers with SPSS because that’s what the professors like to use, but students prefer R Studio. You could cancel the SPSS licenses and move the funds to fill another need in the department.
• Misplaced audience: 
  • A room with speech-to-text software may show very low usage because students are using the room for a quiet place to study. Rather than removing the software, you could designate certain rooms for speech-to-text use only, and other rooms for flexible study space.

In all of these scenarios, rather than just cutting funding, you can identify a need and transition a portion of the budget to fill that need. Data doesn’t have to be threatening. To get buy in, build the picture that data can help your team and other departments reach their goals.

4 – Design Experiments

Now that you have data, goals and buy-in, you can start asking a series of questions. Questions should lead to experiments. Every school is different, but one school may look like this:

1. What do we think we know?  – We’re fairly certain we need more classroom space.
2. What evidence do we have to support it? – Department heads are complaining that there are not enough classrooms for next semester’s proposed classes.
3. Without data, what would we have to guess? – We’d have to guess where to schedule additional classes.
4. How can we get data to fill those gaps? – We can use LabStats data to see which labs are underutilized.
5. How could we use data to design an experiment? – Our experiment will include finding an underutilized lab and scheduling a class when it’s typically not busy. Then we will evaluate if a dual lab-classroom space is feasible at scale.

What if I don’t have the time or resources?

Designing experiments takes time and resources. Many decision makers feel like they don’t have time to design experiments, analyze the results and then make decisions with data. This is something we’re very focused on for future product updates, but we have a few suggestions in the meantime.

Consider people outside of your department that you could partner with. Most institutions have student researchers, data science departments, business research classes, research methods classes and stats courses. Reach out to a professor or grad student to see if you can come up with a project that benefits both parties. 

Many data science professors have difficulty finding data that students understand. LabStats can provide a robust data set that’s relevant to students. Project collaborators can be invited into the LabStats product with limited permissions.

Think outside the box. Interior design students are also a great resource for collaboration. We met a group of students that were trying to understand how accessible resources in computer labs were used. They observed students and manually tracked activity on a notepad. Imagine how much more of an impact they could have had if they were able to use LabStats data to tag accessible hardware and software across campus, and see usage at a larger scale?

Management, education, user experience design and other majors could also be great partners to help you complete data experiments and improve your campus in a variety of ways. 

Know When to Guess

There are going to be times when you have to guess. A good rule of thumb is to be upfront about your assumptions, so that your whole team knows that things could change. Guessing is appropriate in these two very common cases:

1. If the cost of not making a decision right now is higher than the cost of doing the research.
2. If the cost of getting the data is higher than the cost of being wrong.

Having to guess is an opportunity to conduct an experiment. At LabStats, we often don’t yet have the data to drive a decision, so we make an assumption, pick a course of action and make sure to capture results to see if that was the right decision. It’s trial and error with a strategy to track the results.

A data-oriented approach will keep you from needlessly guessing, but you need to remember: “All models are wrong; some models are useful.” –George Edward Pelham Box, British statistician.

While we would never claim that LabStats data could help you create a perfect model, we know from the hundreds of universities we’ve worked with that LabStats can help you create useful data models from which to make decisions. 

As you start to gather data and tackle goals, reach out to the LabStats team with any questions you may have at [email protected].