S1:E26 Suddenly Online

Tyler Jacobson  0:01

Welcome to LabChats, a podcast from the team at LabStats. I’m Tyler Jacobson, your host for today’s episode. Each week we’ll sit down with technology leaders in higher education to get the latest buzz and insights while we discuss current events, trends, problems and solutions. Now let’s get into it.

For LabChats this week, we’re going to do something a little bit differently. I wanted to take an opportunity to review a study that was published, entitled “Suddenly Online: a National Survey of Undergraduates During the COVID-19 Pandemic”. This survey was sponsored by the Bill and Melinda Gates Foundation and the National Science Foundation. And it is taking a look at the abrupt shift from on site to online education during the COVID-19 pandemic. The study looks at student satisfaction when they are transitioning in the middle of a term from on campus to online educations. And they looked at problems that were experienced with technical implications, student struggles and things like that. A lot of the feedback was given on instructors [and] the student’s ability to collaborate with fellow students. Because a lot of students did have problems with their internet connection software computing devices. Now they refer to in the article, they refer to it as the COVID-19 experiment, because a lot of the colleges and universities, all of the colleges and universities had to transition to an online education by pandemic not by choice. And so they entered what they refer to as a triage mode in the spring of 2020, where they had students that were not used to receiving an online education and instructors that were not used to presenting in an online format either. The survey was completed by the digital promise and Langer Research Associates, and they developed the Survey of Student perceptions of remote teaching and learning. The study was composed of 1008 students that were 18 years old or older. And they crossed urban and rural demographics across different racial categories two and four year colleges. And the study statistics came out with a 3.6% margin of error. And so one of the first things that they emphasize is that there is a significant difference between a traditional online education and what happened in 2020. They refer to it more as an emergency remote teaching than an online education. However, for the purposes of the article, and for the study, they did consider what was occurring in 2020 as an online education. Now, to put the findings into context prior to 2020, 43% of the people surveyed in the study had no online experience at all. 21% had had one online course prior to that year, and 35% of the respondents had two or more online experiences. Now, the format of the different courses was very different in certain areas. 67% of the students reported having synchronous classes, so they were held live. 65% had offered recorded lectures, and 64% had frequent quizzes and assessments in order to give regular benchmarks. One of the things that is of interest, though, is only 25% of those reported, students in the survey reported having breakout groups during the live class. Now the grading was another thing that was a significant consideration. Because of the change in format. There were a lot of concerns with consistency and also availability of the required resources to complete the coursework. So some institutions mandated that all courses be considered pass or no pass. Others extended the timeframe in which the students could opt to have a letter grade or go with a pass no pass option.

Now, we can get into some data on how the online format impacted students’ satisfaction. Student satisfaction with their course before and after the move to the remote: before the pandemic 87% of students reported being either “very satisfied” or “somewhat satisfied” with the level of instruction. Now, when that was adjusted to online that shifted to 59%, who were either “very satisfied” or “somewhat satisfied” with the course, only 19% of those are reported being very satisfied in an online format. That gives a difference of 28% of students that considered instruction in class as “satisfactory” or “very satisfactory”, versus after they went online. Fewer than half of the students expressed dissatisfaction with their courses. But there is a very distinct difference between the satisfaction ratings on site versus online.

There were several aspects of the course that they broke down to further assess satisfaction levels. 76% of students were satisfied with their online courses in regards to their instructors’ preparation. 71% were satisfied with the quality of the course content, and 68% were quote, “more satisfied with the quality of instruction”. Now these percentages are an aggregate of the “very satisfied” as well as “somewhat satisfied” ratings. Their learning overall, however, just barely was over 50%. It was 57% with only 17% of those being very satisfied with their learning overall in an online course. When asked to explain some of their frustrations, they broke out several categories of what the experience was prior to the shift from on campus to online. The students reporting that their experience was better online was always a minority. Far more students considered the instruction, their experience to be worse online. So for instance, “opportunities to collaborate with other students on coursework”. 8% of the students reported that that was better when they were online. However, 65% of the students reported that the “opportunities to collaborate” were worse online than they were in person. 12% of students online said that they had a better understanding of what was expected in the course, where 34% assess that they didn’t understand as well, what was expected from them in the course when they were in an online format. Other areas that they looked at that follow a similar pattern is the instructor’s knowledge of the students strengths and weaknesses, the availability of help with the course content, feeling of being included as a member of the class, and being interested in the course content. Now, technology access was one of the other things that was studied. 

The internet availability is very critical for the success of an online coursework, and the internet had more impact on the student success than hardware and software issues did. Internet connectivity, for those that responded to this study were: 95% of the students were using the internet that they had at home. 3% of them went to the school or had other internet arrangements that were provided by the school, and 2% of them used a public space for their internet access. The frequency of internet connection issues that interfered with course participation was varied. 20% of the students reported that they never had a problem with the internet connection that prevented them from participating. However, 6% of the students responded that it often was the case. The devices that the students were using were also varied. 79% of students reported that they had a laptop that they were able to use to complete coursework. 15% were using a desktop, 3% used a tablet, and 2% were using a smartphone. 10% of the students use a device that was shared whether that was shared within their own home or whether they were using a public device at like a library. 3% of them were using devices that were provided by the school. 23% of students experience hardware or software issues that were serious enough to impact their ability to attend or participate in their coursework, even occasionally, which was “very often” or “often” was the 23% number. 

The survey also included several open-ended questions on what the student’s greatest challenges were. There were several themes that did come to the surface. The number one theme was self-motivation. There were a few quotes that were cited in this study publication. For example, self-motivation: “With the world in chaos, it was hard to stay focused and motivated to mentally show up for class”. Another quote, “The greatest challenge was finding motivation to get out of bed and complete assignments. It’s not the same as getting up, getting ready, driving to class and then sitting in class to learn.”

The second topic that was a theme was relating to the missing presence of an instructor, one quote, “Not being able to ask questions during the lecture since it was pre-recorded. That being said, my teacher was very available to help at any time. So I took advantage of that,”. In association with the unavailability of the instructors also means that things like labs were also not possible. And so a lot of students were feeling the pressure of not having the hands-on experience available to them. Many students highlighted the loss of authentic hands-on experiences that helped him to develop understanding in the particular subject that they were studying. Some of those quotes that were included in this study were: “Part of the class was a lab where students could do experiments and see results. After the class went online, all the labs were cut out of the curriculum. I had a harder time understanding the theory and relating to the practical.” Other students felt like they adapted very quickly and did not consider that to be a problem. When they broke out the severity of those problems staying motivated to do well in the course was above the others, as far as being considered a major problem. 42% of students considered it a major problem to stay motivated and to do well in the course. Others that were not as much of a critical issue, for instance, fitting the course in with your work schedule, if you work for pay. 68% said that that wasn’t a problem at all. When they broke some of the information out, they looked at things such as household income, geography, where, whether it was rural or urban, and then also based on racial groupings. One group that seemed to stand apart from the others is the Hispanic population. I’m going to review one of the charts that was included in this study. Under the heading of staying motivated to do well in the course 48% of Asian and other racial groups considered that to be a major problem, where only 31% of the black respondents cited staying motivated to do well in the course as a major problem. In the other categories that they outlined, the Hispanic populations seem to be the most frequent ones to experience the following problems: “finding a quiet place where someone could do the course online.” 27% of Hispanic respondents cited that as a major problem. “Not knowing where to get help with the course,” 24% of Hispanics cited that as a major problem. And then also “fitting the course in with your work schedule. If you work for pay”, 14% of Hispanics were reporting that that was a major problem. The Hispanics were also the most likely to have an internet connectivity issue. 23% cited internet connectivity as a problem that was occurring often or very often. 

Not surprisingly, when they broke down by income, high income households where the household income had $100,000 or more in annual income. 12% of the students living in those incomes had internet issues. Now the low income, which was classified as $50,000 in annual salary or less, had 20% reporting that they had internet connectivity issues often or very often. One surprise was the rural communities were no more likely to have internet connectivity issues as the urban or suburban communities. So that divide was based far more on household income than it was on physical geography. 

Hardware issues were less common. In high income households, which makes sense as they can afford higher quality equipment. Now, there are predictors of student satisfaction as with their online course, and they actually had a list of eight recommended practices for online instruction, I’m going to read the list off, because they broke down the predictors of success based on how many of these different practices were included as part of the online instruction. Number one is assignments that ask students to express what they have learned and what they still need to learn. The second item, breaking up class activities into shorter pieces than in an online course. The third, frequent quizzes and other assessments. Four, live sessions in which students can ask questions and participate in discussions. Five, meeting and breakout groups during a live class. Six, personal messages to individual students about how they are doing in the course or to make sure that they can access course materials. Seven, using real world examples to illustrate course content. And eight, work on group projects separately from the course meetings. Now, the net satisfaction for courses that employed two or less of those recommended practices was 43%. Satisfaction for the courses that offered three to five of those eight best practices was 61%. And 74% of students that [were] offered six or more of those eight practices experienced a net satisfaction. Those satisfaction ratings sound very powerful. However, when you look at those for the students that had classes that had two or less of those best practices that were included, there were 57% of the class attendees that were dissatisfied with their education. The dissatisfaction rate, with the three to five of those eight components was still at 39%. And even those that were very successful in implementing those best practices, and had six or more of them, there were 26% of the students that were dissatisfied. Some of the instructional practices are things that are very controllable by the instructors, such as the three that stood apart from the others were personal messages from the instructor on how the student was doing to make sure that they get access to course materials. 68% of the students who received messages, specifically from their instructors, were considered satisfied with their course overall, compared to 47% of those who did not. The instructors’ use of real world examples to illustrate course content was another strong predictor of student satisfaction among students, when they had instructors that did this 67% were satisfied overall. Assignments that required students to express what they had learned, and what they still needed to learn was a predictor of 68% of the students receiving that as being satisfied in their coursework. Those are the three most closely tied instructional practices with student satisfaction. 

Class size was also something that was assessed. In theory, an online program could have unlimited enrollment. Before COVID while they were on site, the difference in satisfaction and dissatisfaction ratings was largely irrelevant when considered by class size. 11% of students that are on site were dissatisfied in a small class, 12% were dissatisfied in a medium class, and 16% were dissatisfied in a large class, which is only a 5% spread between small and large class sizes. However, when that education transitioned online, the dissatisfaction ratings for students in a small class was 35%, 47% in a medium class size, and 62% in a large class size. So despite the fact that it is online and the students are not sharing a physical space, the spread between student satisfaction in a small class size versus a large class size increased from 5% in an on site environment, to 27% in the online environment. So despite the lack of sharing of physical space, class size was a significant consideration in the online class presentation. Now, when they accounted for all of the challenges that students faced, it makes sense that the more challenges that the students face, the less satisfied they were with their experience. Students that reported never having a major challenge: 81% of them were satisfied or very satisfied with the online course. Now if they had one major challenge, that satisfaction rating dropped from 81 to 52% reporting a satisfactory experience. If they had two or three major challenges, that number was 44%. And then if they had four or more major challenges that they experienced, there was a 32% satisfaction rating, which means 68% of students were dissatisfied with the online experience. 

One other thing that is not surprising, but they were able to verify is that students that participated in an online course prior to the pandemic were more likely to have a satisfactory experience than students that had two or more online courses prior to the pandemic. 23% of them were very satisfied. For students that had never had an online course prior to the pandemic. Only 17% of those reported being very satisfied. There are ways that we can cope with these challenges. Instructors seem to be very key, the course structure to get frequent feedback and frequent benchmarks, personal messages from instructors, were very important in student satisfaction. Online cannot mean alone. Faculty training and content restructure will bridge the gap between online and the emergency remote teaching that we were able to experience last year. 

I hope you enjoyed this very quick review of this research study. I will put the link for the study if you want to see things and read it for yourself. I appreciate you joining us for this episode of LabChats. We hope to have you back for more conversations in the future. That’s all for today’s episode of LabChats. Be sure to subscribe so you’ll be notified when a new LabChats episode is posted each week. We’ll see you next time.