S1:E4 Troy Taysom – The SolarWinds incursion and data security

Tyler Jacobson  0:01

We have joining with us [today] Troy Taysom. You’re with Utah Valley University, correct Troy? 

Troy Taysom  0:07

Right. 

Tyler Jacobson  0:07

Yep. Okay, so give me a little bit of background for our listeners on, in a nutshell, all of your experience and how you got to be where you’re at. And then we’ll talk a little bit more about IT security and some of the risks there.

Troy Taysom  0:24

Sure, I’ve been in the IT field 12, going on 13 years now. I was your network admin, for most of that. And then about two years ago, I took a full time job teaching at Utah Valley University, in the ISNT department. And I exclusively teach IT classes, I have a master’s degree and in network security management, and that’s been my focus. The security field. In particular, the management of security in networks. Yeah, that’s kind of how I got here, I was self taught until I got my master’s degree. Which, I would imagine most IT people, that’s kind of the way they are. 

Tyler Jacobson  1:18

So with a long experience in network security, network management, things like that, we were discussing a little bit earlier about some of the risks that are lingering out there. And in higher ed, when we talk to people, it’s usually the network security guy, or the information security guy [who] is one of the most intense people in the room. They’re the ones that are always looking with a very critical eye at anything that’s going to be put on the network. And I think the discussion today may relate to why they are so passionate about network security, because they understand the risks that are involved. So what are some of the risks and and what are the consequences of a breach of network security or having a problem there? 

Troy Taysom  2:09

Well, the biggest consequence, of course, is what gets taken off of your network. IT people aren’t necessarily worried about what someone is going to put on their network, usually that can be handled pretty easily. The problem is, is that when people start taking stuff off of your network, and that’s where you get into trouble, when they start to steal your information, whether it’s, you know, personally identifiable information, you know, names, addresses, social security numbers, credit card numbers, I mean, anything they can get their hands on. And that can have ramifications that are in the multi-millions of dollars per incident plus, [if] you’re the customer, whoever the customer is of this network, loses confidence in the people that are running the network. And I think one of the things that IT people need to understand…you know, I came from a sales background long before I came into IT, everybody has a customer. And no matter who you are, if you’re a university, your students are customers. And as IT people, the people within our company are also our customers. So the employees at a university are the customers of the IT department, as well as the students, because we’re there to support everybody. And that’s why IT people are so security aware now, because it really comes back on them if anything happens. And the ramifications again, can be not only costly financially, but could ruin relationships. It can even be sensitive information. If it’s proprietary information or secrets, government type stuff, the consequences can even be bigger than we even know. And that’s kind of what we’re seeing right now with the Solar Winds incident that happened and they still really don’t know how big it is. And they may not know for a long time how big it is. 

Tyler Jacobson  4:28

So really quickly outline what the solar winds thing is. I know that you understand that quite well. I’ve just heard to the level of: it was a network incursion, and it’s going to be a huge mess to clean up. So give me a little bit more of an overview there. 

Troy Taysom  4:44

Well, okay, so I’ll tell you what I do know and, and the trick is that most people don’t even know yet. So what happened was that FireEye, which is a cybersecurity company, discovered that there had been a breach into SolarWinds, I think it’s called Orion. And that’s like a network. It’s kind of like an all encompassing IT management type of an app. You monitor traffic and monitor logs, I think and all sorts of different stuff. It’s like it’s an all-in-one, it’s really a powerful tool for an IT manager to manage their network. And what happened was that a malicious actor was able to get into SolarWinds. And then they were able to get into the source code for an update that SolarWinds was working on. And they put malware into the update. And Solar Winds didn’t detect it. And they didn’t detect it, because it was so well done. They’ve been reverse engineering the malware to try to figure out how they did it. And it’s…they did a really, really, really good job of disguising their footprint and making it pass all the normal security tests. So the malicious actors got in there, and they put malware into the update. And then when customers downloaded the update, they got infected. And it was a Trojan horse type thing. Because it came in as something that you thought was okay, which is what a Trojan horse is. And then when it gets there, then it goes through and does things that most of the time, you don’t even know. And we know that Microsoft got hit with it. And we know that a lot of federal government agencies got hit with it. But we don’t know exactly how bad they got it, and how deep it went. They’re still uncovering it and trying to figure it out. But that one’s going to take years to unravel. In fact, some of the people in cybersecurity are saying that networks that have been infected with this may need to be just burned to the ground and rebuilt because of [how] all encompassing this malware was. So I think that’s the change we’re seeing. We’ve gone from the script kiddies, young people in their mom’s basement, in the dark room, messing around just trying to mess with people to: this one seems to be state-sponsored. So they were well funded, well trained, well hidden. It kind of points towards Russia right now. But again, they don’t know that much still. And this started happening in 2019. There’s a timeline out there, that’s pretty good, that shows how they did this. They put it together. And then they tested it. And then they got out of the network. And they went back in. And they ran their stuff and got back out. And I think that they were saying that malicious actors were gone in June of 2020. And so they’re just now discovering this in November. I mean it takes six months to discover that these things were happening. And they still don’t know how bad it’s going to get. So I mean, like I said, the ramifications can just be huge. And that’s why IT guys are nervous about what is on their network [and] who’s on their network. And security has now become…it’s no longer optional. For a long time. Everybody just ignored it. And they just, “Oh, it’ll be okay.” But now, it’s not optional anymore. And I’m trying to remember the statistic [that was] from a couple of years ago, but it’s only gotten worse. There’s like 90,000 security jobs that went unfilled, because they don’t have qualified people. And it’s only getting worse now, especially with this deal. Where you I mean, you had people in Microsoft. Microsoft got infected with it, and they’ve got really good security. And so everybody’s got to reevaluate now what they’re doing. 

Tyler Jacobson  9:29

Yeah, so as far as damages caused: you have, as you had said, a lot of personal private sensitive information could be floating around out there. And then potentially, a complete rebuild of the network. So what does that involve, when you say, “burned to the ground and start over”, from a business perspective, like a university or something like that…what would that look like? 

Troy Taysom  9:59

I don’t even know if I can imagine what it would look like just because everybody of course runs (well, they should), run backups on your stuff. But if you get infected, and it goes six months without you discovering it, your backups have become infected. Now, if you have archival backups, you can go to those. But think about how much data is processed within a six month period, or a year. How much are you going to lose? I mean, if a university lost a year of data, you’re talking about grades and student records and financial records. And it’s overwhelming. It’s crazy. So hopefully, they can figure it out without having to burn these things to the ground. But it’s going to look like billions of dollars just in the cost to get things repaired. Plus, you don’t know what it’s gonna cost you in lawsuits. If FERPA regulations are broken. HIPAA laws are broken. GDPR. What’s the European Union going to do to people? California, the protection act they’ve got, the privacy act that just came out? I mean, it could be….it’s mind numbing, actually. 

Tyler Jacobson  11:22

Yeah, you bring up an interesting [point]. The privacy of information like the FERPA, GDPR, CCPA, PIPEDA. You know, we’ve got several of those there. It’s kind of a transition of topic a little bit, but where do you see the privacy of it? Because with something like a university, you’re dealing with very, very sensitive information anyway, because you’re dealing with people’s financial records, their educational records, if there is a state of accessibility need. You know, you’ve got stuff that is sensitive information? And how do you find the balance and privacy of information as well as being able to effectively provide for the needs of the students and faculty and stuff like that? Where’s that balance? Give me a little bit of background [because] I know that you’ve done quite a bit of research into the privacy of information. What does that look like today? And what do you think the trend is like, for the next few years?

Troy Taysom  12:17

Well, the truth of the matter is that the information that we collect, at the university level or at a government level is necessary. You have to have that stuff. So protecting it becomes a priority. Now, the reason that the legislation and things have been passed and the regulations are passed, really has more to do with people selling information to third parties. That’s where the regulations are targeting. But it really does…the onus of protecting that data falls directly on to the company or the university, whatever the business entity is that collects the information and stores the information, you’re responsible to keep it. So it means you’ve got to have better security, you’ve got to have tighter security. The biggest thing you have to understand is how your company or whoever you work for, how do you collect your data? Number one: how is it collected? And number two: how is it stored? And then number three: how is it used? And you have to be able to have three different states that they call it in the IT world. Data is either at rest, in motion, or in use. And you have to be able to protect that data in all three of those states. And that’s not everyone’s priority now. Because if you think about it, the biggest asset that any company has now is their data. Right? That’s the biggest asset that there is. If you take away someone’s data from their company, they’re done, just like we talked about, you know, if you had to burn your network to the ground and rebuild it, what does it look like? Well, there’s a good chance it looks like you’re going out of business. But that’s the biggest asset that people have now. Because…go ahead.

Tyler Jacobson  14:34

Yeah, I think that, like I wasn’t really anticipating going down this channel, but the two are so closely related because anytime you do business with somebody, they’re collecting information about you. Even if it’s something that’s transactional, they now have potentially your credit card number and your name and your address and all of that. Even something as simple as signing up for a points program at the grocery store, that’s a lot of data that creates great value for the business. And so I think that’s the primary point of the legislation and these rules is: gather the data that’s necessary for your business, but also protect people’s privacy and their rights. And if you’re going to sell that data, or otherwise disseminate it, make sure people know what you’re planning on doing. So [at] the same time, if you’re collecting all this data for a university, or other business, that isn’t necessarily going to be selling that there’s a lot of emphasis on protecting it. I know that with LabStats, a large percentage of our client updates and updates to our product are relating to security, just to make sure that we’re compliant with the different privacy laws, as well as just securing the data because we had that discussion just a little while ago that an incursion could mean a very severe impact of the company. And like you said, if it’s bad enough, it could put just about anybody out of business.

Troy Taysom  16:13

Yeah, even if you don’t end up in a situation like a Solar Winds situation. If you have a breach and your company is responsible for the loss of this sensitive information, you can lose the confidence of your customers, and that would effectively put you out of business. But I think that we also have to face the reality that privacy is kind of…it’s almost a thing of the past now. All of our information has been out there for so long now. And has been sold and stolen. Is privacy really something that everybody has, or is it something that we’re trying to make people feel better about? The truth of the matter is that most companies have been infiltrated, at some level, it’s just a certain percentage of them actually know about it. And that’s what makes this last one with Solar Winds so scary is that the big players in the industry that are security heavy, you know, the US government and Microsoft, and these other real big…and solar winds, you know, they were infiltrated. And that’s what makes it scary is now really there’s nobody that’s untouchable.

Tyler Jacobson  17:38

And I think that’s kind of…to my earlier statement of: “Why is the network security guy, a little bit more intense during a lot of these meetings?” And I think that as you referenced, there’s a lot. It’s kind of a new…not new…but it’s an evolving position, and is evolving very, very quickly. And so you have one individual in the room that lives and breathes these risks, understanding, like you said, this incursion may not be ‘future’, it may be ‘past’ and you just haven’t found out about it yet. And you need to do everything that is realistically possible to protect your data and to put your backups in place. And there’s a lot of moving pieces there because it’s not a one pronged approach. And I agree that this is something that’s going to be a growing part of any business. It is that security component. So it’s great that people are becoming aware of it. What’s the lead time to get people trained up and ready to fill? You said that there were a whole bunch of open positions that weren’t filled? How long would it take somebody to be fit to fill one of those positions?

Troy Taysom  18:53

See, and that’s the other issue is that for so long, and still, it’s a big problem. And that’s that most CEOs and COOs and CFOs…they’ve turned a blind eye to security because it just looked like a big expense. And they didn’t understand it. So they just kind of ignored it. And they ignore it until something happens. And then they grab the IT guy, and everything falls on him. Which is why these guys are so intense in the meetings because they know what’s going to happen if something bad happens. Everybody’s gonna look at him. So why didn’t you tell us this could happen? And one of the problems we have in IT and I actually teach a course on this and that’s: IT people there’s always been this big gap between IT people and business people because they don’t speak the same language. And what I teach the IT people is you have to come to the realization that the business people, the CEOs, and And the C-suite and those guys, they’re not going to learn to speak your language. You have to learn to speak theirs. And then you have to learn to translate from ‘tech’ into ‘business’. And that’s how you have to…and you have to turn into a salesperson, you have to sell them on the idea that you need the security tools. You have to sell it, that it’s not an expense, it’s actually something that is going to be saving revenue. That’s the toughest part right now is getting everybody on board. And that’s why there’s such a vacuum of need, is because people haven’t been focusing on security. That means that people in the field, in IT, haven’t focused on security, because there was no money in it. There weren’t any jobs in the past and security. And now all of a sudden, they’re like, “Oh, we want security.” And they’re asking for all these requirements that nobody has. I mean, they’re looking for unicorns, but they’re paying for mules. I mean, when I read these job descriptions, I kind of have to laugh, because I’m like, “Anybody who has this kind of experience that you’re asking for, you are not going to pay them what that experience is really worth.” And I think that’s where the vacuum is. So you know, you train somebody in IT. And even if they get a degree, say they get a four year degree. When they leave, they’re still really green when it comes to security. And the other thing is that security is so multi-faceted, that companies will try to find somebody that can do all of it. And there’s just not a lot of them out there. They want somebody that can manage a network, that can write all of the different documentation programs that you need, create those, that can reverse engineer malware, that can…and those are all skill sets that…usually a person is really good at one of those. I don’t do any coding, I’m not a computer programmer, I focus mainly on the managerial side. So when they say they want a manager, but then they want somebody that can also reverse engineer. I don’t reverse engineer stuff. So that’s one of the reasons it’s hard. So it’s gonna take a while to get people trained and ready to the point that they’re effective. So, sadly, we’re behind the curve. It’s gonna be hard to catch up.

Tyler Jacobson  22:53

Well, and I also think that we’re dealing. We’ve talked about this a little bit as we’re also chasing a moving target, that if you get people trained up, by the time that they’re ready to go there’s a new set of issues that they need to be ready to deal with. And so I think that that’s going to be a big challenge. Especially for smaller companies, like you said, they don’t have a team, they’ve got a guy. And that guy being ready and up to speed on everything is a huge challenge. And so I think we’re gonna see a lot more of.

Troy Taysom  23:31

Well, you have to accept that, right now, the fact is, that the bad guys will always have the upper hand. They will always be many steps ahead of you. And the reason is, they don’t care what their program does to your system. They don’t care if it works great. If it doesn’t work, they try something different. They don’t have to have a QA department, they don’t have to have to go through different channels to get approval. They put something out there. And if it doesn’t work, they bring it back and they rework it and they send it back out. They don’t, they don’t wait for permission from anybody. So they will always be several steps ahead. And the only thing that you can do, and this is just reality. The only thing that you can do is try to make it so that the prize at the end isn’t worth the effort that it took to get there. And that’s why I think the malicious actors went after something as big as SolarWinds, because they knew that there would be a lot of effort put into this and would take a lot of time and a lot of money. But if it worked, the benefit was going to be huge. So as a small company, one of the things you need to be aware of is that they may not actually be after your data, they might want to get into your system to pivot, and then get into one of your customers’ systems. So if you are a contractor for the government, you’re not a very big company, but your contractor, they may want to get into your stuff, so that they can pivot within your network and get into your customers. So all you can do is make it as difficult as possible within a reasonable financial investment. Not everybody has $10 million to spend on security.

Tyler Jacobson  25:42

Right. 

Troy Taysom  25:43

Right? It’s not worth it for some companies. So that’s just kind of the sad state we’re in right now.

Tyler Jacobson  25:54

Well, and I think that kind of brings it full circle. There’s a lot of things that are in the news that cause great concern. And, there are things that we can do. But it’s finding that balance between what you can do, what you should do, and what is just unrealistic. And understanding that can help people really make the best approach for them being able to provide security for their business, for their data, for privacy, for all of those things, and it’s something that’s going to continually evolve. Because, as you said, the bad guys will be a step ahead. And what we see today is going to be very different two years down the road. And so it’s a constant battle. And it’s gonna be interesting to see how it all plays out with the role of security within IT departments as it grows, and finding that balance between protection and providing resources to all departments that need it.

Troy Taysom  27:00

I mean, right now, at least in our universe at our university, almost every IT degree has a security emphasis in it. And it’s because it’s gotten to the point [that] everybody in IT has to be aware of security, and they have to understand what their role in it is. And not only that, but security is everybody’s job within a company. And that’s the other difficult part is selling people on that. You know, the funny thing is that they still haven’t said how they got into SolarWinds. But I’m gonna go ahead and throw a guest out there that it was a social engineering deal, where somebody clicked on an email they shouldn’t have, because 80 to 90% of all attacks start with some type of social engineering, because humans are always the weak link. We trust too much. It’s so…

Tyler Jacobson  28:05  

Well, and that’s yeah. And that’s where the IT guy stands apart in the meeting, because everybody else is still in a state of trusting oblivion. And they are aware of the reality of the risks. And so there was the parental figure in the corner of the room saying, that’s not a good idea. We need to do things a little differently. So…

Troy Taysom  28:26

I think they’re starting to be listened to more and more, of course, at the big companies. You know, banks, hospitals, universities, government agencies are really tuned into it. And that’s trickling down, but everybody’s vulnerable. And we’re kind of behind. We’re really behind the ball on this thing, because now that it’s moved to state-sponsored. And these guys have…of course they don’t have infinite funding, but they certainly have a lot, you’re going to see even more spectacular attacks like we saw, or are still experiencing with SolarWinds. And I think that it’s gonna be way bigger than anybody is even thinking right now. But by the time you figure it out, how much have you lost?

Tyler Jacobson  29:18

Right. All right. Well, I appreciate all of your input. And appreciate your time. I think that it’s a great opportunity to…you know, [because] some people are not really versed in security as far as the risks and things like that. So I think it was a great conversation. I appreciate you joining.

Troy Taysom  29:37

You bet. It’s a great topic and it’s not going away. That’s for sure. 

Tyler Jacobson  29:42

That is for sure. So, all right. Thanks Troy.