Tunnelling out of corporate networks (Part 2)

Read on to find out more about malware and end-point security failure:

Articles 1 2 3 4 5

Assume for a moment that employees do not have the technical wherewithal to tunnel out of their corporate networks. Let's also assume that your company is like most and over-entitles access to data with Admin privileges on their computers. More over, perhaps management values their employees and as such treats and respects them like professionals. If you're lucky, you work in such an environment of trust and respect. How can this be bad?

Unfortunately, employees are generally not very knowledgeable about security. That coupled with laptops running the standard corporate operating system with a long history of security vulnerabilities and movement in and out of trusted networks are reasons for concern.

As an aside and in contrast, I have found the OpenBSD community to be very security conscious, extra paranoid, maintain a healthy level of skepticism and generally apply a lot of common sense. Now, if everyone was using OpenBSD and had this security mindset, then the Internet wouldn't be the wild west that it is today.

Sysadmins reading this will know that employees do behave differently outside the auspices of their corporate firewalls, proxies, IDS/IPS systems and log servers recording their every action. The problem is not that we shouldn't trust employees. We just can't trust their actions and therefore by implication, we can't trust their computers. By "actions", I mean they seem to click on anything that gets in their way or install seemingly innocuous free software at home. When security prevents this, it's usually not transparent and becomes intrusive to users. If given a button to push, they just turn security off. It is this premise that forms the basis for our solution.

I think that it is really difficult to solve a social problem with technology. I'm not sure that we should. Yet, can we use technology to effect staff behaviour? Perhaps, but maybe only when it is in their face as a constant reminder. You know, the kind that checks their decision and the consequent action they are about to make based on their conscience and security awareness, if any. I tend to think that trying to use technology to this end is more wishful thinking than anything practical security-wise.

Technology that tracks staff actions, surfing usage and behaviour is done behind the scenes. It is usually very transparent and often used in audits or forensics. That is, assuming that raw log data is kept long enough and analysed. It would be better if this was done in real-time and anomalies acted upon in a timely manner but I wouldn't go so far as to use an IPS for this. This technology is very useful but not really good at stopping infection from happening in the first place.

The problem is that over-entitled staff with laptops will install things that they should not and click on things we wish they had not when there are no restrictions to do so. This behaviour is more commonly done outside of the corporate walls. Of course, it is possible to force the staff to use the corporate proxy using white-listed sites for web surfing through a VPN for remote access users too but there has to be a better way.

You can probably see a couple problems with this. What if you are half way around the world from the corporate VPN server but you're trying to access local Internet web sites? Response time won't be as palatable as it could be. The second problem is that travelling staff will need to take along two laptops when travelling: one for corporate use and one for personal use. You can imagine the pushback. Don't they already complain that their laptops are too big and heavy? Yes, you can multiboot from a single corporate issued laptop but corporate policies usually deny such freedoms. Anyhow, imagine how silly that would be, as switching between work and personal related activities would require a reboot. Yes, one could use VMware, XEN or Parallels but that's another quagmire that most IT teams do not want to deal with due to the admin overhead amongst other security related problems.

Putting users aside for a moment, let's talk about endpoint security, the operating systems and applications that employees use to carry out their work. There is an artifact of computer history that is by far the biggest reason many anti-virus and security companies flourish today. You don't have to guess too hard to know what I'm referring to. Now, I'm not advocating that everyone migrate towards using a more "secure by default", heavily armoured operating system built on time tested quality code such as OpenBSD. Rather, it is important to understand the limitations, your business needs, your staff wants and work with them and around these conflicting interests accordingly.

So what do you do with an operating system that doesn't help to protect you like OpenBSD does? Enter the security industry's white knight in shining armour, "endpoint security solutions". It is a bandaid to a big security dilemma. In fact, the security industry, vendors in particular, have been preaching "endpoint security" as if it were the holy grail of security. To be fair, it does help a bit but not nearly enough.

Fortunately, many prominent security professionals have acknowledged that endpoint security is akin to snake oil promoted and pushed by vendors. It's a simple fact; every host connected to the Internet is vulnerable to some extent and at some point in time and some much more than others. Even a "secure" operating system with a reputation and track record like OpenBSD becomes suspect and vulnerable when you attach a user to it. OpenBSD does protect the user to a great extent but all bets are off when buggy third party packages are installed. Most packages have not been security audited, don't use OpenBSD's secure functions, are not privilege separated and/or privilege revoked or chrooted. All of which would help to limit exposure to bugs in software.

Let's drill down into the problem space a bit further. Why is it that we have to deal with bug enhanced operating systems loaded with buggy software in the first place? Maybe some of this is due to bad security designs and/or processes. You've heard the old joke, "Yah! It compiled. Ship it". Maybe there is just not enough testing done. I think that software, hardware and operating system developers would benefit from studying Theo's talk on the OpenBSD Release Process at AsiaBSDCon 2009.

There's another big component to this mess. A great majority of host infections are a direct consequence of some kind of user interaction. Well, it could also be due to inaction such as not updating virus definitions or patching computers in a timely manner to protect against the "known" problems.

The reality is that it is trivial to penetrate corporate networks via the end points. It is trivial to steal data from end points. It can be done without detection. It can be done without leaving any tracks. Most will never know when this happens. Endpoint security falls short more often than many think. It is the IPS approach; catch some but not all.

So we often see employees of companies that are given too much privilege and functionality and over-entitled access to private and confidential data. This might be acceptable if they didn't need access to the Internet at the same time. Herein lies the dilemma. The Web is becoming less standardised, more complex and much more dangerous all because of Rich Internet Applications (RIA).

RIA such as Flash, Air, Silverlight, JavaFX, Gears, Prism, HTML 5, and PDF are making this problem even worse. Web developers are now desktop application developers. They are introducing security flaws in their applications. Users don't understand why this is so dangerous. The combination of privileged access to data, RIAs and user interaction is serious cause for concern.

In the first part of this series, I showed how easy it is to use a SSH VPN to tunnel out of corporate networks but the real problem comes through infected computers. Sophisticated malware infected hosts are known to create back-channel tunnels after infecting them. Mobility and the flexibility to work remotely, however nice and convenient, only means that it is a matter of time before these hosts come back to the office and connect to corporate network resources. Once in the office or connected via a VPN, laptops are given a lot more access to internal resources and they are trusted and assumed to be clean of infection.

How does MS deal with this? I've seen MS presentations showing how they determine the health or posture of a machine that is trying to VPN into their network. They assess the patch level of the operating system and certain applications, check the anti-virus definitions to make sure that they are up-to-date and if not, those remote access laptops are put into a quarantine until they are patched and up-to-date before given the green light into their internal network. This may seem like a very good idea or solution but it is flawed.

Posture checking can't really tell if a machine is infected or compromised because the patches and antivirus definitions are only going to catch a small surface area of publicly known holes. This is essentially low hanging fruit as far as I'm concerned. Hackers and malware miscreants are attacking the BIOS, firmware in hardware and the virtualization technology that so many are depending on these days. The operating system has no chance. Endpoint security solutions do nothing to protect you here. The point being, there are too many weak links or holes for the hackers to choose from and it's almost impossible to fully protect against. We really have to take a different approach.

Most corporate networks, especially in the financial industry, try to prevent hosts from tunnelling out of their offices for very good reasons. However, there is too much evidence showing that companies and governments are not doing a good job in this respect. I doubt that they are incapable or incompetent. There are battles taking place between business needs and wants and those in charge of security, be it policy makers or implementors.

So if you are an IT Manager or responsible for security at your company or organisation what do you do with these conflicting interests? This is perhaps the part where we need to step outside of the box and approach the problem from a different angle. Before doing that, I still think that it is crucial for security professionals to be up-to-date with the latest security issues. Yet, keeping up-to-date and on top of the latest known threats and vulnerabilities is a pretty daunting endeavour.

Over the years at Pacsec, I've seen presentations focus more on these Rich Internet Applications and the exploits that can be done through the browser. This is where I see the biggest battles lies. No matter what you do to the operating system, throw in a user, a few over-privileged applications with access to the Internet and the playing field has dramatically tipped in favour of the bad guys.

As much as I like and learn a lot from attending Pacsec, I often wondered if we would benefit more by having security practitioners come in to demonstrate sensible field tested security design rather than be appraised of the latest zero-day exploits. Sadly, I was told that "sensible security" is not as sexy or as popular as the latest zero-day exploit. Don't get me wrong, I am glad that security researchers are keeping the commercial vendors on their toes with full disclosure. I just believe that they are only exposing a fraction of a small percentage of all the bugs that exist.

I also don't need to see presentations on how to secure an operating system. I think that if any Operating System has demonstrated that, OpenBSD would be the clear winner. I was thinking more on the lines of good security network design and sensible ways to secure networks. There must be a better way than relying on endpoint security.

I think that we can agree on a few simple facts: Systems and software become more complicated over time. Complexity is security's nemesis. Users are the weakest link as they facilitate infections with their affinity to click on anything that comes their way or looks interesting. It is these infections that we want to prevent from happening in the first place. Judging by the frequency of "known" security breaches, this problem is getting worse. I can only imagine what is being done without our knowledge.

Can you see why it is impossible to win this battle with the present methodologies? Should we really trust the endpoints? How do you know the client is malware free? How far can you trust and rely on Antivirus protection? How can you tell whether an endpoint is compromised? All computers with Internet access can be compromised and it is usually trivial, especially when targeted. Really! What to do?

This poses an interesting dilemma. How can we protect that which we can't detect? The simple answer is that you can't. So then what? Assume that all end points are or can be compromised. Then, come up with a way to allow staff access to the Internet and all its distractions whilst protecting data that they also need access to for work related activities.

Before getting into that, here are some flawed endpoint security solutions and security industry dogma: Antivirus, patching, reduce functionality (attack surface), use virtualization, use encryption, two-factor authentication, data loss protection (DLP), network access control (NAC), firewalls, IDS/IPS, etc. They all try and compensate for poor quality code or some weakness in design. Furthermore, they are not cheap especially for smaller companies with limited IT resources. Yet, even with all of this applied, are you still protected and to what extent? I guess that we can continue with more dogma and try and further educate users. Now, I am being facetious.

Do I have the magic bullet solution to this problem? No, but I have one solution that is simple, effective, relatively inexpensive for businesses and does not require Draconian measures to implement while eliminating exposure to your internal network from malware. In the next part, I'll explain our solution and what we do on the infrastructure side of things with OpenBSD to detect and eliminate the problems already discussed. In the meantime, I would be interested in hearing what you do to prevent the spread of malware and protect internal network access to sensitive data.

Mark T. Uemura