“Securing a business involves so much more than plugging in various pieces of computer technology”, Simon Edwards

I met Simon Edwards in January 2007 at the first AMTSO meeting in Bilbao. For many years, Simon dedicated himself to testing security products for Dennis Publishing and, at the time, he was also the technical director of Dennis Technology Labs. The prestige gained over the years has made him a recognized authority in this sector.  Less than a year ago he began a new career path when he started his own business, SE Labs.

1 – Since your time as the editor of the Computer Shopper magazine, your life has been linked with computer security. What has your experience been like in such a changing and innovative industry?

I have always approached the security business from an ethical position because we genuinely want to make a bad situation better. We do much more than testing anti-malware products. We provide threat intelligence to very large companies and, in the UK, the insurance industry uses our information to make important decisions. That is a new diversion from testing, but we do still test security products and that feeds back into the threat intelligence information we provide. We didn’t set out to create a security testing business from day one, though.

When I was first asked to write an anti-virus group test I thought about how to do it, but without any input from other testers or even the companies that made anti-virus programs. In complete isolation from the experts I came up with a method of testing and found that some well-known threats could bypass anti-virus, particularly those that were more like Trojans and hacking attacks rather than standard self-replicating ‘viruses’. That was interesting.

The response from the readers was fantastic and every time we published such a test we sold more magazines than in a usual month. The anti-virus industry was less pleased and I received aggressive phone calls from some people who, today, I actually count as very good friends. We just had to get to know each other and develop trust.

I think that the default position the security industries take, when confronted by challenging results from a new face, is to attack. “We don’t know this guy and he’s saying our product sucks? He must be an idiot, or corrupt!” Nothing much has changed on that front. At least now people know SE Labs creates useful tests and works ethically. Well, most people do. There are some companies, particularly new ones, who are still working out what’s what. They assume that if you don’t support their marketing message then you are an enemy with a biased agenda.

The default position the security industries take, when confronted by challenging results from a new face, is to attack.

One big change is that vendors are starting to see the usefulness of testers really attacking systems, rather than just scanning regular malware that exists on the general internet. We were running hacking attacks in tests back in the days of Back Orifice 2000  and we also used other tools that the bad guys had access to. At the time that was extremely controversial, as the industry had a general view that creating threats was taboo. Many still feel that way, but we’ve been crafting targeted attacks for testing purposes ever since, and it seems fair considering how many products claim to prevent such things.

2- What is like to be an entrepreneur? Are you still able to perform the tests yourself or has management become the main part of your day-to-day?

I personally review every set of data that powers the tests that we publish, and I also develop the test methodologies used by the talented testers who actually sit in front of the systems and put the products through their paces. The ongoing testing and general office tasks are managed by the SE Labs team in London. Once a test is up and running I trust the team and spend most of my time doing one of a million other things. What’s really cool about setting up a company from scratch is that there are so many creative tasks to carry out. But, as we’ll see, there’s also a load of nonsense to contend with too.

When you are running a company on your own you make decisions about literally everything. One day I would be negotiating six-figure finance deals and then I’d be fielding questions about teaspoons. I spent literally half a day in Ikea arguing with colleagues about which cutlery sets to buy.

There is a lot of emotion and some immaturity in this emerging ‘next-gen’ industry.

Back to testing, I have spent a large amount of time trying to work with the newer companies in the industry. Some of them can be reluctant and I understand why. Startups are vulnerable and a poor result could kill a business before it even starts. That said, some of the aggressive marketing we’ve seen very much invites testing to challenge quite extraordinary claims. There is a lot of emotion and some immaturity in this emerging ‘next-gen’ industry. That needs to stop, because it does not serve the customers.

3- As Director of SE Labs, does your work continue to surprise you on a daily basis? Do you have to adapt your tests to the type of attacks that appear frequently? 

A fundamental part of what we do is to seek out and use prevalent threats. Theoretically every product should score 100 per cent in our tests because we’re not using threats from the edges of the internet or zero day threats. So it’s always been quite surprising to me that most vendors don’t score 100 per cent. It’s well-known in the security world that a test in which everyone scores 100 per cent is useless. I don’t think that’s true, as long as the test comes with a good explanation of what it’s trying to achieve.

But regardless, if I throw 100 well-known threats at the leading anti-malware products I know there will be compromises. And that still surprises me. We work with many vendors to help them fix these issues.

4- In addition to traditional security solutions, in the past few years several new solutions have appeared on the market with names like “Next Gen AV” that use a different approach to protect businesses. Have you had the opportunity to try one of these solutions? What has your experience been like?

We have managed to gain access to some so-called ‘next-gen’ products and I know what you’re expecting me to say! But they are not the snake oil that their crazy marketing suggests. They are proving to be competent solutions. I don’t think I’d want to run many on my systems without some other form of anti-malware, but they are not the ‘smoke and mirrors’ fake solution I think many people assume. They are not perfect but neither are they rubbish.

It’s always been quite surprising to me that most vendors don’t score 100 per cent.

5- There are also solutions from “traditional” manufacturers within the EDR category (Endpoint Detection and Response). Have you had the opportunity to try out any of them? 

Indeed we have, and we even run one on these products alongside so-called ‘traditional’ AV on our own systems. Being able to track a breach if/when it happens could be useful. Although we’re a relatively small company, it would be naïve to think that no-one would ever mess with us. We take security seriously, especially considering the nature of some of our clients (we don’t just test anti-malware products, but also provide security advice to some of the largest companies in the world). Our influence extends beyond the basic ‘AV test’ world and, as such, we need to be very careful.

6- You have been involved in AMTSO since the very beginning, and in fact you are currently a member of the Board of Director. In your opinion, what are the major accomplishments AMTSO has achieved since its inception?

The relationship between testers of anti-malware products and the developers of those products is a million times better today than it was. This is important because a good relationship means a productive development cycle of the software that we all use to protect our computers. Once it was the case that vendors hated testers and treated their results as something to work around, rather than use to improve products. I think that AMTSO has largely fixed that problem.

7- What are the challenges that AMTSO has to face in the near future in the testing landscape?

The next-generation companies are opposed to testing. They might claim otherwise, but in my opinion they don’t want to be challenged. Their focus is investment and growth. AMTSO needs to bring these companies into the fold and help them understand that there is something more important than just raising investment funding. Customers count and they need to be protected. Testing actually plays a crucial part in that. They can’t expect to succeed if they operate in a vacuum.

8- In your opinion, what is the biggest challenge that institutions and corporations are up against today in regards to cybersecurity? Does that time lag really exist between adopting new technologies in businesses and applying the proper security measures?

I think the biggest challenge is that securing a business involves so much more than plugging in various pieces of computer technology. Users are potentially the strongest link in the chain, whereas often they are accused of being the weakest. Training can help a lot here. Going back to fundamentals and really understanding what security is would help. It’s easier to spend a few millions on some new types of firewalls, but that’s not going to do the job. CISOs need to understand that.