Email Security Solutions, Part 1: The Challenges of Protecting Email from Advanced Malware

protect email FIThis is part 1 of a 2-part blog post on email security solutions, that examines the challenges. Part 2 will discuss key characteristics of the solutions.

Malware authors can simply alter the signature of their code to avoid detection by signature-based anti-malware tools. Because security tools examine the internal components of an object to generate a signature, modifying even a single bit in any of the malware’s components changes the object’s signature. Some of the malware-building tools available on the dark web include the ability to foil signature-based systems by changing the payload with a simple checkbox.

There are multiple transformation techniques used by malware authors, and applying any of them can alter a signature:

  • Code permutation
  • Register renaming
  • Expanding and shrinking code
  • Insertion of garbage code or other constructs

Unfortunately, it can be several days or even weeks after a new malicious object appears in the wild before security vendors update their signatures. Until the new signature arrives — signature-based security controls will not detect the malware.

In fact, some vendors may never add signatures to their databases for many advanced malware threats. Malware that uses less sophisticated techniques and targets large numbers of victims has a much higher chance of having its signature added to a malware database. Advanced malware, on the other hand, uses sophisticated evasion techniques and often targets fewer victims. This narrower focus greatly reduces the odds that its signature will ever appear in a database of malicious objects.

Indeed our recently released study, the Malscape Monitor Report, found that 65% of samples we analyzed had never been submitted to VirusTotal and were seen only once by Lastline, reinforcing criminals ability to create minor variations in the code for each instance of a malicious payload, resulting in a new signature.

Virtual Environments Used by Next-Gen Tools

Security vendors embraced sandbox technology several years ago to overcome the shortcomings of signature-based technologies for detecting advanced malware. Sandboxes simulate a host system to fool the object into demonstrating its intended behavior, thus allowing the sandbox to identify and block the object before it is delivered to its intended victim.

Because sandboxes use observed behaviors and not signatures to detect malware, they were very effective initially in detecting new malware. However, today’s advanced malware is engineered specifically to detect when it is running in a sandbox. To evade detection the malware avoids exhibiting malicious actions while in the sandbox, resulting in it being released into a network to initiate its malicious behavior.

Sandbox technologies typically utilize virtual machine (VM) environments. In theory, the environment provided by the VM is self-contained, isolated, and indistinguishable from a “real” machine.

Unfortunately, VM technologies insert artifacts that allow advanced malware to discover that it is running in a virtual environment. These artifacts include additional operating system files and processes, supplementary CPU features, and other components necessary for the virtualization to work.

Advanced malware looks for these artifacts to detect the presence of a sandbox. For example, some of the techniques used by malware to recognize a VM environment include:

  • Examining registry keys for values that are unique to virtual systems
  • Detecting if VM tools are installed
  • Checking for certain processes and services that are specific to VM environments

In addition to being easily detected, “next-generation” tools like sandboxes only have visibility down to the operating system level. They can inspect content and identify some potentially malicious code, but they can’t interact with the malware. This means they cannot see what is occurring within the malware itself, nor in other programs, operating system, or kernel functions used by the malware. This limited visibility means that sandboxes miss evasion techniques like encrypted strings that require CPU-level visibility to decrypt. As a result, they have significantly lower detection rates and higher false positives.

Perhaps most challenging is that it’s not only highly skilled hackers who can implement sophisticated evasion techniques like these. Today there are numerous toolkits available that allow novice cybercriminals to create malware that can detect the presence of a VM.


Aside from the fact that signatures are easy to fake and VMs easy to detect, bad actors prefer email because it requires few resources and cost of failure is low; they can just keep trying until they find something that works.

Secure Email Gateways (SEGs) are most useful for blocking spam email, as advertising is easy to detect using common keyword analysis. SEGs are also useful against older, known threats: threats that have been encountered previously and have become commonplace.

Though a SEG can still be an important frontline defense for an organization’s email security, it cannot be relied upon to detect all attacks for the above reasons. As attacks steadily become more sophisticated, the utility of a traditional Secure Email Gateway is reduced.


Please watch this space for part 2, which will be published before the end of this month.

Brian Laing

Brian Laing

For more than 20 years, Brian Laing has shared his strategic business vision and technical leadership with a range of start-ups and established companies in various executive level roles. The author of “APT for Dummies,” he was previously vice president of AhnLab, where he directed the US operations of the internationally known security and software leader. Brian previously founded Hive Media where he served as CEO. He co-founded RedSeal Systems, where he conceived the overall design and features of the product and was granted two patents related to network security. He was also founder and CEO of self-funded Blade Software, who released the industry’s first commercial IPS/FW testing tool.
Brian Laing