Posts

A major challenge in network security is identifying malicious activity among all of the non-malicious activity. Studies on malware find that many malware families generate periodic network traffic, such as pinging a command and control server. However, a lot non-malicious software also generates periodic network activity, such as keepalive messages to keep connections open. In this blog post, I discuss our new algorithm and visualization tool that detects periodic activity and surfaces suspicious activity for investigation. This is joint work I conducted with Lauren Deason from PUNCH Cyber Analytics who handled the algorithm development and worked with me to design the visualization tool.

To solve the detection problem, our paper describes a new algorithm using Short-Term Fourier Transforms to detect activity exhibiting multiple different period lengths within the same series. It can detect period lengths ranging from seconds to days or longer. To help analysts determine which detections are malicious, we also present a new visualization tool for sorting, filtering, and analyzing periodicity detections.

In order to use the algorithm and visualization tool, we first preprocess network logs to extract entities and their time series. The definition of an entity depends on the type of network log, but for DNS query logs, we define an entity to be the source IP address concatenated with the domain query. For example, 192.30.253.112~google.com. Tools can easily parse logs to identify entities and extract their time series.

Finding Suspicious Periodic Activity

In large logs, such as those spanning several months from a large organization, it’s common to have millions of entities. From these, the algorithm may find that several thousand exhibit periodic activity. Our tool aggregates the entities into several tables. These tables help analysts find suspicious activity (Figure 1 shows an example where entities are aggregated by the query domain). Users can sort and filter these entities to hide periodic activity to common, expected domains, such as windowsupdate.microsoft.com. Users can also use the tables to find strong periodic activity to uncommon, unusual domains—this could be activity that is trying to hide in the noise.

Conventional visualization wisdom says to show this data in some sort of overview visualization. However, we found these tables to be an effective way to group thousands of entities into a few hundred meaningful rows. Users can then easily sort or filter the rows by cyber-relevant features, such as domains or port ranges.

Figure 1: The domain aggregation summary table for periodic activity in a DNS network log.

Figure 1: The domain aggregation summary table for periodic activity in a DNS network log.

Visualization Design

After selecting suspicious rows, users can view a detailed visualization to investigate those entities further. Figure 2 shows an example of the visualization. The colored bars indicate when the algorithm detected periodic activity and the associated period length. Below, the visualization shows the entity’s time series from the network log so users can correlate detected behavior with the log data. In Figure 2, it’s easy to see that the period length is increasing over time. This is an insight that could be difficult to see without the use of a visualization.

Visualization of detected periodic activity.

Figure 2: Visualization of detected periodic activity in netflow data. The time series is at the bottom, and the colored bars show the period lengths of the detected activity. In these examples, it’s easy to see that the period length is increasing over time.

One advantage of this tool design is that if users find suspicious activity from a malicious domain or IP address, users can go back to the aggregation table and filter the tables to show only data for that domain or IP. This enables users to easily find other related activity.

Demo and Further Reading

This demo video shows the tool in action, including the visualization’s interaction capabilities, like filtering and details on demand.

For more details on the algorithm and the visualization design, see our paper published in VizSec 2018.

Gove, Robert and Deason, Lauren. “Visualizing Automatically Detected Periodic Network Activity.” Proceedings of the IEEE Symposium on Visualization for Cyber Security, 2018. Preprint PDF.

 

This research was developed with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the author and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. Distribution Statement “A” (Approved for Public Release, Distribution Unlimited).

SELinux and SE for Android can be a crucial part of securing a system, but the size and complexity of SELinux security policies make them challenging for security policy administrators to develop and maintain security policies. For example, the sesearch utility shows 94,420 allow rules in the 20141203 version of the Tresys reference policy. To address these challenges, we built V3SPA, an open source tool for visually analyzing and diffing the allow rules in SELinux and SE for Android security policies.

Today I’m announcing the release of V3SPA v2.1.1. V3SPA can import uncompiled SELinux security policies, or binary policies that can be read by SETools v4, including many SE for Android policy binaries. This post gives an introduction on how to use V3SPA, and I describe some of my research on V3SPA published this year at VizSec.

Figure 1. A screenshot of the Policy Explorer in V3SPA showing the policy from a Nexus 4 device.

The Need for Analysis Tools

Although there are existing tools available for analyzing SELinux policies, such as SETools and apol, policy administrators often have needs not met by these tools. To better understand this, I conducted a survey of SELinux and SE for Android policy analysts and developers. I found that mostly they use text-based or command line tools like grep, seinfo, sesearch, audit2allow, and apol. Meanwhile, I found that some of their most time consuming tasks are searching the files in the policy source tree, and understanding the policy structure and behavior. With many thousands of rules spread across many different files and directories, it can be challenging to perform these tasks using text-based tools that tax the user’s working memory. (See the VizSec paper for more details on the survey.)

V3SPA Overview

V3SPA can import SELinux policies in Reference Policy source format (i.e. not monolithic policies), SELinux compiled binary policies, or SE for Android compiled binary policies. Just zip up your policy, and drag and drop it into the V3SPA UI (see the V3SPA readme for detailed instructions on creating your policy zip, or use the examples provided with V3SPA).

The import process can take several seconds while V3SPA processes the policy. Once it is finished, you can select your policy and load it into the visualization. Initially V3SPA shows your policy in the Policy Explorer, but you can also select the Policy Differ to compare your policy to a different policy. This video gives a brief tutorial on using V3SPA, and the sections below describe V3SPA’s user interface in detail.

Policy Explorer

The Policy Explorer considers your policy as a network of subjects that have permissions on objects and classes. Subjects, objects, and classes form nodes in the network, and permissions are the links that join the nodes together. Subjects are the dark blue nodes, and object-class pairs are orange nodes, where object-class pairs are created if the object and class both occur in the same allow rule. For example, for the rule allow user_t bin_t : file {read write}; we would see a blue node for the subject user_t and an orange node for the object-class node for the bin_t object and the file class. Links connect subject and object-class nodes if the subject has one or more permissions on the object-class pair. This scheme is useful for understanding policy semantics, but it also reduces the number of links compared to some other visual representations, and therefore improves readability. Users can also click and drag nodes to reposition them.

The Policy Explorer contains several controls for filtering the policy and specifying how to view it.

Figure 2. The tabs in the Policy Explorer provide information about the policy and various means for filtering and exploring the policy.

  1. The Statistics tab (Figure 2A) has several range sliders to filter the view. Users can filter nodes by the number of connections, or links, they have (i.e. the number of other nodes that link to them); the authority score, which measures to what degree an object-class pair has permissions on it from a subject that has permissions on many object-class pairs; and the hub score, which measures to what degree a subject node has permissions on an object-class node that has permissions on it from many other subjects. Together, these statistics help users reduce visual clutter, focus on important aspects of the policy, and identify over permissioned nodes.
  2. The Access vector tab (Figure 2B) lists the subjects, objects, classes, and permissions found in the allow rules in the policy. Users can select items to hide or show them from the view.
  3. The Denials tab (Figure 2C) has an input box where users can paste an AVC (Access Vector Cache) denial and filter the policy to show only nodes related to the AVC denial. This can show users allow rules related to the denial, how allowing the denial would affect the policy, and if there is a way to get around the denial.
  4. The Always visible tab (Figure 2D) displays a list of nodes that are always visible, regardless of other filter settings. Users can begin typing the name of something (e.g. a subject or class) and see a list of nodes that contain that text, and then select a node to add to the list of always visible nodes.

The last tab, the Details tab (Figure 2E), shows all the allow rules for a clicked node.

Figure 3. In the Policy Explorer, the context menu for a node displays information about the node and provides controls for displaying related elements of the policy.

Users can right click on a node to view a context menu about the node (see Figure 3). This displays information about the node, including its degree, authority score, and hub score. Users can also see whether alternate versions of the node occur in the graph. For example, if the user right clicked a subject node, the context menu also shows whether or not the subject also occurs as an object in the policy (or, conversely, if the node is an object, whether it also occurs as a subject). Users can then click the button to add those alternate nodes to the always visible list. Users can also add the selected node and/or its neighbors to the always visible list.

There is also a version of the Policy Explorer called the Lobster Explorer, which operates on the SELinux policy source instead of the policy binary. In this visualization, V3SPA imports the policy source by converting it to the Lobster domain-specific language. The Lobster Explorer has very similar functionality as the Policy Explorer, except it also allows users to filter policy elements by their module.

Policy Differ

The Policy Differ allows users to select two policies and see a visual diff between them. This diff is rendered as four groups of nodes: The subjects, the permissions, the objects, and the classes. These groups are arranged from left to right in that order, mimicking how someone might read an allow rule: subject S has permission P on object O with class C. Links connect nodes that appear together in the same allow rule. The objects group is shifted up to reduce the number of permission-to-class links that cross the objects group.

The Policy Differ borrows visual language from other diff tools, where nodes with a left arrow are only in the first policy, nodes with a right arrow are only in the second policy, and nodes with a solid color are in both policies. This indicates to users whether a subject, permission, object, or class has been added or removed in a new version of the policy, or whether it was present in both policies. The nodes are also sorted so that nodes only in the first policy are displayed at the top, nodes only in the second policy are displayed at the bottom, and nodes in common between the two policies are in between.

When the Policy Differ first loads, it only shows the added and removed nodes, because users are probably most interested in seeing the changes between the two policy versions. Users can then choose to also show the nodes that are in common to both policies.

Figure 4. In the Policy Differ, users can compare two versions of a security policy. Here we see two consecutive versions of the Tresys reference policy. We see only subjects and objects that have been added or removed, but we set the filters to show all the permissions and classes. By hovering over the redis_exec_t subject, this allows us to see what permissions it has on what classes.

When users hover the mouse cursor over a node, the Policy Differ highlights all other nodes representing policy components that co-occur in allow rules along with the hovered node, and connects them with links. For example, if a user hovers over a subject node, then the Policy Differ highlights all of the object and class nodes that the subject has permissions on, as well as the corresponding permission nodes.

Note that the nodes alone cannot capture all differences between two policies. For example, if we have the following two rules in version 1 of a policy:

allow user_t bin_t : file read; 
allow user_t bin_t : dir write;

And in version 2 of the policy those two rules change to become

allow user_t bin_t : file write;
allow user_t bin_t : dir read;

There are no new nodes, but the relationships between the nodes changed. Specifically, write is now connected to file, and read is now connected to dir. To see these differences in relationships, users can select to see only links representing relationships only in the first policy, only in the second policy, or links that are in both policies.

Users can click a node to lock the highlighted nodes and links in place. This allows users to pan and zoom the visualization, or hover over other nodes to see their connected nodes and links, while still seeing the clicked node and its connections. Clicking a node also displays all of the allow rules associated with that node and in the Details tab. Users can then see the rules, along with the name of the policy that contains each rule.

Users can also right click a node to get a context menu with several menu options to hide or show all connected nodes of each group (e.g. hide or show all permissions connected to a subject).

More Information

Download V3SPA on Github, and read more about our research in our paper published at VizSec:

Robert Gove, “V3SPA: A visual analysis, exploration, and diffing tool for SELinux and SEAndroid security policies,” 2016 IEEE Symposium on Visualization for Cyber Security (VizSec), Baltimore, MD, 2016. [31% acceptance rate, accepted version PDFpublished version]

Invincea Labs’s research engineers published two papers in this year’s proceedings of Visualization for Cyber Security (VizSec). The papers describe research that the authors and their teammates performed on DARPA’s Cyber Genome program for performing malware analysis at scale.

Similarity Evidence Explorer for Malware, a malware analysis visualization integrated in Cynomix.

In the first paper, Robert Gove and his coauthors describe their work on the Similarity Evidence Explorer for Malware (SEEM). SEEM is a scalable visualization for comparing sets of attributes and components between malware samples. Malware analysts often need to understand if they are analyzing a novel malware sample, or if the sample they are analyzing is a variant of an existing sample, and how the sample is different (e.g. did it add or remove components such as a keylogger). This novel visual comparison tool allows malware analysts to do just that. SEEM is integrated in Cynomix, a scalable system for performing malware triage and analysis. Cynomix is currently in beta, and users can request an account from www.cynomix.org.

 

Process diagram for detecting malware samples with similar image sets.

The second paper describes the research performed by Alex Long and his teammates to analyze the image resources in malware binares, such as icons, background images, and UI skins. Their research extracts image resources from malware, performs a scale-invariant transformation on the images, and then computes the similarity between the set of images in one malware binary and the sets of images extracted from other malware binaries. They conducted a survey with malware analysts to gauge their interest in a tool to analyze malware image sets, and found that analysts were enthusiastic about adding an image set analysis tool to their toolsuite.

 

 

The videos of Robert’s SEEM presentation and Alex’s malware image set analysis presentation are online at the VizSec Vimeopage.

Citations:

  • Gove R, Saxe J, Gold S, Long A, and Bergamo G (2014), “A Scalable Visualization for Comparing Multiple Large Sets of Attributes for Malware Analysis”, VizSec 2014.
  • Long A, Saxe J, Gove R (2014), “Detecting Malware Samples with Similar Image Sets”, VizSec 2014.