Cloud Agents Statistics

Many of you use the MetaFlows Security System to monitor cloud-based instances.  Agents running on the instances send their network packets to the sensor for inspection. We now report the real time bandwidth each agent produces.

The sensors which receive network traffic from agents will now have a clickable button that reveals the agents’ statistics.

Clicking on the button, will open a table with each  agent IP address, the source port being used and the Kbps being sent. These stats change in real time.

Hovering will reveal some passive host discovery information (DNS, MAC, DHCP info, HTTP Agents, Proxies, etc). Clicking on each agent IP will open up an historical report for that IP to see what that instance has been up to in the last day or so.

And, ho, yeah.. we added some links on the dashboard. One can be used to rate us for Amazon (30 seconds survey) and one for Gertner (this will take you 10-15 minutes or so but they give you a $25 gift card). We would appreciate if you could give us feedback.


Feel free to contact us if you have any questions. Happy Hunting!

MineMeld Support

MineMeld is an open source threat feed management system that gathers IP addresses, URLs, and domains which pose a significant network security threat. The threat feed sources can either be free, subscription-based or proprietary.  MineMeld re-scans the feeds at regular time intervals and continuously aggregates and updates the set of all threat indicators to be consumed by fierwalls, IDS/IPS, or any other security device.

MetaFlows now includes MineMeld public threat feeds to augment our existing intelligence sources. The public threat feeds amount to about 200,000 additional indicators updated every few hours. Users also have the ability to add site-specific (either subscription-based or private) MimeMeld sources.

IPv4 and URL/Domain indicators are treated differently.

IPv4 feeds

The default MineMeld IPv4 feeds processed by MetaFlows are below:

SourceCurrent Number of Indicators

MineMeld IPv4 addresses are compiled in a set of IDS/IPS rules designed to alert or block communications to blacklisted addresses.  MetaFlows uses a proprietary technique to quickly look through this huge list of addresses (140,000+) and therefore does not require specialized hardware for hi-speed networks.

reputation ruleset


The MineMeld IPv4 feeds are in the mmreputation.rules configuration file that can be accessed through the existing IDS rule management UI. The feeds are  not activated by default but users can activate them in IDS or IPS mode with just a few clicks. If enabled, these rules can be very useful to detect and/or prevent communication to questionable hosts on the Internet.


All the IP addresses are reduced to approximately 40 separate signatures. Each signature corresponds to a specific feed source (for example blocklist_de) or intersections of sources where the IPv4 address is present in more than one source (for example blocklist_de_alienvault.reputation). This decomposition provides additional operational awareness that can be used to prioritize which set of IPs to alert on or block. Enabling or blocking individual signatures therefore affects  a dynamically changing set of potentially thousands of IPs updated every few hours that map to a single threat feed or the intersection of multiple threat feeds.

Users also have the option of adding site-specify MineMeld IPv4 feeds to enable additional commercial MineMeld subscriptions independently purchased or other proprietary feeds.

Entering the URL as shown above, will automatically add the custom MineMeld reputation feed into the customer’s configuration and the local rule corresponding to the feed can then be managed as the other public MineMeld feeds.

URL and Domain FeedS

The MineMeld domain and URL feeds processed by MetaFlows are below:

SourceCurrent Number of Indicators

These feeds are used to detect when:

  • A user issues an HTTP request to a URL or domain deemed to be malicious or
  • A user receives an email containing a malicious URL or link to a malicious domain whether or not the user clicks on the links.

When either of these two conditions occur, a high priority even is generated that can be used to block those specific communications.

There is also a an additional option to enable real time email notification. When bad emails are detected, users also get a warning email instructing them to discard the email.

MineMeld support will automatically be added next time your system self updates or if the sensor software is restarted.


What is Multi-session Correlation?

   Traditional IDS: A1;A2;A3 are independent                Multi-session: A1, A2, A3 are correlated
Traditional network intrusion detection systems (IDS) generate alerts by finding known threat patterns within a single TCP session. This is very blunt. Important events (A3) are often missed due to the high volume of false positive alerts (A1). To be effective, traditional IDS need constant tuning and expert analysis.

Multi-session correlation is an evolution of dialog-based correlation extended to leverage diverse global threat intelligence. Simply put, it automatically connects the dots between notable TCP sessions between a single internal host and multiple external hosts over time. It produces incident reports containing multiple events related to the same threat (A2+A3) rather than giving you independent alerts. This works much better, it will save you time and money in defending your enterprise.

You can try it on your network for free. Register at and build your own network malware detection appliance within minutes. All you need is some decent hardware with 2 Network Interface Cards and a span/mirror port from your switch.

How to Deploy Open Source Network IDS/IPS in AWS

Open Source Network IDS/IPS in Amazon AWS

The easiest way to deploy a Network IDS to monitor your AWS instances is to setup a Linux security gateway. It does require some amount of IP networking knowledge but it is a very flexible way to manage your cloud assets as if they where in your LAN.

The EC2 security gateway routes IP traffic between the VPC and the Internet and therefore has complete visibility of the full-duplex traffic to and from your protected instances. The Network IDS running on the EC2 gateway instance will then allow you to identify and shut down threats as if it was deployed in a physical network.

Setting up a Linux Security Gateway in AWS

Create a VPC

Launch a VPC (Amazon’s virtual private cloud network) and give it a non-routable network range (ex. Your VPC will need a private subnet (ex. and a public subnet (ex., if you do not already have two subnets then go ahead and create them.

Set up the gateway in AWS:

Launch a Linux EC2 instance on the public subnet of your VPC to be your network gateway, this will probably be the only instance on the public subnet for most deployments. Any Linux OS should be fine, but we prefer and use examples from CentOS.

Your gateway instance will need to be assigned at least one Elastic IP Address (EIP), this will be the public address that people will use to reach your network and the gateway will map that address to the correct instance on the private subnet.

You will need to modify the network adapter for your gateway instance to DISABLE src/dst Checking, this is required for it to properly function as a router.

Configure the gateway as a Router

After it starts, configure the gateway as a router for your private subnet. Execute the following commands assuming your private network subnet is “”:

sudo -s
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s -j MASQUERADE

The above commands first give you a root shell (required to make networking changes), second tell the system to forward network packets that are destined for other networks, and third act as the source for all network traffic originating from your private subnet.

Add additional IP addresses on the public subnet (if needed):

EC2 will automatically assign an address to your instance, that is part of the public subnet, once it is launched. Each instance can have additional IP addresses on the public subnet.

For each of these IP addresses you can assign an Elastic IP Address to correspond to it, thus allowing your router to receive traffic for multiple public IP addresses and route it to multiple internal private hosts. Limits may apply depending on the type of instance you choose.

Set up the routing tables:

The public subnet should have a default route ( to an amazon Internet Gateway device. If your VPC doesn’t yet have an internet gateway, you will need to add one for the public subnet.

The private subnet should have a default route ( to the public facing interface id of the gateway instance. Do not add a route for your private subnet to an amazon Internet Gateway Device, otherwise they will route through it instead of your Linux gateway.

Launch the instances to be monitored

If you haven’t already, launch the EC2 instances that you wish to be monitored in the private subnet.

Add port forwarding

For each of the private subnet instances, add port forwarding rules to the iptables on your linux gateway for their publicly accessible services. You can follow these instructions to do that

Add Network IDS software

Once you have the traffic for your Amazon EC2 assets going through your own Linux gateway, you can deploy any traditional IDS systems in order to monitor your traffic. For example, in our example scenario, the gateway interface for the private subnet is “eth1”, and so we can invoke Snort to monitor all of our amazon traffic by pointing it to that interface:

# snort -f -c /nsm/etc/snort.serv.conf -A console -y -i eth0 --daq-dir /usr/local/lib/daq --daq pfring --daq-var clusterid=88
Running in IDS mode

--== Initializing Snort ==--
Commencing packet processing (pid=22129)
Decoding Ethernet
12/03/18-14:51:21.844473 [**] [1:2022775:1] ET USER_AGENTS BLEXBot User-Agent [**] [Classification: Misc activity] [Priority: 3] {TCP} ->
12/03/18-14:52:40.396441 [**] [1:2025534:10000] ET WEB_SPECIFIC_APPS Drupalgeddon2 8.3.9 8.4.6 8.5.1 RCE Through Registration Form (CVE-2018-7600) [**] [Priority: 0] {TCP} ->
12/03/18-14:52:40.483478 [**] [1:2025534:10000] ET WEB_SPECIFIC_APPS Drupalgeddon2 8.3.9 8.4.6 8.5.1 RCE Through Registration Form (CVE-2018-7600) [**] [Priority: 0] {TCP} ->

If you want to deploy an advanced commercial network IDS in your cloud environment I suggest you take a look at our turnkey solution. It does not require setting up a Linux gateway and provides an unprecedented number of advanced features not available in open source systems.

Qradar Support

The MSS now fully supports the Qradar SIEM from IBM in CEF log format. Qradar is an excellent SIEM but requires classifying and mapping every event type it sees to an internal category. Qradar comes with a large number of common IDS rules (~50,000) already classified but not mapped.  Besides having to manually map all these rules  Qradar-MSS users  would also need to continuously create additional Qradar IDs (Qids) to map to the much larger rule set used by the MSS (which  changes daily). All this required a mechanism to update Qradar dynamically as new rules are published. With this update released today, no manual classification or mapping operations are necessary.

The MetaFlowsCEF log source automatically parses the 13 event types generated by the MSS and presents them in the Qradar default view. All MSS events are automatically mapped to new or existing Qids without any user manual operations. This makes the Qradar SIEM much easier to use.

To setup Qradar for the MSS perform the following steps:

    1. Download the MetaFlowsCEF log source to the Qradar box
    2. Import it with the command /opt/qradar/bin/ -action import -f
    3. Verify the import was successful and assign the MetaFlows sensors to this log source. Also make a note of the log source ID assigned by Qradar to the MetaFlowsCEF log source (something like 400[1-9]).
    4. Edit the file of all sensors and add the line export QRADAR=1.
    5. On one of the sensors you designate as the main Qradar updater, create the file /nsm/etc/qradar.ini to allow the sensor to communicate to the Qradar server (see an example below). Also add the line export QRADARLOGSOURCEID=<logsourceid>; where <logsourceid> is the number you noted in step 3. Probably something like 4001, or 4002, etc..
    6. Restart the sensors

Sample qradar.ini:

certificate_file = /nsm/etc/qradar.pem
auth_token = f3f1201b-3562-46d1-9b8b-9a1623870000
server_ip =


/nsm/etc/qradar.pem is a copy of the file located at /etc/httpd/conf/certs/cert.cert on your Qradar box
auth_token is obtained from your QRADAR application
server_ip is the IP address of your Qradar box.

The Qradar updater sensor will automatically add to Qradar new IDS rules added by the sensor’s rule update  (which will be the same across all your sensors). This will happen through the Qradar API in the background as the sensor is running. The first time, the updater is run, it will have to catch up with about 50,000 definitions; so it will take many hours. Subsequent updates will take less time.

After each Qradar update, the email associated with the sensor owner will receive a summary of the update process.

Qradar integration is a bit complex; so do not hesitate to contact for any questions.


Search In Packet Logs

You can now search for arbitrary strings in the historical packet logs directly. The only requirements for this search is at least 1 IP address in addition to the search string.

For example in the search below we are looking for the IP address in any packet either sent or received by the host The search is also restricted to an hour worth of packets on 5/7/2018.


So why would you look for an IP address string in the packets? Well, this is normally done when there is more than one proxy and the system is not able to properly identify the proxy chain. In that case the offending IP will be recorded in the x-forwarded-for field of the http headers. Once you find the headers, you can find the real flows and then search again to get the data exchanged specifying the source and destination ports.

But this search feature is much more powerful than that; in fact you can also look retroactively in your packet history using full PERL regular expressions!

If you reached this far in this post, and you are an expert user, you will be wondering about the example above. The search string above would actually match more than because the dots really mean any character (for example 139a182b44c203 would also match). To be more precise you would need to enter:


But suppose you wanted to match a specific set of IP addresses

Using a regular expression you could search for:


Just imagine what you could search for when you are hunting down specific strings or patterns.  So, this little new feature (also available through the CLI interface as the option -Q) should really expand the power of our historical packet logging system. It will let you easily dig in your network history for hidden clues of what happened in the past.

Websockets in server mode

You can now use Websockets and disable flash for good. We added support for Websockets instead of Flash for direct real time connection to our sensors to retrieve real time data and payloads. Sensors can be configured in client mode or server mode. In client mode sensors and browser both connect outbound as clients and get connected at the network level by our forwarder. Websockets have been working in this mode for quite sometime. Sensors can also be configured in server mode where the browser needs to connect to the sensor which acts a websocket server.  Since sensors only have a self-signed SSL certificate, secure Websockets would not work. To get around this, we added a small (probably the smallest web server you will ever see) that only serves a root certificate authority to be installed in your browser. Go to:

https://<yours sensor ip> for non-Windows systems

https://your sensor ip>:81 for Windows systems

and install the certificate.

Then configure your sensor as a server and make sure to add its static IP address in the sensor configuration. After restarting the sensor, you will be able to connect in server mode using Websockets.

Proxy Detection Support

This has been awaited for some time. The MetaFlows Security System now detects proxied connections. The original IP is swapped with the proxy IP so that it can be correctly identified in the events. This has a dramatic effect on correlation since most proxied hosts only proxy http and use their real IP for DNS and other communications. Using the real IP for correlation and analysis will correctly correlate IDS http events and file downloads with IDS events and  service discoveries triggered by different protocols.

When a proxied host is detected, a message of the forms xff=<realip><-<proxyip> is appended to the event and the proxy IP is replaced. So, you will see the real IP not the proxy.

When you analyze  the packets data, the system automatically switches back to the proxy IP to look for the packets containing the proxy IP rather than the real IP (since the packets are stored before the IP is replaced).

Here is a real example of two events related to (the real IP) downloading  suspicious content through the proxy


Notice that when we detect a proxy a P is associated with the proxied host.

This feature will be available as soon as your sensor is restarted or self-updates. Let us know if you have questions.

Happy hunting!

Your dedicated MetaFlows Team.


Splunk App

We have developed a Splunk network security app available at


It receives events generated by the MetaFlows sensors and breaks them down by the following types:

  • Multisession Analysis
  • High Priority Events
  • IDS Events
  • Network Logs (3rd party logs sent to the sensors)
  • File Transmission Analysis
  • User Discovery
  • Service Discovery
  • Host Discovery
  • Mac Discovery
  • Suspicious URL Transmission Analysis
  • IPS Notifications
  • User Rankings
  • Modsecurity

From the app you can either drill down on Splunk itself or jump to the MetaFlows console to gather more forensic information like packet payloads.

You can install the app by using the Splunk application management tools. In order to send event to Splunk you need to add a configuration line in your /nsm/etc/ startup script of your sensors.  The SSL-encrypted syslog messages are sent to the MetaFlows Splunk App through TCP port 3015 (please make sure you sensor can communicate on this port).

It is a early beat version, please let us know how you like it.

Please see more details at

Happy Hunting!

The MetaFlows Team.

WannaCry Ransomware Advisory

It has been all over the news this weekend, a surge in Ransomware under the name ‘wannacry’ that has the potential to cripple large portions of networks due to the way that it spreads.

This is a pretty stealthy piece of malware at the network level, little to no CnC has been confirmed, but at an individual level it doesn’t behave much differently from any other Ransomware that we have seen in the past.

What distinguishes WannaCry is that it has a secondary infection vector that prior Ransomware variants lacked. Like any other, the primary infection vector appears to occur via email attachment (zipped javascript). However, once a machine is compromised it begins to behave more like a worm, able to exploit SMB (windows file sharing) on any systems that it can reach in order to spread its self.

This worm like behavior makes it particularly dangerous. While usually* smb (port 445) is not accessible from the outside world, it is often completely unrestricted within a local network, allowing one infected machine to spread the Ransomware across an entire site.

* This is your reminder to do double check firewall rules and run some external scans to make absolutely certain your windows file shares are not reachable from the outside world.


The following signatures are currently indicators to look out for:

2024218: ET EXPLOIT Possible ETERNALBLUE MS17 Echo Response
2024291: ET TROJAN Possible WannaCry DNS Lookup (trojan.rules)
2024292: ET INFO Bitcoin QR Code Generated via (info.rules)

MetaFlows has added 2024291 to our priority alerts category, and may also add 2023218 to add an extra level of alerting for these events.


Many of the windows related scan rules have been updated, and may be treated with greater suspicion, but are not alone indicators of this malware:

2001569 – ET SCAN Behavioral Unusual Port 445 traffic Potential Scan or Infection (scan.rules)
2001579 – ET SCAN Behavioral Unusual Port 139 traffic Potential Scan or Infection (scan.rules)
2001580 – ET SCAN Behavioral Unusual Port 137 traffic Potential Scan or Infection (scan.rules)
2001581 – ET SCAN Behavioral Unusual Port 135 traffic Potential Scan or Infection (scan.rules)
2001582 – ET SCAN Behavioral Unusual Port 1434 traffic Potential Scan or Infection (scan.rules)
2001583 – ET SCAN Behavioral Unusual Port 1433 traffic Potential Scan or Infection (scan.rules)

There are likely to be more updates and more information soon as researchers have time to study the samples collected so far.
Our primary signature provider, Emerging Threats, maintains a mailing list where these issues are discussed as they unfold.