Automatic DNS Resolve

Thank you for your feedback!

We added a new preference setting in Account->Preferences to automatically resolve internal and/or external IP addresses to DNS names. As soon as the browser receives an IPv4 address from the sensor or a historical query, it will try to resolve it to a DNS name using the DNS server(s) used by the sensor (specified in /etc/resolv.conf). Obviously, not all IP addresses can be resolved to a DNS name and, therefore, some will stay in dotted-decimal format.

if you have some IPv4 addresses that are not in the DNS system and you want to resolve them, you can add them to the file /nsm/etc/hosts with the format. If the file does not exist, you can just create it:

#this is a comment 1.2.3.4 <dns_name> 5.6.7.8 <other_name>

Whenever 1.2.3.4 or 5.6.7.8 are seen by the browser (if you have auto resolve turned on), they will be translated to their corresponding names.

As always, let us know if you have any questions at support@metaflows.com.

Improved Packet Logging

We have added support for logging the first 512,1024,2048,4096 bytes of each session rather than the full session. You can select how much of a session length to log in the sensor configuration page under the Store packets on Sensor option.

Unless you have a requirement to log all packets, we recommend changing the default all to 1024 or 2048. This does the following:

Makes much more efficient use of the disk space (you can record important payloads for much larger time windows).
When querying for packet data, you will see both sides of the session (client requests and server replies). This helps in getting better context when doing forensic analysis.

Also, we have added some color coding to better discern the two different sides of the conversation.

The only drawback of selecting a session size other than all is that the file carving function accessible through Look for Files in Flow(s) may not be able to reconstruct files unless one of the two IP addresses was tracked due to an incident report.

If you want to carve files for arbitrary flows (regardless of incident reports), you need to keep the session length set to all.

As always, let us know if you have any questions! Do not hesitate to call us or send us email.

MetaFlows in the Top-20 Security Companies for 2015

CIOReview Magazine has selected MetaFlows as one of the Top 20 Most Promising Enterprise Security Companies in 2015. In the article Cost Effectively Tackling Advanced Security Threats, we outline our approach to the security challenges for the upcoming decade. The internet is shifting from a client/server paradigm to a peer-to-peer, mobile environment.

Your Network Perimeter is Dissolving

Heuristic-based network perimeter defences will become less and less effective because it is like applying medical diagnosis in an environment where new pathogens are created on a daily basis. So, heuristically determining what is bad and what is good may work initially, but it becomes a losing battle unless the network security operators are constantly updating their heuristics. Also, protecting the perimeter is not enough, once something makes it on the inside, the perimeter becomes irrelevant. We have seen that companies adopting this approach can be hacked no matter how much money they spend.

Share Intelligence

Single-vendor network security intelligence feeds have become ineffective due to the sophisticated global cooperation of hackers. Vendors that provide a single box and a single source of network intelligence are selling an inherently flawed promise. Products should be based upon integrating multiple collaborative intelligence feeds. The complexity and interconnectivity of the attacking adversaries requires a similar defense strategy.

MetaFlows has been innovating in these two important dimensions for the past seven years drawing from a thirty year Government-sponsored network security and intrusion detection research. The technical founders of MetaFlows (Livio Ricciulli and Phillip Porras) sharpened their teeth at the Computer Science Laboratory of SRI International, where intrusion detection was first developed back in 1983.

The best part is that these innovations are now commercially available through MetaFlows. The company is improving the security of a large number of networks (big and small) around the world.

MetaFlows at BlackHat 2015

MetaFlows pleased to announce that we will be an exhibitor at BlackHat USA 2015, August 5th-6th. Please visit our kiosk IC7 to see one of the best IDS/Malware detection systems in the world in action. We will be showing an ongoing, live demonstration of our system in action on a university network processing around 200,000 packets per second. You can witness how malware is caught and stopped in real time as if you were running on your own network. We might even be able to let you drive for a while! Do not miss this opportunity to see the secrets of our success.

MetaFlows Inc. develops SaaS-based, network security software appliances that can reliably find and stop malware hidden in your network. False positives are virtually eliminated by correlating multiple independent flows. False negatives are lowered by combining feeds from Emerging Threats, Cuckoo, VirusTotal, SRI, OSSEC, Trustwave, YARA, ClamAV and Web of Trust.

New Packet Data Viewer

Since our first public release, MetaFlows users have been able to use the MetaFlows Real Time and Historical interfaces to download and view raw packet data from their MetaFlows sensors. Today, we are excited to release a major upgrade to the Packet Data page.

New Features:

Packets are now parsed, separated, and colored red or blue depending on which host sent the data. No more digging through raw tcpdump output to find where one packet ends and another begins! Packets are now visibly separated. Server packets use red text. Client packets have blue text.
Packet Data now includes the content of any IDS rules triggered by the flow. Ever opened the Packet Data for a flow and forgotten which alert you were investigating? We now include the full content of all IDS rules that triggered, along with the packet data for the flow.
Packet Data Matcher! This was the most exciting feature to add! The new Packet Data page can highlight the specific content in a packet that caused an IDS alert to trigger! Matching packet data is highlighted in red or blue (depending on whether the packet was sent by the server or client) and includes numbered markers for each condition in the triggered rule. You can even hover your mouse over these markers to see the specific content or pcre condition that matched, along with the condition’s modifiers! For example, the screenshot below shows the packet data for an event matching 1.2003492: ET MALWARE Suspicious Mozilla User-Agent - Likely Fake (Mozilla/4.0) :
An example of packet data in the MetaFlows Packet Data Viewer, with matching content highlighted by the Packet Data Matcher.

The Packet Data Matcher works best for text flows, such as HTTP requests and responses. For encrypted or binary flows, we still attempt to match against non-printable characters by using other conditions and context, but some matches might be inaccurate.

This feature is available to all MetaFlows customers starting today. We look forward to your feedback. Happy hunting!

Network Antivirus White List and Minimum VT Score

We have added support for further customizing the behavior of our network antivirus system. Not all application providers adhere to signing their executables and/or use sound software engineering principles. This causes some Virus Total Antivirus solutions to exhibit some false positives.

White List

To remedy this situation, you can now set up a white list using regular expressions to exclude the virus scanning from certain sources. The user-definable white list is in /nsm/etc/carverwhitelist on your sensor.

Some example white list entries are:

washingtonpost\.com.*\.zip
lavasoft\.com.*\.zip

Notice that you need to escape special characters like ‘.’

The two expressions above would skip content analysis from the washingtonpost.com and lavasoft.com domain. If you see some repeated Virus Total false positives from specific URLs, please add them to this list so that the false positives can stop. You can have comment lines beginning with the character #.

Minimum Score

We also added support for raising the minimum threshold to declare a sample to be malicious. Our default value is 4, meaning that at least 4 out of 55+ antivirus solutions need to report a hit in order for us to generate an incident report. It was noted that this limit might be too low in certain diverse environments. So, we added the ability for customers to change this value by setting the environment variable VTMINSCORE.

To change this on your sensor edit the file /nsm/etc/mss.sh and add the statement anywhere after the first line of the script:

export VTMINSCORE=<minimum vale>

Setting <minimum_value> to anything other than 4, will change the threshold for us to generate incident reports and email alerts. If you set it to less than 4, you will get more reports. If you set it to more than 4, you will get less reports.

As always, do not hesitate to call us at 1-877-664-7774 or send us an email at support @ metaflows.com for any questions.

OpenAppID Support

Cisco released OpenAppID, their answer to Palo Alto Networks’ AppID feature, which allows administrators to know exactly what applications are running in the network.
It has been released as a plugin of the Snort distribution. We have recently upgraded our sensor software to support this feature. OpenAppID results appear as an additional field in the IDS alerts to give better context for the alerts. We also gather this information to associate it with the internal host IP addresses, whether or not they generate an IDS event.

For example, when a user uses Facebook, it will trigger one or more of these:

Facebook Apps
Facebook
Facebook Chat
Facebook Comment
Facebook Read Email
Facebook Send Email
Facebook Status Update
Facebook search
Facebook event
Facebook post
Facebook video chat
Facebook message
Facebook video

If your software has been upgraded, the file /nsm/bin/snort/src/.version should contain 2.7.9.0. If it does not, you can upgrade by executing this command: /nsm/etc/mss.sh restart (Note: MetaFlows UTM appliances do not support OpenAppID yet).

To turn on this feature, check the OpenAppID checkbox in your sensor configuration page and reload or restart the sensor.

Once this feature is turned on, you can look at the daily reports and see the top AppID summary or look at the AppIDs in your IDS events. You can now create user-defined policies that match specific AppIDs!

This new feature requires 40% more memory and in some cases, even though we install it, the system automatically turns it off if you do not have enough memory. You need at least 2 GB RAM per core. For example, if your subscription is for 16 cores and your sensor has 24 GB RAM, the system would disable OpenAppID automatically.

If you do not process a lot of data and have a low memory system, you can force the loading of OpenAppID by adding the line export forceappid=1 at the top of the /nsm/etc/mss.sh script. Note that because it uses about 40% more memory, your sensor might slow down if you do not have enough RAM. Please monitor your drop rate closely if you force the OpenAppID functionality.

We highly recommend using this feature. If you have any questions, please do not hesitate to contact our engineers at support@metaflows.com for more information.

Throttle in Passive Mode

throttling in passive mode!?!? Sometimes users can knowingly, or unknowingly, abuse a network by using a lot of bandwidth. With the proliferation of video on demand services such as Netflix, Hulu, and Amazon Prime, some institutions are once again finding themselves battling bandwidth issues.

Until now, one needed an in-line device, such as a firewall, to throttle traffic by allocating certain bandwidth to certain flows or applications. Any in-line device also adds latency and reduces reliability, especially on high speed links. Wouldn’t it be nice to throttle specific traffic in a way that does not impact performance of the traffic you do care about? This has been an age-old conundrum for network engineers.

Well, continuing on our hot streak of innovations, MetaFlows recently developed (in collaboration with one of our university customers) an unprecedented technique to throttle traffic in passive mode! It works a bit like active response, where spoofed packets are injected into the traffic stream to shut down flows. In this case, we are not shutting down flows, we are forcing them to slow down.

The result is that you can identify any TCP flow using one or more of our 20,000 signatures (appID is coming very soon), and limit its bandwidth. This means you can have zero impact on performance and reliability of your production traffic while you can achieve very fine grain control of the traffic you do not care about!

MetaFlows: SC Magazine Innovators Hall of Fame

Our friends at SC Magazine have inducted us into the SC Magazine Innovators Hall of Fame. It is nice to be recognized for our innovations. Importantly, this is purely based on their journalistic curiosity; we give them props for performing their reviews based on sound technical knowledge. We refuse to pay money for recognition. You might think we are old-fashioned but this is how we roll at MetaFlows.

Their article also points out the importance of monitoring beyond the network perimeter using multi-session correlation. If you are not sure what multi-session correlation can do for you, it is best for you to put it to the test. You will be amazed of what you can find out about your network.

Read the article at SC Magazine’s Website

Get Packet Payloads with Splunk

It is fairly easy to create a workflow action to access the MetaFlows File-Carving and PCAP extraction interface.

Step 1: Extract the flow information from the MetaFlows event feed.

If you already use CEF log output from MetaFlows, or if you want to change to it, then the required fields should already be extracted:
src, dst, spt, dpt, start

Or, if you are using the standard syslog output then you will need something similar to the following extraction regex to make sure each record has those fields:
\{1,(?<rank>\d+)\}\s+\[\d+:(?<sid>\d+):\d+\]\s+(?<msg>[^\{]*)\{IP\}\s+(?<src>[^:]*):(?<spt>\d+)\s-\>\s(?<dst>[^:]*):(?<dpt>\d+)

Additionally, you will need to append “|eval start=_time” to your queries in order to get the start field unless you already have a derived field which gives you a unix timestamp to use in the query.

Or, if you have your own parsing in place that uses different field names which correspond to ‘Source IP, Destination IP, Source Port, Destination Port, Timestamp‘ then you may need to adjust the field names in the URI under step 2 to match. You will still need to make sure that you can provide a unix timestamp field.

Step 2: Create a workflow action.

Go to settings->FIelds->Workflow actions->Newand set the following fields:
Label: Extract PCAP / Carve Files $src$ $dst$ $start$
Show action in: Both
Action Type: Link
URI: https://nsm.metaflows.com/sockets/historical.php?w=carver&srca=$src$&dsta=$dst$&srcp=$spt$&dstp=$dpt$&st=$start$&sensor_sid=<your sensor’s SID>
Open link in: New Window
Link method: Get

Your sensor’s SID should be a hash listed on the view sensors page, or in the file /nsm/etc/UUID on the sensor itself.

Step 3: Test the Setup.

Test by selecting Extract PCAP / Carve Files from the Event Actions menu for any event in a search. You need to be logged into MetaFlows for this to work. It should take you straight to the File Carving interface which will provide a link to the PCAP data as well.

Note that if you have ‘Log All Packets‘ enabled, you will most likely see the PCAP slice as well as all the files that where downloaded/uploaded in that flow or set of flows. If you do not have ‘Log All Packets‘ enabled, you will only see the PCAP slice corresponding to the packets logged by the IDS system.