Hello. My name is Paul Barrett. I’m a director of the deep packet inspection consortium, and I’m also CTO for the enterprise and federal government business of NetScout Systems. And today I’m going to talk about how deep packet inspection or DPI is used in today’s world.
I want to start off by talking about the different types of information that we can extract from a network flow. Now when we talk about deep packet inspection, we’re not talking about looking at the outer layers, not just looking at the IP header or the TCP header, but actually looking at the information in the payload of the packet. Now, the first thing we can do when we look at that is look at protocol specific information‚ or session level information. For example, if I’m looking at an HTTP‚ request, I can see the URL and the user agent. In fact, I can see all of the HTTP headers in both the request and response packet. If I look at a DNS request, I can see the domain name that’s been queried I can see the IP addresses that come back from that, or I can look at a TLS handshake and I can look at the different Cypher suites for example that are being offered and being agreed upon.
Then above that I have conversational information – who is talking to whom and on what protocol‚ where and when. And finally, I can produce roll up data that talks at a high level about what servers have I found on the network, what traffic volumes am I seeing on the network. I can look at things like the performance of applications, the response times of applications, if they’re producing errors.
And I’ve drawn this pyramid because very approximately the width of each slice of the pyramid indicates the sort of the size or granularity of the data we produce. So by storing the packets that’s the most granular data that we generate but it requires the most storage‚ I can extract specific fields from packets for my session level information that produces a great reduction in the amount of information that needs to be stored. That conversation information is more compact yet again and finally the high-level information is deliberately intended to be as compact yet as actionable as possible.
Now let’s just go through some of the application areas of deep packet inspection in today’s world. The first is the infrastructure of the network itself. So, these are actual components of the network. For example, they could be software defined wide area networking, or SD WAN, components that are providing intelligent routing – they’re looking inside the packets to choose the best route. I have things like application delivery and protection. So, I can have a web application firewall that’s protecting my web services I’ve got a load balancer that’s looking inside the packets to intelligently spread the load of request from clients to a farm of servers or workloads. And then I need to protect the business and my users. So obviously, I have firewalls, but next generation firewalls often look inside the packets to provide an extra degree of protection‚ and I have something like a web proxy, so that when users inside my business try and connect to the internet, we look inside the packet to make sure they’re not going to sites that could put the user or the business at risk.
So, these are part of the actual infrastructure of the network itself. Then‚ we want to look at the health of the network, how healthily is it working. I’ve got to worry about running out of capacity for example. I can be performing troubleshooting. So, when we talk about the control plane, this isn’t the part of the network where the protocols are carrying actual user traffic. This is what’s going on in the background to make sure the network can do its job. So, protocols like DNS, DHCP and BGP. To take an example from the mobile phone world, the 3GPP standards organization has defined hundreds if not thousands of protocols that enable mobile networks to work to make sure we can turn our phone on, it can attach to the network, we can move from cell tower to cell tower. All of that involves a control plane in the background. By monitoring that control plane we can make sure that everything is working as it should be and debug it if it isn’t. And we can also look at things like if there are degradations on the network such as packet loss or dropped packets that could affect a service. For example, they could affect voice services such as voice over LTE or VOLTE.
And then we can actually start to look at the health of the applications themselves. This is sometimes called network-based application performance management. Really what I’m trying to do is detect when a fault occurs. I then want to triage the problem in almost the sense of the emergency room. I want to understand‚ how bad the situation is, so in this case maybe how many users are affected, and then make sure that I allocate the problems to the correct team. I want to start to isolate the location of the problem because then I can get to the root cause of the problem, and I can even use deep packet inspection to collect evidence that I can give to the correct team, who own an application that might be in trouble, to help fix that application as quickly and efficiently as possible.
Next, we have a whole world of cybersecurity. I can detect vulnerabilities on the network, so for example, people using old, insecure protocols, for example like telnet or FTP‚ that actually exchange usernames and passwords without any encryption. I can look at solutions like intrusion detection and prevention. These actually look inside the packets. They can look for certain signatures that might, for example, identify the presence of malware.
I can also look for anomalous behavior by analyzing the traffic on the network. There’s a whole class of products that are loosely called network detection and response. I’ve got other application areas such as making sure that nobody is abusing an API in a particular service. And I could also use deep packet inspection to look inside packets, so I’m almost performing a forensic investigation. If I identify that an incident has occurred, I want to be able to go look inside the packets to really understand what happened, maybe what techniques, an attacker was using.
Finally, we have DDoS or distributed denial of service attacks. The art of detecting and mitigating these kind of attacks. And we broadly have two classes of attack. A volumetric attack is where the attackers have taken over lots of hosts on the network. And they focus all of their connections on a particular target and they consume that target’s bandwidth. They take down infrastructure such as, for example, a firewall. And typically the internet service providers or the CDN providers are well positioned to detect and block those kind of attacks. But because of the volume of traffic an internet service provider has to deal with, they can’t afford to perform deep packet inspection. We can do that at the edge of the enterprise and that means we can detect and stop the more low and slow attacks as we call them. These are things like state exhaustion attack where we consume all of the state available in something like a firewall which means that no more connections can be made to that firewall. We have application level attacks where for example we’re finding a very expensive query to run on somebody’s application and we just sit there and we keep running that query so we consume all of the resources of the application.
All of these five areas, from the actual network itself, making sure that I can manage the network, through the applications and services running on top of the network, cyber security and DDoS. All of these areas make a lot of use of deep packet inspection in today’s world‚
So I’m going to talk briefly about cyber security. Now the MITRE organization many years ago developed something called the attack framework. And if you’re familiar with the Lockheed Martin kill chain, the MITRE attack framework is kind of a sort of spiritual successor to that. Now in the attack framework, or the enterprise attack framework, there are fourteen‚ tactics identified that attackers are known to use. And I’ve listed seven of them here. The First is initial access. This is when an attacker has actually been able to install malware inside your network. Now the next thing they have to do is they effectively have to phone home. They establish what’s called a command and control channel so that human users‚ sitting on the dark web, can actually take control of the malware and continue with the attack.
Now the next thing they’re going to do is perform discovery. They’re going to search around on the network again to look for things like DNS servers or maybe LDAP authentication servers, places that have information about the enterprise about the users about the network that an attacker can use to their advantage.
Then the attacker is going to perform lateral movement. They’ve probably got elevated privileges by now they’re going to move from server to server and from workload to workload because ultimately they’re making their way to where the sensitive data is. That’s what attackers are typically after and it could be credit card numbers, it could be intellectual property, source code, all sorts of information that you as an enterprise do not want to be stolen.
And then ultimately they’re looking to exfiltrate that data which is a is a wordy way of basically saying they’re going to steal it and they’re going to get that data out of the enterprise and into their own hands. And we sometimes see a stage before that’s called collection and that’s where the attacker is basically pulling all of the data that they want to steal into one place and then they will try and extract it in one go as fast as possible.
Now the reason I’ve chosen these seven tactics in the framework is that all of these tactics occur over the network. And that means that if we apply deep packet inspection, we can identify and detect and investigate all of these seven types of tactic. So, it’s really valuable to be able to look at what’s happening on the network as part of our defense against cyber attacks.
And you can ask the question well is DPI important? It’s all very clever. In order to answer that question, I’ve shown here the sixteen critical infrastructure sectors that the Department of Homeland Security CISA Organization have identified as being critical to the state of the nation. So, we have things like water, dams, energy generation, nuclear. That’s one group of sectors. We have communications, transportation, emergency services, information technologies. Another group: financial services, fiscal, manufacturing, health care, food and agriculture, and lastly, we have government facilities, the defense industrial base, commercial facilities and chemical.
The company that I’m CTO for, NETSCOUT Systems. We have products that use DPI to look at the health of the network, the health of the applications over the network. We also have cyber security tools, we have DDoS detection and mitigation tools. We sell our products into every single one of these sixteen critical infrastructure areas. And they’re used by the country’s critical infrastructure sectors to protect their networks and their applications. So I’ll let you answer the question in your own mind, but hopefully I’ve made for the case that DPI is very, very important to keeping these areas going.
Just to finish off‚ what is the DPI consortium and why was it formed? The mission of the DPI consortium is to bring to light and document the history of deep packet inspection technology and innovation for new generations. Why do we want to do that? Well, we want to make sure that people don’t try and patent ideas that had already been invented a long time ago. The history of DPI goes back many decades, but oftentimes the terminology has evolved or the use cases have evolved. And we want to make sure that people for example don’t unnecessarily apply for patents when that idea has already been invented, but we also want them to be able to take advantage of the innovation that’s occurred over the last number of decades.
Now the challenge here is that much of the history of deep packet inspection isn’t documented in say patent applications or academic articles. So, one of the things the DPI consortium has done is to create our own database of hard to find DPI. This can be classified as prior art so it can be used by the patent office, for example, when they’re evaluating a patent application. And we put it on a platform called Zotero, which is a database that’s widely used in academic circles, and that’s freely available. Anybody on the internet can go and search that database and all of the content is fully text searchable. And I think I’m right to say there’s about two thousand articles in there and we’re working to increase the number of documents in there. So, I hope you found this helpful, we had a brief overview of how deep packet inspection is used in today’s world. I hope you’ll understand how important the applications of DPI are and I finally spoke a little bit about the DPI Consortium Thank you.
NOTE: This transcript has received minor edits from the original to improve readability.