Real-Time Threat Detection in the Cloud

Organizations have moved business-critical apps to the cloud and attackers have followed. 2020 was a tipping point; the first year where we saw more cloud asset breaches and incidents than on-premises ones. We know bad actors are out there; if you’re operating in the cloud, how are you detecting threats?

Cloud is different. Services are no longer confined in a single place with one way in or one way out.

Traditionally, services have been deployed in data centers on servers that were close to each other, interconnected physically—and data had only one way in or out of that data center. Security was based around a perimeter; our realm was easily protected through firewalls like a medieval town surrounded by high, thick walls, limiting traffic and attacks through solid doors and defended through thin arrow slits.

‍

‍

Nowadays, services are distributed and operate in environments with limited perimeters. Developers, operators and users are located all around the globe. Having all those users access services through a single location will impact productivity and the user experience. Your services are no longer confined to a single place where there’s only one way in or out. If before we compared our infrastructure to a medieval town, now it is more like an amusement park.

‍

‍

An amusement park full of attractions, multiple entrances and exits and many more chances for actors to behave unexpectedly. A distributed infrastructure based on cloud technologies requires detecting threats from myriad sources. So many actors interacting in so many different ways increases the number of potential events and the amount of information that needs to be monitored.

Threat Detection: A Delicate Balance

A common approach to threat detection starts with shipping logs into a centralized repository, then searching for indicators of suspicious behavior or configuration changes that increase risk. That takes time—it’s like trying to identify a moving target. Copying logs outside the cloud and storing an extra copy can be an operational pain in the neck, and it’s expensive. And, most importantly, this approach delays the ability to detect and respond to threats.

Obviously, the closer monitoring tools are positioned to the source of an event, the better the response time can be. This could, however, add complexity and increase costs. Besides, there are still too many steps involved in this pipeline. Couldn’t this be improved somehow?

Inspect Logs in Real-Time with Stream Detection

What if instead of trying to guard a fortified town with a few well-defined entry points, we start thinking about how to watch the activity inside that amusement park? Imagine smart security cameras constantly on alert and looking for anomalous behavior, reacting accordingly and triggering alarms when necessary. Translated to cloud infrastructures, that means the more accurate way to monitor security in the cloud is through stream detection.

Stream detection is a continuous process that collects, analyzes and reports on data in motion. With a streaming detection process, logs are inspected in real-time. This real-time detection allows you to identify unexpected changes to permissions and services access rights as well as unusual activity that can indicate the presence of an intruder or, worst case, exfiltration of data. Based on that idea, the open source community offers a solution: Falco.

Falco is an open source runtime security tool, often described as a security camera for modern cloud infrastructure. Falco is an incubating-hosted level project in the Cloud Native Computing Foundation (CNCF). It was originally designed to watch workloads, so Falco focuses on collecting system calls from running endpoints, like hosts or containers running applications, and collecting granular data from the source in those containers, understanding the details of what applications do.

Obviously, not everything in our infrastructure is hosts and containers. Organizations also benefit from numerous external services offered by their cloud provider(s). Thankfully, the cloud provider also facilitates sharing of valuable information generated by each service, and that information is useful for monitoring. Here is where Falco behaves differently from conventional alternatives: Since the open source project is able to consume additional types of data, it can ingest and digest that information in real-time to generate alerts in the moment.

Consider how this works in an Amazon Web Services (AWS) environment, for example. Almost anything happening in AWS is tracked and logged in AWS’ version of cloud logs, called CloudTrail. By monitoring logs from CloudTrail, you can detect unexpected behavior, configuration changes, intrusions and data theft, not only from existing services but also from newly released ones. Connecting Falco to CloudTrail gives you the flexibility to manage your rules in one single place. Not having to store your logs externally also reduces bandwidth and storage costs.

Respond to Threats Faster

Time is a critical factor in reducing risk. Parsing logs in real-time and leveraging stream detection allows you to immediately detect suspicious activity and trigger an alert for further investigation.

The whole point of stream detection—and of projects like Falco—is to align an effective security approach with the realities of modern architectures. Doing so enables faster time to detection and equips security teams with evolving, community-driven innovation, all of which are critical for effective security with modern stacks. The reality is that the cloud is a new beast, and it enables a new approach to everything, including security. This is your opportunity to get it right!

‍