When our data engineering team was enlisted to work on Tenable One, we knew we needed a strong partner. Here’s how we selected Snowflake to help us deliver on the promise of exposure management.
The Tenable One Exposure Management Platform, introduced Oct. 4, represents a paradigm shift in how organizations can improve their preventive cybersecurity strategies to reduce risk. By giving users a holistic view of their entire attack surface — including vulnerabilities, misconfigurations, web applications, cloud systems, user identities and privileges and attack path analysis — the Tenable One platform is poised to serve as the foundational technology upon which an organization can build an exposure management program.
While the story of Tenable One is, first and foremost, a technology story, the analytics baked into the platform would not be possible without the ability to ingest and process a wide variety of data from a suite of point tools.
When Tenable’s product engineering team came to us in data engineering asking how we could build a data platform to power the product, we knew we had an incredible opportunity to modernize our data stack. By utilizing the benefits of our data platform, we were able to build out the vast majority of features needed for Tenable One in six months. In that time, our data engineering team also scaled from five to 11 engineers. We knew we needed additional support to handle the undifferentiated heavy lifting of managing a distributed data infrastructure so that our in-house teams can be fully focused on delivering customer value.
In this blog, we share the story of our search for a managed service provider who could deliver what we needed while also positioning us for future growth.
The data priorities for Tenable One
Our legacy stack had been based on Amazon Web Services’ (AWS) Elastic MapReduce (EMR) and Simple Storage (S3) and would not have scaled for the requirements given. The industry has moved away from Hadoop-based technologies and embraced the modern data stack. It has swung back to structured query language (SQL) and data warehouses built for and on the cloud.
We believed that if we could leverage these advancements in the data industry, we could build a data platform as a solid foundation for Tenable’s future growth. Any solution we chose would have to give us an advantage in the following areas:
- Time to market - we wanted a fully managed service that abstracts away the undifferentiated heavy lifting of managing infrastructure. This would allow us to focus our efforts on delivering customer value faster.
- Decoupled storage and compute - having the ability to scale compute power independently of storage is a game changer. We knew it would allow us to be more agile in how we dial compute power up and down as needed without scaling our storage.
- SQL support - SQL is one of the greatest abstractions ever built. To support a SQL statement execution on distributed data, an incredible amount of complex processing and data movement occurs. By choosing a platform with SQL support, we knew we would also be able to source engineers with the skills we needed and onboard them quickly.
- Interoperability - Tenable is an AWS partner and we host most of our services on the AWS public cloud. We knew we would need a solution that would integrate well with AWS. Also, we were very certain that neither AWS nor any other single vendor could provide all the services we would need as we grew. Therefore, we needed to choose a solution that would support and work with other data tools in the modern data stack.
- Global footprint - Tenable is a global application with several sites throughout the world. Any solution we chose had to be able to match our footprint.
How we chose Snowflake
After an extensive market scan and running detailed proofs of concept based on Tenable data, we chose Snowflake for the following six reasons:
- Managed services
- Workload isolation
- Variety of features
- Multi-regional and multi-site capabilities
- Continuous integration/continuous deployment (CI/CD)
- AWS compatibility
Below, we provide more details on how Snowflake was able to meet our needs in each area.
1. Managed services
Snowflake is a fully managed service, meaning that we do not have to manage any infrastructure. The service sits on Snowflake’s network in AWS and takes care of the undifferentiated heavy lifting of managing a distributed data infrastructure. This means our teams can be fully focused on delivering customer value. They can focus on developing new product features to keep our customers' assets secure instead of worrying about infrastructural issues.
2. Workload isolation
The separation of compute and storage means we can be extremely flexible in how we analyze our customers' data and serve insights back to them. The ability to spin up segregated compute resources for each workload means we can continue to process data while maintaining a stable and consistent experience for our customers.
Queries used to load and transform data do not interfere with queries used to serve up data to our customers and applications. In effect, Snowflake makes implementing a command query responsibility segregation (CQRS) pattern extremely easy. The only difference between our model and a traditional CQRS is the ability to use shared storage. Customer-facing applications that get their data from Snowflake have dedicated compute resources that are never shared with backend data processing resources. Tenable One customers will receive a consistent experience in terms of performance and load times.
Source: Tenable, October 2022
3. Variety of features
Snowflake has a lot of great features that have made our development process easy and fast. As we continue to add new capabilities to Tenable One, the following Snowflake features will help us reduce the overall time to market for updates to our product:
- Snowpipe allows us to land a continuous stream of unprocessed data from S3 into a landing area within the database.
- Streams allow us to track changes coming into the landing area and only process the changed records.
- Tasks allow us to schedule processes on a cron schedule to begin processing data when we need it.
Having these features supported natively by Snowflake means we don’t need to implement separate tools for orchestration and scheduling. Our developers can continue to focus on delivering the features and improvements that bring value and security to our customers.
Source: Tenable, October 2022
When making our choice to go with Snowflake, we were also excited by a number of other features that were either recently released or are in preview, including:
- Data sharing is an exciting concept for us as it may allow us to share data from our platform back to users via Snowflake's data cloud. It cuts out the need for any data integration or APIs and will help simplify our users' experience.
- Unistore will allow us to support online transaction processing (OLTP) workloads within Snowflake. This will be useful where we support applications that need to join new dimensions with transactional data for filtering metrics and insights.
- Search optimization service will be useful if we need to support exposing transactional data to applications. SOS provides regular expression searches on database fields and semi-structured data types.
4. Multi-regional and multi-site
Tenable One is a multi-site platform. The application is hosted on 15 separate Snowflake databases in 11 Snowflake accounts spread around the globe. This allows our customers to maintain and store their data where they feel comfortable storing it and so they can remain in compliance with their local or regional regulations.
Snowflake’s multi-regional/multi-site capabilities also allow Tenable to maintain the lowest latency possible to ingest and serve data. This ensures that the metrics and insights that our customers value are available in the shortest time possible. The geographical reach of Snowflake was an extremely important factor when we chose them.
Replication is another Snowflake feature that allows users to move shared data between accounts. It allows us to build our data pipeline for certain datasets once and then copy it out to each site using the Snowflake backbone. Replication has also enabled Tenable One’s benchmarking feature, which allows our customers to anonymously share their own exposure score and see how they rank against other customers within the same industry.
Source: Tenable, October 2022
5. CI/CD
Maintaining the same code base is vital in a multi-site platform and Snowflake's support for Terraform and Flyway allows us to deliver changes consistently to each site in a controlled manner. The combination of these tools and Snowflake allows Tenable to deploy features and updates multiple times per day. Having simple and fast deployments means our customers can get the benefit of new or improved features as soon as they are ready. Tenable can gather feedback as to the effectiveness of any changes and adapt to customer feedback quickly.
6. AWS
As previously mentioned, Tenable is an AWS partner, and being able to rely on the AWS infrastructure and services to build Tenable One was important. We have chosen to host our Snowflake accounts on AWS. And while we have chosen Snowflake as our main data storage and processing service, it does not exist in a vacuum. It integrates very well with our existing AWS services to both consume and publish data. By building on Snowflake and AWS, our customers can be assured of a reliable and secure foundation that provides the best in class in terms of uptime and availability.
Source: Tenable, October 2022
Summary
It has been fantastic to work on delivering an exciting new product for our customers this year and we are only getting started. With the benefits of Snowflake on AWS, we can provide our customers with a solid experience, delivering insights and metrics in a timeframe that is meaningful to them. The developer experience and native features reduce the time to market for introducing new features and implementing improvements to existing features. It means we can easily find more developers and train them quickly, enabling us to deliver value for our customers in as short a time as possible. With Snowflake, we believe we have gotten the foundations right for Tenable One and we can only build up from here.
Learn more
- Read the blog, Exposure Management: Our Vision for Securing the Modern Attack Surface
- Download the white paper, 3 Real-World Challenges Facing Cybersecurity Leaders: How an Exposure Management Platform Can Help
- View the infographic, From Risk-Based Vulnerability Management to Exposure Management: The Changing Definition of Good Cyber Hygiene