Release Notes

Open Raven Platform Release: Improved Large File Scanning Control and Speed, Enhanced Maps, New Scan Metrics and Policies

Hamilton Yang
Director, Product Management
August 1, 2022

In our last release, we announced serious changes in our core data scanning capabilities, starting with a dramatic speed improvement for file enumeration and the ability to create scans for buckets of any size, making it easier to analyze large environments.

In this release, we’ve made a few more optimizations to make scans perform even better. We improved scanning throughput for large files. For example, we witnessed a 2x increase in speed when scanning large CSV files and a whopping 7x increase in speed when scanning large TXT files. We’ve also extended the amount of time that a scan can run to 14 days. As always, users can set custom time or cost limits, and scans can continue from where they left off if they decide to restart them. 

New scan metrics and monitoring

Available in Data Scans, we've added a new Scan Metrics feature. Scan Metrics shows key data points about a particular scan, such as the number of enumerated and scanned objects and the estimated cost of that scan job. 

Screenshot of Data Scans page. Scan Runs tab is selected and a table below includes name, asset group, started(pdt), completed(pdt), data to find, and status. Users can view results and create a new data scan job as well.
Screenshot of Data Scans page. Scan Runs tab is selected and and 'Test Run' is selected from the table which opened a panel on the right. Here users can view the scan job or results and see Metrics including no of objects scanned, enumrating bucket, findings, errors, and an estimated cost.

Enhanced Maps provide rapid answers to critical data security posture questions

In Maps, we've added a new way to visualize network connections - with particular attention to parts of the environment that are open to the public internet and VPC peering across regions. First, we combined all external connections into a single point on the map, which greatly simplifies the visualization. Second, we raised the visibility of external connections and VPC peering relationships to immediately see potential data transfer paths on the map without having to drill down. This new view helps security teams quickly answer critical security questions, including "Who can access the public internet?" and "Which of my VPCs are peered?".

Screenshot of Open Raven map. External connections are reaching to US East (N. Virginia), which has 95 assets, and US West (Oregon) which has 796 assets. On the right is a side panel of US East details, showing Security Group Connections.

Also, we added zoom breakpoints that make it easier to view large environments in their entirety and to zoom in on specific details. The breakpoints display different details depending on the zoom level. From a bird's-eye view, regions are abstracted, with the individual assets becoming more apparent upon zooming in.

Screenshot of the Open Raven map showing new zoom breakpoints. This view is very zoomed out with only region names (and the legend) legible, the rest of the map is abstracted.
Screenshot of Open Raven map zoomed in enough to view a singular region and all of its assets. Users can see which assets are backed up, have violations, etc.

The last improvement is the ability to easily share map views using a new export button in the top right-hand corner of the UI. Clicking on the button will export the map area in view as an image file that can be shared.

Screenshot of the Open Raven map zoomed in on US West (Oregon). In the upper right hand corner, there is an 'Export as JPEG' button.

New Policies: Ransomware prevention, geographical data mismatch

We’ve added a new policy called AWS Ransomware Prevention that identifies S3 buckets that may be vulnerable to ransomware attacks. Specifically, the policy looks for three critical configuration conditions — the bucket is exposed publicly due to ACL and policy, MFA Delete is disabled, and Bucket Versioning is disabled — that, if all are true, can expose S3 buckets to ransomware actors. This capability is also available in Magpie, our open-source CSPM tool.

We’ve also added a new “Regional Data Storage Best Practices” policy to determine if country-specific financial data and the AWS region of its data store match. For example, discovering UK financial data in a data store hosted in a non-UK AWS region will trigger a violation.

Bug Fixes & Enhancements

  • Updated showing assets with violations in maps. The asset color is now gray, and the violation color is in a small badge on top of the asset
  • Fixed an issue where asset-related information may have been delayed, outdated, or missing when deploying Open Raven to environments with several hundred thousand assets
  • "Flattened" data catalog findings for compressed files so that child files are no longer rolled up by the parent compressed file
  • Updated the ordering navigation menu items to match users' workflows
  • Updated Scan scheduling to reflect the time zone of the user
  • Fixed an issue where bucket configurations were not displayed during scan creation or edit
  • Fixed an issue where some violations marked as "false positive" were not properly suppressed from the UI
  • Fixed an issue where Violations page filters were unresponsive
  • Standardized the structure for alert description text to be more consistent across the board
  • Improved handling compressed file handling so that more files can be scanned
  • Fixed an issue where a scanner may scan the same page in a large PDF more than once
  • Fixed an issue where only partial results were returned from large JSON and Parquet files
  • Fixed an issue where scans of large JSON and Parquet files returned only partial results
  • Added filtering logic to reduce false positives for some generic data classes
Don't miss a post

Get stories about data and cloud security, straight to your inbox.