AI Smart Scan on Oracle Exadata: Accelerating AI Vector Search for RAG and Similarity Search

1. Introduction: Exadata and the Rise of In-Database AI

1.1. Oracle Exadata: High-Performance Database Platform

Oracle Exadata stands as a premier enterprise database platform, engineered to run Oracle Database workloads with exceptional performance, availability, and security across all scales and criticalities. Its architecture integrates high-performance database servers, intelligent storage servers, and an ultra-fast, low-latency internal network fabric (typically RDMA over InfiniBand). The core philosophy is the co-design of hardware and software, enabling unique optimizations for database operations at both compute and storage layers.  

Exadata is optimized for both Online Transaction Processing (OLTP) and Data Warehousing (DW)/Analytics. It has evolved to support modern workloads like in-memory analytics, Artificial Intelligence (AI), and Machine Learning (ML), facilitating efficient mixed-workload consolidation. Its scale-out design allows balanced expansion of compute, storage, and network resources to meet growing demands.  

1.2. The Trend Towards In-Database AI

Integrating AI and ML capabilities directly into database platforms is a significant trend. This approach minimizes or eliminates the need to move data to separate systems for analysis or model training, reducing complexity, latency, and security risks associated with data movement. Processing data in place allows AI/ML algorithms to leverage the database’s transactional capabilities, security models, and consistency guarantees.  

Oracle addresses this through its Converged Database strategy, managing diverse data types (relational, JSON, graph, spatial, and now vector) and workloads within a single database engine. This aims to eliminate data silos and management complexity associated with specialized databases.  

1.3. Report Focus: AI Smart Scan for Vector Processing

This technical report provides an in-depth analysis of the AI Smart Scan feature on the Oracle Exadata platform, specifically its vector processing capabilities. It defines AI Smart Scan within the Exadata context, explains how it accelerates Oracle AI Vector Search, details the offloading mechanism for vector tasks to Exadata storage servers (especially within the Exascale architecture), and documents claimed performance gains.

The report highlights the importance of this feature for modern database AI applications, particularly Similarity Search and Retrieval-Augmented Generation (RAG) for Generative AI.

2. Understanding AI Smart Scan in the Exadata Context

2.1. Evolution from Traditional Smart Scan (SQL Offload)

Understanding AI Smart Scan requires knowledge of its predecessor, the traditional Smart Scan (or SQL Offload), a cornerstone of Exadata. This technology pushes data-intensive SQL processing from database servers to intelligent storage servers.  

In conventional architectures, full table scans move all data blocks across the network to the database server for filtering (WHERE clauses) and projection (SELECT list), consuming network bandwidth and database server CPU.  

Exadata Smart Scan optimizes this by sending SQL filters and column projections to the storage servers. Storage servers apply these filters as data is read, returning only relevant rows and columns to the database server. This dramatically reduces data transfer and database server CPU usage, boosting database performance.  

2.2. Defining AI Smart Scan

AI Smart Scan is a set of Exadata-specific optimizations designed to accelerate the AI Vector Search capabilities introduced in Oracle Database 23ai. It extends the Smart Scan philosophy to AI vector operations, specifically offloading compute-intensive vector distance calculations and Top-K nearest neighbor filtering to the storage servers.  

Introduced with Oracle Exadata System Software 24.1 and requiring Oracle Database 23ai or later , AI Smart Scan leverages the processing power of Exadata storage servers and the high-speed internal network to optimize vector-based queries.  

2.3. Core Objectives: Performance and Efficiency

The primary goal of AI Smart Scan is to achieve orders-of-magnitude performance improvements for AI Vector Search queries, especially on large datasets. This is accomplished by addressing the computational intensity of vector processing.  

By offloading distance calculations and Top-K filtering to the Exadata storage servers where data resides , AI Smart Scan achieves:  

1. Reduced Database Server Load: Frees up database server CPU resources for other tasks, improving overall system throughput.  
2. Minimized Network Traffic: Only filtered results (e.g., top K vectors) are sent back, significantly reducing data movement, crucial for high-dimensional vectors.  
3. Low Latency and High Throughput: Processing data closer to the source, combined with Exadata’s low-latency RDMA network, results in faster query responses.  

AI Smart Scan represents a logical extension of Exadata’s core principle: moving processing closer to the data. It adapts the proven SQL offload architecture to the demanding requirements of modern AI workloads, particularly the compute-heavy nature of vector processing. The claim of “orders of magnitude” performance gains signifies a fundamental architectural advantage, positioning Exadata as a high-performance platform for vector search, competitive with specialized vector databases.  

3. Accelerating AI Vector Search with AI Smart Scan

3.1. Oracle AI Vector Search Fundamentals

Oracle AI Vector Search is an integrated database capability enabling semantic search based on meaning, not just keywords. It uses vector embeddings – multi-dimensional numerical representations – to capture the semantic meaning of structured and unstructured data (text, images, audio). Semantically similar items have vectors closer in the vector space.  

Key use cases include:

• Semantic Search: Searching documents, products by meaning.  
• Recommendation Systems: Suggesting similar items based on user preferences.  
• Anomaly Detection: Identifying outliers.  
• Image/Video Search: Finding visually similar content.  
• Retrieval-Augmented Generation (RAG): Enhancing Large Language Model (LLM) accuracy with relevant enterprise data.  

Oracle Database 23ai provides:

• VECTOR Data Type: Native storage for vector embeddings.  
• Vector Indexes: Optimized indexes (HNSW for in-memory, IVF for disk-based) to accelerate similarity searches.  
• SQL Operators/Functions: New SQL capabilities (VECTOR_DISTANCE) for performing similarity searches and combining them with other data types.  

3.2. AI Smart Scan’s Role in Acceleration

AI Smart Scan is central to optimizing AI Vector Search query performance, especially for large-scale vector data scans. It addresses the compute-intensive nature of finding nearest neighbors in vast vector datasets.  

Acceleration mechanisms include:

• Compute Offload: Intensive vector distance calculations and Top-K filtering are executed on storage servers, not the database server.
• Parallel Processing: Exadata’s scale-out architecture allows these offloaded operations to run in parallel across multiple storage servers.
• Data Reduction: Only filtered results (top K vectors) are sent back to the database server, minimizing network traffic.
• Hardware Optimization: AI Smart Scan leverages Exadata’s ultra-fast storage tiers (XRMEM, Smart Flash Cache) and low-latency RDMA network.  

These combined mechanisms deliver low-latency responses and high-throughput processing for AI Vector Search queries. Oracle’s strategy of integrating AI Vector Search into the database and accelerating it with AI Smart Scan embodies the “bring AI to the data” approach , simplifying architectures compared to using separate vector databases and offering significant efficiency gains, especially for existing Oracle users. The focus on handling “massive volumes” and “high concurrency” positions this technology for demanding, mission-critical enterprise AI workloads.  

4. Offloading Vector Processing to Exascale Storage Servers

4.1. The Offload Mechanism Explained

AI Smart Scan pushes the most CPU-intensive vector search steps—vector distance calculations and Top-K filtering—down to the Exadata storage servers.  

1. Vector Distance Calculation Offload: Computations using metrics like Cosine Similarity or Euclidean Distance are performed directly on storage servers, distributing the CPU load.  
2. Top-K Filtering Offload: Identifying the ‘K’ nearest neighbors to a query vector also happens at the storage layer. Only these top candidates (or potential improvements) are returned, preventing unnecessary network transfer of irrelevant vectors.  

This storage offload dramatically reduces network traffic between database and storage servers, conserving bandwidth and lightening the load on the database server, especially critical for high-dimensional vectors.  

4.2. Leveraging Exadata Hardware for Vector Processing

AI Smart Scan’s effectiveness is tightly coupled with Exadata’s hardware:

• Exadata RDMA Memory (XRMEM) & Smart Flash Cache: AI Smart Scan processes vector data at “memory speed” using these ultra-fast storage tiers, offering much lower latency than traditional storage. RDMA (Remote Direct Memory Access) allows direct data transfer to database server memory, bypassing network stacks for further latency reduction and throughput gains.  
• Scale-out Architecture: Offloaded vector operations are parallelized across all available storage servers in Exadata’s scale-out design, leveraging numerous CPU cores for rapid processing of large datasets.  

4.3. Enhancements in Exadata System Software 25.1

Oracle continuously refines AI Smart Scan. Release 25.1.0 introduced key improvements :  

• Enhanced Top-K Filtering: Storage servers maintain a running Top-K set locally, only sending results that improve the current best set back to the database server. This significantly reduces network traffic and improves performance, especially for large K values.  
• INT8 and BINARY Vector Support: AI Smart Scan now supports these compact, efficient formats alongside high-precision FLOAT types. BINARY vectors offer up to 32x smaller size and 40x faster distance computation compared to FLOAT32, with minimal impact on search quality in some tests. INT8 offers 4x compression with negligible quality difference in evaluations. This caters to diverse accuracy vs. performance needs.  
• Vector Distance Projection Offload: When a query selects the vector distance (SELECT VECTOR_DISTANCE(...)), this calculation is now also offloaded to storage servers, further reducing network traffic by avoiding the transfer of large vectors just to compute the distance on the database server.  

Support for INT8/BINARY formats broadens AI Smart Scan’s applicability beyond high-accuracy scenarios to use cases prioritizing performance and efficiency. Offloading distance projection demonstrates Oracle’s commitment to refining the offload mechanism for maximum network optimization.  

4.4. Exascale Architecture and Vector Offload

Oracle Exadata Exascale is a next-generation architecture merging Exadata’s performance with cloud elasticity and cost-effectiveness. Its loosely-coupled design, separating compute and storage into shared resource pools , provides an ideal foundation for AI Smart Scan offload.  

The Exascale intelligent storage cloud hosts the storage servers targeted by AI Smart Scan. Shared storage pools and the RDMA fabric enable efficient distribution and parallel execution of offloaded vector tasks. Exascale combines the elasticity needed for AI workloads with the raw performance delivered by AI Smart Scan, making Exadata attractive for dynamic, cloud-native AI applications.  

5. Measured Performance Gains for AI Vector Search

Oracle reports significant performance improvements for AI Vector Search on Exadata using AI Smart Scan.

5.1. General Acceleration Claims

Marketing and technical documents often cite acceleration of up to 30X or up to 32X compared to non-offloaded architectures or potentially older Exadata versions. These figures highlight the fundamental benefit of the offload mechanism.  

5.2. Exadata X11M Platform Gains

Compared to the previous generation X10M, the latest Exadata X11M platform delivers specific gains :  

• Persistent Vector Index (IVF) Searches: Up to 55% faster due to intelligent storage offload.
• In-Memory Vector Index (HNSW) Queries: Up to 43% faster.

These improvements reflect the combined effect of newer hardware (AMD EPYC processors) and software optimizations on X11M.

5.3. Software Optimizations (All Platforms)

Optimizations in Exadata System Software 25.1 benefit all modern Exadata platforms:

• Data Filtering: 4.7X more data filtering capacity in storage servers.  
• BINARY Vectors: Queries using the newly supported BINARY format can run up to 32X faster than with FLOAT32 vectors, due to smaller size and faster distance computation.  
• BINARY Distance Computation: The distance calculation itself for BINARY vectors can be up to 40X faster than for FLOAT32.  

5.4. Summary Table of Performance Claims

Performance Claim	Comparison Point / Context	Relevant Hardware / Software	Source Snippet IDs
Up to 30X faster AI Vector Search	Traditional architecture / Non-offload	Exadata (General) / ESS 24ai+
Up to 32X faster AI Vector Search	Traditional architecture / Previous X10M	Exadata Exascale / ESS 24ai+
Up to 55% faster IVF searches	Exadata X10M platform	Exadata X11M / ESS 25.1+
Up to 43% faster HNSW queries	Exadata X10M platform	Exadata X11M / ESS 25.1+
4.7X more data filtering	Previous software versions	All Exadata Platforms / ESS 25.1+
Up to 32X faster BINARY query	Queries with FLOAT32 vectors	All Exadata Platforms / ESS 25.1+
Up to 40X faster BINARY distance calc	Distance calc with FLOAT32 vectors	All Exadata Platforms / ESS 25.1+

The variety in performance claims highlights that actual gains depend on multiple factors: Exadata hardware generation, software version, vector type (FLOAT, INT8, BINARY), dimensionality, index type (IVF, HNSW), dataset size, and query complexity. Users should evaluate these nuances for their specific scenarios. The strong focus on AI Vector Search performance in recent Exadata X11M and ESS 25.1 announcements underscores its strategic importance for Oracle, positioning Exadata as a key platform for the growing AI/ML market.  

6. Importance for Modern Database Applications and AI

Exadata’s accelerated AI Vector Search via AI Smart Scan has significant implications for modern applications, especially those driven by AI.

6.1. Enabling In-Database AI Applications

AI Smart Scan facilitates running AI logic directly within the Oracle database where the data resides, offering key advantages :  

• Reduced Data Movement: Minimizes costly, complex, and potentially insecure data transfers to external AI platforms.  
• Enhanced Performance: Processing data locally with offload capabilities significantly improves query latency for AI algorithms.
• Improved Data Security: Sensitive data remains within the database boundary, protected by Oracle’s robust security features.  
• Simplified Architecture: Reduces the need for complex integrations between disparate data stores and AI tools.
• Consistency: AI operations can benefit from database ACID guarantees.

6.2. Powering RAG (Retrieval-Augmented Generation) Architectures

Retrieval-Augmented Generation (RAG) enhances LLM responses by grounding them in external, often private or real-time, data. RAG mitigates LLM limitations like knowledge cut-offs and potential inaccuracies (“hallucinations”).  

AI Smart Scan-accelerated AI Vector Search is ideal for the crucial retrieval step in RAG. It efficiently finds semantically relevant information (documents, records) within the Oracle database based on a user’s query (converted to a vector). This retrieved context is then fed to the LLM, enabling it to generate more accurate, relevant, and trustworthy responses based on specific enterprise data. This is vital for building reliable generative AI applications like chatbots and internal knowledge systems.  

6.3. Driving Similarity Search Use Cases

AI Vector Search excels at finding semantically similar items beyond simple keyword matching. AI Smart Scan makes large-scale similarity searches efficient, enabling applications like:  

• Product/Content Recommendations: Suggesting items similar to user preferences.  
• Visual Search: Finding similar images/videos.  
• Document Similarity: Locating related documents in large corpora.  
• Fraud/Anomaly Detection: Identifying patterns similar to known fraud or deviations from normal behavior.  
• Customer Support: Matching queries to relevant knowledge base articles.  
• Bioinformatics/Medicine: Comparing medical images or symptoms to known cases.  

In all these scenarios, AI Smart Scan’s offload capabilities ensure efficient execution on large datasets. This integration of high-performance vector processing within the Converged Database is a key differentiator for Oracle, simplifying architectures and potentially lowering TCO compared to using separate specialized databases. For RAG, performing the retrieval step securely within the database before potentially sending filtered context to an LLM is a critical advantage for protecting sensitive enterprise data.  

7. Conclusion and Summary

7.1. Key Findings Summarized

• AI Smart Scan Defined: A critical, Exadata-specific optimization accelerating AI Vector Search (introduced in Oracle DB 23ai) by offloading compute-intensive vector distance calculations and Top-K filtering to intelligent storage servers.
• Offload Mechanism: Leverages Exadata hardware (RDMA, XRMEM, Flash Cache) to perform vector operations at memory speed near the data, drastically reducing network traffic and database server load.
• Continuous Improvement (ESS 25.1): Enhancements include more efficient Top-K filtering, support for performant INT8/BINARY vector formats, and offloading of vector distance projection.
• Performance Gains: Oracle claims significant speedups (up to 30X/32X) over non-offloaded methods. Exadata X11M offers further gains (up to 55% faster IVF, 43% faster HNSW vs. X10M), and software optimizations provide boosts like 32X faster queries with BINARY vectors. Actual results vary based on configuration and workload.
• Importance for Modern AI: Enables efficient in-database AI, simplifies architectures, enhances security, and is crucial for high-performance RAG retrieval and various large-scale similarity search applications.

7.2. Future Outlook and Strategic Significance

Oracle’s ongoing investment signals the strategic importance of database AI. Future enhancements to AI Smart Scan might include broader operation offload and deeper integration with cloud architectures like Exascale.

The shift towards in-database AI promises more agile, efficient, and secure solutions by processing data where it resides. Exadata AI Smart Scan is central to Oracle’s Converged Database strategy, offering a compelling alternative to specialized vector databases by combining Exadata’s proven enterprise capabilities with cutting-edge AI workload acceleration.