978-1-4244-1694-3/08/$25.00 ©2008 IEEE A Space- and Time-Efficient Hash Table Hierarchically Indexed by Bloom Filters Heeyeol Yu and Rabi Mahapatra Texas A&M University College Station, TX 77843 Email: {hyyu,rabi}@cs.tamu.edu Abstract Hash tables (HTs) are poorly designed for multiple mem- ory accesses during IP lookup and this design flow critically affects their throughput in high-speed routers. Thus, a high capacity HT with a predictable lookup throughput is desir- able. A recently proposed fast HT (FHT) [20] has draw- backs like low on-chip memory utilization for a high-speed router and substantial memory overheads due to off-chip du- plicate keys and pointers. Similarly, a Bloomier filter-based HT (BFHT) [13], generating an index to a key table, suffers from setup failures and static membership testing for keys. In this paper, we propose a novel hash architecture which addresses these issues by using pipelined Bloom filters. The proposed scheme, a hierarchically indexed HT (HIHT), gen- erates indexes to a key table for the given key, so that the on-chip memory size is reduced and the overhead of point- ers in a linked list is removed. Secondly, an HIHT demon- strates approximately 5.1 and 2.3 times improvement in on- chip space efficiency with at most one off-chip memory ac- cess, compared to an FHT and a BFHT, respectively. In ad- dition to our analyses on access time and memory space, our simulation for IP lookup with 6 BGP tables shows that an HIHT exhibits 4.5 and 2.0 times on-chip memory efficiencies for 160Gbps router than an FHT and a BFHT, respectively. 1. Introduction The rapid increase in high-bandwidth usage applications from a huge number of hosts has resulted in a substantial de- mand for high-speed and large-scale routers. It is observed that a class of fast packet processing, such as packet classifi- cation and IP lookup, has become critical data path functions for these networking applications. These functions have en- joyed wide application in networking devices to support fire- wall, access control list, and quality of service in several net- work domains. They look up a packet in a table that fits its associated rule under various matching conditions, such as singleton, longest prefix match, or range match. One in- teresting lookup scheme uses Ternary Content Addressable Memory (TCAM) to achieve deterministic and high-speed packet processing [14, 22, 24]. Unlike TCAM with high cost and power consumption, ap- proaches using a Bloom filter (BF) have been widely docu- mented in literature on networking [8–11, 13, 18, 20]. A BF is a virtually generalized hash mechanism for an approxi- mate membership testing with memory-efficient set repre- sentation. Dharmapurikar et al. in [9] introduced the first algorithm to employ BFs working in parallel for IP lookup. However, any packet processing that simply uses BFs like [9, 11, 18] can only provide an approximate match, so that it will suffer time loss from the later sequential perfect match. Traditionally for a fast perfect match search, a hash table (HT) is widely used to perform fast associative lookups since the search requires O(1) average memory access per lookup. A typical HT application is a network packet processing in a high-speed network router device [13, 19, 20, 23]. In this domain, it is desirable to reduce memory cost and to im- prove lookup speed by using a small amount of fast on-chip memory that takes 1-5ns and cheap off-chip memory that takes 10ns. Beyond a need of having an O(1) complexity for IP lookup with on/off-chip memories, a collision rate for a given lookup must also satisfy the bandwidth requirement of a high-speed router. For example, a 160Gbps router re- quires a deterministic process of 500M packets of minimum size 40 bytes per second. Thus, on average, a chance of col- lision among 500M lookups does not satisfy the bandwidth requirement of the 160Gbps router. Literature [9, 13, 20] on a BF has focused on the deterministic lookup with a very low collision rate, so that for a given lookup a few accesses to off-chip memory are made. These approaches [13,20], however, have the following defects not suitable for a high-speed and large-scale router: 1) Song et al. [20] claimed that for a perfect match a fast HT (FHT) with help of a BF improves the performance over a legacy HT (LHT) by combining hashed linked lists with Authorized licensed use limited to: National Cheng Kung University. Downloaded on April 30, 2009 at 06:11 from IEEE Xplore. Restrictions apply.