Network Topology

The overall topology for the Archival Storage Network is a private network linking three machine rooms at two archival sites, which is in turn connected to the Millennium Backbone Network through a single gateway. The three separate machine rooms spread across two buildings: 290 Soda Hall, 340 Soda Hall, and 165 Cory Hall. The Millennium network was chosen as the connection to the outside world since most of the CITRIS resources are connected there, as well as the Millennium Fast Storage portion of the PSI project.

The network will be set up as a tree of rack-level clusters. Each rack will hold one or two clusters of up to 15 bricks each. Each cluster of bricks is connected by a rack-level concentrator box, which is in turn connected to a concentrator for each room. The two room-level concentrators for 340 Soda and 290 Soda are connected to a site-level router in 288 Soda. It in turn connects to the site-level concetrator for 165 Cory as well as the Millennium network backbone and from there to the outside world. Initially the PSI Archival Storage Network will have just one link to the Millennium Soda Backbone router, but eventually we may wish to connect it to the Evans Hall Backbone router as well.

The use of site-level, room-level, and rack-level concentrators provides several advantages. Rack-level concentrators can be added as needed for expansion. Since bricks added at the same time will most likely be talking to each other for replication, the rack-level concentrator helps isolate this cross-talk from other parts of the network. Smaller switches are also less costly than a large router chassis with high-bandwidth backplanes. Also, since we are hoping to support remote site replication, the hierarchical network design encourages us to develop strategies that can deal with replication across a 1Gbit inter-site connection while allowing higher levels of intra-site bandwidth.

Network Hardware

For the main routers, we have two Extreme Networks Summit5i boxes with 12 SX/MTRJ multi-mode fiber ports and 4 GBIC slots. These routers require 1000LX/SC GBICs for the inter-building connection.

For the room-level concentrators we are starting with Asante 35516-T L2/L3 switches, which have up to 16 1000bT/RJ45 connections, 4 of which can be switched to GBICs. One 1000SX/SC GBIC will be used in each switch to talk to the site-level routers.

Initially we will position the two Extreme routers in 288 Soda, and 165 Cory. The router in 288 Soda will also manage the connections between the private network and the outside world.

The three Asante switches will initially function as rack-level concentrators in 290 Soda, 340 Soda, and 165 Cory. As we expand to a second rack-level cluster, we will switch to using the Asante 35516 switches in 290 Soda and 340 Soda as room-level concentrators.

The room-level concentrators can each support up to 15 rack-level clusters of up to 15 bricks each, for a maximum of 225 bricks per site or 675 bricks for three sites. Higher degrees of scaling could be accomplished by using additional room-level concentrators under each site router. We anticipate this level of fan-out should be more than sufficient to handle the scope of this project.

Additional rack-level concentrators may be Asante 35160 L2 switches since the L3 functionality is not required for this function. However if we discover that 100bT to each brick is sufficient, then less expensive rack-level concentrators can be substituted.

The bricks have 100bT RJ45 connections on the motherboard, but we will be equipping at least the first round of bricks with an Intel PRO/1000 MT gigabit card as well. This will allow us to compare the difference between using 100bT versus 1000bT to each brick.

See Also

• Brick Hardware
• Archival Software