Facebook aims to knock Cisco down a peg with open network hardware | Ars Technica

Two years ago, Facebook set out to change the server and storage industries by creating freely available hardware designs that gave customers more flexibility than those on offer from the HPs and Dells of the world.

Now, Facebook aims to knock Cisco and the other top network vendors down a peg by leading a nearly identical project for switches. Facebook is leading this disruption of the data center hardware industry through the Open Compute Project, which takes the open source approach typically applied to software such as Linux and aims it at hardware.

Thus far, the project had worked on reference designs for servers, motherboards, storage, racks, and interconnects. Facebook itself has used these designs to buy hardware directly from original design manufacturers (ODMs) that build servers exactly to Facebook's specifications. This approach strips out many of the hardware and software features companies like HP and Dell insert into general-purpose products, making them cheaper and more efficient. Going directly to ODMs is easier for companies with the purchasing power of Facebook, but some vendors are embracing Open Compute Project designs, allowing even smaller customers to benefit.

Open Compute embraces networking

Today, Facebook announced at the annual Interop networking conference in Las Vegas that the Open Compute Project will design a top-of-rack switch that can boot pretty much any networking software a customer wants. The switch would provide an alternative to vendors like Cisco, Arista Networks, and Dell's Force 10 division, Frank Frankovsky, VP of hardware design and supply chain operations at Facebook, told Ars last week.

"We still don't have an open source switch variant for the hardware stack and we certainly haven't figured out a way to enable the hardware selection separate from the network operating system selection," Frankovsky said.

Facebook officials acknowledged that switch vendors face a tough task in building products that can meet the needs of a wide range of customers without falling behind the pace of innovation demanded by large Internet companies. But Frankovsky still took a dim view of some commercially available switching products.

"Some of the things we've seen in off-the-shelf switch products lead me to believe that maybe the people that designed these switches have never been in a data center," Frankovsky said. "They do some really weird things with the way they mount into rack enclosures, with air flow direction. For example, we have a row of cluster switches that exhaust heat into each other. I don't know if it's just because the thermal engineer never envisioned an entire row of these things stacked in one row, or whether they just didn't know much about data center thermal dynamics."

The Open Compute Project will create a specification and reference hardware for the OS-agnostic top-of-rack switch. The network project includes hardware and software vendors Intel, Broadcom, Cumulus Networks, VMware, Netronome, Big Switch Networks, as well as the Open Networking Foundation (ONF) and OpenDaylight consortiums, which are working on open source software-defined networking (SDN) systems that could be installed on these switches.

ONF is working on the open source OpenFlow SDN software, while OpenDaylight is a new SDN project led by the Linux Foundation.

Software-defined networking aims to let IT admins configure network equipment from a central server, instead of logging into each switch separately. This would be combined with monitoring and management tools allowing quick and easy changes to networks. As IBM describes it, SDN is "designed for virtual, dynamic and flexible networking that allows organizations to more easily modify, control and manage today's physical and virtual networks." SDN, particularly open source implementations, could help make networks less dependent on the hardware of one specific vendor.

Frankovsky said shipping hardware products based on the Open Compute switch design could come from third-party vendors within a year. This could benefit smaller hardware vendors trying to compete against the network market's big fellas, as well as makers of open source network products that can't install their software onto proprietary hardware.

There are some software-defined networking vendors in stealth mode who are "building their code bases, hoping for the day they could land it on a high volume open source switch," Frankovsky said. "I think it's going to help those kinds of companies flourish and those open source efforts like OpenDaylight or OpenFlow to flourish as well."

Will Cisco be left behind?

Cisco, for its part, is beginning to incorporate OpenFlow into its product line. But it's clearly not the company's preference, with one (now-former) Cisco executive recently calling OpenFlow a "fantasy." As Network World's Jim Duffy writes, "Cisco is critical of OpenFlow, claiming its decoupled control and data planes, linked by the OpenFlow protocol, forces customers to re-architect and re-engineer their networks."

Frankovsky predicts that hardware companies will have to take an open source approach to keep their customers. So far, companies like Hyve and Avnet are on board, selling Open Compute-based hardware.

Frankovksy came on the Interop stage today after the likes of Cisco, VMware, Microsoft, Broadcom, and Juniper Networks, and quickly noted that "I haven't heard anyone mention open source this morning, I guess I shouldn't be too surprised." He went on to tell the Interop crowd that Open Compute is "the greatest opportunity to the incumbent or the greatest threat to their business, depending on whether they embrace it. ... Openness always wins."

"The companies that are going to be successful in the long haul are the ones that can work within an open source framework and provide a service that allows them to customize the product to the end users' needs," he told Ars last week while providing a preview of the announcement. "There are a number of partners that are building Open Compute practices specifically to work with end users, so they can pull the Open Compute building blocks together, certify them, and move them into production. They don't approach the customer with a pre-defined black box solution that can't be modified. They approach the customer with a lot of questions about what would be best for their environment and they work with the Open Compute building blocks to pull something custom together for them."

Proprietary systems allow little customization

Proprietary switches come with their own software stacks that allow little in the way of customization, Facebook officials said. Najam Ahmad, who runs Facebook's network engineering team and is leading the Open Compute Project's network program, believes freeing customers to use whatever software they want is the most important aspect of the open network project.

First of all, customers can avoid "bloated" platforms that try to be all things to all companies, Ahmad said. "More software usually means more bugs," he said.

Second, making changes to software will be easier with Open Compute switches than with most of the ones available today.

"We don't know what feature we may want tomorrow," Ahmad said. "If you look at Facebook culturally, we run into a problem and overnight we say it would be nice if we can solve it this way. Then a bunch of engineers get together and solve the problem, and then two days later it's running somewhere. We don't have that ability in the network stack."

Facebook didn't reveal what networking gear it uses, but when asked if the company uses typical Cisco-type switches, Frankovsky said "those are good guesses… Pretty standard top-of-rack and cluster switches."

The first phase of the Open Compute Project was driven almost entirely by Facebook, with the company creating designs in-house and then releasing them to the world. This time around, it will be closer to a true open source project with collaboration from the start. An OCP Engineering Summit will be held at MIT on May 16 to kick off development.

Frankovsky said Facebook is being careful not to duplicate the efforts already made in the open source software-defined networking market. The OCP switch will come with just a pre-boot environment allowing customers to choose their own OS instead of deciding on one OS for all customers.

OpenFlow and OpenDaylight are likely candidates to run on the OCP switch, but Facebook and its partners want to make sure users can choose what they want.

"We should be able to treat a switch like a server in the rack," Frankovsky said. "We should be able to load a Linux-based operating system, and that server just happens to have a lot of I/O ports on it."