> But that limits what you can do considerably, to the point that an off-the-rack Juniper or CSCO or whatever probably has more features than you can do here without writing your own code to hook into the mellanox sdk?
I mean, I'm not touching any mellanox sdk here, I am using the a very similar stack that someone on a "software router" would use, on a switch that can automatically accelerate it to 800G+ throughputs, while hitting a 60W power target.
You can hit some of those performance/power numbers in vendor hardware like Juniper/Cisco/Arista, however you have to also put up with their software, I (and others in my group of peers) have not had great experiences with vendor software, and in this setup I am able to patch/fix the software on my own terms.
If there is a security vuln in one section, I can fix that, and call it a day, I won't be forced to upgrade parts of the system I do not want to. I cannot do this with Juniper/Cisco/Arista always.
Yes. The switchdev "sw1p[0-9]+" ports are special; the any data the software kernel injects to them is discarded and they never emit packets to the kernel. They exist only to allow you to use `ip bridge` and `ip route` on them. So if you accidentally configure software switching on these ports no data will flow -- it will be totally obvious. You might get "no packets" by accident but you will never get "software switching" by accident.
If you really want software switching you have to use the management port (there's only one or two of these) whose name is "eth0" or "eth1" or something like that. So avoiding "accidental software switching" is really easy -- if you're typing "eth" you're doing it wrong. You can even explicitly delete this interface if you don't need the CPU to be able to snoop/inject traffic to/from the switch ports.
It really depends on how much you know what you are doing, If you stick to:
*) IP Routing that would normally "fit" in a vendor switch
*) Bridging
*) VRFs
You will be fine
If you try and do some weird stuff then it's best to check with "ip route" to see if it was actually installed into hardware or not, but I would simply not do anything weird on such hardware
I mean, I'm not touching any mellanox sdk here, I am using the a very similar stack that someone on a "software router" would use, on a switch that can automatically accelerate it to 800G+ throughputs, while hitting a 60W power target.
You can hit some of those performance/power numbers in vendor hardware like Juniper/Cisco/Arista, however you have to also put up with their software, I (and others in my group of peers) have not had great experiences with vendor software, and in this setup I am able to patch/fix the software on my own terms.
If there is a security vuln in one section, I can fix that, and call it a day, I won't be forced to upgrade parts of the system I do not want to. I cannot do this with Juniper/Cisco/Arista always.