Port mapping and SerDes

Which SerDes lanes form which port, at what speed, with which lane order, polarity and transmit equalisation, is set in config.bcm. These values describe the board’s wiring and the port layout the HWSKU offers. They are read once, when syncd initialises the chip (see Names, scopes and precedence for the name forms).

Physical and logical ports

Trident 3 (BCM56870) has 128 front-panel SerDes lanes in 32 cores of four, numbered as physical ports 1–128, plus one management lane, physical port 129. The chip has two pipelines: physical ports 1–64 belong to pipe 0, 65–128 and the management port to pipe 1.

SDK logical port numbers, the numbers the shell prints and the portmap_<port> suffix, are tied to the physical lane of the port’s first lane:

logical 1–64 are physical 1–64 (same number);
logical 67–130 are physical 65–128 (physical + 2);
logical 66 is the management port, physical 129;
logical 0 is the CPU port, 65 and 131 are the two internal loopback ports; these three are fixed and never appear in config.bcm.

A portmap line that breaks this rule fails initialisation with Port <n>: Invalid port map configuration. The SONiC lane numbers in port_config.ini are physical port numbers.

`portmap`

Each port is one portmap line:

portmap_<logical>=<physical>:<speed>[:<option>[:<option>]]

<speed>: Maximum speed in Gb/s: 1, 2.5, 10, 11, 20, 21, 25, 27, 40, 42, 50, 53, 100 or 106 (the odd values are HiGig speeds). 100 and 106 are only accepted when oversubscribe_mode is set, which is why every 32×100G SONiC file sets oversubscribe_mode=1.
First option: 1, 2 or 4: number of lanes (default 4 for a front-panel port); i: create the port inactive (disabled), as a placeholder for a later breakout; m: management port (on physical 128); or a speed cap in Gb/s for a flex port, for example portmap_1=1:10:100.
Second option: i (inactive) or m (management).

Every physical lane can belong to only one logical port. A port with four lanes uses its physical lane and the next three, so portmap_1=1:100 occupies lanes 1–4 and lanes 2–4 must not appear in another portmap.

The Celestica Seastone2 HWSKU defines 31 ports at 100G, one front-panel port split into four 10G ports, and the management port:

portmap_1=1:100
portmap_5=5:100
portmap_9=9:10:1
portmap_10=10:10:1
portmap_11=11:10:1
portmap_12=12:10:1
portmap_13=13:100
...
portmap_71=69:100
...
portmap_66=129:10:m

portmap_71=69:100 shows the second pipe’s numbering: physical 69 is logical 71. The shell names these ports ce0, ce1, xe0–xe3 and so on (see Ports and port bitmaps).

Breaking out a 100G port into 4×10G

Splitting front-panel port 1 (lanes 1–4) into four 10G ports needs three changes, then a syncd restart (see Configuration variables). The example is the difference between two Dell S5232F-ON HWSKUs, one with port 1 at 100G and one with it broken out.

config.bcm: replace the 100G port by four one-lane ports:
```
-portmap_1.0=1:100
+portmap_1.0=1:10:1
+portmap_2.0=2:10:1
+portmap_3.0=3:10:1
+portmap_4.0=4:10:1
```
The lane map and polarity variables are per core and per lane and stay as they are. If the file sets per-port serdes_preemphasis values, add serdes_preemphasis_2 to _4 for the new ports.

port_config.ini: replace the one 100G port with four ports of one lane each:

 # name          lanes                 alias              index   subport  mtu     speed
-Ethernet0       1,2,3,4               hundredGigE1/1     1       0        9100    100000
+Ethernet0       1                     hundredGigE1/1/1   1       1        9100    10000
+Ethernet1       2                     hundredGigE1/1/2   1       2        9100    10000
+Ethernet2       3                     hundredGigE1/1/3   1       3        9100    10000
+Ethernet3       4                     hundredGigE1/1/4   1       4        9100    10000

config_db.json: replace Ethernet0 in the PORT table with the four new ports (and move any configuration that referred to it), or regenerate the file from the new port_config.ini.

After the restart the shell shows the four ports as xe ports on logical ports 1–4:

Unvalidated

drivshell> config show portmap_
drivshell> show pmap

SONiC’s dynamic port breakout (config interface breakout) changes ports at runtime instead. It needs the core to be flex-enabled with port_flex_enable and the platform’s breakout description; the initial layout still comes from config.bcm.

Inactive ports for later breakout

With i a port exists in the port map but stays disabled. Defining the spare lanes of a 4×10G-capable port this way lets port or FLEXport change the layout without a port-map change. The SDK checks the bandwidth of every core and pipe at initialisation, inactive ports included; a layout that exceeds a limit fails with messages such as PGW_CL4 and PGW_CL5 total line rate bandwidth (230 Gb) exceeds 200 Gb. Changing the chip configuration live shows both cases.

Lane order and polarity

Boards route the four lanes of a core to the front-panel connector in any order and may swap the P and N wires of a lane. config.bcm tells the SerDes how to undo this. The values are fixed by the board design: copy them from the vendor’s file and never change them to fix a link problem.

phy_chain_tx_lane_map_physical and phy_chain_rx_lane_map_physical: One variable per core, on the core’s first physical lane, {<physical>.0}. The value has one hexadecimal digit per lane, lane 0 in the lowest digit; 0x3210 is straight through.
phy_chain_tx_polarity_flip_physical and phy_chain_rx_polarity_flip_physical: One variable per lane: 1 inverts the lane, 0 leaves it.

From the Seastone2 HWSKU, core 3 (physical 9–12):

phy_chain_tx_lane_map_physical{9.0}=0x0132
phy_chain_rx_lane_map_physical{9.0}=0x3210
phy_chain_tx_polarity_flip_physical{9.0}=0x0
phy_chain_rx_polarity_flip_physical{9.0}=0x0
phy_chain_tx_polarity_flip_physical{10.0}=0x0
phy_chain_rx_polarity_flip_physical{10.0}=0x1
phy_chain_tx_polarity_flip_physical{11.0}=0x0
phy_chain_rx_polarity_flip_physical{11.0}=0x0
phy_chain_tx_polarity_flip_physical{12.0}=0x1
phy_chain_rx_polarity_flip_physical{12.0}=0x1

Transmit equalisation

serdes_preemphasis sets the transmit FIR taps of a port as one number, 0x<post><main><pre>. The Seastone2 value serdes_preemphasis_83=0x0d4f08 means post-cursor 0x0d (13), main 0x4f (79), pre-cursor 0x08 (8) for all lanes of logical port 83. To set one lane, use serdes_preemphasis_lane<L>_<port>; a lane value overrides the port value. serdes_tx_taps is an alternative, decimal form.

Good values depend on the trace length to each cage and on the cable or optic; vendors tune them per port. phy diag with the dsc sub-command shows the SerDes state of a port, including the transmit taps in use.

Port types, oversubscription and other port variables

oversubscribe_mode: 1 on every 100G SONiC HWSKU (see portmap). 2 with pbmp_oversubscribe oversubscribes only some ports.
pbmp_xport_xe: Bitmap of logical ports that come up as Ethernet; SONiC files set all front-panel ports, for example 0x3ffffffffffffffffffffffffffffffffe on Seastone2.
port_flex_enable: 1 allows runtime breakout and speed changes on all cores. SONiC files exclude the management port: port_flex_enable_66=0.
phy_an_c73: Clause 73 auto-negotiation mode; phy_an_c73=1 (Seastone2) or 3 (Dell) with the management ports excluded.
serdes_if_type: Interface type number; serdes_if_type_ce=14 (CR4) sets all 100G ports.
dport_map_port: Only renumbers ports in diag-shell commands, see Port bitmap grammar. Dell S5232F files set it for every port.

The port variable reference lists all of them.