I have a global IP by pppoe0 dedicated to the SMTP server.
So outbound traffic on port 25 uses pppoe0 and all other traffic uses the normal route table.
In addition, newly received traffic on pppoe0 needs to be routed to the SMTP server via destination NAT.
So I set "sticky-connections inbound", but this setting did not work correctly.
Specifically, the SMTP server's response to a communication received at pppoe0 uses the normal route table,
not pppoe0, resulting in asymmetric communication.
$ sh conf com | grep load set load-balancing wan interface-health pppoe0 failure-count '1' set load-balancing wan interface-health pppoe0 nexthop 'dhcp' set load-balancing wan interface-health pppoe0 success-count '1' set load-balancing wan interface-health pppoe0 test 0 resp-time '5' set load-balancing wan interface-health pppoe0 test 0 target '1.1.1.1' set load-balancing wan interface-health pppoe0 test 0 ttl-limit '1' set load-balancing wan rule 10 destination port '25' set load-balancing wan rule 10 inbound-interface 'eth1' set load-balancing wan rule 10 interface pppoe0 set load-balancing wan rule 10 protocol 'tcp' set load-balancing wan rule 100 exclude set load-balancing wan rule 100 inbound-interface 'eth1' set load-balancing wan rule 100 protocol 'all' set load-balancing wan sticky-connections inbound
# iptables -n -L -t mangle Chain PREROUTING (policy ACCEPT) target prot opt source destination ISP_pppoe0_IN all -- 0.0.0.0/0 0.0.0.0/0 state NEW WANLOADBALANCE_PRE all -- 0.0.0.0/0 0.0.0.0/0 Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination Chain WANLOADBALANCE_PRE (1 references) target prot opt source destination ISP_pppoe0 tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:25 state NEW CONNMARK tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:25 CONNMARK restore ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain ISP_pppoe0 (1 references) target prot opt source destination CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 CONNMARK set 0xc9 MARK all -- 0.0.0.0/0 0.0.0.0/0 MARK set 0xc9 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 Chain ISP_pppoe0_IN (1 references) target prot opt source destination CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 CONNMARK set 0xc9
After investigating iptables, I noticed that the setting to restore "CONNMARK" for return communication was missing,
even though "CONNMARK" was given in the ISP_pppoe0_IN chain.
I manually added the setting with the following command, and now it works correctly.
I think this is a clear bug, what do you think?
iptables -t mangle -I WANLOADBALANCE_PRE -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff