I've been playing around getting a 4G modem (Sierra AirPrime MC7455 3G/4G LTE Modem) working on VyOS with GoogleFi
It all seemed easy, just set the APN and DHCP/DHCPv6
This led me to this forum post talking about this exact issue: https://forum.vyos.io/t/wwan-not-getting-an-ip/7757
It seems that this card does not support a link layer protocol of `802-3` (ethernet), but `raw-ip` only
We can see this with
```
root@vyos:# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-get-data-format
[/dev/cdc-wdm0] Successfully got data format
QoS flow header: no
Link layer protocol: 'raw-ip'
Uplink data aggregation protocol: 'disabled'
Downlink data aggregation protocol: 'disabled'
NDP signature: '0'
Downlink data aggregation max datagrams: '0'
Downlink data aggregation max size: '0'
```
I tried to set `802-3`, via `qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3`
```
root@vyos:~# qmicli --device=/dev/cdc-wdm0 --device-open-proxy --wda-set-data-format=802-3
[/dev/cdc-wdm0] Successfully set data format
QoS flow header: no
Link layer protocol: 'raw-ip'
Uplink data aggregation protocol: 'disabled'
Downlink data aggregation protocol: 'disabled'
NDP signature: '0'
Downlink data aggregation max datagrams: '0'
Downlink data aggregation max size: '0'
Uplink data aggregation max datagrams: '1'
Uplink data aggregation max size: '4294967295'
```
The protocol doesn't change unfortunately, because it's not supported.
Looking more into it, the reason why DHCP doesn't work is because `dhclient`
and `dhcp6c` doesn't support raw-ip, there seems to be a PR in the works
for dhclient but I believe it's stalled and likely won't ever be picked up https://gitlab.isc.org/isc-projects/dhcp/-/merge_requests/67
```
Jan
19 20:36:4921 09:31:14 vyos
dhclient-script-vyos[25297]: Passing command to /usr/sbin/ip: "link set devsystemd[1]: Stopped [email protected] - DHCP client on wwan0
up".
Jan
19 20:36:4921 09:31:14 vyos
dhclient-script-vyos[25297]: No changes to apply via vyos-hostsd-clientsystemd[1]: Starting [email protected] - DHCP client on wwan0...
Jan
19 20:36:4921 09:31:14 vyos
dhclient[25296]: Unsupported desystemd[1]: Started [email protected]vice
type 65534 for "- DHCP client on wwan0
".
Jan
19 20:36:4921 09:31:14 vyos dhclient[
25296]: Unsupported device type 65534 for "wwan0"3030]: Internet Systems Consortium DHCP Client 4.4.3-P1
Jan 21 09:31:14 vyos dhclient[3030]: Internet Systems Consortium DHCP Client 4.4.3-P1
Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium.
Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved.
Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/
Jan 21 09:31:14 vyos dhclient[3030]: Copyright 2004-2022 Internet Systems Consortium.
Jan 21 09:31:14 vyos dhclient[3030]: All rights reserved.
Jan 21 09:31:14 vyos dhclient[3030]: For info, please visit https://www.isc.org/software/dhcp/
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Current dhclient PID: 3030, Parent PID: 1, IP version: 4, All dhclients for interface wwan0: 3030
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: Passing command to /usr/sbin/ip: "link set dev wwan0 up"
Jan 21 09:31:14 vyos dhclient-script-vyos[3031]: No changes to apply via vyos-hostsd-client
Jan 21 09:31:14 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0"
Jan 21 09:31:15 vyos dhclient[3030]: Unsupported device type 65534 for "wwan0"
Jan 21 09:31:15 vyos dhclient[3030]: If you think you have received this message due to a bug rather
Jan 21 09:31:15 vyos dhclient[3030]: than a configuration issue please read the section on submitting
Jan 21 09:31:15 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file
Jan 21 09:31:15 vyos dhclient[3030]: before submitting a bug. These pages explain the proper
Jan 21 09:31:15 vyos dhclient[3030]: process and the information we find helpful for debugging.
Jan 21 09:31:15 vyos dhclient[3030]: exiting.
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:14 vyos dhclient[3030]: If you think you have received this message due to a bug rather
Jan 21 09:31:14 vyos dhclient[3030]: than a configuration issue please read the section on submitting
Jan 21 09:31:14 vyos dhclient[3030]: bugs on either our web page at www.isc.org or in the README file
Jan 21 09:31:14 vyos dhclient[3030]: before submitting a bug. These pages explain the proper
Jan 21 09:31:14 vyos dhclient[3030]: process and the information we find helpful for debugging.
Jan 21 09:31:14 vyos dhclient[3030]:
Jan 21 09:31:15 vyos dhclient[3030]: exiting.
Jan 21 09:31:15 vyos systemd[1]: [email protected]: Main process exited, code=exited, status=1/FAILURE
Jan 21 09:31:15 vyos systemd[1]: [email protected]: Failed with result 'exit-code'.
```
```
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set client ID (len 18)
Jan 21 09:46:03 vyos dhcp6c[6091]: copyout_option: set identity association
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set elapsed time (len 2)
Jan 21 09:46:03 vyos dhcp6c[6091]: copy_option: set option request (len 4)
Jan 21 09:46:03 vyos dhcp6c[6091]: client6_send: transmit failed: Cannot assign requested address
Jan 21 09:46:03 vyos dhcp6c[6091]: dhcp6_reset_timer: reset a timer on wwan0, state=SOLICIT, timeo=6, retrans=68413
```
This is also noted by ModemManager (which is what VyOS uses to handle starting/stopping the connection) - https://modemmanager.org/docs/modemmanager/ip-connectivity-setup-in-lte-modems/
```
In particular, QMI modems running in raw-ip mode (without ethernet headers in the link layer) will by default be requested to use static IP addressing; not because the DHCP server in the modem doesn’t work, but because not all DHCP clients in Linux can work with network interfaces without ethernet headers (e.g. dhclient can’t).
```
The modem does have an IP address as mentioned above
```
rootvyos@vyos#
sudo mmcli -m 0 -b 1
------------------------------------
General | path: /org/freedesktop/ModemManager1/Bearer/
12
| type: default
------------------------------------
Status | connected: yes
| suspended: no
| multiplexed: no
| interface: wwan0
| ip timeout: 20
------------------------------------
Properties | apn: h2g2
| roaming: allowed
| ip type: ipv4
v6
------------------------------------
IPv4 configuration | method: static
| address: 26.112.
68.182194.94
| prefix: 30
| gateway: 26.112.
68.181194.93
| dns: 10.177.0.34, 106.151.0.253
| mtu: 1500
------------------------------------
Statistics IPv6 configuration |
start date: 2024-01-19T21:54:59Z method: static
|
duration: 1319 address: 2607:fb91:ee1:9e39:f488:eb51:81eb:2902
|
attempts: 3 prefix: 64
|
total-duration: 1707 gateway: 2607:fb91:ee1:9e39:a91c:c31f:4e25:a5e2
| dns: fd00:976a::9, fd00:976a::10
| mtu: 1500
------------------------------------
Statistics | start date: 2024-01-21T09:44:53Z
| duration: 240
| attempts: 1
| total-duration: 240
rootvyos@vyos#
sudo qmicli -d /dev/cdc-wdm0 --device-open-proxy --wds-get-current-settings
[/dev/cdc-wdm0] Current settings retrieved:
IP Family: IPv4
IPv4 address: 26.112.
68.182194.94
IPv4 subnet mask: 255.255.255.252
IPv4 gateway address: 26.112.
68.181194.93
IPv4 primary DNS: 10.177.0.34
IPv4 secondary DNS: 106.151.0.253
MTU: 1500
Domains: none
```
Another note on the ModemManager page is
```
ModemManager does not perform the network interface configuration in the host, as it is exclusively in charge of communicating in the signaling plane between host and modem. The network manager process in use in the system will collect the IP settings exposed in the ModemManager Bearer object, and use them to configure the network interface accordingly.
```
Which leads me to believe NetworkManager will read that IP information (can get JSON output with `--output-json`) then manually configured the IP/default route. I did statically set the IP/route myself and was able to reach the internet via the 4G connection
I was hoping there is a way to use DHCP to do this still and I stumbled on an article (https://forums.raspberrypi.com/viewtopic.php?t=325044) that mentioning udhcpc would work with `raw-ip`, so I tried it and can confirm it works!
```
root@vyos# udhcpc -i wwan0
udhcpc: started, v1.35.0
udhcpc: broadcasting discover
udhcpc: broadcasting select for 26.112.68.182, server 26.112.68.181
udhcpc: lease of 26.112.68.182 obtained from 26.112.68.181, lease time 7200
```
Alternatively, I tried with `systemd-networkd` but did not get anywhere, and noticed this comment on an issue (https://github.com/systemd/systemd/issues/27219#issuecomment-1505640023) that seems to say that systemd-networkd does not support `raw-ip` either
```
vyos@vyos:~$ show version
Version: VyOS 1.5-rolling-202401161743
Release train: current
Built by:
[email protected]
Built on: Tue 16 Jan 2024 19:41 UTC
Build UUID: f7976e02-b740-4027-8c96-6335bbea4315
Build commit ID: 365f10340ec2f1
Architecture: x86_64
Boot via: installed image
System type: bare metal
Hardware vendor: Lanner Electronics Inc.
Hardware model: NCA-1515B-VS1
Hardware S/N: LR202112012772
Hardware UUID: 03000200-0400-0500-0006-000700080009
Copyright: VyOS maintainers and contributors
```
I believe this issue is not seen on modules/modems that support `802-3` for their link layer protocol as DHCP would work normally as expected in that case. It seems to be specifically related to `raw-ip`