Page MenuHomeVyOS Platform
Create Task
Maniphest T5347

Compare commit revision bug
Closed, ResolvedPublicBUG
Actions

Description

To reproduce

set system config-management commit-revisions '100'

Type compare and check completion, there is completion help for 99 commits:

vyos@r14# compare 
Possible completions:
  commands	Compare working & active configurations and show "set" commands
  <Enter>	Compare working & active configurations
  saved		Compare working & saved configurations
  <N>		Compare working with revision N
  <N> <M>	Compare revision N with M

  Revisions:
   0  2023-07-10 14:41:11  vyos  by cli
   1  2023-07-10 14:38:29  vyos  by cli
   2  2023-07-10 14:38:00  root  by vyos-boot-config-loader
   3  2023-07-10 11:33:27  vyos  by cli
   4  2023-07-10 11:33:09  vyos  by cli
   5  2023-07-10 11:27:04  vyos  by cli
   6  2023-07-10 11:24:15  vyos  by cli
   7  2023-07-10 10:03:50  root  by vyos-boot-config-loader
   8  2023-07-09 20:40:08  vyos  by cli
   9  2023-07-09 20:12:39  vyos  by cli
  10  2023-07-09 19:45:13  vyos  by cli
  11  2023-07-09 19:44:08  vyos  by cli
  12  2023-07-09 19:35:55  vyos  by cli
  13  2023-07-09 19:34:09  vyos  by cli
  14  2023-07-09 19:32:00  vyos  by cli
  15  2023-07-09 19:04:48  vyos  by cli
  16  2023-07-09 19:04:29  vyos  by cli
  17  2023-07-09 19:04:08  vyos  by cli
  18  2023-07-09 19:03:46  vyos  by cli
  19  2023-07-09 19:03:21  vyos  by cli
  20  2023-07-09 19:02:06  vyos  by cli
  21  2023-07-09 19:01:33  vyos  by cli
  22  2023-07-09 18:53:22  vyos  by cli
  23  2023-07-09 18:52:48  vyos  by cli
  24  2023-07-09 18:52:36  vyos  by cli
  25  2023-07-09 18:52:29  vyos  by cli
  26  2023-07-09 18:45:34  vyos  by cli
  27  2023-07-09 18:44:15  vyos  by cli
  28  2023-07-09 18:35:25  vyos  by cli
  29  2023-07-09 18:34:49  vyos  by cli
  30  2023-07-09 18:29:03  vyos  by cli
  31  2023-07-09 17:52:03  vyos  by cli
  32  2023-07-09 17:45:04  vyos  by cli
  33  2023-07-09 17:44:23  vyos  by cli
  34  2023-07-09 17:42:12  root  by vyos-boot-config-loader
  35  2023-07-09 17:37:54  vyos  by cli
  36  2023-07-09 17:36:28  vyos  by cli
  37  2023-07-09 17:33:18  root  by vyos-boot-config-loader
  38  2023-07-08 20:47:27  vyos  by cli
  39  2023-07-08 19:29:05  vyos  by cli
  40  2023-07-08 18:05:45  vyos  by cli
  41  2023-07-08 16:57:36  vyos  by cli
  42  2023-07-08 16:57:28  vyos  by cli
  43  2023-07-08 16:56:46  vyos  by cli
  44  2023-07-08 16:09:01  vyos  by cli
  45  2023-07-08 15:59:13  vyos  by cli
  46  2023-07-08 15:58:44  vyos  by cli
  47  2023-07-08 15:58:31  vyos  by cli
  48  2023-07-08 15:58:12  root  by vyos-boot-config-loader
  49  2023-07-08 15:56:55  vyos  by cli
  50  2023-07-08 15:56:40  vyos  by cli
  51  2023-07-08 15:55:48  vyos  by cli
  52  2023-07-08 15:54:43  vyos  by cli
  53  2023-07-08 15:51:11  vyos  by cli
  54  2023-07-08 15:43:37  vyos  by cli
  55  2023-07-08 15:43:29  vyos  by cli
  56  2023-07-08 15:43:18  vyos  by cli
  57  2023-07-08 15:40:30  vyos  by cli
  58  2023-07-08 15:26:50  vyos  by cli
  59  2023-07-08 15:07:39  vyos  by cli
  60  2023-07-08 15:01:26  vyos  by cli
  61  2023-07-08 15:00:52  vyos  by cli
  62  2023-07-08 15:00:45  vyos  by cli
  63  2023-07-08 14:56:29  vyos  by cli
  64  2023-07-08 14:54:23  vyos  by cli
  65  2023-07-08 14:45:19  vyos  by cli
  66  2023-07-08 14:45:14  vyos  by cli
  67  2023-07-08 14:45:04  vyos  by cli
  68  2023-07-08 14:43:33  vyos  by cli
  69  2023-07-08 14:36:38  vyos  by cli
  70  2023-07-08 14:24:28  vyos  by cli
  71  2023-07-08 14:20:29  vyos  by cli
  72  2023-07-08 14:15:53  vyos  by cli
  73  2023-07-08 14:15:46  vyos  by cli
  74  2023-07-08 14:15:35  vyos  by cli
  75  2023-07-08 13:27:42  vyos  by cli
  76  2023-07-08 13:27:36  vyos  by cli
  77  2023-07-08 13:16:42  vyos  by cli
  78  2023-07-08 13:13:33  vyos  by cli
  79  2023-07-08 13:04:02  vyos  by cli
  80  2023-07-08 13:02:55  vyos  by cli
  81  2023-07-08 12:51:16  vyos  by cli
  82  2023-07-08 12:43:31  vyos  by cli
  83  2023-07-08 12:43:20  vyos  by cli
  84  2023-07-08 12:40:49  vyos  by cli
  85  2023-07-08 11:56:30  vyos  by cli
  86  2023-07-08 11:56:23  vyos  by cli
  87  2023-07-08 11:56:08  vyos  by cli
  88  2023-07-08 11:53:24  vyos  by cli
  89  2023-07-08 11:53:14  vyos  by cli
  90  2023-07-08 11:52:20  vyos  by cli
  91  2023-07-08 11:50:51  vyos  by cli
  92  2023-07-08 11:49:02  vyos  by cli
  93  2023-07-08 11:48:50  vyos  by cli
  94  2023-07-08 11:48:38  vyos  by cli
  95  2023-07-08 11:47:13  vyos  by cli
  96  2023-07-08 11:45:29  vyos  by cli
  97  2023-07-08 11:45:09  vyos  by cli
  98  2023-07-08 11:42:52  vyos  by cli
  99  2023-07-08 11:42:28  vyos  by cli

Try to compare 99:

vyos@r14# compare 99 
Invalid revision number 99
[edit]
vyos@r14#

Try to compare 98:

vyos@r14# compare 98
Traceback (most recent call last):
  File "/usr/bin/config-mgmt", line 33, in <module>
    sys.exit(load_entry_point('vyos==1.3.0', 'console_scripts', 'config-mgmt')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/config_mgmt.py", line 688, in run
    res, rc = func(**args)
              ^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/config_mgmt.py", line 315, in wrap_compare
    return self.compare(commands=cmnds, rev1=r1, rev2=r2)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/config_mgmt.py", line 271, in compare
    ct1 = self._get_config_tree_revision(rev1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/config_mgmt.py", line 455, in _get_config_tree_revision
    c = self._strip_version(self._get_file_revision(rev))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/vyos/config_mgmt.py", line 450, in _get_file_revision
    with gzip.open(revision) as f:
         ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/gzip.py", line 58, in open
    binary_file = GzipFile(filename, gz_mode, compresslevel)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/gzip.py", line 174, in __init__
    fileobj = self.myfileobj = builtins.open(filename, mode or 'rb')
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/opt/vyatta/etc/config/archive/config.boot.98.gz'
[edit]
vyos@r14#

The actual files:

vyos@r14# ls -la /opt/vyatta/etc/config/archive/
total 152
drwxrwsr-x 2 root vyattacfg 4096 Jul 10 14:41 .
drwxrwxr-x 7 root vyattacfg 4096 Jul 10 14:37 ..
-rwxrwxr-x 1 root vyattacfg 3000 Jul 10 14:41 commits
-rw-rw-r-- 1 vyos vyattacfg 3850 Jul 10 14:41 config.boot
-rwxrwxr-x 1 root vyattacfg 1532 Jun 30 10:44 config.boot.0.gz
-rwxrwxr-x 1 root vyattacfg 1545 Jun 29 16:40 config.boot.10.gz
-rwxrwxr-x 1 root vyattacfg 1572 Jun 29 16:05 config.boot.11.gz
-rwxrwxr-x 1 root vyattacfg 1670 Jun 29 16:05 config.boot.12.gz
-rwxrwxr-x 1 root vyattacfg 1669 Jun 29 16:05 config.boot.13.gz
-rwxrwxr-x 1 root vyattacfg 1630 Jun 29 16:01 config.boot.14.gz
-rwxrwxr-x 1 root vyattacfg 1669 Jun 29 15:40 config.boot.15.gz
-rwxrwxr-x 1 root vyattacfg 1625 Jun 29 15:40 config.boot.16.gz
-rwxrwxr-x 1 root vyattacfg 1572 Jun 29 15:34 config.boot.17.gz
-rwxrwxr-x 1 root vyattacfg 1634 Jun 29 15:34 config.boot.18.gz
-rwxrwxr-x 1 root vyattacfg 1572 Jun 29 15:32 config.boot.19.gz
-rwxrwxr-x 1 root vyattacfg 1571 Jun 29 18:44 config.boot.1.gz
-rwxrwxr-x 1 root vyattacfg 1638 Jun 29 15:32 config.boot.20.gz
-rwxrwxr-x 1 root vyattacfg 1634 Jun 29 15:29 config.boot.21.gz
-rwxrwxr-x 1 root vyattacfg 1637 Jun 29 15:28 config.boot.22.gz
-rwxrwxr-x 1 root vyattacfg 1637 Jun 29 15:28 config.boot.23.gz
-rwxrwxr-x 1 root vyattacfg 1637 Jun 29 15:26 config.boot.24.gz
-rwxrwxr-x 1 root vyattacfg 1635 Jun 29 15:26 config.boot.25.gz
-rwxrwxr-x 1 root vyattacfg 1632 Jun 29 15:04 config.boot.26.gz
-rwxrwxr-x 1 root vyattacfg 1571 Jun 29 14:57 config.boot.27.gz
-rwxrwxr-x 1 root vyattacfg 1532 Jun 29 14:48 config.boot.28.gz
-rwxrwxr-x 1 root vyattacfg 1532 Jun 29 14:35 config.boot.29.gz
-rwxrwxr-x 1 root vyattacfg 1557 Jun 29 18:43 config.boot.2.gz
-rwxrwxr-x 1 root vyattacfg  942 Jun 29 14:34 config.boot.30.gz
-rwxrwxr-x 1 root vyattacfg  416 Jun 29 14:34 config.boot.31.gz
-rwxrwxr-x 1 root vyattacfg 1545 Jun 29 18:43 config.boot.3.gz
-rwxrwxr-x 1 root vyattacfg 1557 Jun 29 18:42 config.boot.4.gz
-rwxrwxr-x 1 root vyattacfg 1545 Jun 29 18:31 config.boot.5.gz
-rwxrwxr-x 1 root vyattacfg 1532 Jun 29 16:52 config.boot.6.gz
-rwxrwxr-x 1 root vyattacfg 1554 Jun 29 16:42 config.boot.7.gz
-rwxrwxr-x 1 root vyattacfg 1551 Jun 29 16:42 config.boot.8.gz
-rwxrwxr-x 1 root vyattacfg 1554 Jun 29 16:41 config.boot.9.gz
-rwxrwxr-x 1 root vyattacfg  117 Jul 10 14:37 lr.conf
-rwxrwx--x 1 root vyattacfg   93 Jul 10 14:41 lr.state
[edit]
vyos@r14#

It's trying to compare not commits but the current config with saved config.boot.x revisions.
It is incorrect

Expected behavior as it is in 1.3.3:

vyos@r11# set interfaces ethernet eth0 description WAN
[edit]
vyos@r11# commit
[edit]
vyos@r11# compare 1
[edit interfaces ethernet eth0]
+description WAN
[edit]
vyos@r11#

Currently get in 1.4:

vyos@r14# set interfaces ethernet eth1 description WAN
[edit]
vyos@r14# commit
[edit]
vyos@r14# compare 1
[interfaces ethernet eth1]
+ address "192.0.2.1/30"
+ description "WAN"
[interfaces ethernet eth2]
- address "100.64.0.1/24"
[protocols]
+ bgp {
+     neighbor 192.0.2.2 {
+         address-family {
+             ipv4-unicast { }
+         }
+         remote-as "65001"
+     }
+     system-as "65001"
+ }
[service]
+ config-sync {
+     mode "load"
+     secondary {
+         address "192.168.122.11"
+         key "foo"
+     }
+     section "nat"
+     section "firewall"
+ }
[]
+ nat {
+     source {
+         rule 100 {
+             description "1234"
+             outbound-interface "eth0"
+             source {
+                 address "192.0.2.0/24"
+             }
+             translation {
+                 address "masquerade"
+             }
+         }
+     }
+ }
[edit]
vyos@r14#

Details

Difficulty level
Unknown (require assessment)
Version
VyOS 1.4-rolling-202307100526
Why the issue appeared?
Will be filled on close
Is it a breaking change?
Unspecified (possibly destroys the router)
Issue type
Unspecified (please specify)

Related Objects
Search...

Event Timeline

Viacheslav created this task.Jul 10 2023, 12:54 PM
jestabro added a subscriber: jestabro.EditedJul 10 2023, 1:15 PM

As shared with @Viacheslav , I see the following on a 1.4 VM, built last week:

vyos@vyos# set interfaces ethernet eth0 description WAN
[edit]
vyos@vyos# commit
[edit]
vyos@vyos# compare 1
[interfaces ethernet eth0]
+ description "WAN"

[edit]

So we have (1) discrepancies in behavior on different instances (2) I have seen a failure of sync'ing of files in /opt/vyatta/etc/config/archive <--> /config/archive, which may be a clue.

zsdc added a subscriber: zsdc.Jul 10 2023, 5:05 PM

During installation, this happens: https://github.com/vyos/vyatta-cfg-system/blob/bac61854acd40b4eae8b12e47be54306171f5c3c/scripts/install/install-image-existing#L201C3-L201C3

After, all files except those located in the /config/auth dir which has extra attributes sync (https://github.com/vyos/vyatta-cfg-system/blob/bac61854acd40b4eae8b12e47be54306171f5c3c/scripts/install/install-image-existing#L204) have permissions 775.

logrotate ignores config with unsafe permission:

error: Ignoring /opt/vyatta/etc/config/archive/lr.conf because it is writable by group or others.

And this will never work, also as never returns a non-zero exit code: https://github.com/vyos/vyos-1x/blob/91e94036bf808d36399b698fbcecf4b9836f5806/python/vyos/config_mgmt.py#L500

So, the problem is invisible, if you will not check files manually.

Solutions:

a) sync attributes for all files during installation
b) recreate config from scratch, instead of updating existing file.

I think both should be implemented, but for this specific problem b) is preferred. Something like this looks safer than what we have now in https://github.com/vyos/vyos-1x/blob/91e94036bf808d36399b698fbcecf4b9836f5806/python/vyos/config_mgmt.py#L458C1-L471C23:

from textwrap import dedent
from pathlib import Path


def _add_logrotate_conf(self) -> None:
    conf: str = dedent(f'''\
    {archive_config_file} {{
        su root vyattacfg
        rotate {self.max_revisions}
        start 0
        compress
        copy
    }}
    ''')
    conf_file = Path(logrotate_conf)
    conf_file.write_text(conf)
    conf_file.chmod(0o644)
jestabro added a comment.Jul 10 2023, 6:17 PM

There are several issues mentioned in this task so far, and they will be separated out to respective tasks. Firstly, we will address an obvious bug in the implementation that misnumbers entries in the commit log compared to archived commit files ...

jestabro claimed this task.Jul 10 2023, 6:18 PM
pasik added a subscriber: pasik.Jul 11 2023, 10:58 AM
jestabro added a comment.Jul 11 2023, 4:08 PM

@zsdc thanks for the detailed explanation; @Viacheslav confirmed this is the case on update, and I will adopt your suggestion.

jestabro added a comment.Jul 11 2023, 5:50 PM

PR:
https://github.com/vyos/vyos-1x/pull/2086

Viacheslav added a comment.EditedJul 12 2023, 8:03 AM

It seems to work fine (VyOS 1.4-rolling-202307120317).

vyos@r14# set interfaces ethernet eth1 description 123
[edit]
vyos@r14# commit
[edit]
vyos@r14# compare 1
[interfaces ethernet eth1]
+ description "123"

[edit]
vyos@r14# set interfaces ethernet eth1 description LAN
[edit]
vyos@r14# 
[edit]
vyos@r14# compare 1
[interfaces ethernet eth1]
+ description "LAN"

[edit]
vyos@r14# 


vyos@r14# set nat source rule 100 description SNAT123
[edit]
vyos@r14# commit
INFO:vyos_config_sync:Config synchronization: Mode=load, Secondary=192.168.122.11
[edit]
vyos@r14#
jestabro closed this task as Resolved.Jul 12 2023, 2:33 PM
jestabro mentioned this in T5353: config-mgmt: normalize archive updates and commit log entries.
jestabro added a subtask: T5353: config-mgmt: normalize archive updates and commit log entries.
jestabro moved this task from Need Triage to Finished on the VyOS 1.4 Sagitta board.
jestabro changed the status of subtask T5353: config-mgmt: normalize archive updates and commit log entries from Open to In progress.Sep 5 2023, 1:15 PM
jestabro changed the status of subtask T5353: config-mgmt: normalize archive updates and commit log entries from In progress to Backport pending.Sep 6 2023, 6:28 PM
jestabro closed subtask T5353: config-mgmt: normalize archive updates and commit log entries as Resolved.Sep 11 2023, 5:32 PM