Page MenuHomeVyOS Platform
Create Task
Maniphest T1193

libvyosconfig parser cannot handle top level leaf and tag nodes
Closed, ResolvedPublicBUG
Actions

Description

In config mode, if I execute command "run show configuration commands" from the top level of the config, I get the the commands for the complete config.

But when I execute the same command from some other level in the config, instead of getting the commands for that particular level, I get the error message :

Could not parse the config file: Failed to parse config: Parse error

And sometime it gets even more specific :

Could not parse the config file: Invalid syntax at character 40: invalid character g

for example.

Details

Difficulty level
Normal (likely a few hours)
Version
1.2.0RC11
Why the issue appeared?
Implementation mistake

Related Objects

Event Timeline

jmlccdmd created this task.Jan 22 2019, 3:55 PM
jmlccdmd renamed this task from I config mode command "run show configuration commands" generates an error if not on the top level of the config. to I config mode command "run show configuration commands" generates an error if not on the top level in config mode.Jan 22 2019, 7:41 PM
jmlccdmd renamed this task from I config mode command "run show configuration commands" generates an error if not on the top level in config mode to Command "run show configuration commands" generates an error if not on the top level in config mode.
hagbard changed the task status from Open to Confirmed.Jan 22 2019, 10:38 PM
hagbard claimed this task.
hagbard added a comment.EditedJan 22 2019, 10:45 PM

Issue sits somewhere in vyos.configtree

/usr/lib/python3/dist-packages/vyos/configtree.py line 157

@dmbaturin I could need your help here.

hagbard added a subscriber: dmbaturin.Jan 22 2019, 10:58 PM
syncer triaged this task as Normal priority.Jan 23 2019, 12:19 AM
syncer added a project: VyOS 1.2 Crux (VyOS 1.2.0-GA).
dmbaturin added a comment.Jan 23 2019, 12:22 AM

I suppose I can port the fix for |commands to it.

hagbard added a comment.Jan 23 2019, 6:41 AM

I had to pass on libvyos and OCAML, just reading and understanding a few lines took me forever. What would be the fix?

pasik added a subscriber: pasik.Jan 23 2019, 10:25 AM
hagbard reassigned this task from hagbard to dmbaturin.Jan 23 2019, 8:08 PM
hagbard added a subscriber: hagbard.
syncer moved this task from Needs Triage to Backlog on the VyOS 1.2 Crux (VyOS 1.2.0-GA) board.Jan 25 2019, 2:11 PM
dmbaturin renamed this task from Command "run show configuration commands" generates an error if not on the top level in config mode to libvyosconfig parser cannot handle top level leaf and tag nodes.Jan 26 2019, 12:03 AM
dmbaturin changed Difficulty level from Unknown (require assessment) to Normal (likely a few hours).
dmbaturin changed Why the issue appeared? from Will be filled on close to Implementation mistake.
dmbaturin changed the task status from Confirmed to Needs testing.Jan 26 2019, 1:04 AM

This issue uncovered more. The parser was written under an assumption that "bare" leaf and tag nodes at the top level are not allowed. In fact, while VyOS config never have them, it's by convention rather than by design, if you create a command definition for one (set foo bar), it works just fine. So, to be consistent with the original parser, the new parser should allow them.
I've removed the restriction on the grammar to allow them (see https://github.com/vyos/libvyosconfig/commit/1dd05b330f3adfe828ba3ca4c71db2e12b8968f6), and now run show configuration commands seems to work with "partial" configs just fine. The change appears to have no ill effects on parsing "normal" config, according to my testing.

That change also allowed me to get rid of the fixup that original commands pipe used. Now both run show configuration commands and the |commands pipe use the same script, /bin/vyos-config-to-commands.

hagbard added a comment.Jan 26 2019, 2:28 AM

Nice! I will test it tomorrow for sure.

dmbaturin moved this task from Backlog to Finished on the VyOS 1.2 Crux (VyOS 1.2.0-GA) board.Jan 26 2019, 3:54 PM
hagbard added a comment.Jan 26 2019, 5:21 PM

Still same issue on 1.2.0-rolling+201901250337.

conf
edit interfaces
run sh config comm
Could not parse the config file: Failed to parse config: Parse error

libvyosconfig0 v. 0.0.4 but your changelog in github says 0.0.6 so I think there is the issue.

Where do you build the package, I couldn't find it in ci.

hagbard claimed this task.Jan 26 2019, 5:26 PM

Dev.packages has 0.0.06, so something goes sideways during build process, I will work on that and test. I'll take the task back and close it when resolved in ci (looking into it right now). I manually installed the package and everything works as expected.

hagbard added a comment.Jan 26 2019, 5:32 PM

Rebuilding iso, once it finished it will have the correct version.
[...]
Get:152 http://dev.packages.vyos.net/repositories/current/vyos/ current/main libvyosconfig0 amd64 0.0.6 [841 kB]
[...]
Will test it from the iso, just for peace of mind.

hagbard closed this task as Resolved.Jan 26 2019, 6:34 PM
syncer added a project: VyOS-1.2.0-GA.Jan 26 2019, 7:24 PM
jmlccdmd added a comment.Jan 30 2019, 2:17 PM

I confirm that in yesterday's rolling image, the problem is corrected.

Thank you!

dmbaturin merged a task: T1235: "show | commands" don't work from config mode.Mar 26 2019, 12:39 AM
dmbaturin added subscribers: zsdc, WiXZlo.