Page MenuHomeVyOS Platform

Python management library methods give timeouts
Closed, ResolvedPublic

Description

>>> from vyroute.Router import BasicRouter
>>> v = BasicRouter('10.46.1.80','vyos:377vyos350')
>>> v.login()
{'Result': 'Login successfully.'}
>>> v.configure()
{'Result': 'Active configure mode successfully.'}
>>> v.static_route({'config':{'target':'203.0.1113.0/24','next-hop':'192.0.2.1','distance':'1'},})
{'Error': TimeoutException('Timeout while waiting for response from device',)}

After this, commit_config() says "you don't need to commit". Previous time I managed to use it successfully, but commit_config() gave me a timeout, despite it actually worked on the router.

I've never used the SSH lib in question so I can't tell why this may happen and how to fix it, it's up to you.

Details

Difficulty level
Normal (likely a few hours)

Event Timeline

I donno whether you have found the below content in vyroute's readme file(you can visit that file):

Therefore most of methods only have one "data" parameter,but different methods' input data are different.If input data have any mistakes
,due to Exscript,the method will return a timeout exception after few seconds,then the error reason(from VyOS) will be return when you
execute next configuration.So you should execute the former configuration one more time.

It seems that your input data had no mistake.But maybe the problem is the VyOS or Exscript.
Before I pushed vyroute on my github at the first time,I have tested this lib.
When there is a timeout exception raises,you must execute the former configuration one more time.
When you check the VyOS and know that you configuration have success.
But in the router substance("v"),the value of "commit" in self.__status still is "no".
Because if statc_route() return an timeout error,it will not rewrite the value of "commit" to "Yes",
therefore when you execute commit_config() it will return "you don't need to commit".

The solution is when a timeout exception raise,you just execute the former configuration once again.Because
the configure mode in VyOS is idempotent.

Well, you realize that "execute one more time" is absolutely not an option for a library that is supposed to allow making configuration changes unattended from SDN controllers etc. among other things, right? ;)

We'll have a talk about API design later, but until we identify the cause of thess timeouts and find at least a workaround, the library should be considered broken.

I think high-level tools should read the return before they feed back to user,if func return a time out error,high-level tools can ignore it.Now I only find one solution is add a statement to check the return,if func return a timeout error,it will modify the self.__status.

So,what about your opinion

Well,I have find a soluntion.

When I add a func name return_error(),actually,this func just execute a "clear" command in configure mode.It will return the error result like that:

the return_error func like this(Actually the command string in this function is not important,we can use any string) :

def return_error(self):
    try:
        self.__conn.execute('clear')
    except Exception as e:
        return e
>>> from vyroute.Router import BasicRouter
>>> v = BasicRouter('172.16.77.184','vyos:vyos')
>>> v.login()
{'Result': 'Login successfully.'}
>>> v.configure()
{'Result': 'Active configure mode successfully.'}
>>> data = {'config':{'target':'10.20.10.0','next-hop':'10.20.10.1','distance':'1'},}
>>> v.static_route(data)
{'Error': TimeoutException('Timeout while waiting for response from device',)}
>>> v.return_error()
InvalidCommandException('Device said:\nset protocols static route 10.20.10.0 next-hop 10.20.10.1 distance 1\r\nWARNING: terminal is not fully functional\r\n\r-  (press RETURN)\r\r\x07\x1b[m\r\n  "10.20.10.0" is not a valid value of type "ipv4net"\x1b[m\r\n  Value validation failed\x1b[m\r\n  Set failed\x1b[m\r\n\x1b[m\r\n\r[edit]\r\r\nvyos@vyos# lear\r\n\r\n  Invalid command: [lear]\r\n\r\n[edit]\r\r\nvyos@vyos# ',)
>>> v.status()
{'commit': None, 'object': 'login', 'configure': 'Yes', 'save': None}
>>> data = {'config':{'target':'10.20.10.0/24','next-hop':'10.20.10.1','distance':'1'},}
>>> v.static_route(data)
{'Result': 'Configured successfully'}
>>> v.status()
{'commit': 'No', 'object': 'login', 'configure': 'Yes', 'save': 'No'}

In this example I use a error input and staticroute return a timeout error.Then I use return_error to get the error reason:

"10.20.10.0" is not a valid value of type "ipv4net"

This exception is a subclass of Exscript.protocols.Exception,I can use this method to judge the exception type:

>>> type(res['Error'])
<class 'Exscript.protocols.Exception.InvalidCommandException'>
>>> type(res['Error']) == InvalidCommandException

I just import that subclass and compare it with the dictionary's value.

My plan is:
When the functions return a timeout error.It will use return_error() return timeout reason.
If the reason is data error(will return a InvalidCommandException),then user can read the error message and fix the data and try once again.
If high-level tools meet with this error(I think high-level tool should do data validity check first),they can judge by the exception type and decide to feed back to users
and tell them the data has mistakes.

If the reason is the network error or something,this is not the lib's responsibility

Well, I'll reiterate. Interactice SSH works fine to the device in question. The library is giving timeouts. Interactice SSH is not giving me timeouts. From this I deduce that the problem must be in the library, or in exscript.

Handling genuine timeouts is a different story, we should definitely get to it. But before, we should make sure that in situations when interactive SSH works perfectly, non-interactive sessions work too.

When I was testing vyroute,I never met such a timeout.Only I enter a error data the timeout exception will raises

Do you find out the problem in your test?

Should this be rewritten with paramiko?

syncer lowered the priority of this task from Unbreak Now! to High.Jul 17 2016, 9:37 PM

@syncer Sorry,recently I was busy with some research on Machine Learning.

We have two choice:
1.paramiko
2.pexpect

I will start to do write some test and decide to use which library today

@syncer
@dmbaturin

When I try to use paramiko connect to vyos but it doesn't work when I send some command,it only works on the other commands

it works:

>>> stdin,stdout,stderr = ssh.exec_command('uname -r')
>>> print stdout.read()
3.13.11-1-amd64-vyos

it doesn't works:

>>> stdin,stdout,stderr = ssh.exec_command('show interfaces')
>>> print stdout.read()

>>> stdin,stdout,stderr = ssh.exec_command('configure')

I try to use pexpect but I donno why paramiko has such problem

@syncer
@dmbaturin

I have find out a solution.Use pexpect.pxssh to execute the commands and return the result.
pexpect can provide python3 support and maybe deal with the timeout problem(You can try it when I have finish fixing)

from pxssh import pxssh

#Get the address,username and password
address = raw_input("Enter the vyos address: ")
username = raw_input("Enter the login username: ")
password = raw_input("Enter the password :")

#Login vyos
s = pxssh()
s.login(address,username,password)

#Configure mode and make a proper prompt
s.sendline("configure")
s.prompt(0)
s.set_unique_prompt()

#Create the interfaces
command_address = "set interfaces ethernet %s address %s"
command_description = "set interfaces ethernet %s description '%s'"
para_interface = raw_input("Enter the interfaces name: ")
para_address = raw_input("Enter the address you want to attach to interface: ")
#para_description = raw_input("Enter the description you want to use: ")

s.sendline(command_address % (para_interface,para_address))
s.prompt()
result1 = s.before

#s.sendline(command_description % (para_interface,para_description))
#s.prompt()
#result2 = s.before

print result1

#print result2

#Commit but not save
#s.sendline("commit")
#s.prompt()
#s.sendline("exit")
#s.prompt()
s.sendline("exit discard")

s.close()

and try to execute this test program:

root@test1:~# python vyssh.py 
Enter the vyos address: 192.168.225.2
Enter the login username: vyos
Enter the password :vyos
Enter the interfaces name: eth1
Enter the address you want to attach to interface: 192.168.22.4
set interfaces ethernet eth1 address 192.168.22.4

  Invalid IPv4 address/prefix
  
  Value validation failed
  Set failed

It seems that it can provide a perfect return.And I think pxssh will not raise timeout.

I am ready to rewrite the core part of vyroute's router configure

@syncer
@dmbaturin

@dmbaturin

Now you can test it again.I have fix the problem

Anybody can test the improved library?

Hello @Hochikong ,
can you just run basic tests and report back how it goes,
we then can go to next step maybe.

@syncer
OK,I will test it and add a BGP support

@syncer
@dmbaturin

Hi!I HAVE FIXED MOST OF PROBLEMS AND ADD BGP SUPPORT,BY THE WAY I HAVE CHANGED THE API STYLE AND REWRITED THE README FILE.

Test log:
All maintenance method(login(),logout(),configure()...) have tested,no problems.

Static,RIP,OSPF and BGP router functions no problems.I test it on VMware by 4 vms,two VyOS system and two ubuntu server.

No any timeout problem occurs in my whole test.

All codes test by 2to3,support python3.You can only install pexpect(pxssh) for python3 and run the library.

Fix log:
Fix all docstring to a unified format.

Solved the long time waiting problem in save().

Create a new method named rip_redistribute(),split the function from from rip_route().

Change the method API style and return ,please check it on README.

Rewrite a new README file for new API style.

Change some methods' name,"save_config" change to "save" and so on.

New Func:
Add a simple BGP support.

Finish the problem