VyOS 1.2 (jessie) testing spreadsheet
Open, NormalPublic

Description

One of the results from our development meeting on September 14th was that I'd organize a testing matrix for features to be tested in VyOS 1.2 nightly builds.

Testing matrix is here: https://docs.google.com/spreadsheets/d/19jT6_erlS2bKGXJaAWQU-LIrMiWYv2Hok1ZqWK1dC3k/edit#gid=0

If you want to test a feature or submit feedback on a nightly build, please comment here and I'll grant you write access to the spreadsheet.
If you've discovered a bug in a nightly build, please raise a case as well.

Details

Difficulty level
Lunatic (nearly impossible)
trickv created this task.Sep 20 2016, 7:10 PM
trickv created this object with edit policy "All Users".

Hello. I want to participate in testing if possible. Thanks.

Maybe it's interesting to attach the configs to the tested-build data-entry.

syncer edited subscribers, added: VyOS 1.2.x; removed: EwaldvanGeffen, trickv.
3rett added a subscriber: 3rett.Dec 16 2016, 2:25 PM

I too would like to participate in testing..

Brett

aopdal added a subscriber: aopdal.Dec 20 2016, 1:52 PM

I like to participate in the testing. But I think we need to break down the point in a bit more specific tests. We should also have requirements which state what is pass / fail. And we should probably create specific test descriptions ant test cases which could be automated.

Without control of the underlying components there are some problems with the testing. But if we have some requirements to specify what is the expected behaviour we could probably have a easier testing regime.

@aopdal absolutely after previous dev meeting there were some conclusions
I think we just need to repeat meeting and define something more specific
@trickv @dmbaturin @jhendryUK @EwaldvanGeffen @UnicronNL any chance to get quick meeting before 2017?

I could participate in a meeting next week, this week is "getting ready for Christmas" week ;-)

I suggest there should be an agenda for the discussion and probably some good proposals like:

  • standard tool for autoconfigure of VyOS testcases is Ansible
  • when describing testcases we use template XXX
  • testcase description with runbooks, group_vars, host_vars, templates should be checked in to git XXX
  • test results should be checked in to git XXXX

This is just examples so don't take it for "good proposals" ;-)

EwaldvanGeffen added a comment.EditedDec 20 2016, 8:10 PM

In for a quick meeting. I think one of the major points would be 118; what goes in and what not; this shouldn't take more than 10 minutes, I think.

About 'proposals'; document them beforehand. You will probably get valuable feedback on phabricator and others will be able to form an opinion. I'd put them last on the agenda for those willing to stick around.