Suggesting to add monitoring scripts
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html
and cli wrapper for basic configuration (maybe under services aws cloudwatch)
obviously this all for AWS AMI builds
Description
Description
Details
Details
- Difficulty level
- Normal (likely a few hours)
- Version
- -
- Why the issue appeared?
- Will be filled on close
- Is it a breaking change?
- Perfectly compatible
- Issue type
- Feature (new functionality)
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | FEATURE REQUEST | c-po | T5129 Add AWS build flavour | ||
| Resolved | FEATURE REQUEST | c-po | T425 AWS CloudWatch monitoring scripts |
Event Timeline
Comment Actions
Building from source always results in:
goroutine 91 [select]:
net.(*netFD).connect.func2()
/opt/go/src/net/fd_unix.go:118 +0x86
created by net.(*netFD).connect
/opt/go/src/net/fd_unix.go:117 +0x37b
goroutine 90 [IO wait]:
internal/poll.runtime_pollWait(0x7f1c5eaed0d8, 0x77)
/opt/go/src/runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc000419080?, 0x0?, 0x0)
/opt/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitWrite(...)
/opt/go/src/internal/poll/fd_poll_runtime.go:93
internal/poll.(*FD).WaitWrite(...)
/opt/go/src/internal/poll/fd_unix.go:741
net.(*netFD).connect(0xc000419080, {0xf118c8?, 0xc0001153e0}, {0xc00008d150?, 0x40da7f?}, {0xf0bc00?, 0xc00009e260?})
/opt/go/src/net/fd_unix.go:141 +0x719
net.(*netFD).dial(0xc000419080, {0xf118c8, 0xc0001153e0}, {0xf133b8?, 0x0?}, {0xf133b8?, 0xc0000d2d20}, 0x0?)
/opt/go/src/net/sock_posix.go:149 +0x385
net.socket({0xf118c8, 0xc0001153e0}, {0xc6331b, 0x3}, 0x2, 0x1, 0x5?, 0x18?, {0xf133b8, 0x0}, ...)
/opt/go/src/net/sock_posix.go:70 +0x2b2
net.internetSocket({0xf118c8, 0xc0001153e0}, {0xc6331b, 0x3}, {0xf133b8, 0x0}, {0xf133b8, 0xc0000d2d20}, 0xc?, 0x0, ...)
/opt/go/src/net/ipsock_posix.go:142 +0xf8
net.(*sysDialer).doDialTCP(0xc0004297a0, {0xf118c8, 0xc0001153e0}, 0x0, 0xba4780?)
/opt/go/src/net/tcpsock_posix.go:68 +0xa5
net.(*sysDialer).dialTCP(0x14cb5f7?, {0xf118c8?, 0xc0001153e0?}, 0xc00008d3e0?, 0x46f9e5?)
/opt/go/src/net/tcpsock_posix.go:64 +0x69
net.(*sysDialer).dialSingle(0xc0004297a0, {0xf118c8, 0xc0001153e0}, {0xf0e458?, 0xc0000d2d20})
/opt/go/src/net/dial.go:582 +0x214
net.(*sysDialer).dialSerial(0xc0004297a0, {0xf118c8, 0xc0001153e0}, {0xc000477120?, 0x1, 0x66fa35?})
/opt/go/src/net/dial.go:550 +0x312
net.(*sysDialer).dialParallel(0xc000477110?, {0xf118c8?, 0xc0001153e0?}, {0xc000477120?, 0xc0001153e0?, 0xc6390e?}, {0x0?, 0xc6331b?, 0x0?})
/opt/go/src/net/dial.go:451 +0x413
net.(*Dialer).DialContext(0xc0001141e0, {0xf118c8, 0xc000115200}, {0xc6331b, 0x3}, {0xc0000c8168, 0x12})
/opt/go/src/net/dial.go:428 +0x705
net/http.(*Transport).dial(0x38?, {0xf118c8?, 0xc000115200?}, {0xc6331b?, 0x0?}, {0xc0000c8168?, 0x10?})
/opt/go/src/net/http/transport.go:1170 +0xda
net/http.(*Transport).dialConn(0x1470e80, {0xf118c8, 0xc000115200}, {{}, 0x0, {0xc0000c80d8, 0x4}, {0xc0000c8168, 0x12}, 0x0})
/opt/go/src/net/http/transport.go:1608 +0x83f
net/http.(*Transport).dialConnFor(0x0?, 0xc0000f06e0)
/opt/go/src/net/http/transport.go:1450 +0xb0
created by net/http.(*Transport).queueForDial
/opt/go/src/net/http/transport.go:1419 +0x3f2Thus I rather just import the package to our mirrors
Comment Actions
Requires some additional work
we need to preserve configuration between upgrade
alternatively, we need to investigate if default config can be used with VM role
Comment Actions
Notice. Initially this task was about monitoring scripts but they were deprecated. Then aws-cloudwatch-agent emerged.
aws-cloudwatch-agent was successfully added to vyos-build:equuleus. But cloudwatch configuration preservation between image updates is not.
This task was closed mistakenly prematurely thus should be reopen.
Comment Actions
I've created the PR https://github.com/vyos/vyos-documentation/pull/987 as a temporary explanation for users on how to preserve CloudWatch Agent configuration in a semi-automated way, using the SSM Parameter Store.
Comment Actions
@unity when you need AWS credential , will they be automatically deployed from SSM or will we have to add those credentials in the virtual machine? ? shouldn't aws-cli be integrated?
# this command works with out aws-cli $ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c ssm:<your-configuration-name>
Comment Actions
- In order to apply SSM auto-configuration of the CloudWatch agent, an SSM agent must be installed that installs the CloudWatch agent with the necessary configuration. Currently, there is no SSM agent inside VyOS AWS images, and I haven't heard anything about willingness to include it.
- The amazon-cloudwatch-agent package has only one dependency, libc6. Therefore, it does not need the aws-cli to be configured or set up at all.
- Granting access to the CloudWatch service from an EC2 instance is done by applying the corresponding IAM role to the instance. While it is possible to do this via manual credential input, it is an unwanted practice inside AWS.
- The possible scenario of sending data to CloudWatch out of AWS is unique and requires another Phorge task, I think.
Summary: The current workaround solution is for EC2 instances with the IAM role-based permission granting model. In the future there will be platform-specific scripts inside VyOS to preserve any type of CloudWatch configuration (role-based, secret key) between system image updates.
Comment Actions
Thanks for clarifying. Yes , I also saw the possibility of extending role based IAM to add on-premise image (that could be interesting for VyOS).