The WebSocket tests have been very flaky because we weren't able to tell when a WebSocket was fully connected and subscribed to events.
We reworked the websocket subscription code to accept the websocket only after subscribing.
This should eliminate all flakiness in these tests. 🤞 (We can follow-up in an enterprise PR to simplify some of the tests after this fix is merged.)
I ran this locally a bunch of times and with data race detection enabled, and did not see any failures.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
We grab the state lock and check that the core is not shutting down.
This panic mostly seems to happen if Vault is shutting down, usually
in a test.
Also, we try clean up the go-bexpr test by sending duplicates, and
deduplicating in the receive loop.
before starting access loop checker.
We were seeing rare data races because the `core.IsRoot()` check was
being called after the subscriber access loop checker was starter.
The subscriber access loop calls `core.CheckToken()`, which can modify
the values that `IsRoot()` is reading.
Subscribing to events through a WebSocket now support boolean
expressions to filter only the events wanted based on the fields
* `event_type`
* `operation`
* `source_plugin_mount`
* `data_path`
* `namespace`
Example expressions:
These can be passed to `vault events subscribe`, e.g.,:
* `event_type == abc`
* `source_plugin_mount == secret/`
* `event_type != def and operation != write`
```sh
vault events subscribe -filter='source_plugin_mount == secret/' 'kv*'
```
The docs for the `vault events subscribe` command and API endpoint
will be coming shortly in a different PR, and will include a better
specification for these expressions, similar to (or linking to)
https://developer.hashicorp.com/boundary/docs/concepts/filtering
Previously, when a user initiated a websocket subscription,
the access to the `sys/events/subscribe` endpoint was checked then,
and only once.
Now, perform continuous policy checks:
* We check access to the `sys/events/subscribe` endpoint every five
minutes. If this check fails, then the websocket is terminated.
* Upon receiving any message, we verify that the `subscribe`
capability is present for that namespace, data path, and event type.
If it is not, then the message is not delivered. If the message is
allowed, we cache that result for five minutes.
Tests for this are in a separate enterprise PR.
Documentation will be updated in another PR.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Events from multiple namespaces can be subscribed to via
glob patterns passed to the subscription.
This does not do policy enforcement yet -- that will come in PR soon.
I tested this manually as well by pulling it into Vault Enterprise
so I could create namespaces and check that subscriptions work as
expected.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Adding explicit MPL license for sub-package.
This directory and its subdirectories (packages) contain files licensed with the MPLv2 `LICENSE` file in this directory and are intentionally licensed separately from the BSL `LICENSE` file at the root of this repository.
* Updating the license from MPL to Business Source License.
Going forward, this project will be licensed under the Business Source License v1.1. Please see our blog post for more details at https://hashi.co/bsl-blog, FAQ at www.hashicorp.com/licensing-faq, and details of the license at www.hashicorp.com/bsl.
* add missing license headers
* Update copyright file headers to BUS-1.1
* Fix test that expected exact offset on hcl file
---------
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
Co-authored-by: Sarah Thompson <sthompson@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.
We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.
We also bump the eventlogger to fix a race condition that this code triggers.
This checks the request against the `read` permission for
`sys/events/subscribe/{eventType}` on the initial subscribe.
Future work includes moving this to its own verb (`subscribe`)
and periodically rechecking the request.
Tested locally by minting a token with the wrong permissions
and verifying that they are rejected as expected, and that
they work if the policy is adjusted to `sys/event/subscribe/*`
(or the specific topic name) with `read` permissions.
I had to change the `core.checkToken()` to be publicly accessible,
as it seems like the easiest way to check the token on the
`logical.Request` against all relevant policies, but without
going into all of the complex logic further in `handleLogical()`.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.
This can be used by either `wscat` or the new
`vault events subscribe`:
e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
and
```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
