Subscribing to events through a WebSocket now support boolean
expressions to filter only the events wanted based on the fields
* `event_type`
* `operation`
* `source_plugin_mount`
* `data_path`
* `namespace`
Example expressions:
These can be passed to `vault events subscribe`, e.g.,:
* `event_type == abc`
* `source_plugin_mount == secret/`
* `event_type != def and operation != write`
```sh
vault events subscribe -filter='source_plugin_mount == secret/' 'kv*'
```
The docs for the `vault events subscribe` command and API endpoint
will be coming shortly in a different PR, and will include a better
specification for these expressions, similar to (or linking to)
https://developer.hashicorp.com/boundary/docs/concepts/filtering
* reduce calls to DetermineRoleFromLoginRequest from 3 to 1 for aws auth method
* change ordering of LoginCreateToken args
* replace another determineRoleFromLoginRequest function with role from context
* add changelog
* Check for role in context if not there make call to DeteremineRoleFromLoginRequest
* move context role check below nanmespace check
* Update changelog/22583.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* revert signature to same order
* make sure resp is last argument
* retrieve role from context closer to where role variable is needed
* remove failsafe for role in mfa login
* Update changelog/22583.txt
---------
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
Biggest change: we rename `Send` to `SendEvent` in `logical.EventSender`..
Initially we picked `Send` to match the underlying go-eventlogger
broker's `Send` method, and to avoid the stuttering of `events.SendEvent`.
However, I think it is more useful for the `logical.EventSender`
interface to use the method `SendEvent` so that, for example,
`framework.Backend` can implement it.
This is a relatively change now that should not affect anything
except the KV plugin, which is being fixed in another PR.
Another change: if the `secret_path` metadata is present, then
the plugin-aware `EventBus` will prepend it with the plugin mount.
This allows the `secret_path` to be the full path to any referenced
secret.
This change is also backwards compatible, since this field was not
present in the KV plugin. (It did use the slightly different `path`
field, which we can keep for now.)
* Migrate protobuf generation to Buf
Buf simplifies the generation story and allows us to lean
into other features in the Buf ecosystem, such as dependency
management, linting, breaking change detection, formatting
and remote plugins.
* Format all protobuf files with buf
Also add a CI job to ensure formatting remains consistent
* Add CI job to warn on proto generate diffs
Some files were not regenerated with the latest version
of the protobuf binary. This CI job will ensure we are always
detect if the protobuf files need regenerating.
* Add CI job for linting protobuf files
* add APILockShouldBlockRequest to backend proto
* make proto
* add APILockShouldBlockRequest to system view
* Revert "make proto"
This reverts commit 7f33733f185e8a7419590d82150e85abdcc5e707.
* Revert "add APILockShouldBlockRequest to backend proto"
This reverts commit a3bf41f7f2a0811dd323fbff4da45a582c942f2b.
* move APILockShouldBlockRequest to extended sys view
* add changelog entry
* Add stub ACME billing interfaces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add initial implementation of client count
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly attribute to mount, namespace
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor adding entities of custom types
This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.
In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of ACME mounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for billing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better support managed key system view casting
Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of mounts for enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate mounts in ACME billing tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a hopefully unique separator for encoded identifiers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use mount accesor, not path
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename AddEventToFragment->AddActivityToFragment
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* Add header operation to sdk/logical
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for routing HEAD operations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.
We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.
We also bump the eventlogger to fix a race condition that this code triggers.
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.
This can be used by either `wscat` or the new
`vault events subscribe`:
e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
and
```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.
This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
* Adds managed key usages for MAC generate/verify and RNG.
* Remove MAC-related key usages from managed key in favor of sign/verify.
* Remove context from random source managed key interface.
* named MFA method configurations
* fix a test
* CL
* fix an issue with same config name different ID and add a test
* feedback
* feedback on test
* consistent use of passcode for all MFA methods (#18611)
* make use of passcode factor consistent for all MFA types
* improved type for MFA factors
* add method name to login CLI
* minor refactoring
* only accept MFA method name with its namespace path in the login request MFA header
* fix a bug
* fixing an ErrorOrNil return value
* more informative error message
* Apply suggestions from code review
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* feedback
* test refactor a bit
* adding godoc for a test
* feedback
* remove sanitize method name
* guard a possbile nil ref
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Add WriteForwardedStorage to sdk's plugin, logical in OSS
This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.
Notably, these semantics are subject to change and shouldn't yet be
relied on.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Collect paths for write forwarding in OSS
This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add forwarded writer implementation to OSS
Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Wire up mount-specific request forwarding in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify that state lock needs to be held to call HAState in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move cluster sentinel constant to sdk/logical
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose ClusterID to Plugins via SystemView
This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add GRPC ClusterID implementation
For any external plugins which wish to use it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.
Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
* add Link config, init, and capabilities
* add node status proto
* bump protoc version to 3.21.9
* make proto
* adding link tests
* remove wrapped link
* add changelog entry
* update changelog entry
Move version out of SDK. For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString. That will have to wait until we've removed uses of useragent.String from all builtins.
* VAULT-8719 Support data array for alias clash error response so UI can understand error
* VAULT-8719 Changelog
* VAULT-8719 Update alias mount update logic
* VAULT-8719 Further restrict IsError()
Add plugin version to GRPC interface
Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.
I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:
If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.
* Placeholder backend should be external
We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.
So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core
* Fix body handling
* Role resolution for rate limit quotas
* VAULT-6613 update precedence test
* Add changelog
* VAULT-6614 start of changes for roles in LCQs
* Expiration changes for leases
* Add role information to RequestAuth
* VAULT-6614 Test updates
* VAULT-6614 Add expiration test with roles
* VAULT-6614 fix comment
* VAULT-6614 Protobuf on OSS
* VAULT-6614 Add rlock to determine role code
* VAULT-6614 Try lock instead of rlock
* VAULT-6614 back to rlock while I think about this more
* VAULT-6614 Additional safety for nil dereference
* VAULT-6614 Use %q over %s
* VAULT-6614 Add overloading to plugin backends
* VAULT-6614 RLocks instead
* VAULT-6614 Fix return for backend factory
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* PR feedback
* duo to handle preventing passcode reuse
Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
