* Fix OpenAPI spec definitions for PKI EAB APIs
- Do not generate duplicate operation ids for the various new-eab apis
- Fill out proper operation verb for eab delete call
- Pluralize operation verb for list-eab-keys api
- Fill out proper response data for new-eab and list-eab-keys
* Add cl
Add an updated `target_ec2_instances` module that is capable of
dynamically splitting target instances over subnet/az's that are
compatible with the AMI architecture and the associated instance type
for the architecture. Use the `target_ec2_instances` module where
necessary. Ensure that `raft` storage scenarios don't provision
unnecessary infrastructure with a new `target_ec2_shim` module.
After a lot of trial, the state of Ec2 spot instance capacity, their
associated APIs, and current support for different fleet types in AWS
Terraform provider, have proven to make using spot instances for
scenario targets too unreliable.
The current state of each method:
* `target_ec2_fleet`: unusable due to the fact that the `instant` type
does not guarantee fulfillment of either `spot` or `on-demand`
instance request types. The module does support both `on-demand` and
`spot` request types and is capable of bidding across a maximum of
four availability zones, which makes it an attractive choice if the
`instant` type would always fulfill requests. Perhaps a `request` type
with `wait_for_fulfillment` option like `aws_spot_fleet_request` would
make it more viable for future consideration.
* `target_ec2_spot_fleet`: more reliable if bidding for target instances
that have capacity in the chosen zone. Issues in the AWS provider
prevent us from bidding across multiple zones succesfully. Over the
last 2-3 months target capacity for the instance types we'd prefer to
use has dropped dramatically and the price is near-or-at on-demand.
The volatility for nearly no cost savings means we should put this
option on the shelf for now.
* `target_ec2_instances`: the most reliable method we've got. It is now
capable of automatically determing which subnets and availability
zones to provision targets in and has been updated to be usable for
both Vault and Consul targets. By default we use the cheapest medium
instance types that we've found are reliable to test vault.
* Update .gitignore
* enos/modules/create_vpc: create a subnet for every availability zone
* enos/modules/target_ec2_fleet: bid across the maximum of four
availability zones for targets
* enos/modules/target_ec2_spot_fleet: attempt to make the spot fleet bid
across more availability zones for targets
* enos/modules/target_ec2_instances: create module to use
ec2:RunInstances for scenario targets
* enos/modules/target_ec2_shim: create shim module to satisfy the
target module interface
* enos/scenarios: use target_ec2_shim for backend targets on raft
storage scenarios
* enos/modules/az_finder: remove unsed module
Signed-off-by: Ryan Cragun <me@ryan.ec>
* openapi: Fix response schema for PKI Issue requests
* tests
* changelog
* another expiration for generate/rotate root
* more type fixes from @stevendpclark
Vault API endpoints are defined using regexes in instances of the SDK's
framework.Path structure. However, OpenAPI does not use regexes, so a
translation is performed. It is technically possible that this
translation produces colliding OpenAPI paths from multiple
framework.Path structures. When this happens, there has formerly been no
diagnostic, and one result silently overwrites the other in a map.
As a result of this, several operations are currently accidentally
missing from the Vault OpenAPI, which is also the trigger for
https://github.com/hashicorp/vault-client-go/issues/180.
This PR adds a log message, to help catch such accidents so that they
can be fixed. Much of the PR is propagating a logger to the point where
it is needed, and adjusting tests for the API change.
With current Vault, this will result in the following being logged each
time a request is made which triggers OpenAPI generation:
```
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/totp
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/okta
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/duo
[WARN] secrets.identity.identity_0cd35e4d: OpenAPI spec generation: multiple framework.Path instances generated the same path; last processed wins: path=/mfa/method/pingid
```
I will submit a further PR to fix the issue - this one is just to add
the diagnostic.
* Refactor `identity/mfa/method/*` endpoints to fix bad OpenAPI
There is a problem with how the `identity/mfa/method/*` endpoints are
defined, resulting in incorrect OpenAPI generation.
I raised hashicorp/vault-client-go#180 to track a consequence, and
opened #20873 which explains the problem and adds a log message to
detect it.
This PR is now the fix.
It's actually quite an interesting problem, that has come about through
some particular implementation choices, in Vault's first/only case where
REST API objects are created by writing to the collection URL, and have
their ID allocated by the server, instead of the client.
The triggering cause of the malfunction was trying to have a single
framework.Path struct instance which optionally includes or excludes the
method_id path parameter, and also another framework.Path struct
instance handling list operations.
The fix is to simplify the path regexes, and have one framework.Path
which handles the method_id being present, and one that handles it being
absent.
The diff is somewhat large, because the affected code had been
copy/pasted four times (TOTP, Okta, Duo, PingID) - so I took the
opportunity to fix the duplication, creating appropriate helper methods
so that the quadruplicated code could be re-unified.
* Revise documentation
This update refactors how the documentation presents these endpoints to
users, both for clarity, and to align with the new structure of the
code.
From a user perspective, it clears up some unclear presentation of when
the `method_id` parameter should and should not be present, adds
a missing description of the response to create requests, and changes
the `method_id` parameter name to be used consistently (rather than `id`
in some cases, unlike the actual code/OpenAPI).
* Fix incorrect acronym (review fix)
* Accept suggestion of tweaked grammar in documentation
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add changelog
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
We seem to hit occasional capacity issues when attempting to launch spot
fleets with arm64 instance types. After checking pricing in the regions
that we use, it appears that current and older generation amd64 t2 and
t3 instance types are running at quite a discount whereas t4 arm64
instances are barely under on-demand price, suggesting limited capacity
for arm64 spot instances at this time. We'll change our default backend
instance architecture to amd64 to bid for the cheaper t2 and t3
instances and increase our `max_price` globally to that of a RHEL
machine running on-demand with a t3.medium.
Signed-off-by: Ryan Cragun <me@ryan.ec>
Begin the process of migrating away from the "strongly encouraged not to
use"[0] Ec2 spot fleet API to the more modern `ec2:CreateFleet`.
Unfortuantely the `instant` type fleet does not guarantee fulfillment
with either on-demand or spot types. We'll need to add a feature similar
to `wait_for_fulfillment` on the `spot_fleet_request` resource[1] to
`ec2_fleet` before we can rely on it.
We also update the existing target fleets to support provisioning generic
targets. This has allowed us to remove our usage of `terraform-enos-aws-consul`
and replace it with a smaller `backend_consul` module in-repo.
We also remove `terraform-enos-aws-infra` and replace it with two smaller
in-repo modules `ec2_info` and `create_vpc`. This has allowed us to simplify
the vpc resources we use for each scneario, which in turn allows us to
not rely on flaky resources.
As part of this refactor we've also made it possible to provision
targets using different distro versions.
[0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-best-practices.html#which-spot-request-method-to-use
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/spot_fleet_request#wait_for_fulfillment
* enos/consul: add `backend_consul` module that accepts target hosts.
* enos/target_ec2_spot_fleet: add support for consul networking.
* enos/target_ec2_spot_fleet: add support for customizing cluster tag
key.
* enos/scenarios: create `target_ec2_fleet` which uses a more modern
`ec2_fleet` API.
* enos/create_vpc: replace `terraform-enos-aws-infra` with smaller and
simplified version. Flatten the networking to a single route on the
default route table and a single subnet.
* enos/ec2_info: add a new module to give us useful ec2 information
including AMI id's for various arch/distro/version combinations.
* enos/ci: update service user role to allow for managing ec2 fleets.
Signed-off-by: Ryan Cragun <me@ryan.ec>
The upcoming event main plugin will use a very similar pattern
as the database plugin map, so it makes sense to refactor this and move
this map out. It also cleans up the database plugin backend so that
it does not have to keep track of the lock.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* add config changes for name and priority fields in seal stanza
* change env vars and fix tests
* add header and fix func call
* tweak limits on seals
* fix missing import
* add docstrings
* UI: HDS adoption replace AlertBanner part 1 (#21163)
* rename test selector
* replace db banner
* add class
* replace db role edit
* db creds
* generate creds
* simpler class
* license banner component
* oidc callback plash
* raft
* aws
* secret create or update
* change to compact alert for form field
* change back to inline
* combine alert banners
* wrap in conditional
* remove references to message class
* UI: HDS adoption replace AlertBanner part 2 (#21243)
* token expire warning
* delete css
* edit form
* item details distribute mfa step 2 transit verify
* back to secondary
* distribute
* oidc lease error
* sign
* kv obj and repl dash
* more repl
* update test selector
* show, creds
* shamir
* pki csr
* pki banners
* add hds library to ember engines
* woops comma
* fix k8 test
* update message error component for last!
* hold off MessageError changes until next pr
* revert test selectors
* update pki tests
* UI: part 3 remove alert banner (#21334)
* final component swap
* and actual final of MessageError
* update MessageError selectors
* delete alert-banner and remove references
* update next step alerts to highlight color
* finishing touches, auth form test and client dashboard inline link
* fix more selectors
* fix shamir flow test
* ui: part 4 final cleanup (#21365)
* replace AlertPopup
* add test tag
* move tag
* one more message error tag
* delete alert popup
* final css cleanup
* move preformatted flash into <p> tag
* ui: address comments for sidebranch (#21388)
* add periods, move link to trailing
* more periods and typo fix
* remove old references to pki and old adapter
* remove after model hook from list
* remove references to isEngine
* add test
* update test
* delete test
* fix test
* Replace all time.ParseDurations with testutil.ParseDurationSeconds
* Changelog
* Import formatting
* Import formatting
* Import formatting
* Import formatting
* Semgrep rule that runs as part of CI
These don't do anything but reject requests:
> The server will not issue certificates for the identifier:
> role (something) will not issue certificate for name
> xps15.local.cipherboy.com
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
