Vault
Identity: entities and groups
Vault supports multiple authentication methods and also allows enabling the same type of authentication method on different mount paths. Each Vault client may have multiple accounts with various identity providers that are enabled on the Vault server.
Vault clients can be mapped as entities and their corresponding accounts with authentication providers can be mapped as aliases. In essence, each entity is made up of zero or more aliases. Identity secrets engine internally maintains the clients who are recognized by Vault.
Personas
The steps described in this tutorial are typically performed by operations persona.
Challenge
Bob has accounts in both Github and LDAP. Both Github and LDAP auth methods are enabled on the Vault server that he can authenticate using either one of his accounts. Although both accounts belong to Bob, there is no association between the two accounts to set some common properties.
Solution
Create an entity representing Bob, and associate aliases representing each of his accounts as the entity member. You can set additional policies and metadata on the entity level so that both accounts can inherit.
When Bob authenticates using either one of his accounts, the entity identifier will be tied to the authenticated token. When such tokens are put to use, their entity identifiers are audit logged, marking a trail of actions performed by specific users.
Note
If you are running Vault Enterprise or HCP Vault Dedicated, be sure to read the Why create entities? section at the end.
Prerequisites
To perform the tasks described in this tutorial, you need to have a Vault environment.
- Refer to the Getting Started tutorial to install Vault
- jq to process the JSON output for readability
Launch Terminal
This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.
Policy requirements
Unless you are running Vault for test or development purposes, such as dev mode (-dev
),
or with a short lived HCP Vault Dedicated dev tier cluster, it is recommended that you authenticate
with Vault and acquire a tokens with an appropriate set of policies based on your role in
the organization.
If you are connecting to a non-development Vault server, your token must have a policy with following permissions:
# Configure auth methods
path "sys/auth" {
capabilities = [ "read", "list" ]
}
# Configure auth methods
path "sys/auth/*" {
capabilities = [ "create", "update", "read", "delete", "list", "sudo" ]
}
# Manage userpass auth methods
path "auth/userpass/*" {
capabilities = [ "create", "read", "update", "delete" ]
}
# Manage github auth methods
path "auth/github/*" {
capabilities = [ "create", "read", "update", "delete" ]
}
# Display the Policies tab in UI
path "sys/policies" {
capabilities = [ "read", "list" ]
}
# Create and manage ACL policies from UI
path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
# Create and manage policies
path "sys/policies/acl" {
capabilities = [ "read", "list" ]
}
# Create and manage policies
path "sys/policies/acl/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
# List available secrets engines to retrieve accessor ID
path "sys/mounts" {
capabilities = [ "read" ]
}
# Create and manage entities and groups
path "identity/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
If you are not familiar with policies, complete the policies tutorial.
Lab setup
In a terminal, start a Vault dev server with
root
as the root token.$ vault server -dev -dev-root-token-id root
The Vault dev server defaults to running at
127.0.0.1:8200
. The server is initialized and unsealed.Insecure operation
Do not run a Vault dev server in production. This approach starts a Vault server with an in-memory database and runs in an insecure way.
Open a new terminal and export an environment variable for the
vault
CLI to address the Vault server.$ export VAULT_ADDR=http://127.0.0.1:8200
Export an environment variable for the
vault
CLI to authenticate with the Vault server.$ export VAULT_TOKEN=root
The Vault server is ready.
Create an Entity with Alias
You are going to create a new entity with base policy assigned. The entity defines two entity aliases with each has a different policy assigned.
Scenario: A user, Bob Smith at ACME Inc. happened to have two sets of
credentials: bob
and bsmith
. Bob can authenticate with Vault using either
one of their accounts. To manage the user's accounts and link them to identity
Bob Smith
in the QA team, you are going to create an entity for Bob.
![Entity Bob Smith](/_next/image?url=https%3A%2F%2Fcontent.hashicorp.com%2Fapi%2Fassets%3Fproduct%3Dtutorials%26version%3Dmain%26asset%3Dpublic%252Fimg%252Fvault-entity-1.png%26width%3D931%26height%3D366&w=1920&q=75)
For the simplicity of this tutorial, you are going to work with the userpass
auth method. In reality, the user bob
might be a username that exists in
Active Directory, and bsmith
might be Bob's username in GitHub. To mock the
behavior, you are going to enable the userpass auth method at two separate
paths: userpass-test
and userpass-qa
to mimic two different types of auth methods.
User lockout
As of Vault 1.13, the User lockout feature is enabled
by default for the userpass
, approle
, and ldap
auth methods.
Scenario Policies
base.hcl
path "secret/data/training_*" {
capabilities = ["create", "read"]
}
test.hcl
path "secret/data/test" {
capabilities = [ "create", "read", "update", "delete" ]
}
team-qa.hcl
path "secret/data/team-qa" {
capabilities = [ "create", "read", "update", "delete" ]
}
Note
This scenario assumes that the K/V v2 secrets engine is enabled at
the secret
path. If you are not sure, refer to the Versioned Key/Value
Secrets Engine
tutorial.
Create bob
and bsmith
users with appropriate policies attached.
Create
base
policy.$ vault policy write base -<<EOF path "secret/data/training_*" { capabilities = ["create", "read"] } EOF
Create
test
policy.$ vault policy write test -<<EOF path "secret/data/test" { capabilities = [ "create", "read", "update", "delete" ] } EOF
Create
team-qa
policy.$ vault policy write team-qa -<<EOF path "secret/data/team-qa" { capabilities = [ "create", "read", "update", "delete" ] } EOF
List all policies to verify that
base
,test
andteam-qa
policies exist.$ vault policy list base default team-qa test root
Note
If you are using HCP Vault Dedicated, you will also see the
hcp-root
policy.Enable the
userpass
auth method atuserpass-test
.$ vault auth enable -path="userpass-test" userpass
Create a new user named
bob
inuserpass-test
where the password istraining
andtest
policy is attached.$ vault write auth/userpass-test/users/bob password="training" policies="test"
Enable the
userpass
auth method at another paths,userpass-qa
.$ vault auth enable -path="userpass-qa" userpass
Create a new user named
bsmith
inuserpass-qa
where the password istraining
andteam-qa
policy is attached.$ vault write auth/userpass-qa/users/bsmith password="training" policies="team-qa"
Execute the following command to discover the mount accessor for the userpass auth method.
$ vault auth list -detailed Path Plugin Accessor ... ---- ---- -------- ... token/ token auth_token_c5943123 ... userpass-qa/ userpass auth_userpass_8c7b8e0f ... userpass-test/ userpass auth_userpass_264d4705 ... ...
Each userpass auth methods has a unique Accessor value to identify it.
Run the following command to store the
userpass-test
auth accessor value in a file namedaccessor_test.txt
.$ vault auth list -format=json | jq -r '.["userpass-test/"].accessor' > accessor_test.txt
The resulting file contains the accessor value (
auth_userpass_XXXXX
).Similarly, run the following command to store the
userpass-qa
auth accessor value in a file namedaccessor_qa.txt
.$ vault auth list -format=json | jq -r '.["userpass-qa/"].accessor' > accessor_qa.txt
Create an entity for
bob-smith
, and store the returned entity ID in a file named,entity_id.txt
.$ vault write -format=json identity/entity name="bob-smith" policies="base" \ metadata=organization="ACME Inc." \ metadata=team="QA" \ | jq -r ".data.id" > entity_id.txt
The resulting file contains the entity ID for
bob-smith
(e.g.24204b50-22a6-61f5-bd4b-803f1a4e4726
).Now, add the user
bob
to thebob-smith
entity by creating an entity alias. Set a custom metadata on thebob
entity alias named, "account" and set its value to be "Tester Account".$ vault write identity/entity-alias name="bob" \ canonical_id=$(cat entity_id.txt) \ mount_accessor=$(cat accessor_test.txt) \ custom_metadata=account="Tester Account"
Note
To set custom metadata on the entity alias, you must run Vault 1.9 or later. If you are running Vault version earlier than v1.9, run the command without
custom_metadata=account="Tester Account"
.Example output:
Key Value --- ----- canonical_id 24204b50-22a6-61f5-bd4b-803f1a4e4726 id ae2cdd0f-9807-7336-2265-5575c71837e7
To view the created entity alias, use the returned entity alias ID.
$ vault read identity/entity-alias/id/<id>
Example:
$ vault read identity/entity-alias/id/ae2cdd0f-9807-7336-2265-5575c71837e7 Key Value --- ----- canonical_id 971b5ab3-1931-8381-8605-1b9511572278 creation_time 2021-11-15T01:41:03.093366Z custom_metadata map[account:Tester Account] id 10340e8f-64fa-fe7d-0c88-b6172293f788 last_update_time 2021-11-15T01:41:03.093366Z local false merged_from_canonical_ids <nil> metadata <nil> mount_accessor auth_userpass_2cbfe431 mount_path auth/userpass-test/ mount_type userpass name bob namespace_id root
Repeat the step to add user
bsmith
to thebob-smith
entity. Set a custom metadata on thebob
entity alias named, "account" and set its value to be "QA Eng Account".$ vault write identity/entity-alias name="bsmith" \ canonical_id=$(cat entity_id.txt) \ mount_accessor=$(cat accessor_qa.txt) \ custom_metadata=account="QA Eng Account"
Example output:
Key Value --- ----- canonical_id 24204b50-22a6-61f5-bd4b-803f1a4e4726 id 6066f6af-bb1c-5310-58a1-fd9c8f151573
Review the entity details.
$ vault read -format=json identity/entity/id/$(cat entity_id.txt) | jq -r ".data"
Example output: The output should include the entity aliases, metadata (organization, and team), and base policy.
{ "aliases": [ { "canonical_id": "73503625-abcd-db22-08c3-c121d682d550", "creation_time": "2021-11-17T05:33:48.040506Z", "custom_metadata": { "account": "Tester Account" }, "id": "cf073e2e-41af-852f-848d-f67533c8a610", "last_update_time": "2021-11-17T05:33:48.040506Z", "local": false, "merged_from_canonical_ids": null, "metadata": null, "mount_accessor": "auth_userpass_0a6936a7", "mount_path": "auth/userpass-test/", "mount_type": "userpass", "name": "bob" }, { "canonical_id": "73503625-abcd-db22-08c3-c121d682d550", "creation_time": "2021-11-17T05:33:48.107834Z", "custom_metadata": { "account": "QA Eng Account" }, "id": "7add6763-ce53-d92a-c795-c8ae529ce6e7", "last_update_time": "2021-11-17T05:33:48.107834Z", "local": false, "merged_from_canonical_ids": null, "metadata": null, "mount_accessor": "auth_userpass_ef4f8068", "mount_path": "auth/userpass-qa/", "mount_type": "userpass", "name": "bsmith" } ], "creation_time": "2021-11-17T05:33:47.966585Z", "direct_group_ids": [ "49e8b9e8-8933-1e14-f05c-e1c9674b142b" ], "disabled": false, "group_ids": [ "49e8b9e8-8933-1e14-f05c-e1c9674b142b" ], "id": "73503625-abcd-db22-08c3-c121d682d550", "inherited_group_ids": [], "last_update_time": "2021-11-17T05:33:47.966585Z", "merged_entity_ids": null, "metadata": { "organization": "ACME Inc.", "team": "QA" }, "mfa_secrets": {}, "name": "bob-smith", "namespace_id": "root", "policies": [ "base" ] }
Security Consideration
Avoid storing any sensitive personally identifiable information (PII) in the entity metadata. The entity metadata is replicated to other clusters if Performance Replication is configured. This may be a great concern for the GDPR. Vault 1.9 introduced the ability to set custom metadata on each entity alias that does not overlap with the metadata set by Vault. If the auth method is local to the cluster, the metadata will not be replicated to other clusters in the same performance replication group. Therefore, it is recommended to leverage the custom metadata on the entity alias instead.
Test the entity
To better understand how a token inherits the capabilities from the entity's
policy, you are going to test it by logging in as bob
.
First, login as
bob
, and store the generated client token inbob_token.txt
.$ vault login -format=json -method=userpass -path=userpass-test \ username=bob password=training \ | jq -r ".auth.client_token" > bob_token.txt
The
test
policy grants CRUD operations on thesecret/test
path. Test to make sure that you can write secrets in the path.$ VAULT_TOKEN=$(cat bob_token.txt) vault kv put secret/test owner="bob" Key Value --- ----- created_time 2021-11-06T02:12:02.104146Z custom_metadata <nil> deletion_time n/a destroyed false version 1
Although the username
bob
does not havebase
policy attached, the token inherits the capabilities granted in the base policy becausebob
is a member of thebob-smith
entity, and the entity has base policy attached. Check to see that the bob's token inherited the capabilities.$ VAULT_TOKEN=$(cat bob_token.txt) vault token capabilities secret/data/training_test create, read
The
base
policy grants create and read capabilities onsecret/training_*
path; thereforebob
is permitted to run create and read operations against any path starting withsecret/training_*
.What about the
secret/team-qa
path?$ VAULT_TOKEN=$(cat bob_token.txt) vault token capabilities secret/data/team-qa deny
The user
bob
only inherits capability from its associating entity's policy. The user can access thesecret/team-qa
path only if they log in withbsmith
credentials.
Create an internal group
Reset the `VAULT_TOKEN` environment variable
Log back in with the token you used to configure the entity before proceed to create an internal group. (Refer to the Lab setup section.)
Now, you are going to create an internal group named engineers
. Its member is
the bob-smith
entity that you created in the create an
entity section.
![Entity Bob Smith](/_next/image?url=https%3A%2F%2Fcontent.hashicorp.com%2Fapi%2Fassets%3Fproduct%3Dtutorials%26version%3Dmain%26asset%3Dpublic%252Fimg%252Fvault-entity-3.png%26width%3D1028%26height%3D651&w=3840&q=75)
The group policy, team-eng
defines the following in the team-eng.hcl
file.
team-eng.hcl
path "secret/data/team/eng" {
capabilities = [ "create", "read", "update", "delete"]
}
Create a new policy named
team-eng
.$ vault policy write team-eng -<<EOF path "secret/data/team/eng" { capabilities = [ "create", "read", "update", "delete"] } EOF
Create an internal group named
engineers
and addbob-smith
entity as a group member and attachteam-eng
.$ vault write identity/group name="engineers" \ policies="team-eng" \ member_entity_ids=$(cat entity_id.txt) \ metadata=team="Engineering" \ metadata=region="North America"
Example output:
Key Value --- ----- id 0cd76d41-fe36-8c7b-4758-d36bbc212650 name engineers
Now, when you login as
bob
orbsmith
, its generated token inherits the group-level policy,team-eng
. You can perform similar tests demonstrated in the test the entity section to verify that.
Summary
By default, Vault creates an internal group. When you create an
internal group, you specify the group members rather than group
alias. Group aliases are mapping between Vault and external identity
providers (e.g. LDAP, GitHub, etc.). Therefore you define group aliases only
when you create external groups. For internal groups, you specify
member_entity_ids
and/or member_group_ids
.
Create an External Group
It is common for organizations to enable auth methods such as LDAP, Okta and perhaps GitHub to handle the Vault user authentication, and individual user's group memberships are defined within those identity providers.
In order to manage the group-level authorization, you can create an external group to link Vault with the external identity provider (auth provider) and attach appropriate policies to the group.
Example Scenario
Any user who belongs to training
team in GitHub organization,
example-inc
are permitted to perform all operations against the
secret/education
path.
Note
This scenario assumes that the GitHub organization, example-inc
exists as well as training
team within the organization.
Create a new policy named
education
.$ vault policy write education -<<EOF path "secret/data/education" { capabilities = [ "create", "read", "update", "delete", "list" ] } EOF
Enable GitHub auth method.
$ vault auth enable github
Retrieve the mount accessor for the GitHub auth method and save it in a file named
accessor_github.txt
.$ vault auth list -format=json | jq -r '.["github/"].accessor' > accessor_github.txt
Configure to point to your GitHub organization.
Note
Use the actual Github organization name that exists to which you want to connect. The following uses a fictional organization name,
example-inc
as an example. If you do not change this value, the login attempt at the end of this section will not succeed.Example:
$ vault write auth/github/config organization=example-inc
Create an external group, and store the generated group ID in
group_id.txt
.Example:
$ vault write -format=json identity/group name="education" \ policies="education" \ type="external" \ metadata=organization="Product Education" | jq -r ".data.id" > group_id.txt
Create a group alias where
canonical_id
is the group ID and the name must be an actual GitHub team name that exists.Note
The
name
must be an actual GitHub team name that exists in the Github organization that you configured earlier. Use slugified team name.$ vault write identity/group-alias name="training" \ mount_accessor=$(cat accessor_github.txt) \ canonical_id="$(cat group_id.txt)"
Example output:
Key Value --- ----- canonical_id 66818a45-ef85-0ff3-6c1e-37faf12ea55e id 578944f0-dcfd-29fd-a763-d3f9431512d7
Attempt to login as a GitHub user who is a member of the training group and enter the users personal access token.
$ vault login -format=json -method=github GitHub Personal Access Token (will be hidden):
Example output:
WARNING! The VAULT_TOKEN environment variable is set! This takes precedence over the value set by this command. To use the value set by this command, unset the VAULT_TOKEN environment variable or set it to the token displayed below. { "request_id": "fcc10086-9033-dc21-2ddb-d420fbabc474", "lease_id": "", "lease_duration": 0, "renewable": false, "data": null, "warnings": null, "auth": { "client_token": "s.7QKSeasAME.mD5Ve", "accessor": "yUBG78LRdLiZNkBNNTe7OEp1.mD5Ve", "policies": [ "default", "education" ], "token_policies": [ "default" ], "identity_policies": [ "education" ], "metadata": { "org": "example-inc", "username": "bobbysmith_developer" }, "orphan": true, "entity_id": "58a16f6c-ee92-0cd2-f508-4b2fdb093c07", "lease_duration": 3600, "renewable": true, "mfa_requirement": null } }
Summary
At this point, any GitHub user who belongs to training
team within the
example-inc
organization can authenticate with Vault. The generated token for
the user has education
policy attached.
Clean up
If you wish to clean up your environment after completing the tutorial, follow the steps in this section.
Unset the
VAULT_TOKEN
environment variable.$ unset VAULT_TOKEN
Unset the
VAULT_ADDR
environment variable.$ unset VAULT_ADDR
You can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.
$ pgrep -f vault | xargs kill
For Vault Dedicated, delete the Vault instance from the HCP Portal.
Why create entities?
This tutorial used a human user, Bob Smith as an example to demonstrate the use of entities and groups. However, this concept also applies to applications.
Consider a scenario where an application uses multiple auth methods to log in with Vault. Without an entity defined to tie those multiple auth identities back to the application, the Vault server has no way of knowing the same application logged into Vault using different auth methods. As a result, each login gets counted as a new client.
This may not be an issue for Vault Community Edition users; however, this can results in unexpectedly higher client counts that affects your Vault Enterprise or Vault Dedicated billing cost.
Help and Reference
This tutorial focuses on the creation of entities and groups in Vault. Refer to Vault Limits and Maximums to understand the known upper limits when working with Vault Identities.