Administration store and master key

With an administration store, the service keeps its operating data—data sources, rules, assignments, people, keys, and the sign-in policy—in a file of its own instead of in the configuration file. The secrets in it are sealed; names, rules, and the change log are readable. Changes take effect without a restart. This page explains what's stored where, how changes take effect, and what the master key protects.

• Changes take effect without a restart, and an invalid version is never applied.
• Connection strings and second factors are sealed; a copy of the file alone opens nothing.
• The master key can be changed without data loss, with two brief restarts.
• The sign-in log, the load log, and the access log live in a log file of their own with a fixed size limit.

The configuration should have the advantages of a file and those of a database at the same time. SQLite delivers both in one:

The limits further down follow from this: one instance per file, local disk only, and a change log that isn't tamper-proof.

With Gateway:Store, the service keeps its operating data in a SQLite file of its own instead of in the configuration file. A change there takes effect without a restart.

{
  "Gateway": {
    "Store": {
      "Path": "store/dbgwsvc.db",
      "MasterKey": "env:WKL_GW_MASTER_KEY",
      "PollSeconds": 5
    }
  }
}

First start. If the store is new, the service imports the operating data from the configuration file once—with the values that actually apply there, including a connection string from an environment variable. From then on, the store decides. If the file still contains operating data, the startup log lists it in a warning and doesn't use it. For Security:SignInRules there's no warning, even though those, too, then apply only from the store: if you change sign-in rules in the file, you won't see any effect. A store in which all data sources were later deleted does not re-import the entries from the file.

Changes during operation. Every PollSeconds, the service asks SQLite whether another connection has written to the file (PRAGMA data_version); this reads no data. Only after a change does it reload and validate the result with the same checks as at startup. If the result fails validation, the previous version stays in effect, and the log names every error. A loaded version is never modified but replaced as a whole. A data source whose settings didn't change keeps its resource catalog; only a changed one rereads it.

Data sources by name, with a switch. In the store, the name is the key, not the position. Every data source has the Enabled switch (also available in the configuration file, default true): a disabled data source stays in place along with every rule that names it, but its alias isn't found—a request gets the same response as for an unknown alias, and no list of aliases shows it. This is intended for maintenance or for moving a database without deleting entries.

Connection strings and second factors are sealed. They're protected with authenticated encryption using XChaCha20-Poly1305—a subkey derived with HChaCha20 from the first 16 bytes of the 24-byte nonce, followed by the .NET runtime's ChaCha20-Poly1305—from the in-house identity library (Wkl.Identity, CredentialProtection). The key is derived per purpose (DataSourceConnection, TotpSecret) from MasterKey with HKDF, and the purpose is also included in the authenticated data, so a sealed value can be opened only for its own purpose. The master key is never stored in the file. As a result:

• A copy of the file alone—a backup, an attachment to a support request—can't open a connection. The names of the data sources, the rules, and the assignments remain readable.
• With the same master key, the file works on any computer (to move it, transfer the file and the key separately). This is deliberately not DPAPI: DPAPI binds to a computer or an account and would make exactly this kind of move impossible.
• If the key doesn't match, or if a value in the file was tampered with, the check names every affected data source; by the nature of the encryption, the two causes can't be told apart.
• If the master key is lost, the connection strings are lost. Keep it separate from the backups of the file.
• Every value records the ID of the key it was sealed with (envelope format 2 of the identity library). The ID and the format are authenticated as well: a modified ID leads to rejection, never to opening with a different key.

A suitable key is 32 random bytes in Base64:

[Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))

File and access. SQLite runs in WAL mode: a reload reads while writes are in progress and always sees the state of the last commit, and a read covers all tables in one transaction. A folder or file that the service creates can be read only by the service account and the computer's administrators (on Windows, an access control list without inheritance containing these entries; elsewhere, mode 700/600). The administrators need that access for the server commands; without it, init-admin couldn't open the store of a service running as NetworkService. If init-admin creates the store before the service has ever run, --service-account names the account the service will run under. SQLite writes -wal and -shm files next to the database file; they inherit the folder's permissions, which is why the store needs a folder of its own. If an existing file can also be read by another account, the startup log names that account in a warning and changes nothing. Use one file per instance and only on a local disk: SQLite's locks aren't reliable on network shares.

The sign-in log, the load log, and the access log live in a second SQLite file next to the store, the log file. It grows with use; the store itself stays small and is quick to reload and back up. By default, it has the store's name with -logs appended and sits in the same folder—store/dbgwsvc-logs.db for store/dbgwsvc.db. `Gateway:Store:LogPath` names a different location.

People by their ID. The log file refers to a person by the ID of their row in the store, not by name: a renamed person keeps their history. The sets show the current name; if the person no longer exists, they show # and the ID.

Size, not age. The file stays within `Gateway:Store:LogMaxMegabytes` (default 4,096 MB). When it exceeds the limit, the service removes the oldest entries once an hour, in this order: individual slow requests, individual sign-in events, daily profile values, findings, the oldest accesses, and finally the oldest days of the hourly counts and of the load per person. Shapes and stored load hints stay; there are few of them, and they name no person. The retention periods of Gateway:SignInLog:HourDays, Gateway:LoadLog, and Gateway:AuditLog:Days apply in addition.

Access and backup. The log file gets the same access permissions as the store. It contains personal data (addresses, people, times) and no secrets. backup-store backs it up too; both copies, put back side by side, open like the original.

Moving from an earlier version. If the store still held the logs itself, the service moves them into the log file once at the first startup and then removes them from the store. The findings of the sign-in rules start with the events after the move; earlier events aren't evaluated retroactively.

An administration store belongs to exactly one instance of the service. Two instances on different computers can't share a file (no network shares, see above), and two instances on the same computer using the same file aren't supported. If you run several instances—for failover, say—each has its own store and therefore its own people, keys, and sessions; there's no synchronization between them. You get resilience with one instance and a backup you can restore (see Back up, restore, and move the administration store).

Every change made through the administration API writes a row to the change log in the same transaction (the ChangeLog set: time, who, action, set, row, values before and after; a secret such as a connection string appears only as the note “(neu gesetzt)” (newly set), never as a value or an envelope). Through the API, the change log can only be read. It's deliberately not tamper-proof: whoever can write to the file—the service account and the computer's administrators—can also change the log; it doesn't use a hash chain or a signature. If you need evidence that the operator of the computer can't alter, read ChangeLog continuously (the number increases strictly) and write it to a log outside this computer. The service doesn't offer such forwarding.

For a deletion policy under the General Data Protection Regulation (storage limitation, Art. 5(1)(e); right to erasure, Art. 17), the store actually behaves as follows:

People with traces are disabled, not deleted. The sign-in log and the load log refer to a person by the ID of their row, the change log by their name. If the person were deleted, these entries would point to nobody—or to a later person with the same name. The service therefore refuses deletion with 422 as soon as at least one of the following exists for the person:

• a personal key or a session, including revoked or expired ones;
• an individual event in the sign-in log or an entry in the load log for them;
• a change log entry they wrote themselves—the actor there is their name, their name with a key or “(Selbstbedienung)”, their Windows account, or their SQL Server login.

Entries that others wrote about the person, such as creating them or assigning their rules, aren't traces. A person who was created but never used can therefore be deleted. People who left entries in the sign-in log or the load log can be deleted once those entries have been removed from the log file and no other traces remain. Daily values of their profile and findings about them may still exist; they show the person as # with the ID.

Consequence for deleted people. Every change to a person is recorded in the change log with the values before and after—name, Windows account, and email address included. After the person is deleted, these rows remain, as does the person's ID as the actor of changes they made themselves. There's no retention period after which the service removes or pseudonymizes entries. This is a limitation of the service: a deletion policy that requires it can't be implemented with the service alone. It's possible only from outside, with the service stopped and a SQLite tool working on the file; doing so destroys the evidential value of the log and comparability with older backups, and backups still contain the entries.

To change the master key, see Change the master key; for backup, moving, and restore, see Back up, restore, and move the administration store.

• Configuration: Gateway:Store
• Server commands (reference)
• HTTP: administration (reference)