]> ruderich.org/simon Gitweb - safcm/safcm.git/log
safcm/safcm.git
3 years agoMakefile: add lint target to run shellcheck
Simon Ruderich [Wed, 12 May 2021 05:40:56 +0000 (07:40 +0200)]
Makefile: add lint target to run shellcheck

3 years agosync: remove "detected" log message in packages/services
Simon Ruderich [Sun, 9 May 2021 19:19:42 +0000 (21:19 +0200)]
sync: remove "detected" log message in packages/services

Don't waste one full log message to show the detected system. Instead,
append it to an existing log message.

Reorder log call in syncPackagesDebian() so the log message comes before
calling debianInstalledPackages(). This way errors from this function
show clearly where they originate.

3 years agoremote: tests: hide testRunner functions in test output
Simon Ruderich [Sun, 9 May 2021 18:44:02 +0000 (20:44 +0200)]
remote: tests: hide testRunner functions in test output

This is not perfect because they are called from files like
cmd/safcm-remote/sync/commands.go which is not very helpful to figure
out which test case failed. However, it's better than showing
cmd/safcm-remote/sync/sync_test.go which was the previous behavior.

3 years agotests: improve another comment
Simon Ruderich [Sun, 9 May 2021 18:38:37 +0000 (20:38 +0200)]
tests: improve another comment

3 years agochanges: add "(hidden)" to commands with no output and quiet mode
Simon Ruderich [Sun, 9 May 2021 18:20:16 +0000 (20:20 +0200)]
changes: add "(hidden)" to commands with no output and quiet mode

This should make it more obvious why these commands are not shown.

3 years agosafcm: tests: remove empty line
Simon Ruderich [Sun, 9 May 2021 18:15:09 +0000 (20:15 +0200)]
safcm: tests: remove empty line

3 years agorpc: remove empty line
Simon Ruderich [Sun, 9 May 2021 18:14:39 +0000 (20:14 +0200)]
rpc: remove empty line

3 years agoconfig: disallow negative permissions
Simon Ruderich [Sun, 9 May 2021 18:02:31 +0000 (20:02 +0200)]
config: disallow negative permissions

3 years agorpc: replace append to sshOpts with assignment in DialSSH()
Simon Ruderich [Sun, 9 May 2021 17:51:02 +0000 (19:51 +0200)]
rpc: replace append to sshOpts with assignment in DialSSH()

sshOpts is empty by default and assigned only once. Don't suggest that
this line is supposed to add to an existing value.

3 years agosafcm: shorten error message on file conflict
Simon Ruderich [Sun, 9 May 2021 10:55:54 +0000 (12:55 +0200)]
safcm: shorten error message on file conflict

It's obvious that the message talks about "files" because the path is
given immediately afterwards.

3 years agosafcm: use better variable name in hostsToSync()
Simon Ruderich [Sun, 9 May 2021 10:47:40 +0000 (12:47 +0200)]
safcm: use better variable name in hostsToSync()

Having nameMatched and hostMatched is confusing. Follow the comment and
use hostAdded which describes the meaning more clearly.

3 years agosafcm: don't color output if stderr is redirected
Simon Ruderich [Sun, 9 May 2021 10:46:18 +0000 (12:46 +0200)]
safcm: don't color output if stderr is redirected

All log messages are printed to stderr. It's confusing to still get
colored output when redirecting stderr to a file; only redirecting
stdout as well fixed this.

3 years agochanges: change dry-run messages to use "will"
Simon Ruderich [Sun, 9 May 2021 10:44:29 +0000 (12:44 +0200)]
changes: change dry-run messages to use "will"

Don't suggest that the change already took place. "dry-run" is still
printed to make it clear why no changes were made.

3 years agochanges: tests: remove output for dry-run commands
Simon Ruderich [Sun, 9 May 2021 10:39:30 +0000 (12:39 +0200)]
changes: tests: remove output for dry-run commands

Commands during a dry-run have no output. Don't use incorrect test
input.

3 years agoconfig: tests: fix typo in group name
Simon Ruderich [Sun, 9 May 2021 10:34:44 +0000 (12:34 +0200)]
config: tests: fix typo in group name

3 years agoconfig: return map from TransitivelyDetectedGroups()
Simon Ruderich [Sun, 9 May 2021 10:32:53 +0000 (12:32 +0200)]
config: return map from TransitivelyDetectedGroups()

This is less clean than the original slice of strings. However, it
removes unnecessary code as the caller requires a map instead of a
slice.

3 years agoconfig: use more explicit variable name in ResolveHostGroups()
Simon Ruderich [Sun, 9 May 2021 10:31:26 +0000 (12:31 +0200)]
config: use more explicit variable name in ResolveHostGroups()

3 years agoconfig: rewrap line in ResolveHostGroups()
Simon Ruderich [Sun, 9 May 2021 10:31:08 +0000 (12:31 +0200)]
config: rewrap line in ResolveHostGroups()

3 years agoconfig: tests: replace FullPermToFileMode() with fs constants
Simon Ruderich [Sun, 9 May 2021 10:29:08 +0000 (12:29 +0200)]
config: tests: replace FullPermToFileMode() with fs constants

FullPermToFileMode() is necessary in a few places but the tests should
use the common way in Go to set permissions. This is less confusing for
the reader.

3 years agosync: remove duplicate code in triggerPaths()
Simon Ruderich [Sun, 9 May 2021 10:28:33 +0000 (12:28 +0200)]
sync: remove duplicate code in triggerPaths()

3 years agosync: include size in binary "diff"
Simon Ruderich [Sun, 9 May 2021 10:27:28 +0000 (12:27 +0200)]
sync: include size in binary "diff"

3 years agosync: tests: use variable to reduce line wrapping
Simon Ruderich [Sun, 9 May 2021 10:24:12 +0000 (12:24 +0200)]
sync: tests: use variable to reduce line wrapping

3 years agosync: tests: use "..." instead of `...` for regular strings
Simon Ruderich [Sun, 9 May 2021 10:20:39 +0000 (12:20 +0200)]
sync: tests: use "..." instead of `...` for regular strings

3 years agosync: tests: use strict perm for os.WriteFile() in CreateFile()
Simon Ruderich [Sun, 9 May 2021 10:15:58 +0000 (12:15 +0200)]
sync: tests: use strict perm for os.WriteFile() in CreateFile()

This parameter is modified by the umask. The proper permissions are set
by os.Chmod() afterwards. Don't confuse the reader by using a value
which is not relevant.

3 years agosync: tests: wrap overlong line
Simon Ruderich [Sun, 9 May 2021 10:15:42 +0000 (12:15 +0200)]
sync: tests: wrap overlong line

3 years agosync: tests: use CreateDirectoryExists()
Simon Ruderich [Sun, 9 May 2021 10:14:16 +0000 (12:14 +0200)]
sync: tests: use CreateDirectoryExists()

3 years agosync: tests: check return value of setDefaults()
Simon Ruderich [Sun, 9 May 2021 10:13:37 +0000 (12:13 +0200)]
sync: tests: check return value of setDefaults()

3 years agosync: tests: properly scope err variable
Simon Ruderich [Sun, 9 May 2021 10:12:22 +0000 (12:12 +0200)]
sync: tests: properly scope err variable

3 years agosync: tests: go fmt
Simon Ruderich [Sun, 9 May 2021 09:42:31 +0000 (11:42 +0200)]
sync: tests: go fmt

3 years agosync: tests: rename triggers to expTriggers
Simon Ruderich [Sun, 9 May 2021 09:41:57 +0000 (11:41 +0200)]
sync: tests: rename triggers to expTriggers

3 years agosafcm: tests: test detected groups with leading/trailing space
Simon Ruderich [Sun, 9 May 2021 09:38:42 +0000 (11:38 +0200)]
safcm: tests: test detected groups with leading/trailing space

3 years agosafcm: go fmt
Simon Ruderich [Sun, 9 May 2021 09:38:27 +0000 (11:38 +0200)]
safcm: go fmt

3 years agosafcm: strip invalid characters from detected os/arch groups
Simon Ruderich [Sun, 9 May 2021 09:36:44 +0000 (11:36 +0200)]
safcm: strip invalid characters from detected os/arch groups

Handle them like any other detected group because the remote can send
invalid values. The current code can handle arbitrary group names just
fine but it's better to treat all untrusted input the same.

3 years agoImprove and add comments
Simon Ruderich [Sun, 9 May 2021 08:10:56 +0000 (10:10 +0200)]
Improve and add comments

3 years agoci: suppress shellcheck warnings
Simon Ruderich [Sun, 9 May 2021 08:03:34 +0000 (10:03 +0200)]
ci: suppress shellcheck warnings

$flags can contain multiple flags and we must expand them here.

3 years agoREADME: mention root/wheel on BSD systems
Simon Ruderich [Sun, 9 May 2021 08:02:35 +0000 (10:02 +0200)]
README: mention root/wheel on BSD systems

3 years agoREADME: fix typo
Simon Ruderich [Sun, 9 May 2021 08:02:30 +0000 (10:02 +0200)]
README: fix typo

3 years agotests: run commands without side effects in end-to-end tests
Simon Ruderich [Sat, 8 May 2021 08:20:57 +0000 (10:20 +0200)]
tests: run commands without side effects in end-to-end tests

3 years agoREADME: multiple improvements
Simon Ruderich [Thu, 29 Apr 2021 05:43:05 +0000 (07:43 +0200)]
README: multiple improvements

3 years agotests: add end-to-end test with configuration without any changes
Simon Ruderich [Wed, 28 Apr 2021 10:29:50 +0000 (12:29 +0200)]
tests: add end-to-end test with configuration without any changes

3 years agosync: run most tests which modify the host only in CI
Simon Ruderich [Wed, 28 Apr 2021 10:26:18 +0000 (12:26 +0200)]
sync: run most tests which modify the host only in CI

This is especially important in case the user executes the tests as root
(no recommended but not prevented either). Permissions on paths like /
or /tmp which differ from those expected by the test could otherwise be
modified by the tests.

However, the end-to-end SSH tests which write /tmp/safcm-remote-$uid
(but no other paths) are still run so we get proper coverage of basic
features.

3 years agotests: use filetest.CreateFifo() in config tests
Simon Ruderich [Wed, 28 Apr 2021 06:29:49 +0000 (08:29 +0200)]
tests: use filetest.CreateFifo() in config tests

3 years agosync: remove duplicate "priority" from group priority log message
Simon Ruderich [Wed, 28 Apr 2021 05:57:52 +0000 (07:57 +0200)]
sync: remove duplicate "priority" from group priority log message

Now that the configuration option is called "group_priority" we can
remove the second "priority" and spell out "descending" (takes less
space than before).

3 years agosafcm: group_priority: use increasing priority values internally
Simon Ruderich [Wed, 28 Apr 2021 05:51:16 +0000 (07:51 +0200)]
safcm: group_priority: use increasing priority values internally

It feels more natural that groups with higher priority have a higher
numeric value.

3 years agogo fmt
Simon Ruderich [Wed, 28 Apr 2021 05:48:59 +0000 (07:48 +0200)]
go fmt

3 years agoconfig: rename group_order to group_priority
Simon Ruderich [Wed, 28 Apr 2021 05:48:18 +0000 (07:48 +0200)]
config: rename group_order to group_priority

"Priority" describes the actual function better and it was already used
internally in the code anyway.

3 years agosafcm: group_order: higher priority for listed groups over remaining groups
Simon Ruderich [Wed, 28 Apr 2021 05:32:37 +0000 (07:32 +0200)]
safcm: group_order: higher priority for listed groups over remaining groups

The priority for all groups listed in `group_order` was properly
respected: Early entries had the highest priority. However, groups which
were not listed in `group_order` had a higher priority and overwrote
files from all groups configured in `group_order`.

The priority is now as expected (from high to low): host itself, groups
in group_order (from high to low), remaining groups.

3 years agoREADME: multiple improvements
Markus Petri [Sun, 25 Apr 2021 13:33:56 +0000 (15:33 +0200)]
README: multiple improvements

3 years agochanges: display "no changes" when nothing was changed
Simon Ruderich [Sun, 25 Apr 2021 07:57:37 +0000 (09:57 +0200)]
changes: display "no changes" when nothing was changed

It's confusing to show nothing at all when no changes occur on a host.
To inform the user that the sync was successful but nothing changed
display a short message.

3 years agoMakefile: move command to test target
Simon Ruderich [Wed, 21 Apr 2021 06:28:08 +0000 (08:28 +0200)]
Makefile: move command to test target

The permissions are only relevant when running tests.

3 years agogo fmt
Simon Ruderich [Wed, 21 Apr 2021 06:28:00 +0000 (08:28 +0200)]
go fmt

3 years agotests: add very basic end-to-end test with real ssh server
Simon Ruderich [Wed, 21 Apr 2021 06:16:40 +0000 (08:16 +0200)]
tests: add very basic end-to-end test with real ssh server

At the moment only the helper upload without any actual configuration is
tested.

3 years agorpc: always create remote helper with user's group
Simon Ruderich [Wed, 21 Apr 2021 06:16:03 +0000 (08:16 +0200)]
rpc: always create remote helper with user's group

If the group does not match the current user's group then the remote
helper will be uploaded again on each run.

3 years agoRevert "ci: run tests with verbose output"
Simon Ruderich [Tue, 20 Apr 2021 17:53:07 +0000 (19:53 +0200)]
Revert "ci: run tests with verbose output"

This reverts commit 444e9874424580361e00547b597d213370045057.

Turns out this is too noisy and less useful than I expected.

3 years agoci: use ./* in `rm -rf` to guard against option-like filenames
Simon Ruderich [Tue, 20 Apr 2021 17:52:01 +0000 (19:52 +0200)]
ci: use ./* in `rm -rf` to guard against option-like filenames

This won't be an issue in our case but it's good practice anyway.

Found by shellcheck

3 years agosafcm: print unfinished hosts on Ctrl-C
Simon Ruderich [Tue, 20 Apr 2021 14:17:53 +0000 (16:17 +0200)]
safcm: print unfinished hosts on Ctrl-C

3 years agoremote: go fmt
Simon Ruderich [Tue, 20 Apr 2021 09:44:16 +0000 (11:44 +0200)]
remote: go fmt

3 years agoremote: show group/trigger in verbose log for commands
Simon Ruderich [Tue, 20 Apr 2021 09:43:02 +0000 (11:43 +0200)]
remote: show group/trigger in verbose log for commands

This helps the user to figure out why the command was executed.

3 years agosafcm: use Command struct instead of string to run commands
Simon Ruderich [Tue, 20 Apr 2021 09:21:10 +0000 (11:21 +0200)]
safcm: use Command struct instead of string to run commands

3 years agoREADME: mention group and remove obvious sentence
Simon Ruderich [Sun, 18 Apr 2021 09:13:48 +0000 (11:13 +0200)]
README: mention group and remove obvious sentence

3 years agoREADME: mention why YAML was chosen
Simon Ruderich [Sun, 18 Apr 2021 08:24:48 +0000 (10:24 +0200)]
README: mention why YAML was chosen

3 years agoci: run tests with verbose output
Simon Ruderich [Sun, 18 Apr 2021 07:57:30 +0000 (09:57 +0200)]
ci: run tests with verbose output

3 years agoci: check for untracked and unignored files
Simon Ruderich [Sun, 18 Apr 2021 07:52:20 +0000 (09:52 +0200)]
ci: check for untracked and unignored files

3 years agoci: build.sr.ht: run all tests also as root
Simon Ruderich [Sun, 18 Apr 2021 07:51:35 +0000 (09:51 +0200)]
ci: build.sr.ht: run all tests also as root

3 years agoconfig: only skip "invalid sticky" tests for non-root users
Simon Ruderich [Sun, 18 Apr 2021 07:42:43 +0000 (09:42 +0200)]
config: only skip "invalid sticky" tests for non-root users

3 years agoAdd basic support for OpenBSD
Simon Ruderich [Sat, 17 Apr 2021 12:48:42 +0000 (14:48 +0200)]
Add basic support for OpenBSD

Installing packages and starting/enabling services is not yet supported.
There are minor limitations when handling symlinks (see README.adoc).

/var/tmp is a symlink on OpenBSD so just remove this test.

3 years agoMakefile: set -race via GOFLAGS= variable
Simon Ruderich [Sat, 17 Apr 2021 07:43:27 +0000 (09:43 +0200)]
Makefile: set -race via GOFLAGS= variable

3 years agoconfig: groups: use "member" in "not found" error message
Simon Ruderich [Sat, 17 Apr 2021 06:42:52 +0000 (08:42 +0200)]
config: groups: use "member" in "not found" error message

"group" was confusing, especially because "group" occurs at the
beginning of the error marking the group where the error occurred.

3 years agoci: also run tests with strict umask
Simon Ruderich [Sat, 17 Apr 2021 06:36:05 +0000 (08:36 +0200)]
ci: also run tests with strict umask

3 years agoAdd basic support for FreeBSD
Simon Ruderich [Sat, 17 Apr 2021 06:10:17 +0000 (08:10 +0200)]
Add basic support for FreeBSD

Installing packages and starting/enabling services is not yet supported.
There are minor limitations when handling symlinks (see README.adoc).

3 years agorpc: prepare to support more operating systems
Simon Ruderich [Wed, 14 Apr 2021 11:47:28 +0000 (13:47 +0200)]
rpc: prepare to support more operating systems

3 years agoAdd build manifest for build.sr.ht
Simon Ruderich [Thu, 15 Apr 2021 21:23:06 +0000 (23:23 +0200)]
Add build manifest for build.sr.ht

3 years agoREADME.adoc: improve list of supported operating systems
Simon Ruderich [Wed, 14 Apr 2021 12:33:23 +0000 (14:33 +0200)]
README.adoc: improve list of supported operating systems

3 years agorpc: document ConnEventType constants
Simon Ruderich [Wed, 14 Apr 2021 07:06:19 +0000 (09:06 +0200)]
rpc: document ConnEventType constants

3 years agoREADME.adoc: mention full file content is sent to remote
Simon Ruderich [Wed, 14 Apr 2021 06:59:11 +0000 (08:59 +0200)]
README.adoc: mention full file content is sent to remote

3 years agotestutil: show file/line of caller in messages
Simon Ruderich [Wed, 14 Apr 2021 06:54:20 +0000 (08:54 +0200)]
testutil: show file/line of caller in messages

3 years agoconfig: forbid ":remove" groups which conflict with hosts
Simon Ruderich [Tue, 13 Apr 2021 06:17:09 +0000 (08:17 +0200)]
config: forbid ":remove" groups which conflict with hosts

Defining these groups had no effect but might confuse the user. As they
don't make any sense forbid them.

3 years agosafcm: forbid syncing groups which depend on "detected" groups
Simon Ruderich [Tue, 13 Apr 2021 06:04:03 +0000 (08:04 +0200)]
safcm: forbid syncing groups which depend on "detected" groups

3 years agoconfig: add newline between error message and explanation
Simon Ruderich [Tue, 13 Apr 2021 05:47:06 +0000 (07:47 +0200)]
config: add newline between error message and explanation

This makes the result more readable.

3 years agochanges: shorten output to one line with quiet and only silent commands
Simon Ruderich [Sat, 10 Apr 2021 18:24:03 +0000 (20:24 +0200)]
changes: shorten output to one line with quiet and only silent commands

3 years agochanges: hide commands if quiet and dry-run is active
Simon Ruderich [Sat, 10 Apr 2021 18:16:06 +0000 (20:16 +0200)]
changes: hide commands if quiet and dry-run is active

The goal of quiet is to get a quick overview what is happening or what
is going to happen (when combined with dry-run). While quiet worked fine
for non dry-run syncs it was not very helpful for dry-runs. This change
makes the output much more compact and readable with many commands and
hosts.

3 years agochanges: refactor into separate function and add basic tests
Simon Ruderich [Sat, 10 Apr 2021 17:53:05 +0000 (19:53 +0200)]
changes: refactor into separate function and add basic tests

3 years agoREADME.adoc: mention quoted strings in limitations
Simon Ruderich [Sat, 10 Apr 2021 17:52:02 +0000 (19:52 +0200)]
README.adoc: mention quoted strings in limitations

3 years agoMakefile: add test target
Simon Ruderich [Fri, 9 Apr 2021 05:27:51 +0000 (07:27 +0200)]
Makefile: add test target

3 years agoconfig: permit "all" and host group in .InGroup of templates
Simon Ruderich [Thu, 8 Apr 2021 17:02:51 +0000 (19:02 +0200)]
config: permit "all" and host group in .InGroup of templates

These special groups were not handled properly.

3 years agoAdd README.adoc
Simon Ruderich [Thu, 8 Apr 2021 06:35:05 +0000 (08:35 +0200)]
Add README.adoc

3 years agoconfig: forbid duplicate host names
Simon Ruderich [Wed, 7 Apr 2021 17:11:45 +0000 (19:11 +0200)]
config: forbid duplicate host names

3 years agosync: abort if stdin/stdout is a terminal
Simon Ruderich [Tue, 6 Apr 2021 09:50:58 +0000 (11:50 +0200)]
sync: abort if stdin/stdout is a terminal

Notify the user that is probably not what they intended to do.

3 years agoremote: go fmt
Simon Ruderich [Tue, 6 Apr 2021 09:47:55 +0000 (11:47 +0200)]
remote: go fmt

3 years agoremote: add "sync" sub-command to perform changes
Simon Ruderich [Tue, 6 Apr 2021 09:44:47 +0000 (11:44 +0200)]
remote: add "sync" sub-command to perform changes

Performing the changes when no command line arguments were given became
problematic in ddd21f0 (remote: add ainsl sub-command ("append if no
such line"), 2021-04-04). With "ainsl" as sub-command a user might be
tempted to run the helper manually to test it. But running it without
any arguments causes it to hang (waiting for commands from stdin) which
is confusing. Instead, use the new separate "sync" sub-command to
perform the changes and abort when no sub-command is given.

3 years agosafcm: add tests for logEvent()
Simon Ruderich [Tue, 6 Apr 2021 09:27:32 +0000 (11:27 +0200)]
safcm: add tests for logEvent()

3 years agosafcm: add "version" sub-command
Simon Ruderich [Tue, 6 Apr 2021 08:57:57 +0000 (10:57 +0200)]
safcm: add "version" sub-command

Embed current Git and Go version at build time.

3 years agochanges: remove unused code
Simon Ruderich [Mon, 5 Apr 2021 10:06:14 +0000 (12:06 +0200)]
changes: remove unused code

3 years agochanges: add tests for output with terminal
Simon Ruderich [Mon, 5 Apr 2021 10:05:56 +0000 (12:05 +0200)]
changes: add tests for output with terminal

3 years agotests: add and use testutil package to reduce duplication
Simon Ruderich [Mon, 5 Apr 2021 09:20:02 +0000 (11:20 +0200)]
tests: add and use testutil package to reduce duplication

This is not as explicit as the previous code but still good enough. And
it gets rid of quite some boilerplate.

3 years agoconfig: remove unnecessary os.Chdir in test
Simon Ruderich [Mon, 5 Apr 2021 08:47:40 +0000 (10:47 +0200)]
config: remove unnecessary os.Chdir in test

3 years agotests: go fmt and rewrap
Simon Ruderich [Mon, 5 Apr 2021 08:44:52 +0000 (10:44 +0200)]
tests: go fmt and rewrap

3 years agotests: use subtests
Simon Ruderich [Mon, 5 Apr 2021 08:36:02 +0000 (10:36 +0200)]
tests: use subtests

This removes the need to print the name of the test in each t.Errorf()
call which reduces unnecessary code. It also permits skipping tests
which will be used in the future.

3 years agosafcm: add -q (quiet) command line option
Simon Ruderich [Mon, 5 Apr 2021 07:35:37 +0000 (09:35 +0200)]
safcm: add -q (quiet) command line option

3 years agoremote: add ainsl sub-command ("append if no such line")
Simon Ruderich [Sun, 4 Apr 2021 21:35:50 +0000 (23:35 +0200)]
remote: add ainsl sub-command ("append if no such line")

It is preferred to deploy complete files by putting them in the files/
directory of a group. However, sometimes this is not possible because
parts of the file's content are unknown or managed by other programs or
users. An example is .ssh/authorized_keys which should contain certain
keys but which is also managed manually. `ainsl` permits adding a key to
the file without rewriting it completely.

`ainsl` can be used by specifying the following command:

    $SAFCM_HELPER ainsl /path/to/file line-to-add

Per default non-existent files are an error. To create the file if
necessary use:

    $SAFCM_HELPER ainsl -create /path/to/file line-to-add

The environment variable $SAFCM_HELPER is set when executing commands
and contains the absolute path to the remote helper.